[0.21.1] Flight Recorder

[Benache]

I wanted to analyze data post-flight, and i stumbled upon crwper's Logomatic. But it didn't work with the new unity format of parts, so i first took on myself to modify the part (as it was CC BY SA).

But then i felt the need to add more data to the log file (orbit data like ApA e.g.), so i decided to make my own (first) mod. I only realized afterwards that his source code was available...

Anyway, here is FLIGHT RECORDER.

I'm quite satisfied with the look of the parts

DOWNLOAD

SpacePort

LICENSE

CC BY SA 3.0

Source is in the archive

DESCRIPTION

There are two parts available, Flight Recorder and Sensor Recorder (as seen on the picture above).

Flight Recorder

Saves a bunch of data to a log file (see below).

The data to save is configurable in the Data.cfg file located in GameData/FlightRecorder/Plugin/PluginData/

Explanations are contained in the file. Basically, you can had any method, property and field of the PartModule class.

 There are two syntaxes available 

// SINGLE VALUE
//  Header : member1[.member2[.member3[...]]]
//   Will look into the Flight Recorder members for the specified 'member1' (and sub-members) and return its value.
//   'member1' must be a member (or an inherited member) of the class PartModule, which is the class of the Flight Recorder.
//   Don't worry, there are a lot of examples below.

// SUM
//  Header : <moduleType>.member1[.member2[.member3[...]]]
//   will look in every part of the vessel for a PartModule of the type 'ModuleType', get the value of its specified member
//   'member1' (and sub-members) and return the sum of their value (in a float object, so limits are ±1.5E−45 and ±3.4E+38).

// In both syntaxes, members can either be Methods, Properties and Fields .
// You can't add arguments to a Method. You can had parenthesis to Methods for ease of reading, but they will be ignored.

Sensor Recorder

Saves the data of a scientific sensor attached to it (in the white hatched zone if you want it to be pretty).

If you want to record multiple sensor, use as many Sensor Recorders, a file will be created for each Sensor Recorder.

Remember to turn on your sensor !!

Currently supported :

- [squad] GRAVMAX Negative Gravioli Detector

- [squad] PresMat Barometer

- [squad] Double-C Accelerometer

- [squad] 2HOT Thermometer

- [KSP Interstellar] Dual Technique Magnetometer

Log Files

All log files are located in GameData/FlightRecorder/Plugin/PluginData/FlightRecorder/

The file is created when the ship is loaded on the launchpad, but recording begins at liftoff.

Data are saved every 200ms (not editable for the moment, although it would be easy now i added the Data.cfg file...).

CHANGELOG

2013/10/21

- added compatibility with Tech Tree (you will need advanced electrics).

2013/10/15

- added support for the methods

- added possibility to add a value from different modules of the ship.

- improved the cfg file

2013/10/13

- added the ability to choose log values with the Data.cfg file.

- increased resistance of the flight recorder to crash

2013/10/12

- added totalThrust and totalMass

- removed a lot of useless fields from the FlightRecorder. The log files are much more lighter now !

- added toggle to enable/disable recording.

- added support for [KSP Interstellar] Dual Technique Magnetometer

TODO

- allow user to add checkpoint

- allow user to choose recording interval

- DONE : allow user to choose what data is recorded

- add support for more scientific instrument (tell me which)

- OBSOLETE : log RCS usage, engine gimbal angles, throttle setting (not just thrust amounts) and reaction wheels usage

- OBSOLETE : remove data that can be obtained by an instrument / Add those instruments.

DATA DOCUMENTATION : I need your help !!

I didn't find any comprehensive documentation. Quite annoying. If anyone can provide a link or a document, that would be greatly appreciated.

For the moment, I use :

- the public interface of the KSP c# assembly (\KSP\KSP_Data\Managed\Assembly-CSharp.dll)

- the KSP wiki community API documentation which is really poor for the moment

- examples of code on github (mostly Telemachus and MechJeb) or on the wiki

I need your help to improve the part.cfg file. If you find new usefull data, please share here.

The goal would be to have a default file with sections and comments. A bit like that :

//VESSEL DYNAMICS
// Orbital velocity : velocity measured in a geocentric reference
obt_velocity              : vessel.obt_velocity.magnitude
//obt_velocity.x            : vessel.obt_velocity.x
//obt_velocity.y            : vessel.obt_velocity.y
//obt_velocity.z            : vessel.obt_velocity.z

// Surface velocity : velocity measured in a local reference
srf_velocity              : vessel.srf_velocity.magnitude
//srf_velocity.x            : vessel.srf_velocity.x //east
//srf_velocity.y            : vessel.srf_velocity.y //north
//srf_velocity.z            : vessel.srf_velocity.z //up

//ORBITAL VALUES
// Basic orbital data
orbitalEnergy           : vessel.orbit.orbitalEnergy
period                  : vessel.orbit.period
inclination             : vessel.orbit.inclination
//argumentOfPeriapsis       : vessel.orbit.argumentOfPeriapsis
//semiMajorAxis             : vessel.orbit.semiMajorAxis
//semiMinorAxis             : vessel.orbit.semiMinorAxis
//semiLatusRectum           : vessel.orbit.semiLatusRectum
eccentricity            : vessel.orbit.eccentricity

In this exemple, i added comment to surface velocity, but it must be false, I don't know anything about that

Edited October 21, 2013 by Benache

[Gristle]

Good idea. I can see this creating some huge files though. Is there a way to toggle the log file creation on and off? A future feature would be a way to insert "checkpoints" into the log file to quickly locate events.

[Benache]

I will add a toggle, good idea, i'm mostly using it for optimization of the launch so i didn't think of it.

You can use the third column ("situation") as a checkpoint. It says FLYING / SUBORBITAL / ORBITAL etc. But i'll think about a way of manually adding checkpoint without corrupting the file.

I posted a new release : i just added totalthrust and totalmass, which are quite usefull...

I think i'm gonna remove 80% of the columns though. They are useless to me, so please feel free to ask for a specific data you need.

Edited October 12, 2013 by Benache

[lincourtl]

Ooo, awesome! The parts look fantastic too. I'm going to try it out right now.

One minor suggestion for a future version -- allow the user to set the recording interval, and toggle which data to record, ala Graphotron 2000. I'll let you know any other notes I might have after my test flights.

On edit: This is fantastic. It's going to be so useful. You're right that large swaths of that data could go. I don't have any specific recommendations though, apart from what I said above about allowing the user to set the recording interval, and toggle which data to record (of those which you decide to keep).

Edited October 12, 2013 by lincourtl

[mwlue]

Look coooooooool~! Nice tools for analyzing your rocket performance.

[drtedastro]

good stuff. have been looking for this. thanks.

[Benache]

updated : added toggles, and removed useless data.

[Scotius]

Hey, more science to do Benache, may i have a suggestion/request of sorts? Will you be adding input data from stock science equipment? If yes, would you be so kind to add input from KSP Interstellar magnetometer? Level of magnetic field flux compared to orbit's height. I tried to do this with Graphotron 2000, but it didn't work.

[jpkerman]

If you want players to be able to actually use this to analyze their flights and why they may be unstable or failing to achieve their goals, RCS usage, engine gimbal angles, throttle setting (not just thrust amounts) and reaction wheels usage would be nice to have. If this was a flight recorder on real life that is the input info that engineers and crash investigators would want to know. What was the crew/flight computers having to do to fly the ship.

[Benache]

I initially supported only the four basic (squad) instruments.

As you asked nicely, I just added support for the KSP interstellar plugin's "Dual Technique Magnetometer".

[Benache]

If you want players to be able to actually use this to analyze their flights and why they may be unstable or failing to achieve their goals, RCS usage, engine gimbal angles, throttle setting (not just thrust amounts) and reaction wheels usage would be nice to have. If this was a flight recorder on real life that is the input info that engineers and crash investigators would want to know. What was the crew/flight computers having to do to fly the ship.

Noted in the todo list, thx.

[Scotius]

Thank you very much Benache ()b

[sirkut]

This is awesome. Thanks for your work!

[Benache]

I think i'm gonna remove the atmPrssure from the flight recorder, as you can obtain it from the barometer.

On the same spirit, I will make a radar altimeter part, and move terrainAltitude and terrainNormal into it.

And the same for geeForce based on this design :

If you have any other scientific instrument in mind, existing or not, please suggest.

Edited October 13, 2013 by Benache

[Benache]

Thanks for your support !

[TelluriumCrystal]

This is an awesome data collecting tool! Nice work!

I am having some issues with the volume of data though... What are you guys using to view this?

As far as science instruments go, here is my suggestion: Latitude & Longitude = GPS

Edited October 13, 2013 by Thrfoot

[GraphTheData]

Firstly Benache, thanks for taking your time to put this together, I've been looking for something like this for a while, and I really like the volume of data to be honest. It's far easier truncating or expunging the data than not having it if you wanted it. Would it be possible to maybe get a cfg file we could comment/uncomment required fields?

I did have a question regarding some of these numbers, obviously the game is returning integers, and most can be extrapolated (example, acceleration.x,y,z was not immediately clear, but the velocity magnitude numbers were too high to be measured in G's) are the returns documented anywhere?

Again thank you for the mod, getting some experience graphing.

[TheZofDeath]

Just a quick question, how exactly do you go from raw data in the logs to the graphs? I can get a graph using excell, but it isn't really helpful. What sort of settings and such did you use for the example antimatter graph you put up?

[drtedastro]

Before you remove items, is there not a way that the user can / could decide what to use and what not to ??? Just because something is available in another location does not mean that it is not needed or wanted here. I am just starting to use and test this and i hope that you don't start cutting too much out of it so soon.

I appreciate what you have done and i look forward to integrating it with some stuff that i am working on.

Regards,

[Benache]

I am having some issues with the volume of data though... What are you guys using to view this?

I'm using excel ftm, but I will be using XnGrace when I need to do Science

As far as science instruments go, here is my suggestion: Latitude & Longitude = GPS

Thanks, good idea. Though it is part of the nav system... I will add it but not remove it from the flight recorder.

Firstly Benache, thanks for taking your time to put this together, I've been looking for something like this for a while, and I really like the volume of data to be honest. It's far easier truncating or expunging the data than not having it if you wanted it. Would it be possible to maybe get a cfg file we could comment/uncomment required fields?

I'm currently searching the best way to allow user customization. Maybe the cfg file is the best way. Looking into serialization now...

I think i'm gonna allow users to make several cfg file, and add a selection box in the gui. If i can do it, honestly i didn't look much into the GUI and it's kind of obscure to me right now.

I did have a question regarding some of these numbers, obviously the game is returning integers, and most can be extrapolated (example, acceleration.x,y,z was not immediately clear, but the velocity magnitude numbers were too high to be measured in G's) are the returns documented anywhere?

Well, that's my problem.

I use :

- the Visual Studio object explorer to view the public interface of the KSP c# assembly (\KSP\KSP_Data\Managed\Assembly-CSharp.dll)

- the KSP wiki community API documentation which is really poor for the moment

- examples of code on github (mostly Telemachus and MechJeb) or on the wiki

But no, i didn't find any comprehensive documentation. Quite annoying. If anyone can help...

Just a quick question, how exactly do you go from raw data in the logs to the graphs? I can get a graph using excell, but it isn't really helpful. What sort of settings and such did you use for the example antimatter graph you put up?

I added to the zip an excel sheet with macros, in the logs directory. Use the tools window that will pop up. It doesn't handle sensor detectors, but you'll get the idea by right clicking on the graph and looking at the data source.

Before you remove items, is there not a way that the user can / could decide what to use and what not to ??? Just because something is available in another location does not mean that it is not needed or wanted here. I am just starting to use and test this and i hope that you don't start cutting too much out of it so soon.

Already thinking about that.

I appreciate what you have done and i look forward to integrating it with some stuff that i am working on.

Please do. Don't forget to quote me.

Again, thanks for all your kind cheerings. Please rate the mod on the SpacePort, the better grade it gets, the more it will be downloaded, and we will have more suggestions / help from kerbonauts. Also it will cheer me up and i will work on it.

Edited October 13, 2013 by Benache

[m4ti140]

May I suggest something? - Give flight recorders high crashTolerance, enabling them to survive catastrophes; black boxes are meant to be durable - just an immersion thing, but since crashing is an important part of gameplay in KSP, it would be fun to search for recorders around wrecks.

[Benache]

Would it be possible to maybe get a cfg file we could comment/uncomment required fields?

Done !!

That wasn't easy, i had to learn serialization and it's quite tricky.

Config file is such :

// FLIGHT RECORDER by Géraud Benazet
// License : CC BY SA 3.0 http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB
// See forum thread for questions http://forum.kerbalspaceprogram.com/threads/52403-0-21-1-Flight-Recorder

// SYNTAX
// Header : Value 
//  - Header will be written at the first line of your csv file
//  - Value must be a member of the PartModule class. For the moment only Properties and Fields are supported. I'll add Methods later
// You can add comments at the end of the lines
// You can use white spaces, tabulations and blank lines
// 

// LIMITATIONS
// Until I take the time to code it properly, the first 4 columns aren't customizable
//  Date : computer date the line was written
//  TotalMass : vessel.GetTotalMass()
//  TotalThrust : total thrust (calculated)
//  GeeForce : FlightGlobals.getGeeForceAtPosition(this.transform.position).magnitude

// Please keep these comments in your config files. It will be easier to share.

missionTime             : vessel.missionTime
situation               : vessel.situation
currentStage            : vessel.currentStage
latitude                : vessel.latitude
longitude               : vessel.longitude
altitude                : vessel.altitude
acceleration            : vessel.acceleration.magnitude
specificAcceleration    : vessel.specificAcceleration
verticalSpeed           : vessel.verticalSpeed
orbitVelocity           : vessel.obt_velocity.magnitude
surfaceVelocity         : vessel.srf_velocity.magnitude
angularVelocity         : vessel.angularVelocity.magnitude
atmDensity              : vessel.atmDensity
terrainAltitude         : vessel.terrainAltitude
terrainNormal.x         : vessel.terrainNormal.x
terrainNormal.y         : vessel.terrainNormal.y
terrainNormal.z         : vessel.terrainNormal.z
orbitalEnergy           : vessel.orbit.orbitalEnergy
period                  : vessel.orbit.period
inclination             : vessel.orbit.inclination
eccentricity            : vessel.orbit.eccentricity
ApA                     : vessel.orbit.ApA
timeToAp                : vessel.orbit.timeToAp
PeA                     : vessel.orbit.PeA
timeToPe                : vessel.orbit.timeToPe

Here is the code that gets the values. Feel free to use but quote me please.

        USAGE : getValue(this, "vessel.acceleration.magnitude")


        private String getValue(Object parent, String valueName)
        {
            Object o;
            String value;
            int period = valueName.IndexOf(@".");

            //Parsing value name
            if (period == -1)
                value = valueName;
            else
                value = valueName.Substring(0, period);            

            //Serializing object
            try
            {
                o = parent.GetType().GetProperty(value).GetValue(parent, null);
            }
            catch
            {
                o = parent.GetType().GetField(value).GetValue(parent);
            }

            //print(@"Created " + o.GetType() + " " + o);

            //Recursing
            if (period == -1)
                return o.ToString();
            else
                return getValue(o, valueName.Substring(period + 1));

        }

[Benache]

May I suggest something? - Give flight recorders high crashTolerance, enabling them to survive catastrophes; black boxes are meant to be durable - just an immersion thing, but since crashing is an important part of gameplay in KSP, it would be fun to search for recorders around wrecks.

I already did, but maybe not enough. I tryed a few crashes and had fun looking for the flight recorder with my rover

Ok, i'll adjust that.

If you don't want to wait, you can change the crashTolerance value in the \FlightRecorder\FlightRecorder\part.cfg file. I already multiplyed it by 10. Try 100 000 !!

[Benache]

OK, two afterthoughts :

I need your help to improve the part.cfg file. If you find new usefull data, please share here.

The goal would be to have a default file with sections and comments. A bit like that :

//VESSEL DYNAMICS
// Orbital velocity : velocity measured in a geocentric reference
obt_velocity              : vessel.obt_velocity.magnitude
//obt_velocity.x            : vessel.obt_velocity.x
//obt_velocity.y            : vessel.obt_velocity.y
//obt_velocity.z            : vessel.obt_velocity.z

// Surface velocity : velocity measured in a local reference
srf_velocity              : vessel.srf_velocity.magnitude
//srf_velocity.x            : vessel.srf_velocity.x //east
//srf_velocity.y            : vessel.srf_velocity.y //north
//srf_velocity.z            : vessel.srf_velocity.z //up

//ORBITAL VALUES
// Basic orbital data
orbitalEnergy           : vessel.orbit.orbitalEnergy
period                  : vessel.orbit.period
inclination             : vessel.orbit.inclination
//argumentOfPeriapsis       : vessel.orbit.argumentOfPeriapsis
//semiMajorAxis             : vessel.orbit.semiMajorAxis
//semiMinorAxis             : vessel.orbit.semiMinorAxis
//semiLatusRectum           : vessel.orbit.semiLatusRectum
eccentricity            : vessel.orbit.eccentricity

In this exemple, i added comment to surface velocity, but it must be false, I don't know anything about that

I think it might be quite easy now to had a value to search in all the parts of the vessel.

For example : to calculate the total thrust, i look in each part of the ship for a module of type ModuleEngine , and i add the result of the CalculateThrust() Method.

I will work on that later.

Edited October 13, 2013 by Benache

[Nostromo]

Just wanted to thank you for taking the time to make this. I love the look & functionality!