[PLUGIN, PART, 0.15+] crxTelemetryMk2 Module alpha

[Corax]

What it is:

• a plugin that sends data over IP.
  
• a part to put on your rocket
  
• a sample sounding rocket for the impatient
  

What it does:

• send information about the module's state as records of fields; the field separator (default is TAB), the data fields, and the sequence in which they are sent are customizable
  
• use TCP or UDP
  A very basic UDP listener is included, you can start it with mono UDPReceiver.exe from a Linux shell. It should run under MacOS and Windows, too, but I haven't tested that.
  Alternatively I recommend ncat, which can be found at http://nmap.org/ or probably with your Linux packet manager.
  Or you could write your own ground control station software.
  
• format the data or send it raw (for increased precision and post-processing)
  

What data you can get:

// possible fields are:

// rRoll, rPitch, rYaw: rotational velocities

// velX, velY, velZ: linear velocities

// hdgX, hdgY, hdgZ: heading vector

// accX, accY, accZ: accelerations along the axes

// accAbs: magnitude of acceleration ('total')

// ASL, AGL: altitude above sea level, above ground level

// Elev: ground elevation (=ASL-AGL)

// Lat, Lon, Ref: latitude and longitude, reference body (Kerbin, Mun, ...)

// rho: atmospheric density

// pStat, pDyn: static/dynamic pressure

// pMult: pressure multiplier (purpose unknown)

// t_ext: external temperature

// t_int: module's temperature

// UTC, MET: universal time, mission elapsed time

// AP, PE: apoapsis, periapsis

// tAP, tPE: time to apoapsis, periapsis

// SMA: semi-major axis

// ECC: eccentricity

// INC: inclination

// LAN: longitude of ascending node

// EPH: epoch

// modID: unique ID for each telemetry module

// fName: flight name, given in VAB or taken from ship file at launch

// vName: vessel name, may change on stage separation (eg. 'XYZ Debris')

// crew: an aggregate for crew data, more precisely a sequence of

// {crwNm,crwID,crwPnc,crwWee} times the number of kerbals on board,

// if any.

// crwPnc and crwWee are the 'panic' and 'wheee' factors of each kerbal.

// It's important to note that if there are kerbals in the vessel at the

// activation time of the telemetry module and the module gets decoupled

// from the command pod, the crew data will no longer be present in the

// output. This may (will) break the column count, so it is advised to

// put the 'crew' field as the last field, if at all.

// Non-existant (invented or misspelled) fields _will_ show up in the header,

// but will output 'NaN', i.e. invalid data.

What it doesn't:

• Provide a means of processing telemetry data; you have to supply your own method, or simply look at the numbers scrolling by.
  The data can be visualized using gnuplot (see images attached to this post) or similar graphics packages, or imported into spreadsheet programs like OpenOffice Calc or MS Excel.
  I suggest having a look at Warringer's Elderberry Ni, a software package to process and graphically display the data.
  

How to use:

• download from Kerbal Spaceport
  
• unpack the archive in your KSP directory, or unpack it elsewhere and move manually
  
• read the README file
  
• optionally edit the config file./Parts/crxTelemetryMk2/part.cfg to your preferences;
  basically adjust the address/port/protocol, and whether you want to have the data raw (recommended for further processing) or formatted (if you just want to watch the data pour in); defaults are udp, localhost:54345, unformatted
  
• You can also configure the data fields you want to receive, and the sequence in which they are sent. I recommend always including a way to identify the transmitter and packet, i.e. modID and at least one of UTC and MET. It is also advisable to include the body of reference if you plan to go further than LKO.
  
  • The data fields, formatting, and network configuration parameters are stored for each module.
    
  • Modules will use the configuration that was active at their launch, even if the part.cfg is changed later on.
    
  • Editing the part.cfg only affects new modules pre-launchpad, once you have a flight scene, your 'flight' along with the current module configuration is stored to the persistence.sfs (and quicksave.sfs if you use it). I've tried to strike a balance between which options are saved with the module, and which are global for all modules; I'll also try and differentiate more clearly between the two in the next update. In the long run, I envision an in-game graphical interface where you can configure each module individually.
    

  [*]start up a TCP or UDP listener (see 'What it doesn't' above)

  [*]start up KSP and either launch the supplied Sounding Rocket, or build your own ship - the module can be found under Utility&Scientific

  [*]the telemetry module is activated like any other part; you can put it in a separate stage or activate it together with other parts

  [*]you may modify and improve the source code, provided you share the results, including your modified code, in the same way and do not use it commercially (CC BY-NC-SA, ), or you may contact me for individual license terms.

Known issues:

• If activated, the formatting is very basic, don't expect it to look too pretty
  
• The icon still looks like a tricoupler icon...
  

I haven't had any KSP crashes caused by this plugin, but it's always a possibility.

Take your precautions, back up your data, and don't use this on a production system.

1-X Sounding Rocket telemetry transmitter: online
1-X Sounding Rocket
Rpitch Rroll Ryaw Vx Vy Vz HDG HDG HDG Gx Gy Gz Alt: Lat: Lon: Ref: Pstat: Pdyn: Pmult: tmp: UTC MET UUID EOT
0.0016113652382046 1.29521840790403E-05 0.086711198091507 3.83874535560608 166.643569946289 -0.0534871891140938 0.995006084442139 0.0266666412353516 0.0961865484714508 -8.6301456074803 0.039719425200456 -4.54973377009973 1655.81667019054 -0.233266033670277 27.7977881278362 Kerbin 0.718429643093482 0.743062292309161 0 877.2913 811455.329306119 15.3799996562302 5fa171a5 <eot>
0.0046341591514647 1.21397288239677E-05 0.0872729644179344 4.04619789123535 166.753814697266 -0.0626779720187187 0.995170474052429 0.0267431139945984 0.0944480299949646 -8.63006530274464 0.0397173714324583 -4.54966955820886 1658.93072434736 -0.233256386716709 27.797674477903 Kerbin 0.717980764855356 0.742615105360699 0 877.7861 811455.349306118 15.3799996562302 5fa171a5 <eot>
1-X Sounding Rocket telemetry: 5fa171a5 carrier lost

The ISA_RAM_Mapper is back from hibernation; kept here for historic reference

ISA's format is

LAT;LON;ALT

so you would have to edit the following lines in part.cfg:

statusMessages = false
formatting = false
fieldSeparator = ';'
packetRate = 10

dataFields = Lat, Lon, Elev

You can play with the packetRate, but if anything, I'd recommend increasing the value, unless you want to go crazy with resolution (and consequently get gargantuan data files). You have been warned

Capture the telemetry data to isa_RAM_{Kerbin|Mun|Minmus|whatever}_Data.csv.

Linux/Mac:

$ mono UDPReceiver.exe > isa_RAM_XXX_Data.csv

Win:

:\> UDPReceiver.exe > isa_RAM_XXX_Data.csv

EDITS:

2012-03-15: UUID persistence, configurable field separator, fixes

2012-03-19: UUID fixes, simple UDPReceiver included in download, header format slightly modified

2012-04-08: New part model and texture, user-customizable data fields (now with AGL and atmospheric density), example gnuplot files.

2012-05-22: More robust address resolution, new data fields (orbital elements), more configuration options, sound.

2012-06-10: Added info to configure for use with the ISA RAM Mapper

2012-06-12: Modified and updated a few passages for clarity

2012-07-26: Updated for KSP0.16

2013-04-22: Picking up bits and pieces from the latest and greatest forum crash - déjà vu...

It is strongly advised that you start with a clean installation of KSP or at least back it up before upgrading. Anything may happen.

Edited April 22, 2013 by Corax

[togfox]

Nice. But you just KNOW ppl want to know how to run a \'ground control\' session on the same PC they\'re playing on.

Keep it up. 8)

[Warringer]

I\'d like to note something that popped up while looking at the output of three different flights and rerouting the output into a text file.

The first actual line of data when getting the raw data is interrupted by the line that displays the data header. (Line Bolded)

1-X Sounding Rocket telemetry transmitter: online

1-X Sounding Rocket telemetry:

Rpitch Rroll Ryaw Vx Vy Vz HDG HDG HDG Gx Gy Gz Alt: Lat: Lon: Ref: Pstat: Pdyn: Pmult: tmp: UTC MET UUID

-0.00111347425263375 0.00104163156356663 0.00473867589607835-0.000913132098503411 0.000183493975782767 -0.00254069524817169 0.88559502363205 -0.00436675548553467 0.464437782764435 -8.67394910496355 0.0404764404002766 -4.57798930069471 60.9449598112842 -0.236453299847054 27.8244672293657 Kerbin 0.987676563205042 0.987677848543253 0 19.57567 204010.304691722 0 ffffadf8<EOT>

1-X Sounding Rocket telemetry: -0.000721193617209792 0.000447666068794206 -0.005537188611924650.00226346706040204 6.3967534515541E-05 -0.00234417477622628 0.885543465614319 -0.00438076257705688 0.464536011219025 -8.67394910496355 0.0404764404002766 -4.57798930069471 60.9449598112842 -0.236453299847054 27.8244672293657 Kerbin 0.987676563205042 0.987677445608102 0 19.57579 204010.324691722 0 ffffadf8<EOT>

Would be nice to have that line moved to above the actual data, so that we can get just the pure data from the beginning.

Something like this (Lines 138 - 147):

// prepare data packet:

String szData = '';

// intro for CSV:

if (formatting || sendHeader) {

szData += '\nRpitch\tRroll\tRyaw\tVx\tVy\tVz\tHDG\tHDG\tHDG\tGx\tGy\tGz\tAlt:\tLat:\tLon:\tRef:\tPstat:\tPdyn:\tPmult:\ttmp:\tUTC\tMET\tUUID\n';

sendHeader = false;

}

szData += myVesselName + ' telemetry:\t';

That way the header isn\'t in the way for programs to read the datastream.

[Corax]

The first actual line of data when getting the raw data is interrupted by the line that displays the data header.

Would be nice to have that line moved to above the actual data, so that we can get just the pure data from the beginning.

Actually the vessel name was not meant to be part of the packet at all, and only added as an afterthought when I realized there could be potential demand for more than one module per rocket, or several (rockets with) modules in close vicinity. I\'m still not satisfied with the way I\'m handling this (vessel name and session-local module UUID), but I haven\'t come to a conclusion yet.

In the meantime, I\'ll go with your suggestion, and I\'ll also add the corresponding field to the header...

If you want this to be fixed urgently, feel free to shout and I\'ll see to it; otherwise it may take a while.

[Katalliaan]

Nice. But you just KNOW ppl want to know how to run a \'ground control\' session on the same PC they\'re playing on.

Keep it up. 8)

Then use a loopback address.

Looks interesting. Probably not the kind of thing I\'d use, but it would be fun to set up a computer on the network purely for this.

[Meatsauce]

This would prove helpful if there was a GUI or built in listener that translated some information, or if there was a LUA shell we could use to execute commands.

This plugin might be ahead of it\'s time since the game core doesn\'t have a way to display this info yet. An in-game MFD would be ideal.

Damn I wish we had cockpit views. This information would be critical for engineering data and visual cues, and a launch from first person view would be epic.

[Corax]

This would prove helpful if there was a GUI or built in listener that translated some information, or if there was a LUA shell we could use to execute commands.

The data is made available as tab-separated values. It\'s no big deal to import them into a spreadsheet for post-flight analysis, or whip up some GUI using LUA, Python, Processing or what have you... of course, you could have that sit next to your KSP window for an almost-in-game GUI

This plugin might be ahead of it\'s time since the game core doesn\'t have a way to display this info yet. An in-game MFD would be ideal.

You wouldn\'t need to send telemetry across the network for an in-game MFD. Just code a plugin with a GUI.

Damn I wish we had cockpit views. This information would be critical for engineering data and visual cues, and a launch from first person view would be epic.

Again, this is not what this plugin is about. Its purpose is to allow a 'remote' (i.e. anywhere on the internet, even on your own PC ) 'ground station' (i.e. a separate program) record (and process if it wants to) flight information. Also, I hear cockpit views are coming at some point in the future.

[Warringer]

Actually the vessel name was not meant to be part of the packet at all, and only added as an afterthought when I realized there could be potential demand for more than one module per rocket, or several (rockets with) modules in close vicinity. I\'m still not satisfied with the way I\'m handling this (vessel name and session-local module UUID), but I haven\'t come to a conclusion yet.

In the meantime, I\'ll go with your suggestion, and I\'ll also add the corresponding field to the header...

If you want this to be fixed urgently, feel free to shout and I\'ll see to it; otherwise it may take a while.

Well, I could try and build it myself via Mono, through then I\'d jump into cold water with working on C#.

The data is made available as tab-separated values. It\'s no big deal to import them into a spreadsheet for post-flight analysis, or whip up some GUI using LUA, Python, Processing or what have you... of course, you could have that sit next to your KSP window for an almost-in-game GUI

Another separator would be better for pushing it into an external program to read the data.

Mainly because I now trying to develop a Java program to externally read the data stream and then do stuff with it.

EDIT:

I just tried out to put multiple telemetry modules into the same rocket and activating them at the same time. I noted that the UUID field in the data read out indicates the different modules. So there is no read need to send the vessel ID more then once. (May change later on)

EDIT2:

Important!

I have found a small error that preventy me from correctly use the data in the datastream!

Line 154

szData += linearVelocity.x.ToString (f32) + '\t' + linearVelocity.y.ToString (f32) + '\t' + linearVelocity.z.ToString (f32);

should be

szData += '\t'+linearVelocity.x.ToString (f32) + '\t' + linearVelocity.y.ToString (f32) + '\t' + linearVelocity.z.ToString (f32);

To allow correct slitting of the string for later use!

Please correct so that I can use the data for a program that is already in the works!

[Corax]

I just tried out to put multiple telemetry modules into the same rocket and activating them at the same time. I noted that the UUID field in the data read out indicates the different modules. So there is no real need to send the vessel ID more then once. (May change later on)

The problem was when the part went on rails and came off again (like when switching vessels, or restarting KSP), another UUID was randomly assigned. On the other hand, the stage name only changes when the ship is generated, or a stage is separated from the rest of the rocket. It still was not perfect, but it gave you two, not entirely redundant yet not sufficiently reliable, ways of identifying a transmitter module.

I have redone the whole stage name/module ID stuff; it should now take the initial flight name and the inital ModuleID, both should now get saved and reloaded. In case there were no values assigned (with previous save files), the last known stage name and a new ID are assigned and stored the next time the game saves its state.

I have also added an option so you can set a custom field separator in the part.cfg.

Currently doing some final tests, new version should be up tomorrow.

EDIT:

I have found a small error that preventy me from correctly use the data in the datastream!

Please correct so that I can use the data for a program that is already in the works!

And fixed, thanks! Now I\'m curious about what you\'re up to

[Warringer]

The problem was when the part went on rails and came off again (like when switching vessels, or restarting KSP), another UUID was randomly assigned. On the other hand, the stage name only changes when the ship is generated, or a stage is separated from the rest of the rocket. It still was not perfect, but it gave you two, not entirely redundant yet not sufficiently reliable, ways of identifying a transmitter module.

I bet there is a way to get the number of the stage.

I have redone the whole stage name/module ID stuff; it should now take the initial flight name and the inital ModuleID, both should now get saved and reloaded. In case there were no values assigned (with previous save files), the last known stage name and a new ID are assigned and stored the next time the game saves its state.

I have also added an option so you can set a custom field separator in the part.cfg.

Currently doing some final tests, new version should be up tomorrow.

Nothing like getting a kick in the rear end to get going, eh?

EDIT:And fixed, thanks! Now I\'m curious about what you\'re up to

Now that would be telling.

[Meatsauce]

The data is made available as tab-separated values. It\'s no big deal to import them into a spreadsheet for post-flight analysis, or whip up some GUI using LUA, Python, Processing or what have you... of course, you could have that sit next to your KSP window for an almost-in-game GUI

You wouldn\'t need to send telemetry across the network for an in-game MFD. Just code a plugin with a GUI.

Again, this is not what this plugin is about. Its purpose is to allow a 'remote' (i.e. anywhere on the internet, even on your own PC ) 'ground station' (i.e. a separate program) record (and process if it wants to) flight information. Also, I hear cockpit views are coming at some point in the future.

Appreciate your response. I follow you on remote access, and it certainly is an exciting plugin. Great work!

I have some experience building GUIs in LUA, and would make an attempt if when we have some documentation. If you know of any...

I wonder, could future iterations be used to accept flight commands from a ground station application?

My thinking here is that the parts are addressable, and enable two-way comms between satellites, and remote probes. This way you can monitor fleet telemetry, and send correction commands to wayward probes.

[Corax]

Update available, please let me know if you run into any issues.

[actieveling]

I Like this very much ^^

I can see this working for \'multiplayer\' somehow. (a bit)

---------------------------------------------------------------

9 users online with multiplayer command pod

- User N sends his data to the server.

= Server broadcasts Users N\'s data to all other users.

.. All users get updated shit positions.

---------------------------------------------------------------

Problems?

-example: \'User 2\' has lots of lag and slow/interupted connection, he misses 10 sec of broadcasts. so all other users will see \'User 2\' warp, and \'User 2\' will see all other warp.

- Debris will not be tracked. Only the active module/command pot ( +side is that everyone\'s space will be clean except with your own debris)

- What if I am \'user 0\' and i have lots of space junk, and other ships will fly somewhere there is no debris ( on their side ) .. will they explode/collide on my client

There are lots more problems, but rendezvous with friends in space looks like fun to me

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Best Solution?

( not possible now )

Save file would be server-side. so every junk, bit, byte, star, mun, minute will be stored online and shared with eachother.

Problem here? Yes.. Timewarp maybe keep multiplayer 'sandbox mode' and make time clientside only. hehe You will see ships fly very fast from point A to B ???

Just some thoughts, trying to think of how multiplayer Minecraft works.

But since multiplayer is 2 galaxies away at least, im just happy with this and I am trying to make a website which displays the data in nice graphs or something =) or nice '3D' Canvas which shows locations and orbits

SO MUCH WIN ^^

[Corax]

I Like this very much ^^

Thanks, much appreciated

I am trying to make a website which displays the data in nice graphs or something =) or nice '3D' Canvas which shows locations and orbits

It\'s always interesting to see how this plugin could be used, just be aware of the massive amounts of data it can produce

[actieveling]

Was Thinking about log file reader first for test purpose. Later an automated hourly script that reads log file and writes to database, Clear log after while only saving what you want. But yes this can be MASIVE <3 amazon cloud haha

[Warringer]

Little info on my little project.

The base program runs now and saves the data it receives to a csv file. But those files get so big so fast that I have had to cut off some data. At the moment it only saves the data to the file every 0.1s. Later for longer missions that may be reduced to 1 second. Maybe I even make it dynamic in the sense that in orbit the data is only saved every 5 or 10 seconds. But that is for later.

Then I have a request!

Please find a way to integrate the number of the stage the sensor is into the readout!

That way its easier to deal with multiple sensors in the same rocket. Like getting data from a freshly discarded stage...

[Corax]

Little info on my little project.

The base program runs now and saves the data it receives to a csv file. But those files get so big so fast that I have had to cut off some data. At the moment it only saves the data to the file every 0.1s. Later for longer missions that may be reduced to 1 second. Maybe I even make it dynamic in the sense that in orbit the data is only saved every 5 or 10 seconds. But that is for later.

Glad to hear you\'re making progress. As I just said, massive amounts of data are produced - one packet of close to 300 characters per 20ms, you do the math...

Then I have a request!

No problem, requests are free... no need to shout though.

Please find a way to integrate the number of the stage the sensor is into the readout!

That way its easier to deal with multiple sensors in the same rocket. Like getting data from a freshly discarded stage...

I\'ll look into how this could be done. I don\'t think it makes sense to send the stage number with every packet, but maybe when the sensor reports online.

For the time being, you can detect staging events by monitoring the MET, it resets when the stage is decoupled.

[Warringer]

Glad to hear you\'re making progress. As I just said, massive amounts of data are produced - one packet of close to 300 characters per 20ms, you do the math...

I need to find a way to reduce the amount of data... Maybe convert the String contents to actual numbers that are easier saved.

4 byte per float surely bear about 19 or 20 characters... But that makes saving them as CVS a tad harder...

No problem, requests are free... no need to shout though.

Only making sure that I\'m heard.

I\'ll look into how this could be done. I don\'t think it makes sense to send the stage number with every packet, but maybe when the sensor reports online.

For the time being, you can detect staging events by monitoring the MET, it resets when the stage is decoupled.

You are already sending the parts UUID with every package, why not send the stage number in the header?

Something like this:

Untitled Space Craft telemetry transmitter: online

Untitled Space Craft Stage N <UUID>

Rpitch Rroll Ryaw Vx Vy Vz HDG HDG HDG Gx Gy Gz Alt: Lat: Lon: Ref: Pstat: Pdyn: Pmult: tmp: UTC MET UUID EOT

With some RegExp the stage number can then be extracted.

[N3X15]

I need to find a way to reduce the amount of data... Maybe convert the String contents to actual numbers that are easier saved.

4 byte per float surely bear about 19 or 20 characters... But that makes saving them as CVS a tad harder...Only making sure that I\'m heard. ;)You are already sending the parts UUID with every package, why not send the stage number in the header?

Something like this:

With some RegExp the stage number can then be extracted.

Using a binary stream would help.

[Warringer]

Using a binary stream would help.

Harder to read in plain text through.

I have now made some changes to my code to make it more modular.

A row of the data is pushed into an object containing all the data in binary format, which is contained in a list and on request dumps the whole stuff into CSV.

Maybe I\'ll make a binary file format as well to dump the data into. We\'ll see...

[Warringer]

There appears to be a problem with multiple telemetry sensors per rocket.

Since the Update I only get one data stream from the sensors.

SHAMELESS PLUG:

See my Elderberry Ni for all your telemetry needs...

[Corax]

There appears to be a problem with multiple telemetry sensors per rocket.

Since the Update I only get one data stream from the sensors.

You\'re right, it looked like only one data stream, because every module got the same ID assigned. If you had several modules on a rocket, all would send their (correct) data, but it would be very hard to programmatically discern the individual modules.

I\'ve fixed this in the latest download.

Also please note that the way the headers are sent has changed in the new version:

[list type=decimal]

[li]Each module reports online[/li]

[li]Each module sends its ID along with the list of data fields[/li]

[li]From here on, every module sends its data packets[/li]

This change is in order to make it possible for individual modules to send different data fields than other modules. This isn\'t in yet, and probably quite a while away.

[Warringer]

You\'re right, it looked like only one data stream, because every module got the same ID assigned. If you had several modules on a rocket, all would send their (correct) data, but it would be very hard to programmatically discern the individual modules.

I\'ve fixed this in the latest download.

Also please note that the way the headers are sent has changed in the new version:

[list type=decimal]

[li]Each module reports online[/li]

[li]Each module sends its ID along with the list of data fields[/li]

[li]From here on, every module sends its data packets[/li]

This change is in order to make it possible for individual modules to send different data fields than other modules. This isn\'t in yet, and probably quite a while away.

Need to modify my Elderberry Ni software then.

But thankfully you provide the sourcecode of the plugin so I can take a look at what is actually sent.

[Corax]

[...] you provide the sourcecode of the plugin so I can take a look at what is actually sent.

I think you should, too, so others can see you\'re up to no evil

[Warringer]

I think you should, too, so others can see you\'re up to no evil

I just included the source code in the package for my Elderberry Ni. Not very commented through...