[Hardware, Plugin] Arduino based physical display + serial port io+ tutorial (24-11-19)

[MrOnak]

I tried asking for one [edit]subforum[/edit], it was mostly ignored. All this work being shared (and code along with it) certainly seems to be a completely different topic than one single "mod"

Honestly, I disagree. There are only a handful of active hardware controller projects - each with their own thread, plus this one on the shared software side and Mulbins "collection of hardware threads" thread. If you compare that to other shared mods (life support, resources, OKS/MKS come to mind, also RasterPropMonitor and maybe even kOS) they all have more posts per day than we have combined most of the time.

I think it's fine as it is. I'm subscribed to every hardware thread that I can find so finding new posts is as easy as going to "settings" on the top right.

That said, what we could do to keep Zitronens life a bit easier is add feature requests and their respective discussions on his github page instead of this thread.

[Freshmeat]

Right now the controls are only "Synced" during scene changes (going back to VAB, tracking station), I can make it sync on vessel switching, but that can cause other problems like on ship A you have control group 10 set to solar panels and you switch to ship B which has it set to jettison engines, so when you switch vessel, well you get the idea... Docking, EVA adds even more complexity so I don't think sync on vessel switching will be a good idea. Once you have the status information your LEDs will reflect the actual states in the game, this will be less of a problem I hope.

I understand where you're coming from. If I may, I'd then suggest adding a "force sync" to the ControlPacket, so you could get immediate sync after an undock. It is a bit stressful to loose throttle and RCS during these maneuvers.

On the bug I wrote about, I think the issue is fuel lines. My pod has fuel lines linked directly to the engine, and the fuel in these tanks is not counted.

Hope you have a happy new year.

[MrOnak]

All you builders, makers, tinkerers and rocket instrumentation engineers!

Have a great launch into 2015! (and don't forget the 'chutes!)

See you on the other side

[hakan]

See you on the other side

Now *that* is a nice Kerbaly picture

[MrOnak]

@Zitronen: Am I right that the Arduino side of your plugin is not on your github?

Reason why I'm asking is because I've applied a useful patch to my AVR-c "fork" of your code that adds a return value to kspio_input() and I believe that might be useful for your official code as well. My kspio_input() function matches your input().

The return value allows to run code that depends on vesselData numbers to be run only when the numbers actually have changed.

See my latest commit in my branch for details:

https://github.com/MrOnak/avr_KSPIO/commit/6d3c1bb5d483c28d6feb542f67f1b07ed9a99ce4

If I'm not mistaken your code with this patch would be this (changes in bold):

[B][COLOR="#FF0000"]int[/COLOR] [/B]input() {
  now = millis();
 [B] [COLOR="#FF0000"]int returnValue = -1;[/COLOR][/B]

  if (KSPBoardReceiveData()){
    deadtimeOld = now;
   [B] [COLOR="#FF0000"]returnValue = id;[/COLOR][/B]

    switch(id) {
    case 0: //Handshake packet
      Handshake();
      break;
    case 1:
      Indicators();
      break;
    }

    //We got some data, turn the green led on
    digitalWrite(GLED,HIGH);
    Connected = true;
  }
  else
  { //if no message received for a while, go idle
    deadtime = now - deadtimeOld; 
    if (deadtime > IDLETIMER)
    {
      deadtimeOld = now;
      Connected = false;
      LEDSAllOff();
    }    
  }

  [B][COLOR="#FF0000"]return returnValue;[/COLOR][/B]
}

In case you're interested in a bit of collaboration, I'll try and send you pull requests whenever I find useful changes / additions and your Arduino code is on github

[MrOnak]

Okay, I did some thinking about the amount of time the microcontroller spends to receive the data packets from KSP and I wanted to start a little discussion if anyone has tried faster baud rates than the default 38400 to have more time available doing something other than receiving.

If my maths is right then a single data packet consists of 177 bytes, more to come with the SAS/RCS/... settings and the SOI information. With 38400 baud, that takes (177*10/38400 =) 0.046 seconds to receive - not counting software overhead on either side.

The default transmission frequency seems to resolve to 12.5 transmissions per second which in turn means that the microcontroller spends (12.5 * 0.046 =) 0.57 seconds each second to receive data. Add a bit of overhead and you have 60% MCU time spent. In other words you have 40% of overall MCU time available to do something.

Now that percentage may very well go up a bit if you're using lots of interrupts but in the end you need to read the buffer before it gets overridden so lets keep the maths simple .

Is my maths right so far?

If it is:

Let's assume we keep the transmission frequency fixed at 12.5 Hz, but increase the transmission baud rate:

- with a baud rate of 57600 the overall time spent receiving would come down to roughly 0.38s per second, giving us around 60% free MCU time.

- with the highest "standard" baud rate setting of 115200 the amount of "free" MCU time goes up to around 80% (0.19s spent receiving).

Now... receiving at 115200 baud is generally not an issue for the ATMega328 but I haven't tested it with KSP yet.

Has anyone tried this? Any surprising side effects or noticeable drawbacks? How does KSP handle that ( I suspect I wouldn't notice the effect on KSP)?

@Zitronen: Any specific reason for setting the default baud rate to 38400?

Thanks a lot in advance for any helpful comments, insights into things I didn't take into account or plainly pointing out that my maths is borked

[stibbons]

I am a lowly system administrator who generally thinks in gigabytes not bits, so take my thoughts with a reasonable grain of salt. Also, nitpicking, but the flow rate is specified as bits per second, not baud.

The Arduino core library allocates 64 bytes for the Hardware Serial receive buffer. Boards that ship their own cores can increase this apparently, but I have no idea how many do. (similarly, my five minutes googling says that if you want a bigger buffer for your Mega2560, you'll be hacking up a slightly modified core).

This means that, at 38400bps, you need to have polled and started reading (64*10/38400) = 0.016 seconds after the packet started to arrive, otherwise the buffer overflows and the packet is corrupt. And after you've read that 64 bytes, you've only got another 0.016 seconds to come back and check again for the next 64 bytes. It'll take a minimum of three batches of reads to get the full packet, possibly more depending on how much time the controller is spending on other tasks.

At 57600bps, the buffer overflows in 0.011 seconds. And at 115200bps, you've got 0.005 seconds before the buffer overflows. Even though you've got a lot more idle CPU time, you need to keep individual tasks quite short because you literally need to poll for new serial data every 5 milliseconds.

You can relax some of those. I'd be happy letting processing and display update tasks run longer than 5ms after a new packet has arrived on the assumption there won't be another one for 70-odd milliseconds. But to do it without a significant number of dropped packets I think would require much more careful structure.

I've been approaching this from the other direction. I've put a lot of thought in to how long I'd need to poll my inputs, run business logic, update displays. And a fair bit of effort in to optimising how well they run and how often they need to run, so I'd have plenty of idle time for serial IO. Haven't yet done any analysis of how busy my arduino is and how many packets are being dropped from the CPU. But I was planning on integrating a second microcontroller soon, and using that to benchmark the main processor's performance sounds like an excellent plan.

[MrOnak]

I too usually think in terms of how many cores per server blade and GHz/GB so yeah, I get where you're coming from .

The thought about how often to poll in order to not corrupt a packet is actually very valuable, so thanks for that!

The thing is I'm having a tough time to wrap my head around how long my business logic will take to execute - I'm simply lacking some monitoring tools and counting number of commands & calculating execution time from that is really not something I'm looking forward to, so if you have any hints on how to do that I'll be more than happy.

I should note here that I'm running on a "naked" ATMega328p, not a full Arduino so I do have a bit more control over what's happening... not that I necessarily have the knowledge to do so but at least its not all hidden behind abstraction layers .

Some thinking about the poll rate issues that you brought up, and I really mean thinking, not teaching - all of this might be rubbish:

As far as I see it there are two aspects to reading the data on the MUC:

First: Fetching the UART packets quickly enough to avoid corrupted data.

Second: Reading the data from the UART buffer and putting it into the KSPIO data structure where it becomes usable.

On the first aspect:

- The UART library I'm using has one 32 byte circular buffer each for transmitting and receiving but their sizes are simple #defines and thus easily changeable as long as they fit into the SRAM.

- The UART library also works with interrupts to send/receive.

I believe from that follows that the serial packets will be fetched in time and written to the UART buffer as long as my own interrupt routines - during which other interrupts are disabled - finish fast enough. Wouldn't that mean that the UART buffer is always valid?

On the second aspect - and I do find it hard to put in words:

If the business logic runs fast enough to be able to run KSPIO input() often enough to fetch every transmission into the KSPIO data structure everything is good anyway so no point discussing that situation.

But if the business logic takes so much time that we write only every second transmission into the KSPIO data structure, is there any communication breakdown?

I mean, missing a transmission means the code runs with "old" data which of course is suboptimal for calculations. But I doubt that updating displays with changing values 12 times per second is much better than updating the values lets say 5 times per second. So how bad a missed transmission is depends on what I'm doing with the data I guess?

Question for me is whether or not the serial communication will recover in case packets have been dropped - but I believe the answer is "yes" since resetting the MUC (not the USB2serial circuit, which for me is seperate!) causes no obvious effects - numbers still change afterwards.

Hmmm... I'll have to do more thinking on this I believe . Any more comments?

----

Quick addition on your numbers: 0.005s time between packets at 115200 baud - running 16MHz that is still 80000 clock cycles in between, right?

Edited January 2, 2015 by MrOnak

[Mulbin]

Right, I'm back! What did I miss?... oh.. about 10 versions of the plugin

My first new batch of parts has arrived, but as many already know, while I'm not bad at designing a panel, I'm useless at electronics!

I have just got 6 of these 7 segment boards which each have a MAX7219 chip. Any advice on how to wire these up to my arduino without frying them would be gratefully received! I will be using this batch for AP, PE, Time to AP/PE, Altitude, Excentricity and Inclination. I will no doubt buy more later for other data such as MET

Here are some pics of the components, thanks in advance!

Edited January 2, 2015 by Mulbin

[Daid]

Hmmm... I'll have to do more thinking on this I believe . Any more comments?

Just a small one.

I have a ton of experience on Arduino USB Serial communication. (From a 3D printer perspective. As I've build and maintain one of the leading open source 3D printing applications)

From this I can tell you that you are missing one important factor, USB. USB is polling based, there are wierd driver interactions happening, that even I do no fully understand. But you won't get full speeds, and there are some strange "bulk" transfers going on behind your back. So while the serial port might look like a stream from A to B, there is actually a polling based bulk transfer in between. So sending data really quick does not do much except for making it difficult on you MCU side.

I would recommend updating at 10Hz. That's quick enough for you to make it feel like "real time", but won't make stuff extremely complex on the MCU side.

Also, for anything serious on Arduinos, you'll better use your own UART driver. As the Arduino drivers pretty much suck at just about everything.

[MrOnak]

Heh, thanks Daid. I'll keep that "bulk communication" hint in mind once I start benchmarking things seriously.

Like I said, the UART driver I'm using is a custom one, not done by me but by Peter Fleury who seems to know what he's doing . And I do share your thoughts about the Arduino drivers as you put it, too much bulk, too much abstraction. Nice for the beginner but the losses are staggering.

*takes mental note to bug Daid when the 3d printer project is on the desk*

[Mulbin]

Ok, it was far easier than I expected to get a 7 segment running! Wired the first one up and have also added the LedControl library and got the display to fire up a simple sketch.

Next request... can anyone give me a sample arduino code to include in my code? Something to display some data, such as MET so I can start to get my head around how to code outputs for 7 segment.

I am using LedControl plugin via 8 digit 7 segments (can daisy chain them.. I have 6 boards currently!) each is running on a MAX7219 chip. Thanks in advance for any help!

[MrOnak]

Hey Mulbin, glad you got it to work that easily.

Displaying KSP data on the 7segs should be fairly easy:

Download Zitronens demo code for Arduino: https://sites.google.com/site/zitronfiles/KSPIODemo8.zip to get started.

Add the #includes (...) for the LedControl library to the top part and LedControl initialization stuff in setup() as you already did with your test code.

Inside the loop() function you want to add your stuff:

void loop()
{  
  input();
  output(); // not needed for displaying only

  updateDisplays(); // your new method
}

void updateDisplays() 
{
  // this is to drive an LCD display, your code will differ, depending how the LedControl library works
  lcd.setCursor(0, 0); lcd.print("alt: ");
  lcd.setCursor(5, 0); lcd.print[B][COLOR="#FF0000"]((long) VData.Alt[/COLOR][/B]);

  lcd.print("LF: ");
  lcd.setCursor(4, 1); lcd.print([COLOR="#FF0000"][B](long) VData.LiquidFuelS[/B][/COLOR]);

}

...and that should be it. Not very elegant and not very high performance but since you're offloading the multiplexing to the MAX chips on your LED boards it should be good enough.

For a list of available data and their variable names, see Zitronens first post (or the struct VesselData) on top of the KSPIODemo8 code.

[Mulbin]

I'm just getting errors thrown up from this...

KSPIODemo8.ino: In function 'void updateDisplays()':

KSPIODemo8:159: error: 'lcd' was not declared in this scope

Also... I'm looking for code for a 7 segment, not an LCD

Thanks for the help though!

[MrOnak]

Erm yeah that's what I meant when I wrote the code is for an LCD . You'll have to replace the "lcd.*" commands with whatever LedControl uses. Since you have a working demo code for LedControl (the one that you used to output "87654321" on that one display it shouldn't be too hard.

Sorry I can't help you more at the moment, I haven't used the LedControl library yet.

[Mulbin]

Hmm, think it's all waaaaaay beyond my understanding. I've managed to get switches and voltmeter dials working but things seem far to complicated for 7 segments (just looks like random words and numbers to me ).

I'll start putting some feelers out to see if anyone is willing to write some code for me in exchange for some panel graphics! When you don't have a skill, trade the skills you do have

[katton]

Hmm, think it's all waaaaaay beyond my understanding. I've managed to get switches and voltmeter dials working but things seem far to complicated for 7 segments (just looks like random words and numbers to me ).

I'll start putting some feelers out to see if anyone is willing to write some code for me in exchange for some panel graphics! When you don't have a skill, trade the skills you do have

I was around where you are a while ago, and got some code up and running for my display they you will probably be able to reuse. I was only using 4 digit 7segment displays, but was using k- or m- modifier lights to extend their range. The main problem that I ran into was that the arduino did not have enough power to translate a dozen float values into what I needed to send to the displays (basically just formatting the numbers into four digits, plus the two modifiers and the decimal position) in a timely enough fashion. If you were only using one or two displays it shouldn't be a problem.

I eventually decided that I would have to format the values on the computer before sending them over serial, but that would require forking zitronen's plugin, so I kind of gave up for a bit. I have a feeling that if I were more clever at coding, I might be able to optimize it enough, but I got busy right around that time anyways.

This is where I got to, and was able to get all the values populated, but the refresh rate was down to one or two seconds, so basically unusable: http://i.imgur.com/JYEeGEm.jpg

[MrOnak]

@Mulbin: Paste the code that you have that displays the "87654321" on a single display, it shouldn't be too hard to adapt that to display something from KSP.

[Mulbin]

Here you go...

#include "LedControl.h"

// Arduino Pin 7 to DIN, 6 to Clk, 5 to LOAD, no.of devices is 1

LedControl lc=LedControl(7,6,5,1);

void setup()

{

// Initialize the MAX7219 device

lc.shutdown(0,false); // Enable display

lc.setIntensity(0,10); // Set brightness level (0 is min, 15 is max)

lc.clearDisplay(0); // Clear display register

} void loop()

{

for(int i=0; i<8; i++){

lc.setDigit(0,i,i+1,false);

}

delay(1000);

}

[MrOnak]

Alright.

- Download the KSPIODemo8 that I linked earlier (link also in Zitronen's first post), unpack it somewhere and open it in your Arduino IDE.

- open HandShake.ino and remove line 4 (digitalWrite(GLED,HIGH); )

- open Input.ino and remove lines 10-12 (case 1: [...] break;)

- still in Input.ino, remove line 16 (digitalWrite(GLED,HIGH);)

- still in Input.ino, remove line 26 (LEDSAllOff();)

- open KSPIODemo8.ino and replace the whole thing with this:

#include "LedControl.h"
// Arduino Pin 7 to DIN, 6 to Clk, 5 to LOAD, no.of devices is 1
LedControl lc=LedControl(7,6,5,1);

//---- START initialization stuff from KSPIO - NOT cleaned by me, you can probably get rid of a lot
//pins for LEDs
#define GLED 5
#define YLED 6
#define RLED 7

//pins for input
#define SASPIN 8
#define RCSPIN 9
#define CG1PIN 10
#define THROTTLEPIN 0

#define THROTTLEDB 4 //Throttle axis deadband

#define SAS 7
#define RCS 6
#define LIGHTS 5
#define GEAR 4
#define BRAKES 3
#define PRECISION 2
#define ABORT 1
#define STAGE 0

//macro 
#define details(name) (uint8_t*)&name,sizeof(name)

//if no message received from KSP for more than 2s, go idle
#define IDLETIMER 2000
#define CONTROLREFRESH 25

//warnings
#define GWARN 9                  //9G Warning
#define GCAUTION 5               //5G Caution
#define FUELCAUTION 10.0         //10% Fuel Caution
#define FUELWARN 5.0             //5% Fuel warning

unsigned long deadtime, deadtimeOld, controlTime, controlTimeOld;
unsigned long now;

boolean Connected = false;
byte caution = 0, warning = 0, id;
//---- END initialization stuff from KSPIO - NOT cleaned by me, you can probably get rid of a lot

struct VesselData
{
  byte id;                //1
  float AP;               //2
  float PE;               //3
  float SemiMajorAxis;    //4
  float SemiMinorAxis;    //5
  float VVI;              //6
  float e;                //7
  float inc;              //8
  float G;                //9
  long TAp;               //10
  long TPe;               //11
  float TrueAnomaly;      //12
  float Density;          //13
  long period;            //14
  float RAlt;             //15
  float Alt;              //16
  float Vsurf;            //17
  float Lat;              //18
  float Lon;              //19
  float LiquidFuelTot;    //20
  float LiquidFuel;       //21
  float OxidizerTot;      //22
  float Oxidizer;         //23
  float EChargeTot;       //24
  float ECharge;          //25
  float MonoPropTot;      //26
  float MonoProp;         //27
  float IntakeAirTot;     //28
  float IntakeAir;        //29
  float SolidFuelTot;     //30
  float SolidFuel;        //31
  float XenonGasTot;      //32
  float XenonGas;         //33
  float LiquidFuelTotS;   //34
  float LiquidFuelS;      //35
  float OxidizerTotS;     //36
  float OxidizerS;        //37
  uint32_t MissionTime;   //38
  float deltaTime;        //39
  float VOrbit;           //40
  uint32_t MNTime;        //41
  float MNDeltaV;         //42
  float Pitch;            //43  
  float Roll;             //44  
  float Heading;          //45  
};

struct HandShakePacket
{
  byte id;
  byte M1;
  byte M2;
  byte M3;
};

struct ControlPacket {
  byte id;
  byte MainControls;                  //SAS RCS Lights Gear Brakes Precision Abort Stage 
  byte Mode;                          //0 = stage, 1 = docking, 2 = map
  unsigned int ControlGroup;          //control groups 1-10 in 2 bytes
  byte AdditionalControlByte1;        //other stuff
  byte AdditionalControlByte2;
  int Pitch;                          //-1000 -> 1000
  int Roll;                           //-1000 -> 1000
  int Yaw;                            //-1000 -> 1000
  int TX;                             //-1000 -> 1000
  int TY;                             //-1000 -> 1000
  int TZ;                             //-1000 -> 1000
  int Throttle;                       //    0 -> 1000
};

HandShakePacket HPacket;
VesselData VData;
ControlPacket CPacket;

long alt;
byte digit;

void setup()
{
  // initialize KSPIO - read-only! if you want to control the vessels more is needed
  Serial.begin(38400);
  InitTxPackets();

  // Initialize the MAX7219 device
  lc.shutdown(0,false); // Enable display
  lc.setIntensity(0,10); // Set brightness level (0 is min, 15 is max)
  lc.clearDisplay(0); // Clear display register
} 

void loop()
{  

  // update KSP data
  input();

  // fetch current altitude as long value, not as floating point
  alt = (long) VData.Alt; // casting to long is needed to be able to fetch individual digits I believe

  // update displays - for loop staring at 1 to make pow() work
  for(int i = 1; i < 9; i++) { 
    // this isolates the i-th digit, but note that pow() is expensive. a big ol' switch(){} is much faster
    digit = (alt / pow(10, i)) % 10;
    lc.setDigit(0, i, digit, false);
  }

  delay(70); // roughly matches the default update frequency of KSPIO
} 

- compile and upload to your Arduino

- open the GameData\KSPSerialIO\PluginData\KSPSerialIO\config.xml and replace the "DefaultPort" value with the com port that's listed in the bottom-right corner of your Arduino IDE.

- Start KSP

Once you put a vessel on the pad you should see a message on the right of your monitor confirming that KSP and the Arduino talk to each other.

If that happens you have a slim chance that I didn't make any mistakes in the code / instructions about KSPIO above and you see your current altitude on your display.

[disclaimer]It's Saturday, I'm not exactly sober [/disclaimer]

Note, if your Arduino is connected to a COM port higher than 9 then you have a chance that KSP won't find it, even if you configure the .xml correctly. Find the Arduino in the device manager and change the COM port to a single-digit number.

On the off-chance *cough* that it doesn't work right out of the box, the KSPIODemo8.ino I posted above is a mix of your code, Zitronen's demo and something I have running on an LCD display. I left comments wherever I changed / extended your code. You might find clues there.

If not, report any issues here (or PM) and I'll see what I can do.

God speed sailor!

[Mulbin]

That's a lot of code for a Saturday Thanks for your time, let me know if you want any help on facia panel design/production techniques!

Still getting an error though...

KSPIODemo8.ino: In function 'void loop()':

KSPIODemo8:152: error: invalid operands of types 'double' and 'int' to binary 'operator%'

[MrOnak]

Oh... yeah...

replace line 152 of KSPIODemo8 with this:

digit = (long) (alt / pow(10, i)) % 10;

And don't worry, I'm in a polar orbit around Minmus spamming biome EVA reports so... there's time

[Freshmeat]

Oh... yeah...

replace line 152 of KSPIODemo8 with this:

digit = (long) (alt / pow(10, i)) % 10;

And don't worry, I'm in a polar orbit around Minmus spamming biome EVA reports so... there's time

As you said, pow() is expensive. In this case, I think you can replace it with a running counter that gets multiplied by 10 on each run:

void loop()
{  

  // update KSP data
  input();

  // fetch current altitude as long value, not as floating point
  alt = (long) VData.Alt; // casting to long is needed to be able to fetch individual digits I believe

  // update displays - for loop staring at 1 to make pow() work

[COLOR=#008000]  int [I]digitPlace = 1;[/I][/COLOR]

  for(int i = 1; i < 9; i++) { 
    // this isolates the i-th digit, but note that pow() is expensive. a big ol' switch(){} is much faster

[COLOR=#008000]    [I]digit = (alt / digitPlace) % 10;[/I][/COLOR]
[COLOR=#ff0000]    //digit = (alt / pow(10, i)) % 10;[/COLOR]
    lc.setDigit(0, i, digit, false);


[COLOR=#008000]    [I]digitPlace = digitPlace *10;[/I][/COLOR]
  }

  delay(70); // roughly matches the default update frequency of KSPIO
}

Something like this, I have not tested it.

[MrOnak]

Yep that should work

There are many ways to improve from pow()... one is what you posted, another one is to get rid of the for() {} loop altogether and simply have 8 lines of lc.setDigits() "hardwired".

Aaaanyway, point of the exercise is to get something showing up on Mulbins rig

[Mulbin]

Progress! The sketch now uploads to the Arduino. Com port is working fine and KSPserial is detecting the device but...

All that happens is the display shows 0000000 leaving the last digit unlit. I'm assuming from the code that it should be showing Altitude?

Tried both MrOnak's original version and Freshmeat's edit (keen to be efficient as will be running 2 daisy chains each with 8 Max chips... so a total 128 digits!), both codes have the same effect.

- - - Updated - - -

Also, just dug this up from earlier in the thread by Zitronen if it's any help! (all greek to me )

I also use the LEDControl library katton used. My code for displaying is really simple:

Code:

void write7Segment(byte display, char value[7], byte decimal)

{

for (byte j = 0; j<6; j++) {

lc.setChar(display,j,value[j],decimal==j);

}

}

Display is the display number if you have daisy chained more

value is what you want to display, it's char so you can write "-BOOB-"

decimal is which digit you want the decimal to show, set it to 10 or some hight number if you don't want it.

This is obviously only for 6 digits, just modify it for more or less digits.

Edited January 4, 2015 by Mulbin