Staging stack fuel level overlays cause slowdown

[Phreakish]

My craft is about 350 parts, with ~50 engines in one stage. 

When the engines are staged/enabled, my FPS drops to ~13. With the engines deactivated, it runs in excess of 30. If I decrease my engine count, I don't see this behavior, but at 2300t I kinda need quite a few engines.. 

Is there a way to disable these overlays? I don't need them with the fuel level readout in the upper right. Thanks.

[Atkara]

Maybe the problem isn't the fuel overlays, but the 50 engine exhausts, hitting the ground and generating the smoke effects. Try with surface FX disabled (you can find it in the graphics options)

[Phreakish]

All my testing is with the ship in orbit - I expect to pay a performance penalty with the smoke/exhausts - but my FPS drops even with the engines throttled to 0% - the only difference is the overlays.

[Phreakish]

I think I may have tracked it down to autostrutting (not surprisingly, I should have tried this more extensively first!)

[KerikBalm]

have you tried staging with throttle set to zero? to see if its the engine thrust calculations or the fuel overlays?

 

[Dafni]

12 hours ago, Phreakish said:

I think I may have tracked it down to autostrutting (not surprisingly, I should have tried this more extensively first!)

Hmm, interesting. Let us know how your research turns out.

Intrigued myself, as hunting down mysterious performance drops seems to be a hobby of every seasoned KSP player.

[Phreakish]

I should mention up-front: 1.5.1, no mods (none, zero, zilch, nada). Launching from steam, or killing steam and launching stand-alone makes no difference. 

TL;DR: if multiple clusters of engines are attached using radially mounted tanks to create nodes to make my own clusters, my performance is a disaster. But if I use stock clustering options, performance does not take as big of a hit, but will still start to 'stutter'. It's not garbage collection, and it definitely has to do with engines being active (inactive, or unstaged engines do not have an effect).

I did some more playing around and am even more confused now. 

I installed memgraph last night too, and my 'stutters' don't line up with garbage collection. So it's not that. 

I've played with every graphic setting, Nvidia setting, etc with little to no change (lowest graphics settings were actually the worst performance too). I've cold-booted, rebooted, applied updates and rebooted. I've also tried my desktop (i7-3770 3.4ghz, 16g ram, P4000 8gb video card on Win10) and my laptop (2.8ghz, 32g ram, M5000 video on Win7). Both exhibit the same behavior - my desktop seems to deal slightly better and will deal with ~24 engines before it acts up. 

I've tried different engines (Vector, nukes, wolfhound, swivel) and that doesn't seem to matter either. Even if it's a combination. Any more than ~18 engines, and things begin to 'stutter'. I have noticed though that if I radially attach tanks to a parent tank (or engine plate) and then cluster engines on THAT - the performance hit is the worst. 

An example is: if I use the kerbodyne engine cluster tank, I can put 5 wolfhounds on it. I then radially attached 4 jumbo-64 tanks and attached 4x cluster tanks to those. This gives me an engine cluster with 25 wolfhounds. I then alt-copied the root cluster tank and attached this same 25 engine cluster using symmetry to either side of my craft. This gives me 75 engines. This setup has moderate performance once in-orbit. It stutters slightly, but only just. It's playable though.

I replicated the above example by radially attaching 4x FLT-TX220 tanks to an S4-64 tank. This also allows me 5x wolfhounds. I put them in the same array as the kerbodyne engine cluster tank yielding 75 engines on the craft. Same auto-strut arrangement. On-orbit, my FPS in the performance window is similar, but the screen updates every 3-5 seconds (Real time). The MET clock will show green, then flash yellow, then flash green again before the clock advances by 1 second. Totally unplayable. 

Even if my FPS is higher (performance graph showing high of 40, low of 25-30) it will visibly 'stutter' along. I think what's happening is that active engines run through the physics differently from inactive ones. I reached this conclusion because if I use F2 and hide the interface, the stutter remains but if I bind engine activation to an action group, I can shutdown engines and the performance improves noticeably. When I reactivate, it slows back down. I think the engines within the physics calcs are causing me to tip into the physics time-delta and time is slowing down and then catching back up. The MET clock flashes yellow/green in cadence with the stutter. I've changed the max delta, and tried .03, .05, .10 and all have behaved roughly the same. Increasing it to .10 seemed to help 'smooth' things over, but didn't eliminate it enough to be considered playable. 

I'm thinking there's got to be a bug somewhere causing a massive slowdown in the physics calcs when engines are attached to radially mounted tanks. The same # of engines on stock clustering attachments doesn't cause as big of a slowdown. I have to imagine that there's still something else going on with regard to engines as well, because I can't think of a reason a deactivated engine should behave differently with regard to physics vs one at 0% throttle.

Edited October 30, 2018 by Phreakish

[ZL647]

21 minutes ago, Phreakish said:

I should mention up-front: 1.5.1, no mods (none, zero, zilch, nada). Launching from steam, or killing steam and launching stand-alone makes no difference. 

TL;DR: if multiple clusters of engines are attached using radially mounted tanks to create nodes to make my own clusters, my performance is a disaster. But if I use stock clustering options, performance does not take as big of a hit, but will still start to 'stutter'. It's not garbage collection, and it definitely has to do with engines being active (inactive, or unstaged engines do not have an effect).

I did some more playing around and am even more confused now. 

I installed memgraph last night too, and my 'stutters' don't line up with garbage collection. So it's not that. 

I've played with every graphic setting, Nvidia setting, etc with little to no change (lowest graphics settings were actually the worst performance too). I've cold-booted, rebooted, applied updates and rebooted. I've also tried my desktop (i7-3770 3.4ghz, 16g ram, P4000 8gb video card on Win10) and my laptop (2.8ghz, 32g ram, M5000 video on Win7). Both exhibit the same behavior - my desktop seems to deal slightly better and will deal with ~24 engines before it acts up. 

I've tried different engines (Vector, nukes, wolfhound, swivel) and that doesn't seem to matter either. Even if it's a combination. Any more than ~18 engines, and things begin to 'stutter'. I have noticed though that if I radially attach tanks to a parent tank (or engine plate) and then cluster engines on THAT - the performance hit is the worst. 

An example is: if I use the kerbodyne engine cluster tank, I can put 5 wolfhounds on it. I then radially attached 4 jumbo-64 tanks and attached 4x cluster tanks to those. This gives me an engine cluster with 25 wolfhounds. I then alt-copied the root cluster tank and attached this same 25 engine cluster using symmetry to either side of my craft. This gives me 75 engines. This setup has moderate performance once in-orbit. It stutters slightly, but only just. It's playable though.

I replicated the above example by radially attaching 4x FLT-TX220 tanks to an S4-64 tank. This also allows me 5x wolfhounds. I put them in the same array as the kerbodyne engine cluster tank yielding 75 engines on the craft. Same auto-strut arrangement. On-orbit, my FPS in the performance window is similar, but the screen updates every 3-5 seconds (Real time). The MET clock will show green, then flash yellow, then flash green again before the clock advances by 1 second. Totally unplayable. 

Even if my FPS is higher (performance graph showing high of 40, low of 25-30) it will visibly 'stutter' along. I think what's happening is that active engines run through the physics differently from inactive ones. I reached this conclusion because if I use F2 and hide the interface, the stutter remains but if I bind engine activation to an action group, I can shutdown engines and the performance improves noticeably. When I reactivate, it slows back down. I think the engines within the physics calcs are causing me to tip into the physics time-delta and time is slowing down and then catching back up. The MET clock flashes yellow/green in cadence with the stutter. I've changed the max delta, and tried .03, .05, .10 and all have behaved roughly the same. Increasing it to .10 seemed to help 'smooth' things over, but didn't eliminate it enough to be considered playable. 

I'm thinking there's got to be a bug somewhere causing a massive slowdown in the physics calcs when engines are attached to radially mounted tanks. The same # of engines on stock clustering attachments doesn't cause as big of a slowdown. I have to imagine that there's still something else going on with regard to engines as well, because I can't think of a reason a deactivated engine should behave differently with regard to physics vs one at 0% throttle.

Attach a craft file? I'll see if the same happens on my machine. That would help at least rule out a few hardware related things.

[Phreakish]

3 minutes ago, ZL647 said:

Attach a craft file? I'll see if the same happens on my machine. That would help at least rule out a few hardware related things.

https://drive.google.com/open?id=1KQCF48lkibkvvgYFhHUtd24zYA-wz_BF

[ZL647]

@Phreakish

Awesome. I'll look at it when I get home.

I'm thinking it's some kind of CPU bottle neck. Would fit with your physics engine theory.

When you lower graphics settings without enforcing a lower frame rate, your GPU will ask the CPU for more so it can crank out more frames since the GPU isn't working as hard. That would slow the CPU down even more if it's already bottlenecked by the physics calcs.

Your part count is really shooting up when you use your cluster method vs the stock cluster method. Each part has it's own physics so that does quickly scale up calculation cost.

Just out of curiosity try your clustering with engines that DONT have gimbal and see if that reduces the issue. 

Edited October 30, 2018 by ZL647

[Phreakish]

I've tried gimbal and not - no change. 

Here's what's odd. On a test craft, I used FL-C1000 tanks, radially mounted, to get to 5x Wolfhound engines - same part count as with an FL-TX220 but the same performance as using the kerbodyne clustering tank. 

Part count doesn't seem to make a huge difference - I've tested with just a MK3 capsule with a ton of engines under it, and performance does improve - but even modest counts (~100 parts) and it'll happen with 20~30 engines. 

Honestly, I could deal with it if it only happened with the engines running. But having to shutdown engines between burns just to keep performance marginal, seems like a pain. 

[Phreakish]

Ran it again with the craft in the file above. Processor utilization runs about 25% when drifting in orbit. Turn on SAS, it goes up 1-2%. Stage the engines, processor utilization drops to 15-18%. None of the 8 cores shows any spikes or pegging when I enable engines or stage them. There's a temporary spike on a couple cores, but nothing huge - not even half of the headroom left. 

GPU runs about 10-12%, but when I stage or activate the engines GPU drops to 0-2%. 

[Phreakish]

Eureka! Ish.. 

So I've got all my engines on an action group to toggle them on/off to see what happens to the frame rate and performance as they switch. 

If I never use the staging, and use the action group to toggle my engines on - I never have this performance issue. But as soon as I hit space, whether to activate my current stage engines, or even if there's nothing to stage - the stutter shows up! 

I can run the same craft, cheat it to orbit, and toggle the engines on with an action group - 17fps constant, no stutter. 

Do the same thing, hit space, and it stutters. 

I can cheat to orbit, toggle engines with the action group - 17fps, no stutter. Hit space (there are no other stages on this craft) and it immediately begins to stutter. 

So I put a decoupler on it, I attached a single engine to the decoupler. I then engaged that engine with spacebar. It light, no stutter. I then activated the decoupler with space. It decoupled, still no stutter. I activate my bazillion engines with the action group - still no stutter. Full throttle, 17fps, no stutter. Hit space (already on last stage, no stages left) - stutter. Toggle engines with action group - stutter remains whenever engines are active.

So I put smaller groups of engines into stages - maybe it's so many engines activating at once? nope. 

This HAS to be some sort of bug, right?

[Francois424]

I would say to report it.

Any improvement will gladly be taken in this game

 

[ZL647]

Messed with the version you posted that's using the adapters, It seems normal to me. Can you post up the one with the clusters done the other way that make it lag bad?

Edited October 31, 2018 by ZL647

[Phreakish]

36 minutes ago, ZL647 said:

Messed with the version you posted that's using the adapters, It seems normal to me. Can you post up the one with the clusters done the other way that make it lag bad?

https://drive.google.com/open?id=1867O-GF56wRADckV-h0JewCZsjGDMkyE

Try this file. 

Launch it, f12 to orbit. Then use action group 1 to ignite the engines. Advance to full throttle. Once you establish that everything is 'normal', hit spacebar. If it's not just me, it should begin to stutter pretty bad. 

I tried this on my desktop at work and just now tried it on my laptop at home. The only thing these machines have in common is that the display adapters are Nvidia, so I have to assume it's not just me. 

[Phreakish]

I've made a video which shows it pretty clearly. 

I hit space at 6 seconds in. There are two other GC steps prior to that - I'm not talking about those (though they're also annoying).

 

[Phreakish]

Even more testing... 

I previously built a large booster setup which has over 90 engines at launch. It does not exhibit the behavior of my ship in the example. This megabooster uses vector engines.

I ruled out the mk3 parts. I ruled out manned/unmanned. Then, finally, breakthrough. 

I built a testbed that resembled my megabooster that does not stutter. It ran smoothly, just like my megabooster. I replaced the vectors with the wolfhound used on the ship in my previous video. Still runs smooth. I then placed a small fuel tank between the engine and the kerbodyne adapter. Bingo! lag city. If I remove that tank and replace it with an octagonal strut - still smooth (and the same part count). I then added 4 more engines per cluster with a radially mounted octagonal strut. Lag strikes again, but it feels different so it may not be the same. 

So, if I'm right. The steps to reproduce this are below: 

1. Place any capsule.
2. Stack 3-5 S4 tanks under it.
3. Terminate the stack with a kerbodyne engine clustering tank.
4. Attach small tanks to the engine nodes on the kerbodyne cluster tank. 
5. Attach engines (Vector and wolfhound are what I've tested, nukes also) to those tanks. Assign 'toggle engine' to action group 1. 
6. Pattern the engine block (kerbodyne adapter with patterned small tanks and engines) so that you have 5-9 clusters of engines.  
7. Alt+F12, hack gravity to .01, launch ship and cheat to orbit. 
8. Turn on SAS and set retrograde. 
9. Activate action group 1. 
10. Throttle up and allow craft to 'come around'. Note performance as ship renders and animates. 
11. Toggle action group 1 several times and note performance. 
12. With engines deactivated, hit hit spacebar to stage. 
13. Watch for stutter. 

I'd be curious if anyone else can confirm this behavior. You can take the same craft and replace the tanks which are between the engines and the kerbodyne adapter with an octagonal strut. This yields the same part count, but the stuttering behavior is gone. 

Why this is strange: 

• If the tanks between the engine and the adapter were causing the slowdown due to additional physics for their 'squishiness', then it shouldn't matter HOW the engines are activated. Staging the engines into action seems to yield a different mathematical model than 'activating' the engines instead. 
• If the tank 'squishiness' was the cause, then moving to zero throttle and warping (NOT physics warp) should result in static parts again, and the behavior eliminated. This is not the case. The stuttering will begin to happen even if the engines are never moved off idle once staged. 

This is a bummer because not all engines are radially attachable, and so making large engine blocks for interplanetary ships becomes a challenge, but I think the octagonal strut will work for me. I'll have to update the original craft that started this investigation and see if it helps. 

[Phreakish]

Not sure if this is a bug or not, but if it is (it looks like one), I've made an entry in the bug tracker:

https://bugs.kerbalspaceprogram.com/issues/20306?next_issue_id=20289&prev_issue_id=20308

[HvP]

I wonder if this has anything to do with the new delta-v-per-stage indicator that was added in 1.5 for maneuver nodes. A large number of engines attached to a large number of tanks might complicate the delta-v computation which attempts to show if you have sufficient fuel in your stages for maneuvers.

Edited November 1, 2018 by HvP

[Phreakish]

2 minutes ago, HvP said:

I wonder if this has anything to do with the new delta-v-per-stage indicator that was added in 1.5 for maneuver nodes. A large number of engines attached to a large number of tanks might complicate the delta-v computation which attempts to show if you have sufficient fuel in your stages for maneuvers.

Good question. 

But what's odd is that it's not just performance-related. I've spammed a ton of engines (I just got done toying with one with ~500 parts and 112 nukes) to see if it will 'stutter'. It goes slow, and UT slows down, but it does not 'stutter'. 

If the engines are attached to a smallish tank, and THEN patterned onto a larger tank, and THEN staged - it will stutter. I think the engines are being dealt with differently in the physics engine depending on whether they're staged or not. It could be that the Delta-V calculator get's stuck somewhere and has a hard time resolving... That would explain why it happens when staging, but not when 'activating' using an action group. 

But now that I know how to work around it, I just need them to fix the burn time indicator for nukes... I can't take this behemoth anywhere without it!