Stalling / Hitching / Pauses in KSP

[kithylin]

So I'll start off saying I have a rather above average computer..

i7-3770k @ 4.8 ghz & +20% overclocked water cooled R9 290X 8GB card.

Not sure why but KSP seems to have these 1-2 second "stalls" or "hitches" when launching a space ship and during orbital maneuvers, once per 5 seconds.

 

Has anyone else experienced anything like this?

 

Mod's I'm using: MechJeb (Of course, everyone uses mechjeb), KIS & KAS. That is all.

 

It actually didn't start having these issues with an earlier vessel I had.. with part count around 100 - 120, then I upped part count to 296 and.. this happens.

 

I've tried adjusting the max physics delta time per frame slider all the way in either direction it seems to have no effect.

 

Any ideas folks?

[T.A.P.O.R.]

I've got an i5 6600k and it's been stutter in a lot on 1.2.1

I'm using heaps of mods including the ones you mentioned.

never happened in previous versions on this rig though.

[katateochi]

I've been aware of this issue since around KSP 0.25, mods exacerbate it (and some more than others) but it is still there in a pure stock game. Large part counts can also make it more apparent, so does having lots of inflight craft or doing lots of switching between flight and VAB/SPH.  The stutter happens when the garbage collector runs.  Some work was done on the stock game to reduce the amount of garbage produced per frame but (afaic) it's still far from ideal.  

There is a mod called MemGraph which (with some tweaking) can reduce the frequency of the stuttering.  If you've got stutters every few seconds it will probably change that to every couple of minutes. see the thread on it here
Once installed open up MemGraph\PluginData\MemGraph\padheap.cfg and increase the last line "total" to 2048 and in settings.cfg change applyPadding to true (so it applies the padheap automatically).  I also set visible to false.  You can set "total" in padheap.cfg higher and have longer delays between stutters, but I find that the trade off is that each stutter is a longer pause, 2GB seems to be a good balance for me.  In game; alt + numpad * will open/close the graph and alt + end will reapply the padheap (I find that periodically you need to hit this).

[DocMoriarty]

I have that stutter too. I think its mainly caused by some massive memory leak in conjunction with particle effects or sound effects or both, especially but not limited to nuclear engines. It's not really the part count causing this, but bigger ships with more parts tend to have more engines and thus more effects and leak.

There is a mod that helps with that stuttering which was featured a few days ago: http://forum.kerbalspaceprogram.com/index.php?/topic/139128-12x-memgraph-1103-with-stutter-reduction/

I hope despite this mod the devs are going to fix the reason for the stuttering.

Edited November 22, 2016 by DocMoriarty

[Sharpy]

Most of these "stalls" are actually autosaves.

Although I'm finding it weird that they happen e.g. 1.5s before the moment a maneuver burn should start...

[Val]

1 hour ago, Sharpy said:

Although I'm finding it weird that they happen e.g. 1.5s before the moment a maneuver burn should start...

That's because autosave waits until you come out of time warp. It won't save while you're in warp.

[sal_vager]

Moved to modded support.

The Unity engines garbage collector will run when the memory heap is close to full, this can cause a slight pause as the whole application has to stop for it to work, this is more pronounced with some mods including mechjeb which allocate many heap objects such as lists or strings, compared to a simpler mod that just needs a few integer and floating point variables.

I suggest you try Memgraph, which has the option to pad the heap memory, making more available, this can make the pauses further apart but cannot eliminate them.

 

[kithylin]

18 hours ago, sal_vager said:

Moved to modded support.

The Unity engines garbage collector will run when the memory heap is close to full, this can cause a slight pause as the whole application has to stop for it to work, this is more pronounced with some mods including mechjeb which allocate many heap objects such as lists or strings, compared to a simpler mod that just needs a few integer and floating point variables.

I suggest you try Memgraph, which has the option to pad the heap memory, making more available, this can make the pauses further apart but cannot eliminate them.

 

Thanks for confirming it.. I was wondering if I was alone in this. Sometimes it's -very- annoying every few seconds. I'll try the memgraph thing and we'll see how that goes.

Does anyone know if the devs can acknowledge this as a performance issue and that they're trying to work on improving it?

 

EDIT: If it's caused by a lot of in-flight objects well.. 1.2 added the comms network and we're going to have a lot of satellites in the system to make the network functional.. if that's going to make it worse then 1.2.xx is going to ruin KSP. I only have 8 satellites in orbit around KSP so far (4 @ 1.5 million meters & 4 @ 3.2 - 3.6 million meters) for a modest comms network but I'm already expecting to have to put up many more as my new play through progresses in the future.. I think it's going to make stalling/pauses worse

Edited November 23, 2016 by kithylin

[Dafni]

1 hour ago, kithylin said:

Does anyone know if the devs can acknowledge this as a performance issue and that they're trying to work on improving it?

Good question. I hope they will try to fix this. It is very annoying, up to the point where it kills all the immersion and makes me switch off the game.

BTW, I only started to notice this in 1.2... and after all the talk of game optimization this is a big disappointment.

[Val]

1 hour ago, kithylin said:

Does anyone know if the devs can acknowledge this as a performance issue and that they're trying to work on improving it?

It is acknowledged and mentioned in the Devnotes and Release notes for KSP versions from 1.1 and on.

Examples:
1.1.2 - Remove some garbage creation in Part.Update.
1.1.3 - Optimize Part.GetConnectedResources and Vessel.GetActiveResources for speed and to not create garbage.
1.2 - Heavily reduced garbage generation caused by foreach loops, Linq usage, and various other things.

It's seldom mentioned directly as stutter or stalls. Just as performance optimizations and garbage reduction.

It's ongoing and hopefully improves with each new version. It's certainly gotten much better since 1.0.x, where it was much, much worse in my opinion.

[Dafni]

14 minutes ago, Val said:

 

Examples:
1.1.2 - Remove some garbage creation in Part.Update.
1.1.3 - Optimize Part.GetConnectedResources and Vessel.GetActiveResources for speed and to not create garbage.
1.2 - Heavily reduced garbage generation caused by foreach loops, Linq usage, and various other things.

 

Thank you for highlighting my point. Even though they did all these optimizations I only started to see the stutter in stock 1.2.  My modded 1.0.5 and 1.1.3 did not have this annoying stutter at all.

But it's true that they kept on introducing new features and whatnot, I'm just wondering if these additional workloads outweigh the optimization gains in such a dramatic way.

[kithylin]

30 minutes ago, Val said:

It is acknowledged and mentioned in the Devnotes and Release notes for KSP versions from 1.1 and on.

Examples:
1.1.2 - Remove some garbage creation in Part.Update.
1.1.3 - Optimize Part.GetConnectedResources and Vessel.GetActiveResources for speed and to not create garbage.
1.2 - Heavily reduced garbage generation caused by foreach loops, Linq usage, and various other things.

It's seldom mentioned directly as stutter or stalls. Just as performance optimizations.

It's ongoing and hopefully improves with each new version. It's certainly gotten much better since 1.0.x, where it was much, much worse in my opinion.

Well at least they're working on it. I'll try the mem heap mod up there above later and see if it helps. I have 16GB ram in my system maybe I can set it to something huge like 4096 or something. At least it's something to play with. They should give us a slider in game to adjust it directly.

[steve_v]

22 hours ago, sal_vager said:

I suggest you try Memgraph, which has the option to pad the heap memory, making more available, this can make the pauses further apart but cannot eliminate them.

It does, but a bigger heap also means longer pauses. Almost as annoying IMO.

3 hours ago, kithylin said:

If it's caused by a lot of in-flight objects well..

Yup, sure is.

1 hour ago, Val said:

It's ongoing and hopefully improves with each new version.

Noticed pretty much zero improvement here. Modded, but pretty much the same mods. See it in stock too, to much the same extent as 1.0.5. As far as I can tell, clean up existing code + add new code ~= back where we started.
Reducing garbage creation will help, but it will never fix the problem (Unless you create zero disposable objects, yeah, right.) Rip out that ancient garbage collector already, mono has a much better one now.

Yeah, I know. Unity. Third party game engine. yada yada. Heard it lots. Saying it again doesn't fix anything.

Edited November 23, 2016 by steve_v

[Val]

23 minutes ago, steve_v said:

Noticed pretty much zero improvement here. Modded, but pretty much the same mods. See it in stock too, to much the same extent as 1.0.5. As far as I can tell, clean up existing code + add new code ~= back where we started.
Reducing garbage creation will help, but it will never fix the problem (Unless you create zero disposable objects, yeah, right.) Rip out that ancient garbage collector already, mono has a much better one now.

Mods can create garbage, too. Hypothetically, if the mods used haven't done any steps to reduce garbage or even increased garbage with new features, then garbage reduction in the stock game would not make a noticeable difference.

Everyone will have a different experiences regarding stutters, depending on what mods they have and what features of the stock game they use.

 

[sal_vager]

The devs already have worked hard to reduce this over the last few releases, but there's only so much that can be done as everything in heap memory will be walked over by the garbage collector when the heap is near full, whether it's referenced or not, though better GC's will split things into tiers so they spend less time on objects that are less likely to be orphaned.

And the GC can only release memory that is actually orphaned, if most of the objects have references then the GC won't release a lot, so it'll take less time to fill the heap again, Unity should automatically expand the allocated memory and mods like memgraph work by padding the heap, forcing more memory to be allocated.

By the way, the garbage collector is a feature of the C# language used in the Unity3D game engine, which is implemented in Unity using the open source Mono framework.

To use a car analogy here, if garbage were the cars exhaust you can only do so much to reduce it, you can burn leaner, add a catalytic converter etc, but if you stuff bananas up the tailpipe the engine won't run.

[katateochi]

On 22/11/2016 at 9:58 AM, Sharpy said:

Most of these "stalls" are actually autosaves.

Although I'm finding it weird that they happen e.g. 1.5s before the moment a maneuver burn should start...

If you use the MemGraph mod to inspect what's going on you can see this isn't the case, yes there is a pause for autosaves but that's not nearly as frequent as the "stutter" which coincides with GC activity shown on the MemGraph display. 

3 hours ago, kithylin said:

I can set it to something huge like 4096 or something

Give it a try at that setting, but what I've found is that while that will lower the frequency of stutters, it will make each stutter more pronounced (as it's simply more memory to deallocate and reallocate)

 

2 hours ago, Val said:

Mods can create garbage, too. Hypothetically, if the mods used haven't done any steps to reduce garbage or even increased garbage with new features, then garbage reduction in the stock game would not make a noticeable difference.

This is going to become a defining characteristic of what makes a "good" (or rather, well made) mod.  Some modders are very experienced C# coders who will know how to best manage/minimise garbage production while others may not know how to deal with that at all.  It would be nice if the KSP dev team and some of the more experienced modders could put together a guideline for best practices, dos and don'ts, for minimising garbage production in mods. 

 

[Sharpy]

58 minutes ago, katateochi said:

If you use the MemGraph mod to inspect what's going on you can see this isn't the case, yes there is a pause for autosaves but that's not nearly as frequent as the "stutter" which coincides with GC activity shown on the MemGraph display. 

 

Well then, that means I'm not experiencing it. 16GB RAM might be responsible.

[kithylin]

4 hours ago, Sharpy said:

Well then, that means I'm not experiencing it. 16GB RAM might be responsible.

16GB ram here and the stalling is terrible. I might try and make a video later today and show y'all what I'm experiencing, it's just ridiculus, every 1-3 seconds, pause, pause, pause, pause. Very distracting and difficult to play with to the point I'm thinking of just (yet again) leaving the game for another few months until they can sort out this bull---- so it's actually playable.

[steve_v]

17 hours ago, sal_vager said:

better GC's will split things into tiers so they spend less time on objects that are less likely to be orphaned.

Any non-ancient mono version has generational GC, which should improve this situation significantly. The mono Unity/KSP uses is based on a 7 year old upstream release. That we are still stuck with this fossil beggars belief.

[katateochi]

16 hours ago, Sharpy said:

Well then, that means I'm not experiencing it. 16GB RAM might be responsible.

11 hours ago, kithylin said:

16GB ram here and the stalling is terrible.

How much RAM you have doesn't make a difference to the stuttering. I've got 32GB (high grade, 3200MHz corsair dominator) and without MemChart I'd be suffering it every few seconds (with memchart and with most fairly simple craft it's about 3-4 mins between stutters, on more complex craft it's more like every minute. *complex not necessarily meaning part count, certain combinations of parts, mods etc)

11 hours ago, kithylin said:

Very distracting and difficult to play with to the point I'm thinking of just (yet again) leaving the game

Have you tried the MemChart mod? It doesn't remove it, but it really makes it much more bearable. 

1 hour ago, steve_v said:

That we are still stuck with this fossil beggars belief.

I'm surprised there isn't more pressure from the Unity community to upgrade the GC (well, maybe there is, I'm not part of that com so I don't really know).  How can we put pressure on them? Or should we try and put pressure on Squad to put pressure on Unity?

[steve_v]

14 minutes ago, katateochi said:

Have you tried the MemChart mod?

MemGraph, FWIW. And yes, it helps. But padding the heap is a bandaid at best, and a bigger heap means the GC runs less often, but takes longer when it does.

14 minutes ago, katateochi said:

How can we put pressure on them?

Buggered if I know. Not (directly) a Unity customer so have no leverage.
I'm sure I've heard this story before: Commercial entity meets open-sorce project... Forks it, makes some money off it, doesn't bother updating it or contributing back to the original. Ends up with an inferior and incompatible version, then starts talking about ditching it and doing the same thing over again with another project.
Unity's fork is still open source, so in theory anyone could try to add the Unity changes to a newer mono release, or backport the Sgen GC... but it sure sounds like a lot of work, and the repo is a mess.

Edited November 24, 2016 by steve_v

[kithylin]

I've used Memgraph and tonight it works great. I run with 6144 (6 GB) in the config file and pad it once I get a ship loaded in the VAB and it's great for hours. With my 3770k system when it does do a mem dump it's quick and I barely notice it and it's about once every 10-15 mins even on complex ships.

 

Thanks for the suggestion guys I Can get back to enjoying KSP now.

[sal_vager]

On Unity updating Mono and the GC, they can now do this at any time, as Microsoft has acquired Xamarin and have relicensed Mono under the more permissive MIT license.

Unity Technologies have had plans to update Mono since at least May 2014, and newer versions do have improved garbage collectors, Unity look like they are still on the 2.x version of Mono.

As to when they upgrade, it looks like they won't until IL2CPP is working on all platforms.

[kithylin]

20 hours ago, sal_vager said:

On Unity updating Mono and the GC, they can now do this at any time, as Microsoft has acquired Xamarin and have relicensed Mono under the more permissive MIT license.

Unity Technologies have had plans to update Mono since at least May 2014, and newer versions do have improved garbage collectors, Unity look like they are still on the 2.x version of Mono.

As to when they upgrade, it looks like they won't until IL2CPP is working on all platforms.

So we're just waiting for it to be updated to work on all the platforms KSP runs on before KSP can upgrade to use the newer garbage collector schemes?

[sal_vager]

4 hours ago, kithylin said:

So we're just waiting for it to be updated to work on all the platforms KSP runs on before KSP can upgrade to use the newer garbage collector schemes?

No, we're waiting for Unity Technologies to finish the work on IL2CPP, after that they intend to upgrade Mono to the most recent version in a new version of their Unity3D game engine, this isn't likely to be trivial as changes to Mono may break existing Unity3D scripting methods, and Unity's custom changes will have to be ported to the new Mono version.

This could take a long time.

Then all developers who use the Unity3D game engine will be able to upgrade to that new engine and see what happens when they build their project.

For Squad this could be relatively painless as Squad has kept pace with Unity upgrades, and is usually not far behind the latest version of Unity3D when KSP updates are released, garbage collection is abstracted from the developer so it should not matter that the GC has changed.

But the changes to Mono may be as big a change as the move from Unity 4 to Unity 5, we won't know until it happens.