Serious answer to the stutter issue.

[Scientia1423]

Is the stuttering when you approach the KSC for a Landing (flying a plane(where it pauses for a moment)) or is it when you launch a space craft/carry on a mission? Because both are annoying. (Experienced with a slow and a fast computer, still the same thing. But It'll need to get fixed at some point).

[basic.syntax]

So you are unconcerned with a bug you notice and the devs will not fix...

Suggestion: "will not fix" >> "haven't fixed yet"

@Scientia1423: This performance issue can be noticed anywhere. The simplest method to see the problem, is to take a capsule to the Launchpad, hold down the right or left arrow to spin around it a few times - and watch for a tiny catch, in a movement that should be completely smooth. The conclusion I reach from the previous pages of discussion: very complex problem that is difficult to solve.

[CaptainTurbomuffin]

When I first red this I said What?! Than I looked at my game and tried to observe it. It was there, altough I got used to it so much that it doesn't annoy me.

[Rocket_Tamer]

I get a regular, slight stuttering too - perhaps just in space and not on the ground. It happens sort of regularly every 5 seconds but generally I haven't got into high part count territory. My humble i5 CPU never seems taxed and my GTX670 isn't high spec. I am running the 64 bit Linux version on 8GB of RAM with about 25 mods plus a 4k skybox (the only custom texture). I've found that the screen updates more smoothly if I press F2 to remove the GUI, that said, it is only that the FPS seems to be increased. I still get the problem on stock.

[Broco]

I think I have to clarify what exactly is happening here again: The stuttering described by OP has NOTHING to do with the power of his hardware. I never had an issue with this since KSP moved to Unity 5 (1.1 that is) now I experience it, too. It's not the framerate, it was fine before 1.1 and is even BETTER now. The problem is the garbage collection (GC). Garbage collection is basically an automated function of the engine which searches your memory for data that is not used any more and deletes it. For example in games this means different models, variables, textures, etc.

Usually a GC runs in the background and goes active in a specified interval or when a specific memory threshold is reached. The program itself doesn't call the GC in this case. However, you can also let the program call the GC.

This makes sense in games like Skyrim, when you leave a cave fore example, then you don't need the textures of the cave in your memory anymore. Since leaving/entering a level requires a loading screen anyways the user/player won't notice anything anyways. However, in games like KSP where you have an entire planet to be rendered on the fly you have to constantly load and unload data.

If I'm not wrong I think KSP calls the GC if you leave the proximity of a vessel for example (2.6km I think). There is no need to keep the models for all parts + textures + other stuff for that vessel in your memory because you're not close to it anyways. When you leave the proximity of a vessel you will experience a short stutter that is either the GC or the program itself unloading it from your memory to make space for new stuff.

There is a mod for KSP (GCMonitor that shows you exactly when the GC becomes active:

These ups and downs represent the used memory, these straight drops are when the GC cleans up and the stutters (it's like a hold for 0.2 seconds) occur at THIS EXACT moment. The thing is, OP isn't the only one to have pointed this out and when I stumbled upon this problem myself I did a search just to find out that this problem has been around for ages and affects thousands of players.

To be honest, in this case I have to share OPs opinion that this is not a debateable problem, this really needs to be fixed asap and it's a shame that it has been around for so long. Oh and before you start talking about mods used, this also occurs in vanilla KSP for me, installing mods doesnt make the problem better or worse.

I really love KSP and i've been playing it since 2011 but I'm also a programmer and there is one key rule of programming: Stability comes before features. Since KSP is now unity 5 and that means no major updates are coming, things like this or the wheels must be adressed. I wouldn't add any new feature before the basics are working.

 

Oh and before you ask about my poor framerate: I had Dota running, too when I made this screenshot, my hardware is fine, thanks.

 

Another thing to add: a GC is not a bad thing. It's needed to make sure unneeded data gets unloaded from your memory. A GC normally runs in the background and "parallel" to the software, which means it doesnt hold your regular program. In KSP however it seams that the GC pauses the game, collects the garbage and then resumes the game. This is no good behaviour, and I would really like to know what causes this since I didnt have the problem in 1.05 but others had. I didn't change my hardware in between btw.

Edited April 23, 2016 by Broco

[rkman]

A Porche with a square wheel is in the end a bit of a crappy vehicle.
It's not ok just because there's other nice things about it. Sorry but that's how these things work.  

 

[Broco]

5 hours ago, rkman said:

A Porche with a square wheel is in the end a bit of a crappy vehicle.
It's not ok just because there's other nice things about it. Sorry but that's how these things work.  

 

Pretty much this. I also made a short video to showcase this problem:

https://www.youtube.com/watch?v=nq0Z2Up1_Uw

Oh and thanks to @Padishar to filling me in to this problem.

Edited April 23, 2016 by Broco

[Renegrade]

18 hours ago, Broco said:

Another thing to add: a GC is not a bad thing. It's needed to make sure unneeded data gets unloaded from your memory. A GC normally runs in the background and "parallel" to the software, which means it doesnt hold your regular program. In KSP however it seams that the GC pauses the game, collects the garbage and then resumes the game. This is no good behaviour, and I would really like to know what causes this since I didnt have the problem in 1.05 but others had. I didn't change my hardware in between btw.

Garbage collection actually cannot ever be totally parallel to execution.  Even the most sophisticated systems (read: complicated and bug-prone), sooner or later, are going to have to have to throw on a bunch of mutex locks or enter a critical section to alter the allocation tables/trees/whatever.   Also, these more sophisticated systems that try to reduce stalls are not efficient: they have much higher overhead overall than stall-oriented algorithms, so while they might reduce the hiccups somewhat, they'll reduce the overall performance/framerate too.

Squad can't directly impact any of this anyways, they're using C#/mono, which runs on the CLR or Mono's version of the CLR, and that handles the garbage collection.  To significantly change it, they'd have to move away from the CLR entirely, which ALSO means a new language (the CLR VM supports many languages, but C# only supports the CLR), which would be even more work than porting away from Unity.  The only thing they COULD do is try to reduce the number of allocations/deallocations (well, implicit deallocations) they do.  So, fewer objects, buffers, etc, and/or pack them together in some way (if that's even possible with "modern" languages?).  NB: There are some tweaks that can be used for GC configuration, but they'd be largely platform specific (MS-CLR vs Mono-CLR), and only have a handful of options with limited capabilities, and tend to default to "as parallel as possible" anyhow (nobody likes those pauses in any sort of environment).

GC isn't magic. It's also over twenty years old (Java dates back that far and has had GC since day one), and still causing problems.  Sure it's nice not having to manage heap manually, but you pay for that convenience in performance and smoothness.  Trade-off will always be trade-off.  Also remember how GC was supposed to get rid of memory leaks?  *snerk*

#programmingdirectlyinANSI-CandOpenGLmasterrace~

[Mr Shifty]

I've been playing for 3 years and never noticed this before, but I started seeing it today (it was probably always there) and now I can't unsee it. I've tried to pare down mods to make it better, with limited success. I notice on the fps indicator on the debug menu, it'll show an average drop of 5-10 fps just after the glitches. Good to know it's not a mod issue; I was tearing my hair out trying to figure out the cause. It was making some of my craft nearly unflyable.

[sarbian]

So much wrong in those last few post...

[rkman]

4 hours ago, sarbian said:

So much wrong in those last few post...

What an unusually useless contribution from you.

 

[Mr Shifty]

4 hours ago, sarbian said:

So much wrong in those last few post...

I'm curious what you mean. There's this article in the Unity development manual, which mentions the problem and offers a couple ways to manage it, and there's this (long) article on Gamasutra, which suggests that the problem is even more intractable than the dev manual suggests somewhat due to Unity using an outdated version of the CLR, and some due to implicit object creation. There's a list here of how to minimize it. Unity appears to be working on the situation with the slow introduction of IL2CPP (not available in Unity 5 for non WebGL platforms.)

Edited April 24, 2016 by Mr Shifty

[sarbian]

Ok, I need a short break from coding so here is a more substantial comment. Also I won't smooth the angle in my comments, so don't take them as an insult. I am just stating fact (from my PoV). Also I won't go into the details of managed memory so if you want to know more go on Wikipedia

First you need to understand that Unity does NOT run on mono. Unity is a pure C/C++ engine that integrate mono as a scripting engine to run the game logic (so the graphic, physic, texture, ... are not done in mono). This is a really important point to understand when talking about memory in Unity. So now I will talk about unmanaged memory (handled by Unity native code) and managed memory (mono).

When Broco post a GCMonitor graph that tracks the mono memory and explain it is the model and texture unloading this is wrong.

• The texture, model, and most of the game are in loaded in unmanaged memory. Some are loaded/unloaded by Unity on scene change (those in the KSP_DATA asset files) and some always stay here (those in GameData).
• The managed memory is used by the game logic. Squad (and modders) KSP code runs in mono and constantly create new objects while running. This is PERFECTLY normal for code that uses mono (or .NET, Java, and many of the lang that uses managed memory and GC). Each time you want to display a number on screen you need to convert that number in a list of characters and that list takes memory. All those operation use more and more managed memory (the graphe going up). At some point later the manged memory reach a certain size and the Garbage Collector starts to cleanup object that are not used anymore (the graphe going down).

Now there are multiple things to keep in mind :

• Mono and its version is tied to Unity. It is an OLD version (2.something from around 2011). The Garbage Collector in that version is quite simple and does not use really advanced strategies to clean the memory. More modern version have multiple memory space to separate short and long lived object and clean the short object more often to avoid the large cleanup pause. Unity dev said they want to upgrade it but they could not for licensing issues (they need to use mono on their phone & consoles version of Unity and the company that has the license for that ask for big $$). They now have an alternative for the console/phone (IL2CPP) and they said they want to upgrade mono (or maybe use CoreCLR) in a future Unity release (Roadmap) 
• You can not run Unity without mono. It is part of the engine.
• You do not call the GC. It does it's job without any need to call it. If you call the GC manually you are doing it wrong. But then we have an old GC...
• Squad can not do that much about the managed memory use. They can try to redo part of the code to use less memory but again the allocation/GC cycle with managed memory is perfectly normal. They already lowered the memory use by a great deal when they converted the UI to the new UI system. The old one use tons of memory and removing it sure help a lot. The actual culprit is Unity, not KSP.
• There are ways to lower the allocated memory use but they make the code a lot more complex and can introduce a lot of new bugs. I did it for 2 sections of MJ code but it took a lot of time to make it stable again and some part of the code are now hard to read for a new dev (and for me a couple of month later). I am sure some part of the code could use more work but I doubt the remaining bits would have as much impact as removing the old UI.

And finally : give the dev a break. Sure 1.1 is not as perfect as we want it but they did an amazing work on the Unity 5 conversion and they will keep making it better.

[rkman]

21 hours ago, sarbian said:

Each time you want to display a number on screen you need to convert that number in a list of characters and that list takes memory. All those operation use more and more managed memory

I suppose that explains why even with only one small craft in the entire Kerbin universe, just orbiting a planet, no new geometry nor new textures loaded, memory use still increases and we get a CG stutter every 5 seconds or so.  

I'd think it does not really need new additional memory every time for instance the altitude meter ticks up or down, instead a change of that number displayed on screen could use the same memory as the previous change, and it could just overwrite that bit of memory instead of first cleaning it. 

 

Edited April 25, 2016 by rkman

[rudi1291]

1 hour ago, rkman said:

 

I'd think it does not really need new additional memory every time for instance the altitude meter ticks up or down, instead a change of that number displayed on screen could use the same memory as the previous change, and it could just overwrite that bit of memory instead of first cleaning it. 

 

Thats not possible with C#/Mono, unless you do some extremly ugly hacks.

Edited April 25, 2016 by rudi1291

[Padishar]

1 minute ago, rudi1291 said:

Thats not possible witch C#/Mono, unless you do some extremly ugly hacks.

Well, things like the stock altitude display have no real need to allocate memory on every frame or when the numbers change, Sarbian was referring more to textual items that need to be specified to the Unity API as a string.  Strings are immutable under the CLR so if your code just blindly takes a numeric value and converts it to a string on every rendered frame then this will create a new string object every time.  The usual way to combat this is to only recreate the string when the numeric value changes, though this only makes a significant difference for things that do not change very often.  For things that do change very often it will make a lot of difference simply not updating it on every frame, even every other frame would reduce the number of new strings created by a factor of 2 and updating a flight data display 30 times a second is still probably far more than necessary.  Between 3 and 5 times a second would probably be plenty for most vessel state readouts (between 12 and 20 times less than 60 times per second).

In any case, there are still several bits of stock KSP code that are creating far more garbage than is sensible.  At least a couple of these should be addressed by the reworking of the resource handling code that got pushed back to 1.2.  I would like to see the memory allocation rate down at something like a few kbytes per frame (that should be plenty to cope with creating new immutable strings) rather than the megabytes per frame it seems to do sometimes now...

[Thorbard]

On 24/04/2016 at 5:47 AM, Mr Shifty said:

I've been playing for 3 years and never noticed this before, but I started seeing it today (it was probably always there) and now I can't unsee it. I've tried to pare down mods to make it better, with limited success. I notice on the fps indicator on the debug menu, it'll show an average drop of 5-10 fps just after the glitches. Good to know it's not a mod issue; I was tearing my hair out trying to figure out the cause. It was making some of my craft nearly unflyable.

Likewise. I notice this on a clean install of 1.1 but never before. Seems worse on the map view for me.

[sarbian]

I will add that I experimented with one of those "ugly hack" that let you use string without allocation a couple of days ago and I have something working in the Unity editor. I ll deploy it in GCMonitor (since having a monitor that does not use memory sounds like a good idea) and other can use it (if it does not explode stuff on contact).

The strange thing if that I don't have those stutter myself in 1.1, but I had them before... Maybe we could play with manually calling the GC every x seconds (my ex-java performance profiler heart cringe) or allocate more heap on start (you tried this in the past @Padishar I think ?)

[Padishar]

8 minutes ago, sarbian said:

The strange thing if that I don't have those stutter myself in 1.1, but I had them before... Maybe we could play with manually calling the GC every x seconds (my ex-java performance profiler heart cringe) or allocate more heap on start (you tried this in the past @Padishar I think ?)

Yes, the strangest part of all this is how it affects different people in different ways.  I've been looking into it for a long time and I am yet to find any definite pattern.

Yes, I've tried calling the GC directly (shudder) and it generally doesn't help much.  I've also tried various things to pad the heap and, while these do help initially, Unity (at least in Unity4) very quickly frees up the padding due to how the heap management works (all still used blocks get shifted down as part of the GC, moving all the free space to the end and when it sees it has several hundred megs it isn't using it gives most of it back to the OS).  I also built a version of your GCMonitor that defaulted to Mono mode and 0.125s and only had the graph in the window because the set-up of the controls at the top of the window was doing quite a lot of string concatenation and it was skewing the results of some tests I was running.

If the allocation rate could be got down to a few k per frame (or even a few k per second) then the garbage collection would run much less often (e.g. once every few minutes rather than once every few seconds) and, I suspect, almost no-one would complain about it.  At that point it would probably make sense to manually run the GC at points the user won't really notice, e.g. going into and out of map view, pausing the game, saving (including autosave), switching between vessels etc.

[Broco]

I'm sorry if I got some things wrong here I'm not that deep into Unity, I just tried to find a general explanation.

Just to make sure that I understand it correctly (correct me if I'm wrong):

1.  The Garbage Collector is automatically called when a certain amount of memory is used. SQUAD has no influence over this because it is called by the engine.
2.  If you reduce the allocation rate, the peak is reached much slower so the GC runs less often.

A friend of mine who is using Unity a lot for some of his projects told me that one of the biggest causes for these rapid GC calls is when you instantiate and destroy a lot of objects because this causes a lot of garbage to be created. He said it makes more sense to instantiate most objects at the start of your game and enable/disable them. Only destroy them when the Game is closed.

Also string concatenating seems to be a big issue.

 

God I really wish KSP was Open Source (and no, this means not for free). I would love to try things out.

[sarbian]

35 minutes ago, Broco said:

A friend of mine who is using Unity a lot for some of his projects told me that one of the biggest causes for these rapid GC calls is when you instantiate and destroy a lot of objects because this causes a lot of garbage to be created. He said it makes more sense to instantiate most objects at the start of your game and enable/disable them. Only destroy them when the Game is closed.

AFAIK this is not the case here. And anyway this solution works for some case (the common one being projectiles) but not all. You can't have all your world loaded in memory all the time.

[Broco]

On 27.4.2016 at 10:58 AM, sarbian said:

AFAIK this is not the case here. And anyway this solution works for some case (the common one being projectiles) but not all. You can't have all your world loaded in memory all the time.

Uhm I thought the data causing the GC to awake isn't things like textures but variables and arrays etc.?

If it wasn't the case, could there be a setting to increase the heap on machines with more memory? I mean I really got a lot of RAM to spare here...

[sarbian]

And where did I mention texture ?

 

A big heap does not solve the problem. You just move from a frequent small pause to a less frequent large one.

[Majorjim!]

I'm curious to know if the OP experiences the stutter all the time or just with a certain number of parts of after a certain amount of play time elapsed.

 I get no stutter at all during initial gameplay. It only becomes game breaking when building complex craft in the VAB for an hour or so. It gets so bad I have to restart the game to stop it happening.

[Svm420]

38 minutes ago, Majorjim said:

I'm curious to know if the OP experiences the stutter all the time or just with a certain number of parts of after a certain amount of play time elapsed.

 I get no stutter at all during initial gameplay. It only becomes game breaking when building complex craft in the VAB for an hour or so. It gets so bad I have to restart the game to stop it happening.

I don't even think the OP looks at this thread it is almost a year old and he hasn't replied to the thread in almost a year. I don't get why this needed to be bumped there is no info in this thread that made it worth keeping. Why not make a new thread to discuss this...