It's been 6 months since I asked about the stutter

[Padishar]

On 1/14/2016 at 0:23 PM, Padishar said:

Actually, this isn't quite true.  Forcing the heap to be larger would also reduce the frequency as it would take longer for the heap to fill up.  This is also described in the Unity docs I linked in another post.  It should be possible to write a mod that forces a large allocation of memory (but not so much that it kills KSP) and then discard it.  If Unity doesn't automatically shrink the size of the heap (and there are other tricks that may prevent this if it tries) then this should result in the heap having much more free space and the stutters should stay further apart.  I'll try to knock this up if I get a few minutes...

Edit: Even better, make it a button on the stock toolbar that triggers the extension of the heap by allocating and discarding a large amount of memory.  That way, the user can just click it if the glitches are too bad and they should get better (assuming more memory can be allocated without crashing the game).

P.S. I hereby place all these ideas in the public domain so any other person is free to implement them before I do if they feel like it...

Just in case anyone following this thread hasn't seen it yet, I have now implemented this "heap padding" mechanism in my MemGraph mod linked in my sig below.  It took a little longer than "a few minutes" to get it working reliably, the initial tests worked but only for one or two runs of the GC, after which Unity/Mono would discard the padding and the improvement would disappear.  After research into the low-level details of how the Mono heap management works, I have devised a way of padding the heap in such a way that Mono can't get rid of it.  This mechanism can result in the interval between stutters being increased significantly though it is highly dependent on the pattern of memory allocation (I've measured between 3 and 12 times in various situations and others in the thread have also reported similar and that they consider it a significant playability improvement).

[Kethevin]

I recently returned to the game again and have really noticed the stutter.  Mine is every 5 seconds.  It's a brief freeze, but it's enough to ruin a close docking attempt or course correction.  Worse, it's driving me crazy (granted, I don't have far to go there, walking distance really...)

I've played the game off and on for several years, I don't remember the stutter happening or at least being this noticeable before.  It has seriously become game-breaking for me.  

[HebaruSan]

1 hour ago, Kethevin said:

I recently returned to the game again and have really noticed the stutter.  Mine is every 5 seconds.  It's a brief freeze, but it's enough to ruin a close docking attempt or course correction.  Worse, it's driving me crazy (granted, I don't have far to go there, walking distance really...)

I've played the game off and on for several years, I don't remember the stutter happening or at least being this noticeable before.  It has seriously become game-breaking for me.  

There's some potential good news to share on this front. According to several recent devnotes, the developers have worked on an initiative for the 1.2 release to reduce the amount of garbage the game generates, specifically by changing the style of many loops and rewriting certain notorious core API calls. This suggests that they're at least taking it seriously, and we may see significant improvements if the changes are well-done (hard to tell for sure before the changes are released, of course).

Quote

removing old foreach() loops, Singletons and generally de-linqing the game. We recommend that modders to the same in anticipation of the next update to help increase performance for all KSP players. Linq and foreach() loops in particular create a lot of garbage, which is what causes the stutter that some people experience while running the game.

Quote

Resource requests and GetConnected requested are now cached per set of crossfeeding parts, and per resource ID. On top of that the event system created in 1.1 for when PartResources change state (flow mode, empty, nonempty, full, etcetera) is used so that the caches are updated in place rather than recreated. All in all, this means there is a small spike on staging but so long as the part count doesn’t change and crossfeed rules don’t change the actual lookups are extremely fast and garbage-free.

Quote

PartModules take advantage of caching where possible and have lower garbage hits, and CrewManifest was completely rewritten to lower garbage creation and avoid loops.

 

Edited September 3, 2016 by HebaruSan

[katateochi]

3 hours ago, HebaruSan said:

There's some potential good news to share on this front.

This is really good to hear, I'd not read some of those dev notes so thanks for collecting that info together. That sounds promising. 

[PSU_Jedi]

I was having a ton of problems with stutter in the game. It got to the point where it would freeze up for a second about every two to three seconds and it was driving me absolutely bonkers. I tried all kinds of tweaks that have been suggested, to no avail. In fact, just for S's & G's I bumped up the graphics to the max and it didn't make anything worse.

I just happened to be combing through my contracts and noticed that I had one to capture an E class asteroid and one for an A class asteroid. When I went to the tracking station, I noticed that I was tracking a ton of C class asteroids. So I stopped tracking them. All of a sudden, voila! No more stuttering!

So for me it seemed to be fixed because I was just tracking way too much junk in the tracking station. I should also note that my methodical debris removal (using KIS and the K4 explosive when it's too much delta-v to try to bring it back for reentry) has not made any noticeable difference...it's only been the removal of tracking asteroids that drastically improved my gameplay. I hope this helps someone else!

[Sybok]

I suffer from this problem too. It is very painful and my biggest issue with KSP by far!
Bugs and some game crashes are not that big of a deal, but this stutter (~0,2s every 2s) ruining my whole game experience

[Padishar]

4 hours ago, Sybok said:

I suffer from this problem too. It is very painful and my biggest issue with KSP by far!
Bugs and some game crashes are not that big of a deal, but this stutter (~0,2s every 2s) ruining my whole game experience

Try the MemGraph mod linked in my sig...

[Broco]

On 28.12.2016 at 0:27 AM, Padishar said:

Try the MemGraph mod linked in my sig...

It somehow worsens it for me. Any settings suggestions? I have a lot of ram to spare. I think it has to do with the sheer amount of flights I have (~50, refineries on different moons, relays, space stations and missions) since it doesn't occur in an empty sandbox save. I try to reduce it but that's not really the point of a space program. Why are inactive missions creating so much garbage?

[Padishar]

5 hours ago, Broco said:

It somehow worsens it for me. Any settings suggestions? I have a lot of ram to spare. I think it has to do with the sheer amount of flights I have (~50, refineries on different moons, relays, space stations and missions) since it doesn't occur in an empty sandbox save. I try to reduce it but that's not really the point of a space program. Why are inactive missions creating so much garbage?

Your best bet would be to post an output_log.txt and some screenshots showing the MemGraph window in a number of situations (with the scale set reasonably so it isn't all solid green).  Also, some more details of your machine would be helpful, e.g. at least your total RAM and your free physical memory once you've loaded your save in KSP.  Inactive missions shouldn't be creating large amounts of garbage as Squad fixed the main cause of this for 1.1.x but there have been reports that seem to imply that having a large number of tracked asteroids does nasty things.  Do you have lots of asteroids?  If the save game is (mostly) stock (at least in terms of part mods) then you could upload that so others could try it.  Even if it uses quite a few part mods, someone may find the time to investigate it (or already have most/all of them installed)...

[Mike106]

I have 2x GTX 1080's and a 7700k @4.4 Ghz. I still have these stutters. it goes from 30-40 FPS to 1 every 10-15 seconds when using larger aircraft. If you quick save + reload it helps. relaunching the game makes them temporarily stop. It doesn't matter what settings you use and it is NOT a hardware thing. I think the Unity game engine is struggling to refresh after V-sync causing certain frames to take longer than others. My dual core XP machine runs this game the same as my friends rendering server as well as my gaming rig. HE has 6 quadros and 2 xeons and it still stutters after a couple hours of gaming (400-500 FPS then drops to 6 for 10-15ms). Windows, linux, unix, doesn't matter. the engine is broken and can only be resolved by patching or upgrading the engine. The only thing i haven't tried is Windows 7 or an apple based OS. Playing with your driver setting may help. engaging triple or dual buffering seems to make it worse. SLI cuts the stutter time in half but increases the frequency. 

There has been talk on steam of a new patch recently. will wee what happens otherwise its a permanent thing. 

[Val]

@Mike106, I moved your post to a more recent topic, than the dusty old 2015 one, you had found.

[sal_vager]

Hi @Mike106, the pause is from the old version of Mono's Boehm garbage collector, this is built into the Unity3D engine.

Lots of work has been done by Squad to reduce garbage but it cannot be eliminated, and add-ons can create their own garbage.

Thanks to Microsoft relicensing Xamarin as MIT software, Unity Technologies are now able to update their version of Mono, the newer versions of Mono make the SGen garbage collector available, check the Unity3D site for news on when this might occur.

To be clear, this isn't an FPS issue, the Unity engine is literally pausing the application to manage memory, Squad have done all they can to alleviate this but it's unlikely to ever be eliminated, as it's a feature of the engine.

[kokospl]

On 2.03.2017 at 11:21 AM, sal_vager said:

@Mike106 Squad have done all they can to alleviate this but it's unlikely to ever be eliminated, as it's a feature of the engine.

It's a very elite feature, I haven't seen it in any other Unity games. Aparently, we are very lucky.

[sal_vager]

1 minute ago, kokospl said:

It's a very elite feature, I haven't seen it in any other Unity games. Aparently, we are very lucky.

Elite? No, you are welcome to check the Unity3D documentation yourself, this is a core component of the engine and it is present in every Unity3D game made, from the Unity tutorials to the most extravagant projects.

Here's some links that may interest you.

https://docs.unity3d.com/Manual/UnderstandingAutomaticMemoryManagement.html

https://github.com/Unity-Technologies/mono

https://unity3d.com/learn/tutorials/temas/performance-optimization/optimizing-garbage-collection-unity-games

There are many more.

[kokospl]

1 hour ago, sal_vager said:

Elite? No, you are welcome to check the Unity3D documentation yourself, this is a core component of the engine and it is present in every Unity3D game made, from the Unity tutorials to the most extravagant projects.

Pardon me, I forgot I'm not supposed to use sarcasm on the internet.

What I meant was, not a single Unity game I played had this stutter nonsense. Before 1.2 rolled out, I thought garbage collector was just a guy waking me up in the middle of the night with his dump truck. When you look at it this way, KSP is an unique enlightening experince, where one cannot simply play the game and enjoy it. You need to understand software design witchcraft, peaks, memory leaks, padheaps and whatnot. You never get that sort of depth playing Battlefield.

Edited March 6, 2017 by kokospl

[sal_vager]

Just now, kokospl said:

Pardon me, I forgot I'm not supposed to use sarcasm on the internet.

What I meant was, not a single Unity game I played had this stutter nonsense. Before GC 1.2 rolled out, I thought garbage collector was just a guy waking me up in the middle of the night with his dump truck. When you look at it this way, KSP is an unique enlightening experince, where one cannot simply play the game and enjoy it. You need to understand software design witchcraft, peaks, memory leaks, padheaps and whatnot. You never get that sort of depth playing Battlefield.

Oh I see what you mean, well it's the nature of the game unfortunately, a physics simulation in C#, there's a lot (or there was a lot, things are a lot better now) of usage of the kind of memory that needs a garbage collector, unless it's rewritten in C and C++ and memory is handled manually.

You just don't get this with Battlefield or most other games, there's no real physics, what there is is faked or handled by a physics engine that's designed to do limited things very fast, and it doesn't exactly take a lot of code to move a soldier forwards when you hold the W key.

I am not sure anyone tried a game like KSP in Unity before this, or will again, every game I see with physics of some sort is either small enough that this isn't a problem or it uses a custom engine.

[kokospl]

16 minutes ago, sal_vager said:

I am not sure anyone tried a game like KSP in Unity before this, or will again, every game I see with physics of some sort is either small enough that this isn't a problem or it uses a custom engine.

So what will it take to fix it? I assume that reducing the amont of garbage created is resource intensive process. Like many things in life - it's not impossible, just expensive. If we create a crowdfunding campaign "Make KSP great again", what the magic number would be?

[sal_vager]

Just now, kokospl said:

So what will it take to fix it? I assume that reducing the amont of garbage created is resource intensive process. Like many things in life - it's not impossible, just expensive. If we create a crowdfunding campaign "Make KSP great again", what the magic number would be?

The devs who left last year worked extremely hard to reduce garbage as much as they were able to, and this has had a huge positive effect on KSP, but it can't be eliminated.

At some point the work required is too great and the return too small.

Also, mods add to the memory garbage, not parts but plugins, anything that uses strings in C# will allocate heap memory, so that's anything with text.

There's newer ways to handle text, stringbuilder for example, though performance will vary, and there's awkward workarounds using char arrays, which is already done in KSP.

Sorry but there's really no way to 'fix' this as it's not really 'broken', this is how the garbage collector in Unity works, but when Unity updates Mono they will make the SGen GC an option, and that is much newer and smarter than the garbage collector Unity currently provides.

[kokospl]

9 minutes ago, sal_vager said:

At some point the work required is too great and the return too small.

I agree, hence my question in post above. Assuming the "return" is additional revenue from more copies sold, I asked what the crowdfunding campaign target should be?

I for one, would like to play this awsome game without stutter. To that end, I'm willing to pay for KSP for a second time and I don't belive I'm alone in this matter.

 

Edited March 6, 2017 by kokospl

[sal_vager]

Just now, kokospl said:

I agree, hance my question in post above. Assuming the "return" is additional revenue from more copies sold, I asked what the crowdfunding campaign target should be?

I for one, would like to play this awsome game without stutter. To that end, I'm willing to pay for KSP a second time and I don't belive I'm alone in this matter.

 

I don't think you quite understand, it would take a complete rewrite of KSP at this point and that could take years, or for example a KSP 2, even then there is no guarantee because a 'stop the world' garbage collector is how Unity works, even SGen won't eliminate this, it's just smarter about what it checks and deletes so pauses are potentially shorter.

Unity themselves offer advice for memory management, but I'm afraid in this case the only fix might be a simpler game, and none of us want to lose the complexity that KSP does have, many want to add to it.

[kokospl]

Keeping my fingers crossed for KSP 2.0, hopefully I won't be to old to enjoy it. Thank you for your honesty.

[cantab]

For what it's worth I've not noticed it in a while, not playing near-stock (only mod being RSS and its hard dependencies). As sal_vager mentioned dealing with the problem isn't just on Squad, modders need to be careful too. It's a problem that affects every Unity developer, evidently some deal with it better than others. Code that might be natural and acceptable in another application can need to be avoided in a Unity game.

Game developers, mod developers, and players won't have a true 100% solution to this until Unity change their C# runtime.

(My understanding is that Unity got stuck in this problem because they made their own modifications to a once-current version of Mono, which meant that when Mono improved their garbage collector Unity couldn't upgrade. Legal issues with the software licenses were also a problem for Unity.)

[sal_vager]

Well, it's all hearsay and but there's plenty of this kind of post around if you look.

[paulprogart]

1 hour ago, sal_vager said:

Well, it's all hearsay and but there's plenty of this kind of post around if you look.

That's a bit old and actually a lot has changed that gives reason to be optimistic.  Microsoft is now involved with both Unity and Mono.   (There's already code sharing underway between Mono and .NET Core, and actually I fully expect at some point for the two projects to just merge.)  It will take time to upgrade Unity though; they've got to replace the very core of the Unity player on multiple platforms after all.

For the Unity-bashers, remember that while Unity may be far from ideal for KSP (and indeed many other games), if it weren't for Unity then we probably would not even have a KSP.  It let the developers focus on the simulation rather than low-level stuff.  Writing a new custom engine for something as complex as KSP could take years -- and an 8-figure budget.

[sal_vager]

8 hours ago, paulprogart said:

That's a bit old...

Yes, it's old, Microsoft have actually bought Xamarin and have released Mono under the MIT license, so Unity are free to upgrade to the latest version and I hear this is happening.