[1.1.2] GrumpyCollector 1.0.1 - Stutter ? What Stutter

[sarbian]

This is an experiment. Some people keep complaining about stutter in the game and other don't get this problem. This addon is for the first category.

If you have stutter in gamer please install this and tell us if things are better or worse. You even have a amazing UI that you can open with Ctrl + * (the one on the numpad). Faster may lower stutter but also lower the frame rate. Slower may make the stutter come back but has less impact on the frame rate.

Please share your experience with using this and if it make the stutter better or worse.

GrumpyCollector-1.0.1.0.zip

License MIT

Source

 

Edited May 10, 2016 by sarbian

[JPLRepo]

7 minutes ago, sarbian said:

This is an experiment. Some people keep complaining about stutter in the game and other don't get this problem. This addon is for the first category.

If you have stutter in gamer please install this and tell us if things are better or worse. You even have a amazing UI that you can open with Ctrl + * (the one on the numpad). Faster may lower stutter but also lower the frame rate. Slower may make the stutter come back but has less impact on the frame rate.

Please share your experience with using this and if it make the stutter better or worse.

GrumpyCollector-1.0.0.0.zip

License MIT

 

 

Assume this is an aggressive Garbage Collector... is there source?

[sarbian]

Of course. You even have a 1.0.1 since I forgot to remove some debug spam

Edited May 10, 2016 by sarbian

[Chris97b]

I've been having the stuttering issue pretty badly, so I figured I'd give this a try. Basically what I'm seeing with this mod is the game stutters every time the GC runs, no matter how fast I set the GC interval. Basically if I set it very low, the game simply stutters every time GC runs. If anything, faster GC has made things worse, as it means the game stutters more frequently.

[ve7oko]

Tried it but it made it much worse...

[drtedastro]

7 hours ago, sarbian said:

This is an experiment. Some people keep complaining about stutter in the game and other don't get this problem. This addon is for the first category.

If you have stutter in gamer please install this and tell us if things are better or worse. You even have a amazing UI that you can open with Ctrl + * (the one on the numpad). Faster may lower stutter but also lower the frame rate. Slower may make the stutter come back but has less impact on the frame rate.

Please share your experience with using this and if it make the stutter better or worse.

GrumpyCollector-1.0.1.0.zip

License MIT

Source

 

Would this be considered the "Gunther" effect, perhaps?

Cheers.

[stk2008]

When i went to bed last night i see this and thought wow i am going to try this in the morning.

But reading replies it may not work?.

Thanks op for attempting to fix this though if you manage to then we all owe you big time.

Other than stutter when unity does GC the game runs ok for me even with 50 mods

[nedlee]

I've had this problem for years, and the game's lagging even worse in 1.1.2. I'm glad someone is looking for a solution, but the fix is making it worse for me, unfortunately.

[stk2008]

Yep tryed it and its worse.

As said adjust it to lowest just makes it skip every second

[Padishar]

1 hour ago, nedlee said:

I'm glad someone is looking for a solution, but the fix is making it worse for me, unfortunately.

People have been looking for a solution for years, the problem is that the only practical solution is to reduce the rate of memory allocation by both core KSP code and mods.

Also, I don't see any mention by @sarbian that this was any sort of "fix".  He clearly stated it was an experiment and also clearly stated it could make things better or worse.

I have also tried various things along these lines (including patching mscorlib.dll to be able to profile exactly how long a manual run of the GC takes) but they do not help.  The frequency of the pauses is directly related to the rate of garbage creation, the collector runs automatically when the Mono heap doesn't have enough space when some C# code tries to allocate some memory.  The length of the pauses depends on a number of factors but the amount of garbage to be cleaned up is quite a minor one.  The total number of allocated blocks of memory is the main factor as the collection basically works as follows:

• Step 1, loop through every block of memory allocated on the heap and mark it as unreachable.
• Step 2, loop through every global and stack reference variable and mark the memory block as reachable.  If the block contains any references to other blocks (that are still marked as unreachable) then mark those as reachable and continue recursively.
• Step 3, loop through every block and collect any that are still marked as unreachable.  The collection usually involves moving the memory blocks down to fill up the collected spaces as this greatly increases the allocation efficiency.

It should be obvious that, even if there are no unreachable blocks at all (no garbage to be collected), running the collection will still take a considerable time as the entire heap is still scanned through twice plus the entire recursive "reachable" scan still has to run.  The only bit you save on is the shuffling around of the still-allocated blocks and fixing up of the references.

So, the only sensible ways to reduce the severity of this issue are:

• Reduce the overall number of "live" memory blocks.  Various parts of KSP (and mods) use significantly more memory (or more complex data structures) to represent things than is required.  E.g. the PartResource class is a simple class used to represent a resource in a part but it is derived from MonoBehaviour which is a complex Unity class that has many other reference members.  No base class would make this much simpler and would reduce the number of memory blocks used by loaded vessels considerably (and making it a struct instead along with some tweaks to the Part class would make it even better still).  This would reduce the length of each pause but would have no direct effect on the frequency.
• Reduce the amount of garbage being created, especially on a regular basis (e.g. in Update/FixedUpdate).  As stated before, the collection runs when the free space in the heap runs out.  If the game is allocating 10 MB/s then the free space in the heap will run out much sooner than if it were only allocating 10 KB/s and hence the collections would be happening much more often.  Reducing the amount of garbage created has the potential to reduce the collection frequency to such an extent that the collections could be triggered at times when the user is unlikely to notice the pause, e.g. switching to/from map view, pausing the game, etc. or to attribute it to something, e.g. run it after every save and load.

Unity do recommend manually running the garbage collection at a high rate as a way of preventing this issue but, only in games where the total heap size is small (I seem to recall it mentions < 50 or 100 MB).  KSP uses several times this just getting to the main menu and the GC runs simply take too long to be deliberately triggering them during scenes.

Even Unity upgrading the version of Mono used to a version that has a much more efficient garbage collector would not eliminate the problem though it should reduce the length of the pauses.  I have numerous saves where the memory allocation rate is constantly over 50 MB/s which results in the GC running every 2-3 seconds.  Even the latest version of Mono (or the very latest version of .NET) would have problems collecting the garbage from that without causing any delays longer than a frame or two.

Edit: one other thing, to eliminate the somewhat garbage happy GUILayout code in this mod I recommend you make sure the window is closed before evaluating the stutter.

Edited May 11, 2016 by Padishar

[stk2008]

Wow thats a details explanation.

Also I was in no way expecting the OP`s program to fully fix the issue nor do I expect him to.

was just excited to see some one was dabbling with the possiblitys of such.

Any way all we can do is prey and hope Squad or unity fix this...other than the GC issues the game runs good for me....untill I try to use an 800 part rocket that is

[sarbian]

Well, it was worth trying. I don't have any stutter on my system so it is kinda hard to know how it will react.

[Padishar]

Just now, sarbian said:

Well, it was worth trying. I don't have any stutter on my system so it is kinda hard to know how it will react.

Can you try loading the Platonic Stations save in my MemGraph thread and flying the KerbalX and then just holding down left cursor to rotate the view continuously?  If you really can't see any stutter in that then I will be very surprised...

[LordAssault]

I'm not expert at this, but there was something similar with mods for other game Cities Skylines it's using Unity 5 too. One of the modmakers did a great job with his own mod and posted solution here:

https://www.reddit.com/r/CitiesSkylinesModding/comments/2zk3ys/bordercities_2_edge_detection_done_right/cpjnweb

Maybe that will give you some ideas :/

[genericeventhandler]

Hi, 

Forcing a garbage collect is a very brute force way of freeing up memory, but the whole .net stack will pause when you do it, so that's not what you want. 

Simple rules to follow, 

• Avoid huge arrays of objects, use arrays of primative types instead. 
• Don't create List<T> before you use them, if a list is in memory for a long time, it skips the first (quick) garbage collect and goes straight to the slow jerky version 
• When you have done with an object set it to null as soon as possible, don't hold onto it if you don't need it, a method that takes 2 seconds to fall out of scope means that that object is in memory for 2seconds longer than needed and will probably be slow garbage collected.
• If the object you are using supports IDisposeable wrap it in a using block to close and free it as fast as possible..
• If you are using an IDisposeable interface, implement it correctly and call GC.SuppressFinalize(this) after you cleaned all references
• Avoid Statics, 
• See previous rule
• Seriously avoid statics

GE. 

 

1 hour ago, Padishar said:

 

So, the only sensible ways to reduce the severity of this issue are:

• Reduce the overall number of "live" memory blocks.  Various parts of KSP (and mods) use significantly more memory (or more complex data structures) to represent things than is required.  E.g. the PartResource class is a simple class used to represent a resource in a part but it is derived from MonoBehaviour which is a complex Unity class that has many other reference members.  No base class would make this much simpler and would reduce the number of memory blocks used by loaded vessels considerably (and making it a struct instead along with some tweaks to the Part class would make it even better still).  This would reduce the length of each pause but would have no direct effect on the frequency.
• Reduce the amount of garbage being created, especially on a regular basis (e.g. in Update/FixedUpdate).  As stated before, the collection runs when the free space in the heap runs out.  If the game is allocating 10 MB/s then the free space in the heap will run out much sooner than if it were only allocating 10 KB/s and hence the collections would be happening much more often.  Reducing the amount of garbage created has the potential to reduce the collection frequency to such an extent that the collections could be triggered at times when the user is unlikely to notice the pause, e.g. switching to/from map view, pausing the game, etc. or to attribute it to something, e.g. run it after every save and load.

[sarbian]

4 minutes ago, genericeventhandler said:

Forcing a garbage collect is a very brute force way of freeing up memory, but the whole .net stack will pause when you do it, so that's not what you want.

Just to be clear I know that calling the GC manually is an heresy but given the limited option we have with Unity I wanted to see if it helped for those with the problem.

[Padishar]

44 minutes ago, LordAssault said:

Maybe that will give you some ideas :/

It appears to me that the mod in question was doing something incorrectly with shaders and materials causing a memory leak which he then fixed (and, by the look of it, not by a particularly sensible method).  Why is he creating and destroying a shader and material instance on every frame?

13 minutes ago, genericeventhandler said:

Forcing a garbage collect is a very brute force way of freeing up memory, but the whole .net stack will pause when you do it, so that's not what you want. 

Yes, but it isn't really about freeing up memory, it is more about splitting up the work required into smaller packages so the pauses they cause are less noticeable.

13 minutes ago, genericeventhandler said:

Don't create List<T> before you use them, if a list is in memory for a long time, it skips the first (quick) garbage collect and goes straight to the slow jerky version 

There is no "first (quick)" garbage collect in the version of Mono used by Unity (as far as I am aware).  In fact, the only one of your points that really makes sense is the first one...

Edited May 11, 2016 by Padishar

[genericeventhandler]

16 minutes ago, Padishar said:

 

There is no "first (quick)" garbage collect in the version of Mono used by Unity (as far as I am aware).  In fact, the only one of your points that really makes sense is the first one...

Mono garbage collection

You need to aim all objects at what the article describes the "Nursury" level 0 garbage collect, Nulling objects will qualify them for quick collect. 

GE

[Padishar]

1 minute ago, genericeventhandler said:

Mono garbage collection

You need to aim all objects at what the article describes the "Nursury" level 0 garbage collect, Nulling objects will qualify them for quick collect. 

That article describes the generational collector in recent versions of Mono, not the collector in the old version of Mono that Unity uses...

[sarbian]

To be precise Unity uses the Boehm GC. So if you have some magical trick to make it work please enlighten those who have been looking into it for month. 

 

[Blacks]

28 minutes ago, Padishar said:

Yes, but it isn't really about freeing up memory, it is more about splitting up the work required into smaller packages so the pauses they cause are less noticeable.

o yees what result in better comunication between hardware & soft IF request are made proper, on a simple test lower Grumpy to 0.250 you will see straight / instant in MemGraph efect, then up rise to +2 or more depend by each rig configuration agen effect it will be seen instant to MemGraph

by the way GrumpyCollector + MemGraph work perfect to see result how it goo

[LordAssault]

42 minutes ago, Padishar said:

It appears to me that the mod in question was doing something incorrectly with shaders and materials causing a memory leak which he then fixed (and, by the look of it, not by a particularly sensible method).  Why is he creating and destroying a shader and material instance on every frame?

His mod had many options of changing shaders(as i can understand) http://steamcommunity.com/sharedfiles/filedetails/?id=410196151 It's easier to see, than explain.

But this isn't the only mod, which had this issue, every mod which worked with shaders was causing this leaks i remember at least 5 mods from 4 different authors with same issues. So i would blame Unity

I remember how you can diagnose this issue(because this doesn't make any exceptions) just install mod with custom shaders save game and reload it 5-10-20 times based on how heavy mod is, your framerate will drop for 1-60 fps.

I'm not a modmaker, neither programmer, just a mod user and maybe a beta tester, so i'm not 100% sure that this will be helpful for KSP modmakers. :/

P.S. I'm not native English speaker, so i'm sorry.

[Maars]

Sarbian, thank you for thinking of the community and coming up with this experiment 

[sarbian]

12 minutes ago, LordAssault said:

So i would blame Unity

I would blame bad mod code. The coder in that thread clearly did not understand how Unity object works and that would not be an isolated case.

 

[Jansn67]

Sarbian, thanks you very much for letting us experimenting with and initiation to make an amazing and hyperfun game "betaR than evaR".

Tried GrumpyCollector with a copy of my 1.112-longtime-savegame (including around 90 mods).

Want to tell my personal experience (Lubuntu 16.04, KSP64, Intel [email protected] -> Hyper Threading off, nVidia 560TI, 32GB Ram@2400):

Ingame Time (upper left corner) runs faster, quite earth realtime, but stuttering feels unusual and strange.

I tried settings 0.3, 1 and 5 in settings file. I started KSP with and without the wiki-Linux-Launcher-script.

 

PS: i would say, also that "hacking around" in cfgs, trying different mod mixtures, keep an installation cosy, makes that game so much "fun" too.

And yes, over the times and years, i learn/ed a lot of "handling ksp" :-) (It sometimes really can be a.. "poodley and uptight", specially heavy modded.)

But now, MINE actually runs really smooth.

Thanks again and keep up your important business. Without your participation KSP would never got so great how it is today.