Jump to content

[1.12.x] MemGraph Updated with Stutter Reduction


linuxgurugamer

Recommended Posts

Originally written by @padishar, original thread is here:  https://forum.kerbalspaceprogram.com/index.php?/topic/139128-12x13-memgraph-1103-with-stutter-reduction/

Note New Dependencies

This started as a simple plugin that displays a graph of the Mono heap allocation rate and garbage collection, mainly intended as a troubleshooting and development aid rather than for general use (hence why I originally released it in the Add-on Development sub-forum rather than this one).  However, I have since devised a way to force Mono to keep significantly more free space in the heap, which can significantly reduce the frequency at which the heap fills up and the Mono garbage collection causes a stutter, so I have added it to this mod and released it more publicly.  I will be continuing to work on this mechanism to improve it further.

The code has been highly optimised to create as little garbage as possible (given the constraints of the old OnGUI based API) so that it interferes with what it's measuring as little as possible.  I do have plans to add various "usability" type enhancements but only things that can be implemented cleanly and are directly useful for the mods purpose.

To install, simply download the zip from the link below and copy the MemGraph folder from the zip file into the GameData folder of your KSP installation.

  • Mod-KeypadMultiply toggles the display of the window.
  • Mod-KeypadPlus increases the vertical scale of the graph.
  • Mod-KeypadMinus decreases the vertical scale of the graph.
  • Mod-KeypadDivide runs a bit of test code controlled by MemGraph\PluginData\test.cfg
  • Mod-End pads the Mono heap with a configurable amount of headroom to reduce frequency of garbage collections.

Every second the plugin totals up all the memory allocated on the heap and whether any garbage collections have run.  It also displays the current total heap allocation, the maximum heap allocation just before the last garbage collection, the minimum just after the last collection, the number of "render" and "physics" updates and the time between the previous two collections.

The graph is 600 pixels wide and shows the last 10 minutes of the memory allocation in green.  If a garbage collection happens during the interval, that column of the graph will have a red background.

The test code basically allocates a number of blocks of the specified size and displays which allocations actually cause the allocated heap to change.  This allows us to deduce various characteristics of the memory allocator and garbage collection mechanisms (and was a crucial part of developing the heap padding mechanism).

Mod-End activates the heap padding.  The amount of padding is controlled by the MemGraph\PluginData\padheap.cfg.  The format of the file is very simple.  Each line controls the amount of padding allocated for blocks of each size range.  The first value is the size of each block allocated and the second is the number of blocks.  The first values are only present for illustration, they don't actually control the size of the blocks, these are hardwired to the sizes in the default configuration.  The total value at the end is the total number of megabytes of padding allocated (note, the memory usage of KSP will increase by more than this due to the free space headroom maintained by Unity).

Logging

The mod now has the ability to write the data to a set of log files. The files are written in the root of the game directory.  Logging can be enabled at startup by editing the settings.cfg file. Set the following to true to turn on logging, false to turn it off:

enableLogging = true

Logging Controls

  • Mod-Home writes a marker to the log file (if logging is enabled, see below)
  • Mod-PageUp toggles logging

The first set of files will only have a single value in them at any time. This will be the last value which was plotted. The name of the file corresponds to the data item as shown on screen:

  • dataLog.Cur.txt
  • dataLog.HeapMin.txt
  • dataLog.Interval.txt
  • dataLog.Last.txt
  • dataLog.Max.txt
  • dataLog.P.txt
  • dataLog.R.txt

This last file is a sequential log, and is overwritten every time the game starts. The log is formatted as a CSV file, and the first line contains the column headers

  • dataLog.Log.txt

I recommend that you run the game normally and load up a situation that has noticeable stutter.  Display the graph, setting the scale so the regular allocation rate fits nicely and the garbage collection red lines can be seen.  Let it run for several runs of the garbage collector and then hit Mod-End.  After a short pause, the game should continue with a considerably larger gap between the collections.  After another few collections hit Mod-End again and it may improve further.  I would appreciate feedback about how well this works, e.g. screenshots of the graph taken during this process and after another few collections would be very helpful along with details about your setup (and preferably, an output_log.txt/player.log file).  Many thanks to @Tig for the testing and very well presented data he posted over in the Grumpy Collector thread.

One other thing I should add, though it should be obvious with only a little thought, is that the heap padding mechanism is only intended for 64 bit versions of the game.  Trying to allocate 900 MB of extra heap space on the 32 bit version is unlikely to be successful and, if it is, then it will probably cause the game to crash before long due to running out of address space. It is also unlikely to work effectively if your machine has only 4GB of RAM as the total usage of KSP is likely to grow close to 4GB even without loading a save, resulting in virtual memory paging which will seriously hurt performance.

Regarding the padheap.cfg file:

If the default settings aren't working well for you, try the following steps:

Increase the padheap size in your install: \GameData\MemGraph\PluginData\MemGraph\padheap.cfg, change the "total" number to 4096 or even 6144. See the next section for an example

For example, if you have 16 GB:

  1. Look at your task manager to find out how much free ram you have when the game is running. 
  2. Do this calculation 16 Gb - ~5/6 GB from KSP - 1/2 GB Windows = 8 GB free
  3. This is the amount of RAM you can use for your padheap total size (in MegaByte 4GB=4096MB).  
  4. Always have some Ram to spare of course. 

Padheap is the volume memgraph is using to save the junk and than deletes it when its full.

Obligatory picture of mod in use:

tUUtX2i.png

Availability

Older versions

Previous releases on GitHub (versions 1.0.0.x are for KSP 1.1.x, versions 1.1.0.x are for KSP 1.2).

There's no obligation and it won't entitle you to any special treatment but, if you feel like it, then you can make a donation using the link below.

Donate Here

 

Edited by linuxgurugamer
Link to comment
Share on other sites

  • 4 weeks later...

hoo men... did you sleep some time?

your work is pure gold.

this mod works perfect for me...

Tested on i7-4700HQ - RAM 8GB - GTX 680m
Ubuntu 18.04 Nvidia Drivers 390.48
Runing perfect (almost zero stuttering) with KSP.x86_64 (64bit) and more than 60 mod's

Link to comment
Share on other sites

  • 2 weeks later...
1 minute ago, AngrybobH said:

Wow! I just stumbled into this because of @4x4cheesecake's post(thank you). downloading now. I need a serious stutter reduction (too many mods, too many parts, too many ships) and I have headroom for days (32GB). 

:D

Don't use too much of your RAM to increase the mono heap. If more memory is used, the GC will need more time so your game will 'freeze' for a second or two (tested with roughly 20gb added to the heap) At least, this will happen just every 10min^^ But feel free to test it on your own ;)

Link to comment
Share on other sites

7 hours ago, 4x4cheesecake said:

:D

Don't use too much of your RAM to increase the mono heap. If more memory is used, the GC will need more time so your game will 'freeze' for a second or two (tested with roughly 20gb added to the heap) At least, this will happen just every 10min^^ But feel free to test it on your own ;)

Yup. One approach would be to add a little RAM. I think hitting the add RAM button will add RAM each time you hit it ie first time say 0.5Gb, next time another 0.5Gb, so the total would be 0.5Gb then 1.0Gb etc. Once you work out what value works for you, you can replace your increment value with that.

Link to comment
Share on other sites

So this works, such great work! Much Props. For a long time this was an insoluble, hard-wired problem. This is a small miracle.

 

The default heap settings extend the gap between GC from about 20s to about 50s, so far so good.

Im not certain on how to increase the amount of "heap", I changed the values in the heap textfile from 1 to 10, is that how you do it? Thats megabytes right?

Is 10 reasonable? I am not RAM limited as yet, 24gb installed. Could I do more?

I have anecdotal evidence howver, that the 10-10-10-10...etc setting in the heap textfile reduced stability? Is that to be expected? I had a few extra crashes happen soon after adding extra heap with those settings.

On the other hand, Im running something like 200mods and my system crashes randomly every couple of days anyway (I think its a hardware mismatch as it can happen at any time, but its rare enough that fixing it isnt really worth the bother) so it could easily be coincidence.

 

 

10 hours ago, 4x4cheesecake said:

:D

Don't use too much of your RAM to increase the mono heap. If more memory is used, the GC will need more time so your game will 'freeze' for a second or two (tested with roughly 20gb added to the heap) At least, this will happen just every 10min^^ But feel free to test it on your own ;)

I can easily live with a 2s pause if the gap is 10min! What settings did you use for this?

 

***

One more question - does it matter when you add the heap? Does it work the same from space centre or VAB that it does in-play? I get a lot of pauses in the VAB when building larger vessels.

Link to comment
Share on other sites

11 minutes ago, p1t1o said:

I can easily live with a 2s pause if the gap is 10min! What settings did you use for this?

I can't remember, sry. Haven't used the mod for a while since I'm no longer running 80+ mods.

You might want to read through the original thread, i.e. on page 13 is an discussion about different settings for the padhead.cfg (the last number in this file is basically the increment of your heap everytime you press 'alt + end', you can ignore the other values) Don't forget to enable 'applyPadding' in your settings.cfg.

Link to comment
Share on other sites

4 minutes ago, 4x4cheesecake said:

I can't remember, sry. Haven't used the mod for a while since I'm no longer running 80+ mods.

You might want to read through the original thread, i.e. on page 13 is an discussion about different settings for the padhead.cfg (the last number in this file is basically the increment of your heap everytime you press 'alt + end', you can ignore the other values) Don't forget to enable 'applyPadding' in your settings.cfg.

Ah, the last number you say, where it says "total" I think? I did not actually touch that number....maybe that explains the crashes perhaps? I havent touched "settings.cfg" but assume it is setup ok as the default heap settings work.

Thanks, I'll try some stuff later tonight :)

 

Link to comment
Share on other sites

  • 4 weeks later...
  • 2 weeks later...
  • 2 weeks later...

Pretty much a invaluble tool for a smooth sailing experience, so thank you for this mod.

It's pretty much optimal here after some fiddeling with no reds in sight, although im running a bare minimum asortment of mods on v1.4.5

Before:

screenshot0.png?raw=1

After:

screenshot3.png?raw=1

Edited by Cratzz
Link to comment
Share on other sites

  • 4 weeks later...

My game didn't stutter much (only in flight, every 5-10 seconds) but oh boy did it annoy me. I have pretty good specs (R7 2700X, 16GB 3200mhz ddr4, GTX 1080, MX500 1TB SSD) and yet it stutters as much as it di on my 2600K rig. That is until I found this mod :D Linuxgurugamer saves the day, again

 

Edit: Do I need to pad the heap every time I play or does the mod do it automatically after setting it up to my liking?

Edit 2: If I understand correctly, enabling "applyPadding" in the settings.cfg does what I want. 

Edited by kendoka15
Link to comment
Share on other sites

  • 4 weeks later...
  • 2 weeks later...
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...