[1.9.x-1.12.x] HeapPadder - Pads the heap to minimize the garbage collection

[linuxgurugamer]

HeapPadder

This mod is designed to force Mono to keep significantly more free space in the heap, which can significantly reduce the frequency at which the heap fills up and the Mono garbage collection causes a stutter,

 Installation

1. Copy the HeapPadder folder from the zip file into the GameData folder of your KSP installation.

Regarding the padheap.cfg file:

1. If the default settings aren't working well for you, try the following steps:
2. Increase the padheap size in your install: \GameData\HeapPadder\PluginData\padheap.cfg, change the "total" number to 4096 or even 6144. See the next section for an example
3. For example, if you have 16 GB:
   1. Look at your task manager to find out how much free ram you have when the game is running. 
   2. Do this calculation 16 Gb - ~(5 to 6 GB from KSP) - (1 to 2 GB Windows) = 8 GB free
   3. This is the amount of RAM you can use for your padheap total size (in MegaByte 4GB=4096MB).  

 Always have some Ram to spare of course. 

Configuration

• The mod uses a config file called:  padheap.cfg.  This file is not distributed with the mod, it is created the first time running the mod based on the amount of system memory found.
• The mod has several default config files for different sizes of system memory.  These files are used if no padheap.cfg file is found, usually on the first time running.
• If the config file padheap.cfg is found, then it will be used.  The mod will NOT overwrite the file, this is done so that any future updates to the mod won't wipe out any local config changes
• There is nothing preventing you from use one of the suggested files, even if your system memory is different.  To use any of them, simply copy the desired file to:  padheap.cfg
• Current files supplied allocate the following memory to the heap:
  • SuggestedFor_32g.cfg    8 gig of memory, used by default if system memory is greater than 20 gig of memory
  • SuggesetdFor_16g.cfg    4 gig of memory, used by default if system memory is between 8 and 20 gig of memory
  • SuggestedFor_4g.cfg        1/2 gig of memory, used by default if system memory is less than or equal to 4 gig
  • default_padheap.cfg        1 gig of memory, used by default if system memory is greater than 4 and less than or equal to 8 gig of memory
• In the unlikely event that no files are found, a default configuration will be written out.

While this can be used with Memgraph, you risk doubling the memory usage if you allocation memory with Memgraph as well as HeapPadder.  To address this I've released an update to MemGraph, which disables the heap padding functionality of Memgraph if HeapPadder is installed

Pics

HeapPadder has no UI, so nothing to show

Availability

• Download:
  • https://github.com/linuxgurugamer/HeapPadder/releases
  • https://spacedock.info/mod/2190/HeapPadder
• Source Code: https://github.com/linuxgurugamer/HeapPadder
• License: MIT license (see https://github.com/linuxgurugamer/heapPadder/blob/master/Graph.cs)

 HeapPadder is also available via CKAN

 

Edited November 19, 2021 by linuxgurugamer

[jefferyharrell]

Is this just the heap padding part of Memgraph pulled out into a standalone mod?

[linuxgurugamer]

15 minutes ago, jefferyharrell said:

Is this just the heap padding part of Memgraph pulled out into a standalone mod?

Yes

[jefferyharrell]

@linuxgurugamer Excellent. Thanks very much for this.

[HuubinNZ]

This looks like a useful Add-on to have.

You have mentioned "Mono" and "task manager". The "task manager" means you are working on a Windows system and not a MacOS or Linux system. I was not aware that the Windows version of KSP ran on Mono, which is used on the Mac as a Windows runtime for CKAN. 

Does KSP then run under Mono on a Mac as well and would this work on MacOS and Linux systems?

[Nightside]

Thanks for this @linuxgurugamer, does it make the stutter longer when it eventually occurs?

[linuxgurugamer]

2 hours ago, Nightside said:

Thanks for this @linuxgurugamer, does it make the stutter longer when it eventually occurs?

Possibly, but I haven't noticed it yet 

[A_name]

Pardon the noobness, what does this do?

[Daniel Prates]

On 7/19/2019 at 8:48 PM, linuxgurugamer said:

 

@linuxgurugamer just to be sure, if I am using memgraph merely for its anti-stutter effects, can I use this and uninstall memgraph?

[DStaal]

18 hours ago, HuubinNZ said:

This looks like a useful Add-on to have.

You have mentioned "Mono" and "task manager". The "task manager" means you are working on a Windows system and not a MacOS or Linux system. I was not aware that the Windows version of KSP ran on Mono, which is used on the Mac as a Windows runtime for CKAN. 

Does KSP then run under Mono on a Mac as well and would this work on MacOS and Linux systems?

Mono is a language framework written in C, based on .Net.  Unity uses parts of Mono in it's scripting framework, and KSP was written in Unity.

These are all timesaving layers for programmers, and depending on how they're used they may need translation/libraries to work in different places.

In this case - CKAN uses Mono as a primary library, and therefore needs it's translation libraries to work on systems other than Windows.  (Where it can use the .Net libraries I assume, which are built into the system.)

KSP works in Unity, which uses Mono as a scripting library, so it has the needed translation libraries as part of Unity.

That help explain things?

[linuxgurugamer]

20 minutes ago, Daniel Prates said:

@linuxgurugamer just to be sure, if I am using memgraph merely for its anti-stutter effects, can I use this and uninstall memgraph?

Yes, exactly

Edited July 22, 2019 by linuxgurugamer

[Dunbaratu]

This makes good sense.  I suspect that the majority of people using memgraph are just going:

"Okay, let's set this up: alt-*, alt-end, alt-end, alt-end, alt-end, alt-end, alt-end, okay I hope that is enough padding.  Now to close the window and never open it again."

 

[Tonka Crash]

3 hours ago, Steven Mading said:

This makes good sense.  I suspect that the majority of people using memgraph are just going:

"Okay, let's set this up: alt-*, alt-end, alt-end, alt-end, alt-end, alt-end, alt-end, okay I hope that is enough padding.  Now to close the window and never open it again."

This is NOT for those people. There is no window to open and no alt-end to increase the padding. This is for the people (like me) that had the padheap.cfg set to always apply padding without ever needing to interact with the MemGraph GUI at all.

Edited July 22, 2019 by Tonka Crash

[Kerbalwerks]

In layman terms - If I increase the RAM on my machine, will the SPH/VAB  stutter go away, lets say I have 8GB now and install 8GB more.

 

Thanks

[Dunbaratu]

17 hours ago, Tonka Crash said:

This is NOT for those people. There is no window to open and no alt-end to increase the padding. This is for the people (like me) that had the padheap.cfg set to always apply padding without ever needing to interact with the MemGraph GUI at all.

Way to miss the point.

What I was describing is somebody who only opened the Memgraph window long enough to use it to pad the heap space and for no other reason, and then never looked at it again.  The whole point of this mod is to effectively do that without the window at all and without the overhead of tracking that graph data, like you just said.

Edited July 23, 2019 by Steven Mading

[linuxgurugamer]

18 hours ago, Tonka Crash said:

This is NOT for those people. There is no window to open and no alt-end to increase the padding. This is for the people (like me) that had the padheap.cfg set to always apply padding without ever needing to interact with the MemGraph GUI at all.

It is EXACTLY for those people.  With the memory identification I added, it makes it simple to install and forget.

1 hour ago, Steven Mading said:

What I was describing is somebody who only opened the Memgraph window long enough to use it to pad the heap space and for no other reason, and then never looked at it again.  The whole point of this mod is to effectively do that without the window at all and without the overhead of tracking that graph data, like you just said.

Exactly.  The default settings will work nicely for most people

[Dunbaratu]

I think this line in the Readme.md could use a re-phrasing:
 

Quote

 

    b. Do this calculation 16 Gb - ~5/6 GB from KSP - 1/2 GB Windows = 8 GB free

 

This just didn't make much sense to me at all.  I was pretty sure KSP takes a lot more than "five sixths" of a GB and Windows takes more than "one half" of a GB.  It took a while to realize the slash meant "or".  The "fraction" meaning of slash is the first thing you'd naturally think of when it's being used in the context of digits like that.

Edited July 24, 2019 by Steven Mading

[Stone Blue]

1 hour ago, Steven Mading said:

I think this line in the Readme.md could use a re-phrasing:
 

This just didn't make much sense to me at all.  I was pretty sure KSP takes a lot more than "five sixths" of a GB and Windows takes more than "one half" of a GB.  It took a while to realize the slash meant "or".  The "fraction" meaning of slash is the first thing you'd naturally think of when it's being used in the context of digits like that.

EXACTLY... thats wot I thought too... I was like, "Wot??... I think LGG is highly underestimating Windows using 1/2GB of RAM... Thats Win XP/Win7 level... " lol
My Win8.1 uses ~1.6GB at idle..

And by the way, I have 16GB installed, and I changed (hopefully correctly), the amount in the cfg from 4GB to 6GB, which I think serves *me* a little better...
Only tried it out once, but it put my usual RAM usage from 5~6GB with KSP running, to just over 12GB... still leaving around 3~5GB overhead ..

Edited July 24, 2019 by Stone Blue

[linuxgurugamer]

47 minutes ago, Stone Blue said:

EXACTLY... thats wot I thought too... I was like, "Wot??... I think LGG is highly underestimating Windows using 1/2GB of RAM... Thats Win XP/Win7 level... " lol
My Win8.1 uses ~1.6GB at idle..

That is a direct copy from what Memgraph has, I'll look into rewording it

[sjsharks39]

I threw this at my 1.6.1 install and got no errors. Do you suppose it should be working? I don't want to fall victim to placebo but it seems to have a positive effect

[linuxgurugamer]

1 hour ago, sjsharks39 said:

I threw this at my 1.6.1 install and got no errors. Do you suppose it should be working? I don't want to fall victim to placebo but it seems to have a positive effect

It’s a totally silent mod.  You can look at the task manager to see the memory usage before and after installing it to see the difference.

[Jognt]

I wouldn’t go much higher than 2gb padding tbh.

For one, a modded KSP can easily use 10GB, and windows can use several GBs for some users that run ‘stuff’ in the background. 

Ignoring that bit, the reduction in GC intervals is pretty good even for 0.5-1GB. If not, then the user probably has mods spamming garbage that is better off being reported than hidden. ( Kopernicus currently creates a LOT of garbage when having a medium amount of parts in the scene )

Going for 4-8GB on 16GB systems IMO is asking for trouble since most end users will take it for granted and run into lots of pagefile thrashing. 

TL:DR: I suggest not going higher than 2GB with the note that end-users can increase the amount based on how much free memory they have when playing KSP, with the recommendation to keep ~10% of total RAM free  

Edited July 27, 2019 by Jognt

[linuxgurugamer]

5 hours ago, Jognt said:

I wouldn’t go much higher than 2gb padding tbh.

For one, a modded KSP can easily use 10GB, and windows can use several GBs for some users that run ‘stuff’ in the background. 

Ignoring that bit, the reduction in GC intervals is pretty good even for 0.5-1GB. If not, then the user probably has mods spamming garbage that is better off being reported than hidden. ( Kopernicus currently creates a LOT of garbage when having a medium amount of parts in the scene )

Going for 4-8GB on 16GB systems IMO is asking for trouble since most end users will take it for granted and run into lots of pagefile thrashing. 

TL:DR: I suggest not going higher than 2GB with the note that end-users can increase the amount based on how much free memory they have when playing KSP, with the recommendation to keep ~10% of total RAM free  

If no changes are made, the defaults are:

• For a 4gig system:  512meg
• For an 8 gig system: 1 gig
• For a 16 gig system: 4 gig
• For a 32 gig and larger: 8 gig

 

[Jognt]

9 hours ago, linuxgurugamer said:

If no changes are made, the defaults are:

• For a 4gig system:  512meg
• For an 8 gig system: 1 gig
• For a 16 gig system: 4 gig
• For a 32 gig and larger: 8 gig

 

I’d set the 16GB amount to 2GB, but maybe I’m overestimating how many big mods people use.

(memgraph is set to 512MB locally, with 16GB ram, and running Dx11, and my ram still hits 15,5GB after long VAB sessions. I manually removed IVAs for some big mods to free RAM, but I would be surprised if anyone else with 16GB didn’t experience this too.)

[linuxgurugamer]

I may reduce it in the future, but for now, will see what happens.