[1.2.x/1.3] MemGraph 1.1.0.3 - with Stutter Reduction

[Padishar]

11 hours ago, Auriga_Nexus said:

This isn't necessarily an ideal fix - the IDEAL fix would be unity and/or Squad fixing their crap so that a workaround like this isn't necessary - but we all know that's never going to happen.

Errr, have you been following the devnotes for the last few weeks?  Squad are "fixing their crap".

11 hours ago, Auriga_Nexus said:

I do have a few questions, though. Mainly, I'm still a little lost on what Memgraph actually keeps track of, besides the obvious. Like, if I increase the scale of the graph to 64 MB, does that mean that the entire graph is from 1 to 64 MB, or is there a grid system in place and each tick is 64 mb? Also, what do the numbers up top mean. I'm assuming the "interval" metric is the period of time between the last GC operation and the one before that, but I'm having trouble making sense of the other numbers on the window.

Have you read the first post in the thread?  The scale shows the maximum value on the Y axis of the graph.  It adds a new point to the graph each second (approximately).  The value plotted is the "Last" value from the top of the window, which is the amount of memory allocated in the last second.  The cur value is the current size of the Mono heap.  The max is the heap size just before the last GC, the min is the heap size after the last GC.  The R and P values are the number of "rendered" and "physics" frames during the last second.

If you read various other posts in the thread you will see that the Contract Configurator mod can cause a significant amount of garbage.  I would try copying your entire KSP installation, removing that mod and all the contract packs and then trying your save again.

[charliepryor]

I see your listed key configuration for displaying the windows and stuff.... mind telling me how a person using a tenkeyless keyboard, which doesn't have that numberpad, is supposed to use this? I don't want it displaying all the time. Either tell me how to modify the key bindings, or make the default something that everyone has.

Edited August 23, 2016 by charliepryor

[WeirdCulture]

usualy have small keyboards on laptops some function key to emulate the numeric  block. search for it. or there are other possibilities like this: 

 

[charliepryor]

10 hours ago, WeirdCulture said:

usualy have small keyboards on laptops some function key to emulate the numeric  block. search for it. or there are other possibilities like this: 

 

Yeah a standard tenkeyless PC keyboard doesn't have a Fn key I don't have any idea how using a Fn key is going to help with this mod requires keys specifically on a keypad, nor am I going to buy some bulky accessory to use a single mod in KSP. Honestly, speaking with zero coding experience here however, the ability to rebind a key isn't difficult to do, but an even easier answer would have been to bind it to a key that everyone has in the first place... like F keys, or Mod+number (instead of keypad number). It's just silly... at the VERY minimum, an "X" on the window to close the damn thing. I cannot use this mod while it shows 100% of the time, especially as a YouTuber. No thanks until its usable without a numberpad.

Edited August 23, 2016 by charliepryor
Strike a comment, because I actually do have one on my keyboard.

[TinoR]

Thank you Padishar for this stutter reduction mod.

I first tried with the default values in padheap.cfg, after padding was executed stutter intrerval changed from 4 seconds to 15 seconds.

Then I multiplied by 4 the values in padheap.cfg and the interval increased to 46 seconds. The overall RAM usage is at 72% of 8 GB installed, KSP takes now ~3.4 GB. By increasing the values more the stutter gets reduced to 60 seconds interval, but brings other performance issues, because of page faults I suppose. If I add more RAM to my computer I will be able to reduce the stutter even more, right?

Here are the graphs:

And thank you again, now the stutter is much less annoying.

Edited August 23, 2016 by TinoR

[Padishar]

19 hours ago, charliepryor said:

Either tell me how to modify the key bindings, or make the default something that everyone has.

As I mentioned in this post:

...on the previous page, at the moment you would need to modify the source and recompile the mod.  I haven't got around to making the keys configurable yet, I've just been too busy with other things for several weeks...

 

1 hour ago, TinoR said:

Thank you Padishar for this stutter reduction mod.

You're welcome.

1 hour ago, TinoR said:

If I add more RAM to my computer I will be able to reduce the stutter even more, right?

Yes, there are a number of posts in this thread that mention running with more memory.  You would just need to adjust the padheap.cfg to suit the amount of memory you have though I recommend leaving a fair amount of headroom or Windows (especially Win10) may start doing strange things like memory compression.

Anyone out there using this on Linux or OS/X?

Edited August 23, 2016 by Padishar

[charliepryor]

Temporary Workaround for those that use tenkeyless keyboards: Software keyboard in Windows. The On-Screen keyboard can be access in the Ease of Use settings in Windows 10, and there is a toggle there to turn on the keypad within this as well.

[steve_v]

On 24/08/2016 at 11:42 AM, Padishar said:

Anyone out there using this on Linux or OS/X?

Was planning on it (GNU/Linux), but:

On 6/05/2016 at 9:52 PM, Padishar said:

Mod-KeypadMultiply toggles the display of the window.

Doesn't. All the other hotkeys appear to work fine but I can't toggle the window.

As for actual functionality... testing now. Yep, does what it says on the tin. Anything specific you wanted feedback on?

Edited August 26, 2016 by steve_v

[Padishar]

31 minutes ago, steve_v said:

Was planning on it (GNU/Linux), but:

Doesn't. All the other hotkeys appear to work fine but I can't toggle the window.

As for actual functionality... testing now. Yep, does what it says on the tin. Anything specific you wanted feedback on?

Hmmm, weird, I assume you're using the correct "Mod" key or the various other things wouldn't work but it's strange if it's only NumpadMultiply that doesn't work.  Do the other numpad keys works, e.g. Mod-NumpadPlus/NumpadMinus to adjust the scale?  Perhaps something in your window manager is trapping the key combination.  Can you try setting some default KSP function to use that key and see if that works?  Looks like another reason for me to get around to making the keys configurable.

There's nothing specific I wanted to know about, just a general "does it work"?  There isn't anything that should be platform specific but weird issues do sometimes crop up when you make assumptions like that...

[steve_v]

24 minutes ago, Padishar said:

Can you try setting some default KSP function to use that key and see if that works?

Huh, now that I actualy test it... methinks my numpad mult key is bung. Nevermind.

24 minutes ago, Padishar said:

Looks like another reason for me to get around to making the keys configurable.

Uh, yeah. That would be neat. Meantime I'll just recompile.

Edited August 26, 2016 by steve_v

[Fat Gordy]

Sorry to clutter up the forum with this post, but I'm just so happy about this mod!!  I must have searched for about 8 hours trying to figure out what was wrong with my computer and why it was stuttering every 4 seconds.  I used GPU-Z and could see when the frame rate and GPU dropped out but couldn't figure out why!!  I have a pretty powerful PC and graphics card and was getting to the point where the stuttering was physically bothering me and made it almost uncomfortable to play.  

I use a lot of mods, including visual enhancements and 4k resolution.  I have 16Gb of RAM and just added a zero to all of the values in the padheap file.  That took me from around 40% mem usage to close to 85%.  But the stutter is reduced from 4 seconds to 41 seconds in the worst cases!!  And up to 1.5 minutes in the best!  

Thank you so much for making this!   

Edited August 28, 2016 by Fat Gordy
added detail

[dr_evil]

I first saw this mod in one of Roverdude´s streams, didn´t exactly pay attention what it was capable of when it comes to debugging and performance improvements.

I am running KSP with a bunch of mods and I am pretty sure that at least one of them is producing some memory overhead. Even though I have a decent rig (Core I7 4790k, 16GB of memory), I had a micro stutter occuring every couple of seconds which I didn´t have before in 1.0.5. So I got here and grabbed your mod. I am pretty happy with the results. First testing revealed that I went from a stutter every 2-4 seconds up to one in every 30-40 seconds. I multiplied the values in the padheap config file with nine, as adding a zero would cause my machine to run out of memory. I have the paging file set to zero MB manually and always have the browser open on my second monitor, which both limit available memory.

With that said, now I can enjoy KSP again, thanks for that!   When I get bored of playing I will use your mod to find out, which mods require large ammounts of memory.

[Padishar]

16 hours ago, dr_evil said:

I first saw this mod in one of Roverdude´s streams, didn´t exactly pay attention what it was capable of when it comes to debugging and performance improvements.

Do you happen to remember which one of Roverdude's streams it was?

[dr_evil]

1 hour ago, Padishar said:

Do you happen to remember which one of Roverdude's streams it was?

It was a couple of months ago. He was debugging one of his mods and a specific part, but don´t ask me which one. I just remember that he was stacking a few pieces of that specific part and hyperediting them to orbit to track down a memory leak / high memory usage and potential lag issue. I remember it because of the specific graph pattern that your mod displays. 

Is there a way to have your mod adjust the padding heap size allocation and optimization on KSP startup without having to press Mod+End each time? I see the point for devs to have the GUI up all the time while testing but as a player I have to wait for a couple of cycles then press Mod+End and Mod+* each time I fire up KSP to enable the lag reducing effects.

[linuxgurugamer]

@Padishar Might I make a small suggestion?

Add a .version file and the MiniAVC.dll so that it is easier to know when the mod gets updated (see http://ksp-avc.cybutek.net/ for details)

CKAN would also be nice

 

 

[jamie]

Hi Padishar, and thanks for your work on this.

I not much of an expert on memory utilisation and only understand what garbage collection is and what MemGraph does in a very basic manner, so this might be a dumb question.

I've noticed that using the alt+end function to pad the heap also increases the minimum heap size. This can be quite sizeable; for example in my current modded build, the Heap Min value jumps from 1392MB to 3280MB following the first iteration of heap padding (the first alt+end activation). Following iterations increase the Heap Min value only fractionally. Although heap padding increases the Heap Max value by a larger amount and I therefore get a longer GC interval, a sizeable chunk of the extra allocated memory is eaten up instantly due to the increase in the Heap Min value, and the gains are therefore significantly diminished.

My question is: is the Heap Min value meant to increase during the padding process, and is there any way to minimise this increase?

FYI, I found that Contract Configurator was creating a disproportionately large amount of garbage in my modded build. I think it was creating around half as much garbage as all of my other mods (around 60) combined, and therefore removed it from my build. This mod has been mentioned elsewhere in this thread, but I just thought I'd mention it again as it had such a significant impact on the stutter problem for me.

Thanks!

Jamie.

 

[Padishar]

13 hours ago, jamie said:

I not much of an expert on memory utilisation and only understand what garbage collection is and what MemGraph does in a very basic manner, so this might be a dumb question.

I've noticed that using the alt+end function to pad the heap also increases the minimum heap size. This can be quite sizeable; for example in my current modded build, the Heap Min value jumps from 1392MB to 3280MB following the first iteration of heap padding (the first alt+end activation). Following iterations increase the Heap Min value only fractionally. Although heap padding increases the Heap Max value by a larger amount and I therefore get a longer GC interval, a sizeable chunk of the extra allocated memory is eaten up instantly due to the increase in the Heap Min value, and the gains are therefore significantly diminished.

My question is: is the Heap Min value meant to increase during the padding process, and is there any way to minimise this increase?

That's ok, there are lots of quite experienced programmers that don't fully understand this stuff either.  I explained some of it in this post:

The heap padding basically creates a large number of 4k pages that only have a single block in, e.g. a 113-144 byte page will have 1 of the 25 available slots used but the other 24 slots free.  The same is true of the other size ranges.  So, some of the memory that you think is allocated is really allocated (it is necessary to avoid the GC discarding it again and losing the benefit) and the rest is the empty padding that is what actually increases the GC interval because lots of allocations don't cause the heap to run out of empty space.

[Eiktyrner]

Looking at some of the screenshots here I get a feeling my game runs with an unusual amount of GC?

Graph http://imgur.com/a/Xoz0c

16gb of ram, currently using about 8gb. How would I modify the padheap to utilize like 5gb more?

modlist: https://paste.ee/p/J8nah

Edited August 31, 2016 by Eiktyrner

[dr_evil]

4 minutes ago, Eiktyrner said:

16gb of ram, currently using about 8gb. How would I modify the padheap to utilize like 5gb more?

Since the values in the padheap config file provide approximately 900MB of padheap you divide five GB by 0.9 and then multiply the outcome with the values in the padheap config. In this example it would be 5.5 times those values.

8 minutes ago, Eiktyrner said:

Looking at some of the screenshots here I get a feeling my game runs with an unusual amount of GC?

Hm, I am not an expert but it looks completely similar to my graph

[Padishar]

47 minutes ago, Eiktyrner said:

Looking at some of the screenshots here I get a feeling my game runs with an unusual amount of GC?

Seriously excessive perhaps, but, unfortunately, not unusual.  The most important value is "Last" as this indicates how much memory is being allocated each second.  In your case this is over 110 megabytes every second.  Does the value change significantly if you close the KAC alarms window?  A lot of the garbage is probably due to contract configurator, hopefully the author (or somebody) will be able to make some significant improvements to it soon.

56 minutes ago, Eiktyrner said:

How would I modify the padheap to utilize like 5gb more?

Exactly as @dr_evil said above (thanks).

[Eiktyrner]

Thanks, going to try out multiplying all values with 5.5 now. Will also try toggling KAC, I'm just used to have it open at all times.

Would I benefit from adding some blocks of the larger sizes (664-2032) ?

[Padishar]

1 hour ago, Eiktyrner said:

Would I benefit from adding some blocks of the larger sizes (664-2032) ?

You may benefit a little but the 4k pages for the larger sizes can only hold a small number of blocks so the single block in each page wastes a larger proportion of the memory, e.g. for 1345-2032 each page only holds 2 blocks and one is used to keep the page allocated, so half of the memory is effectively wasted.  It really depends on the pattern of block sizes allocated by your particular game which varies according to installed mods, the scene the game is in, how complex your save is and even things like which particular parts are used on your vessel.  There certainly isn't a config that will suit all situations.  I did do some work on tuning the padding to suit the current usage pattern but it's a tricky problem because the only way to determine information about the current usage pattern is by allocating and releasing memory to see what happens to the heap and this obviously affects the usage pattern itself.  My current plan is to change how the config works so that the block size values set the "shape" of the padding allocated but have the total amount of padding controlled separately.  This will make it very much easier to change it for machines with more memory and changing the block size values to better fit the usage pattern (e.g. if you somehow happen to know that your install allocates a lot more blocks of 145-176 bytes than usual then you would want to increase that value in the config) wouldn't affect the total amount of padding (well, not by much).  However, this is still in the future and I've not got a lot of spare time at the moment so it will probably be at least a couple of weeks before I get around to it.

[UltraJohn]

Hi, I've been having some problems with stuttering since a few days ago. I've been using this mod for several weeks, and it has worked very well reducing my stutter from every 10 seconds down to every 50 seconds. But since the other day my stuttering suddenly got way worse. Now I'm sitting at a stutter every 2 seconds and using this mod will only reduce it to around 10-15 seconds, and I have no idea why it suddenly got so bad. I've tried deleting all my crafts, but it didnt change anything, though creating a new savefile will fix the stuttering (see image). My only guess is that I might have updated a mod or something, but I have no idea which and I can't see which of the mods would be producing a lot of garbage. I could really use some help on this.

[linuxgurugamer]

37 minutes ago, John007qwe said:

Hi, I've been having some problems with stuttering since a few days ago. I've been using this mod for several weeks, and it has worked very well reducing my stutter from every 10 seconds down to every 50 seconds. But since the other day my stuttering suddenly got way worse. Now I'm sitting at a stutter every 2 seconds and using this mod will only reduce it to around 10-15 seconds, and I have no idea why it suddenly got so bad. I've tried deleting all my crafts, but it didnt change anything, though creating a new savefile will fix the stuttering (see image). My only guess is that I might have updated a mod or something, but I have no idea which and I can't see which of the mods would be producing a lot of garbage. I could really use some help on this.

How big is the save file?

[UltraJohn]

1 minute ago, linuxgurugamer said:

How big is the save file?

4MB.