[1.2.x/1.3] MemGraph 1.1.0.3 - with Stutter Reduction

[joebopie]

Anybody got any new news on weather squad is working on this problem.?

Its sad because KSP would be basically a perfect game if it didn't have this problem. but now its just a stutter fest of frustration.......

Edited March 26, 2017 by joebopie

[HebaruSan]

17 minutes ago, joebopie said:

Anybody got any new news on weather squad is working on this problem.?

Here's the closest to an official statement that you're likely to see:

On 3/6/2017 at 2:36 PM, sal_vager said:

The devs who left last year worked extremely hard to reduce garbage as much as they were able to, and this has had a huge positive effect on KSP, but it can't be eliminated.

At some point the work required is too great and the return too small.

Also, mods add to the memory garbage, not parts but plugins, anything that uses strings in C# will allocate heap memory, so that's anything with text.

There's newer ways to handle text, stringbuilder for example, though performance will vary, and there's awkward workarounds using char arrays, which is already done in KSP.

Sorry but there's really no way to 'fix' this as it's not really 'broken', this is how the garbage collector in Unity works, but when Unity updates Mono they will make the SGen GC an option, and that is much newer and smarter than the garbage collector Unity currently provides.

On 3/6/2017 at 2:50 PM, sal_vager said:

I don't think you quite understand, it would take a complete rewrite of KSP at this point and that could take years, or for example a KSP 2, even then there is no guarantee because a 'stop the world' garbage collector is how Unity works, even SGen won't eliminate this, it's just smarter about what it checks and deletes so pauses are potentially shorter.

Unity themselves offer advice for memory management, but I'm afraid in this case the only fix might be a simpler game, and none of us want to lose the complexity that KSP does have, many want to add to it.

On 3/7/2017 at 1:30 AM, sal_vager said:

Yes, it's old, Microsoft have actually bought Xamarin and have released Mono under the MIT license, so Unity are free to upgrade to the latest version and I hear this is happening.

You can click the arrow icons at the top of any of those boxes to jump to the original thread for more details.

Edited March 26, 2017 by HebaruSan

[joebopie]

Thanks for that info, its good to know that it might get some improvement in some way.

The only thing that really bothers me, is why did it suddenly become a thing that we noticed only in the U5 update.

Before that all that really happened when I added more mods was a lower frame rate. but I do not remember stutters of any kind.

we seem to have exchanged a smooth lower FPS with a higher FPS with stutters.

 

Edited March 26, 2017 by joebopie

[HebaruSan]

3 minutes ago, joebopie said:

The only thing that really bothers me, is why did it suddenly become a thing that we noticed only in the U5 update.

Before that all that I really happened when I added more mods was a lower frame rate. but I do not remember stutters of any kind.

I think it still happened but was harder to notice when frame rates in general were lower. We would not push the game to its limits as much, and would experience big FPS drops before the stuttering became obvious. But there were complaints about stuttering on Unity 4, including reports that it had gone on since 2013:

 

[John FX]

On 06/05/2016 at 10:52 AM, Padishar said:

I would appreciate feedback about how well this works, e.g. screenshots of the graph taken during this process and after another few collections would be very helpful

For me it increases the garbage collection rate and stutter as a result. Here is a screenshot where I press alt-end about 1/5 from the left side, then again after the same time. As you can see the green bars get lower and garbage collection happens more often.

Win-7 24GB I7 - lots of mods.

EDIT :

On 15/01/2017 at 0:26 PM, Padishar said:

The way to increase it is to edit the padhead.cfg file in the MemGraph/PluginData/MemGraph folder and change the value on the total= line.  The value is in MB so you may be able to increase it from the default of 1024 to something like 1200 or even 1500 without causing immediate crashes.

I tried changing the value to 8192 and it works great now. I also noticed GC happens every time you screenshot.

 

EDIT 2: A suggestion. Maybe have a couple of hotkeys for doubling and halving the padhead and another hotkey to store a value?

1024MB is great for some but for those of us with more memory it is counterproductive and increases GC. If hotkeys were in place it would make getting the right padhead easier, also if it was possible to store settings from the previous time then it would not need doing every time the game is launched.

Edited April 6, 2017 by John FX

[Padishar]

On 05/04/2017 at 9:29 PM, John FX said:

1024MB is great for some but for those of us with more memory it is counterproductive and increases GC.

There is no reason I can think of for any amount of padding to increase the GC in the way you describe.  The heap min, cur and max values in your first screenshot aren't consistent with those in the second screenshot (the second one makes sense given what you describe but the first ones are all 4050-4099, which doesn't make sense if 1024 MB of padding was added and then re-added).  Any chance you could run the first test again but take a screenshot before applying the padding and a little while after?

Incidentally, for anyone interested in the KSP 1.2.9 pre-release, MemGraph 1.1.0.3 appears to work correctly without being recompiled.  I will probably release a new version once 1.2.9 is officially released but it can be used in the meantime...

[John FX]

2 hours ago, Padishar said:

Any chance you could run the first test again but take a screenshot before applying the padding and a little while after?

Sure, no problem.

I will revert the mod to vanilla by changing the current value back to 1024 then load KSP, go to the same craft and get a full graph of data then take a screenshot, then pad, wait for another dataset, take a screenshot, then I will try to repeat the same situation after that, repeated alt-ends and another screenshot when a fullgraph of data is there.

Can post a log as well if you like.

EDIT :

It would seem all was OK this time, don't know what caused things to do what they did last time. As you can see, numbers stay sane and GC is far less often.

 

Edited April 17, 2017 by John FX

[Blackline]

Did anyone already figure out, why hovering over parts in the VAB editor causes soooo much Garbage? Its about 5Mb per part, and if you scroll through a long list, while having your mouse pass a number of part icons, it comes to 20-40Mb/s !!!

[jd284]

I like this mod for its side effect of increasing heap and reducing GC. However it's slightly annoying that I manually have to close the graph every time I start KSP.

So could I ask for a cfg option to start with the window closed? Or to remember the last state of the window?

[steve_v]

9 minutes ago, jd284 said:

So could I ask for a cfg option to start with the window closed?

It's in settings.cfg, try visible = false

Edited April 22, 2017 by steve_v

[jd284]

2 hours ago, steve_v said:

It's in settings.cfg, try visible = false

How embarassing. Thanks.

[Gordon Dry]

Hmm, the keypad keys seem not no work, neither with Numlock on or off...

Keypad-Minus does not change the vertical scale, neither while loading nor in main menu.

Is there a difference in keypad codes in other localizations than english?

[Padishar]

7 hours ago, Gordon Dry said:

Hmm, the keypad keys seem not no work, neither with Numlock on or off...

Keypad-Minus does not change the vertical scale, neither while loading nor in main menu.

Is there a difference in keypad codes in other localizations than english?

Are you holding down your configured "modifier" key?

[Gordon Dry]

19 hours ago, Padishar said:

Are you holding down your configured "modifier" key?

Hmm, nooo, sorry.
I guess I forgot that since last summer - I had a 9 month break from KSP to calm down ...

I guess KSP is the only game I know where people talk about "modifier keys" and have the guts to write it down as "Mod-" ...

Is there too much Apple in it?

In my world it's called "ALT"

[Padishar]

11 hours ago, Gordon Dry said:

In my world it's called "ALT"

Yes, but in the linux and Mac worlds it is different keys...

Edited May 7, 2017 by Padishar

[Cetera]

Is there any way we can investigate which mods might be responsible for the most garbage generation and collection, in an effort to cut down on some of the root cause of the issue from a gameplay perspective?

 

 

[HebaruSan]

1 hour ago, Cetera said:

Is there any way we can investigate which mods might be responsible for the most garbage generation and collection, in an effort to cut down on some of the root cause of the issue from a gameplay perspective?

Not without substantial effort. I've tried to imagine such a process:

1. Create two standalone installs with Memgraph, one including the mod to be tested, the other stock
2. Run a standard suite of test cases; they should be clear and simple enough that different people can easily do them the same way every time (e.g., build a specific 10-part ship in the VAB, switch to a specific 100-part ship in orbit, load a save with 100 flights, fly supersonic in atmosphere, etc.), possibly via a standard downloadable save file
3. For each test case in each install, capture stats from Memgraph; bytes per second of garbage generated?
4. Collect all of it in a big spreadsheet with one mod per row and one test case per column so you can sort to find the worst offenders

The trickiest part would be accounting for variations in hardware of different testers, but we could at least hope that truly severe problems would stand out on any system, and using measurements of stock as a control should help. Then it's just a question of defining the test cases and setting up a repository for the data. Interactions between mods may matter, but we really need to tackle the individual case first.

Some modders might take issue with being "called out" in this way; ideally any such project should take the approach of providing information for potential users, not trying to bully someone with bad stats. Finding and fixing a few bugs would be nice, but right now we can't even answer simple questions like, "Which mods should I uninstall to try to reduce my stuttering?"

Edited May 20, 2017 by HebaruSan

[Cetera]

5 hours ago, HebaruSan said:

Finding and fixing a few bugs would be nice, but right now we can't even answer simple questions like, "Which mods should I uninstall to try to reduce my stuttering?"

Yeah, this is where I'm at.  I used to run pretty well.  Memgraph helped, a lot.  I ran CKAN the other day, and let it update several mods.  I didn't make a note of which were being updated.  One of the updates killed my install, and I'm not sure which.  I can't find a CKAN log of which were installed when, either, to just remove the offending update and roll back.

I went from using the memgraph option to increase heap after playing for a while (at least 30-60 minutes) with garbage collection every 30-45 seconds, to now I can't load a vessel onto the launchpad, no matter how complex, without garbage collection running every 4-9 seconds.  Memgraph still helps push out the garbage collection to around 60-80 seconds, which is OK, but not as good as it used to be.  However, it really doesn't last past a scene change anymore either.  It used to last through a couple of launches.  Something made a change, for a serious negative outcome on my install, and I can't find out which one.

I was hoping there might be some kind of output log for the garbage collector that you might be able to get it to track which items it saw as no longer active, and dumped, and see if we can take that output and start mapping it backwards to origins.

I guess the moral of the story is, "Kids, make backup copies of your installs and GameData directories before you make changes!"

*sigh*  IT 101, frequent backups, changes one at a time, document your changes.  Do we do it?  Do we remember?  No, it's just a game, what could possibly go wrong???

It doesn't help that excluding the Squad folder, my GameData directory is roughly 2.5 GB, and my module manager patches load is pushing 20k.  Maybe I should pare things down?  But where to start, lol.

[Teasecake]

Thanks Padishar, your stutter reduction worked well for me. I'm running 1.3 with about 10 mods and the stutter was driving me nuts just in the vehicle assembly building.

[Sokar408]

This actually made my stuttering worse. The stutters happens more and more often, the more I hit Alt + End. Is where anyway to undo this?

[Padishar]

17 hours ago, Teasecake said:

Thanks Padishar, your stutter reduction worked well for me. I'm running 1.3 with about 10 mods and the stutter was driving me nuts just in the vehicle assembly building.

You're welcome.

7 hours ago, Sokar408 said:

This actually made my stuttering worse. The stutters happens more and more often, the more I hit Alt + End. Is where anyway to undo this?

Have you followed the advice/instructions in the first post (mostly about needing a decent amount of free memory)?  Nothing is remembered across a game restart so just restart KSP and don't hit Alt+End.

If you want any, more helpful, advice then you will need to provide some details of your machine and KSP install...

[Sokar408]

5 minutes ago, Padishar said:

You're welcome.

Have you followed the advice/instructions in the first post (mostly about needing a decent amount of free memory)?  Nothing is remembered across a game restart so just restart KSP and don't hit Alt+End.

If you want any, more helpful, advice then you will need to provide some details of your machine and KSP install...

I late the game run for a while, then hit Alt + End. Did I miss something obvious?

[orcaman]

On 6/7/2017 at 8:21 PM, Teasecake said:

Thanks Padishar, your stutter reduction worked well for me. I'm running 1.3 with about 10 mods and the stutter was driving me nuts just in the vehicle assembly building.

Does this mean it works in 1.3, or that it doesn't? Also, yours is my new favourite username.

[Kobymaru]

On 9.6.2017 at 7:24 AM, orcaman said:

Does this mean it works in 1.3, or that it doesn't?

Gotta say, works pretty darn well for me:

 

[HafCoJoe]

Yo! I've got a quick question... is this reading good or bad?