sarbian

[1.1.2] GrumpyCollector 1.0.1 - Stutter ? What Stutter

Recommended Posts

2 minutes ago, Padishar said:

Yes, that was the entire purpose of my MemGraph mod.  I've already passed on information about a number of stock areas of code that create excessive garbage and am continuing to look for them as reducing the amount created is still, by far, the best way to reduce the annoyance of the stutters...

While targeting code that creates 1.5 KB/frame will have an effect (and it will be considerable if lots of such code is fixed), I'm, initially, more concerned about situations where stock code allocates much more.  I have a number of saves where the garbage creation adds up to 20+ MB/s resulting in the GC running every 3 to 4 seconds.  This padding of the heap extends that to approx. 20 seconds between collections which, even with the pauses being noticeable longer, is considerably nicer to play (in my opinion).

Well, I meant fixing them ourselves, not just passing the info on. I haven't had any numbers nearly as high as yours, but for example, the Learstar A1 sitting on the pad with engines running (no other vessels) generates about 7kB of garbage per update and ~15kB per fixed update on my testbed install so wasting 1.5kB in a LateUpdate is significant

Share this post


Link to post
Share on other sites

WOW thanks so much I shall try this and report back.

Also just to say I seem to get the most stutter when in the aircraft hanger.
I do get it when flying to but its most frequent in hanger etc.

Has this been noticed before?

I assume it could be to do with all the parts are stored in memory and seeing as I have a lot of mods its probably not helping much :P

Share this post


Link to post
Share on other sites
19 minutes ago, xEvilReeperx said:

Well, I meant fixing them ourselves, not just passing the info on. I haven't had any numbers nearly as high as yours, but for example, the Learstar A1 sitting on the pad with engines running (no other vessels) generates about 7kB of garbage per update and ~15kB per fixed update on my testbed install so wasting 1.5kB in a LateUpdate is significant

So, you mean by patching or overriding various bits of stock code?  Yes, it should be possible to fix a lot of stuff but the extra hassle to actually override some stock code from a mod rather than just sending the code changes to Squad for possible inclusion in the next update, don't really seem worth it to me.  However, I would support and probably contribute to such a mod, especially if it were a single mod containing multiple such fixes.

Edit: oh, I meant to add that even though the garbage creation you describe adds up to 1170 KB/s for a machine doing 60fps, if you had another few (large) vessels in that save (e.g. a couple of thousand parts in total) then that would be a minimal part of the total usage (see the platonic stations examples in the MemGraph thread).

9 minutes ago, stk2008 said:

WOW thanks so much I shall try this and report back.

Also just to say I seem to get the most stutter when in the aircraft hanger.
I do get it when flying to but its most frequent in hanger etc.

Has this been noticed before?

I assume it could be to do with all the parts are stored in memory and seeing as I have a lot of mods its probably not helping much :P

Yes, the editors generate a lot of garbage displaying (or even not displaying because it still happens when switched to crew or action groups) the part categories.  The more parts in the category, the more garbage is created.  There is a post somewhere in this thread or the MemGraph one that mentions this. This is (I presume) the "main editor GC problem" that xEvilReeperx mentioned... 

Edited by Padishar

Share this post


Link to post
Share on other sites

:) well thanks to all who is helping with this issue means a lot to me as I suffer with it a lot even with 16GB of ram :(

Share this post


Link to post
Share on other sites

It would be possible to dynamically load and unload just the displayed textures in the editor? Maybe the loading will be slower but with better long term results.

Share this post


Link to post
Share on other sites

Textures have nothing to do with the stutter 

Share this post


Link to post
Share on other sites
11 minutes ago, Nansuchao said:

It would be possible to dynamically load and unload just the displayed textures in the editor? Maybe the loading will be slower but with better long term results.

4 minutes ago, sarbian said:

Textures have nothing to do with the stutter 

Not to mention that the code to dynamically unload and load them would probably generate more garbage itself unless it were very carefully written...

 

Share this post


Link to post
Share on other sites
2 hours ago, Padishar said:

I suspect, though I haven't got any real evidence to back it up, that they have tried to use some of the more advanced features of the Boehm collector to make it less conservative, e.g. by using the Mono type information to determine that a block can't contain any references to other blocks and therefore doesn't need to be scanned, or even, to make the scanning itself use the type information to only check those memory locations within a block that actually are reference types.  It wouldn't surprise me if these changes were not thread-safe as, with the option turned off, they don't need to be.

Yes, there is a degree of set limiting which occurs in the Unity GC which has had a positive effect on GC time however collection cycles are still pretty heavy and noticeably so in some cases.

(I happen to work at Unity (opinions my own etc) and recently spent a bit of time in the GC related code looking in to an overhead issue reported by another customer... it's a bit of a confusing rats nest at times.)

11 hours ago, Chris97b said:

Anyone know if the Unity GC is multi-threaded? If not, would it be possible to shuffle the GC off onto its own CPU core by way of prioritization? Or does it just run as a part of an existing thread?

You can't put a GC on it's own thread; at least with not any real degree of sanity.

The GC has to be able to Stop The World for at least the thread in question; trying to run the GC code would involve that thread stopping another and then going over it's memory details. The thread is still stalled (at least; you might still end up stalling the whole system if someone else tries to do something which has any impact on the memory of the thread being GC'd), but now you are burning another thread doing the work.

The "best" way to do it would be to pause all the threads and have each thread/core chew on it's own set of memory before coordinating at a global level for aliveness, which might still only gain you a small amount back depending on how the work is split.

Ultimately this wouldn't help here however; most of Unity's work is done on a single thread - yes, Physics is threads, graphics has some threading options and there is a task based job system which is getting increasing amounts of work, but the scripting/C# side of things lives in a single thread and from there most of the memory will be allocated and thus tracked.

Ultimately the solution is "don't make garbage" - mods might well be Doing Things Wrong, but there is a chance that both them and Squad have just hit code paths which can't do non-allocating ways of doing things which is what is causing the memory to be allocated.

Share this post


Link to post
Share on other sites
1 hour ago, Padishar said:

Yes, the editors generate a lot of garbage displaying (or even not displaying because it still happens when switched to crew or action groups) the part categories.  The more parts in the category, the more garbage is created.

Huh, guess this explains the effect of part-only mods on editor performance I have been seeing. Confirms my hunch as to the cause too, thanks. :)

Share this post


Link to post
Share on other sites
59 minutes ago, Padishar said:

So, you mean by patching or overriding various bits of stock code?  Yes, it should be possible to fix a lot of stuff but the extra hassle to actually override some stock code from a mod rather than just sending the code changes to Squad for possible inclusion in the next update, don't really seem worth it to me.  However, I would support and probably contribute to such a mod, especially if it were a single mod containing multiple such fixes.

Yes, at least the big ticket or easy pickings items. I do get what you're saying though, I just wanted to know if it was something you were looking at. Are you reporting these on the bug tracker? I have a whole mountain of them I could contribute. Some are incredibly easy to fix, such as the one related to GC being proportional to number of on rails parts. If you crystal ball the reason for this one, it's so dumb it makes me wonder if they're profiling at all:

// TetraTetra save

f12c7d5124.png

Share this post


Link to post
Share on other sites
39 minutes ago, bobvodka said:

Yes, there is a degree of set limiting which occurs in the Unity GC which has had a positive effect on GC time however collection cycles are still pretty heavy and noticeably so in some cases.

(I happen to work at Unity (opinions my own etc) and recently spent a bit of time in the GC related code looking in to an overhead issue reported by another customer... it's a bit of a confusing rats nest at times.)

Thanks for confirming that, it probably explains some of the inconsistencies in GC timing I've seen. Nice to see a Unity dev (I assume you're a dev if you risked delving into that code :wink:) in this forum...

44 minutes ago, bobvodka said:

Ultimately the solution is "don't make garbage" - mods might well be Doing Things Wrong, but there is a chance that both them and Squad have just hit code paths which can't do non-allocating ways of doing things which is what is causing the memory to be allocated.

Indeed.  While there are certain things that are quite difficult to do without creating some garbage, there are very few things that couldn't be optimised to generate considerably less than at present.  Most of the really rampant garbage creation (in both stock KSP code and in mods) is simply badly written code that ignores the garbage issue completely (e.g. using complex LINQ queries that result in temporary iterators and allocation of intermediate result arrays, writing functions that create and return a new List even if it's empty, etc).

52 minutes ago, steve_v said:

Huh, guess this explains the effect of part-only mods on editor performance I have been seeing. Confirms my hunch as to the cause too, thanks. :)

Yes, Squad are aware of the issue, exactly what bits of code are at fault and have plans to look at it soon™.

41 minutes ago, xEvilReeperx said:

Are you reporting these on the bug tracker? I have a whole mountain of them I could contribute.

I usually report them to an internal tracker I have access to (due to being on the experimentals team) or just discuss them directly with a Squad dev in IRC to see if they are already known about.

41 minutes ago, xEvilReeperx said:

If you crystal ball the reason for this one, it's so dumb it makes me wonder if they're profiling at all:

No comment...  Actually, I will comment, I think it's obvious that very little profiling gets done given that some of the code is so obviously bad just looking at the source.  There are lots of bits of code that flat-out ignore all the guidelines for reduction of garbage creation and general CPU efficiency.  Hopefully it will continue to improve...

 

Edited by Padishar

Share this post


Link to post
Share on other sites
On 5/23/2016 at 8:35 PM, Padishar said:

Great data again, thanks.  I'm a bit disappointed by only just over three times, [snip]

Your welcome, and I'm not disappointed :)  I think what you said later is right on, totally depends on each person's mod configuration.  I'm in the same boat as you, prior to starting to focus on this problem I was getting 15-20MB/sec being added to HWM and GC running every 4-5sec. ick.

On 5/23/2016 at 6:05 AM, stk2008 said:

WOW thanks so much I shall try this and report back.

Also just to say I seem to get the most stutter when in the aircraft hanger.
I do get it when flying to but its most frequent in hanger etc.

Has this been noticed before?

I assume it could be to do with all the parts are stored in memory and seeing as I have a lot of mods its probably not helping much :P

Yes, and to be clear with my testing I certainly notice this as well.  I've only chosen to run my tests in high orbit because I get the most consistency in my frames out there.  IMO, its better for testing, my fps is more all over the place in the VAB/SPH which I think is fairly common for most players.  Also, again personal opinion, lag in the VAB/SPH although annoying won't result in dead kerbals... :)

On 5/23/2016 at 6:56 AM, Nansuchao said:

It would be possible to dynamically load and unload just the displayed textures in the editor? Maybe the loading will be slower but with better long term results.

@sarbian already gave the correct and short answer.  But in case you were wondering, the GC is only run on the heap.  That's not how textures are stored/loaded.  I think in terms of GC an objects location in the memory we are a lot more concerned with the heap vs stack, textures (not stored in either) just don't matter for GC performance.

On 5/23/2016 at 7:45 AM, Padishar said:

No comment...  Actually, I will comment, I think it's obvious that very little profiling gets done given that some of the code is so obviously bad just looking at the source.  There are lots of bits of code that flat-out ignore all the guidelines for reduction of garbage creation and general CPU efficiency.  Hopefully it will continue to improve...

  emphasis mine.  lol.

On 5/23/2016 at 8:35 PM, Padishar said:

Another thing that could reduce your stutter would be to set a lower fps limit.  Currently, the 16-20 MB/s is being generated by 120 rendered frames and 50 physics updates.  It is hard to say how much is being done per rendered frame and how much is per physics update but changing to a 60 fps limit could significantly reduce the rate of garbage creation...

Challenge accepted. :) Results, well maybe very important for your design going forward with MemGraph.  Maybe important for any dev (hopefully?  :) )reading this thread.

Important changes from my prior testing: A few mods updated, including TWP which had kept popping up at strange times, TriggerAu fixed that though.  I had a another craft still on-rails in the background in my prior testing, due to statements made in this thread I deleted it.  @Padishar I think that might be what caused the strange decrease in stutter frametime between B) and C) in my prior post - I think that craft hit the atmosphere in between and was deleted by the game.  Not sure though.

The executive summary:

My prior setting fps limited to 120 fps.                     Stutters every ~15.8s, 325ms stutter.  Mod-End x3.  Stutters every ~37.5s, 430ms stutter.  7.3GB GCMonitor memory usage

New setting fps limited to 60 fps. (only change)      Stutters every ~20.7s, 265ms stutter.  Mod-End x3.  Stutters every ~59.5s, 370ms stutter.   6.9GB GCMonitor memory usage  

HWM memory usage didn't appear significantly different... I can recheck that if needed.

 

Obligatory picture from the 60fps limit run:

lp1mJ0b.jpg

In any event, that's a pretty huge increase.  I think its safe to say the rendered frame update is a relatively large portion of my heap MB/s increase....

 

--------------*****************-----------------

EDIT :

To anyone interested in my further testing, I'm moving the discussion to the MemGraph thread, since most of the discussion on this thread is now about that mod.  Thanks to @sarbian again for starting this mod, and sorry for the slight thread jacking, but hopefully everyone will appreciate the common goal of no-more-stutter. 

 

 

Edited by Tig
Moving my testing

Share this post


Link to post
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.