Are memory errors acknowledged yet?

[kithylin]

I've been out of the KSP loop since about december 2016 / jan 2017 and haven't checked in... so sorry if this has been answered already. But I'm curious.. have SQUAD staff actually acknowledged the nearly game-breaking memory leaks that read to rapid garbage collection with running KSP on 64-bit and really large (300+) part-count ships in space? I know there's a mod to extend and "pad" the memory allocation so garbage collection is less frequent, but it's a band-aid to the situation.

So.. has there been any official progress on this? Have SQUAD Acknowledged it's existence and said they'll fix it some day at the very least?

[Capt. Hunt]

um, I'm pretty sure they said, even before you were "out of the loop," that this was a Unity problem, and not necessary something they can fix.

[kithylin]

54 minutes ago, Capt. Hunt said:

um, I'm pretty sure they said, even before you were "out of the loop," that this was a Unity problem, and not necessary something they can fix.

I had a whole thread on this with like 4 pages on this before the recent patch.. no one found anyone from SQUAD actually admitting it's an issue or actually anything they can (or can not) fix. The only thing we could do was a community mod to try and bandage the issue and that's it. I would of thought after a few months there'd be some sort of official statement.

[The Aziz]

But there is a mod bug tracker for that. If the issue was already there, it was probably seen by Squad. Now it comes to the point where they can fix it or not.

[Starman4308]

First: it's not a memory leak unless it isn't garbage collected and just accumulates.

Second, if the memory accumulation coming from Unity classes, there's basically nothing Squad can do about it. Remember that Unity isn't the ideal option for KSP: it was simply the least bad option, and that a "good" engine for KSP doesn't exist, because KSP strays so far outside the norm for video games.

Third, there's a lot of background calculations that are going to generate a lot of temporary variables. While in theory, you could do something to try to keep those memory addresses open for the next physics tick... that's basically manual memory management, something that takes a lot of programmer time not just to make sure it works, but also that it doesn't cause an actual memory leak.

[Rocket In My Pocket]

It's an issue with Unity.

Squad can't fix or change it.

Can we please let this poor horse rest? It's been dead for ages.

[LordFerret]

5 hours ago, Rocket In My Pocket said:

Can we please let this poor horse rest? It's been dead for ages.

Yes, it has been. Unfortunately the stink of its rotting corpse still wrinkles the noses of many, myself included. A patch is just that - a patch... temporary, inconsistent, and not a solution. For the time being, it suffices (marginally anyway).

The last I bothered reading on this, it was a well known and documented Unity/Mono issue. I also recall reading about how developers (including KSP modders) could work their way around this by managing their memory manually and not relying on Unity to do it. That however is a lot of work.

[GarrisonChisholm]

So, my KSP game is still on 1.1 (i think)-  the version where things can't roll straight at 80+kph.  I've been out of the loop for most of 6 months; does this mean there is a potential viable reason to delay or *not* upgrade my game to the current latest version?

I realize it is a poor thing to ask such a question without knowing the exact version I am playing on- I will provide that tomorrow via edit.

[Alshain]

21 minutes ago, GarrisonChisholm said:

So, my KSP game is still on 1.1 (i think)-  the version where things can't roll straight at 80+kph.  I've been out of the loop for most of 6 months; does this mean there is a potential viable reason to delay or *not* upgrade my game to the current latest version?

I realize it is a poor thing to ask such a question without knowing the exact version I am playing on- I will provide that tomorrow via edit.

No.  1.2 received a massive optimization to reduce the occurrence of garbage collection.  You definitely want 1.2.  It's pretty much the definitive version you are going to want for a long-long time.  After it was finished there were relatively few wide-spread bugs left and all the developers then quit.  No telling what 1.3 is going to screw up, I would get it while you can.

Edited May 15, 2017 by Alshain

[GarrisonChisholm]

6 minutes ago, Alshain said:

No.  1.2 received a massive optimization to reduce the occurrence of garbage collection.  You definitely want 1.2.  It's pretty much the definitive version you are going to want for a long-long time.  After it was finished there were relatively few wide-spread bugs left and all the developers then quit.  No telling what 1.3 is going to screw up, I would get it while you can.

Asked and Answered!  Thank you!

[Spraki]

On 13.5.2017 at 0:43 AM, kithylin said:

I've been out of the KSP loop since about december 2016 / jan 2017 and haven't checked in... so sorry if this has been answered already. But I'm curious.. have SQUAD staff actually acknowledged the nearly game-breaking memory leaks that read to rapid garbage collection with running KSP on 64-bit and really large (300+) part-count ships in space? I know there's a mod to extend and "pad" the memory allocation so garbage collection is less frequent, but it's a band-aid to the situation.

So.. has there been any official progress on this? Have SQUAD Acknowledged it's existence and said they'll fix it some day at the very least?

I can't quite say much other than the fact that I am playing on a VERY LOWEND LAPTOP with an even lowerloooowend graphic card. The game will run quite stable for HOURS. And even with 300+, sloppy fps included and even recording with OBS.

 

Again. Not a useful comment on my side, but memory issues have rarely been an issue for me since 64bit.

[steve_v]

On 14/05/2017 at 0:45 AM, Rocket In My Pocket said:

It's an issue with Unity.

Squad can't fix or change it.

Can we please let this poor horse rest? It's been dead for ages.

No, no we cannot. These GC stalls are, AFAIAC, the single biggest problem with the game right now.

I don't want more parts, more languages, graphical improvements or a DLC. For all these things we have mods.
What I want is for the game engine performance problems and bugs to be fixed. This is as it has always been.

Squad can at fix it, or at least improve the situation, by buying a Unity source licence and upgrading the mono runtime. It may even be possible without the unity sources, as Unity's mono fork is open source.
But it's a lot of work and won't make them any money, so they won't. Just like any other effort-intensive bug / feature that can't be monetised in some way.

Aside, mono is a horrible idea for realtime performance-sensitive applications anyway... but it's free, which makes Unity cheap, which is apparently all that matters.

I sure hope there's nothing garbage heavy in this new localisation code (string manipulation in mono being nasty for this and all), if there is, 1.2.2 will be the last update I download. Won't be buying the accompanying DLC either.

I would, however, gladly pay for a DLC that is nothing but a garbage reduction or GC performance improvement.

[klgraham1013]

11 hours ago, Alshain said:

No telling what 1.3 is going to screw up, I would get it while you can.

Hopefully not much.  I need those terrain seam fixes.

[Starman4308]

1 hour ago, steve_v said:

No, no we cannot. These GC stalls are, AFAIAC, the single biggest problem with the game right now.

I don't want more parts, more languages, graphical improvements or a DLC. For all these things we have mods.
What I want is for the game engine performance problems and bugs to be fixed. This is as it has always been.

Squad can at fix it, or at least improve the situation, by buying a Unity source licence and upgrading the mono runtime. It may even be possible without the unity sources, as Unity's mono fork is open source.
But it's a lot of work and won't make them any money, so they won't. Just like any other effort-intensive bug / feature that can't be monetised in some way.

Aside, mono is a horrible idea for realtime performance-sensitive applications anyway... but it's free, which makes Unity cheap, which is apparently all that matters.

I sure hope there's nothing garbage heavy in this new localisation code (string manipulation in mono being nasty for this and all), if there is, 1.2.2 will be the last update I download. Won't be buying the accompanying DLC either.

I would, however, gladly pay for a DLC that is nothing but a garbage reduction or GC performance improvement.

You're sounding amazingly entitled there. Coding projects on that scale are not cheap and they are not easy. Manual memory management is a pain in the neck to deal with. Coder productivity drops like a stone when you have to deal with that nonsense, and KSP is at a minimum hundreds of thousands of lines of code, quite possibly over a million at this point, built on top of Unity and its built-in rigid-body physics integrator, which is itself a non-insubstantial amount of code.

It's easy to say "Squad should deal with the GC collection pauses" when you're not the one who will have to go through hundreds of thousands to over a million lines of code and de-allocate every scrap of memory yourself.

[MaxwellsDemon]

5 hours ago, Spraki said:

I can't quite say much other than the fact that I am playing on a VERY LOWEND LAPTOP with an even lowerloooowend graphic card. The game will run quite stable for HOURS. And even with 300+, sloppy fps included and even recording with OBS.

 

Again. Not a useful comment on my side, but memory issues have rarely been an issue for me since 64bit.

I've experienced the same thing on my own low-end system.   The main thing I have to watch out for is using mods that load too many/too big textures, I think-- that seems to have a noticeable impact.  But my CTDs have been nearly eliminated.

[AlamoVampire]

Ive myself never really experienced the GC jitters (GCJ) but I can imagine how irritating it could be or is. That being said, any fix regardless of simplicity or complexity can be monitized by the simple fact it COULD make the game more "playable" <defined by your perception of how bad the GCJ is> and increase the public reputation, but Im speculating only here on a theoretical outcome.

But bottom line: time constraints, monetary constraints, license issues, potential bugs etc make this complicated. Speculations ofc

[Tyko]

Are there ways to reduce the GC pauses? It seems like they get worse as my space program grows. I keep wondering if Life Support and / or running asteroid mining ops are significantly increasing my pauses. 

[Starman4308]

42 minutes ago, Tyko said:

Are there ways to reduce the GC pauses? It seems like they get worse as my space program grows. I keep wondering if Life Support and / or running asteroid mining ops are significantly increasing my pauses. 

There's the MemGraph mod, which lets you forcibly allocate more memory to the heap, but that's only viable if you have plenty of spare RAM. This should result in the GC running fewer, bigger pauses, with less time wasted scanning over memory that is in active use.

The ideal solution is to write a game engine that resembles high performance computing code written for use on a supercomputer, but having helped write HPC code for a supercomputer, it is a gigantic pain in the neck. It's easy to say "replace Unity" if you've never had to deal with a program containing a half million lines of code.

EDIT: Though, if Mono can implement a real time garbage collector like Java, that could help a bit. Those consume more CPU cycles than a simple stop-the-world GC, but can give a smoother user experience. 

Edited May 16, 2017 by Starman4308

[LordFerret]

3 hours ago, Starman4308 said:

There's the MemGraph mod, which lets you forcibly allocate more memory to the heap, but that's only viable if you have plenty of spare RAM.

...and if you're running the 64-bit version.

[Starman4308]

17 minutes ago, LordFerret said:

...and if you're running the 64-bit version.

I have no idea whatsoever why anybody with enough RAM to consider MemGraph would be running 32-bit, but okay, granted.

EDIT: There are apparently reasons why one would consider that. I still have no idea what they are, but I can only presume they are valid.

Edited May 17, 2017 by Starman4308

[Tyko]

2 hours ago, LordFerret said:

...and if you're running the 64-bit version.

I'm running MemGraph with the 64-bit version and 32GB of RAM...Still getting the pauses. They're less frequent, but oh so annoying. The definitely become more common as I have more stuff going on (ships in space, landers landed, mining going on, etc). I'm trying to figure out if certain of the activities are more responsible for the pauses than others so I can steer clear of those activities.

[steve_v]

17 hours ago, Starman4308 said:

Coding projects on that scale are not cheap and they are not easy.

I never suggested it was either. Being difficult and/or expensive does not make the problem go away.
 

17 hours ago, Starman4308 said:

Manual memory management is a pain in the neck to deal with.

And yet that's the way it's been done since the 1970s.

 

17 hours ago, Starman4308 said:

Coder productivity drops like a stone when you have to deal with that nonsense, and KSP is at a minimum hundreds of thousands of lines of code, quite possibly over a million at this point, built on top of Unity and its built-in rigid-body physics integrator, which is itself a non-insubstantial amount of code.

Sure. Again, saying "it's hard" doesn't fix the problem. Coders have been dealing with "nonsense" manual memory management for decades. Yes, you need to know what you're doing, but you also have control over your memory (de)allocations.

Squad decided to use Unity3D, and in doing so sacrificed performance for cheap and easy.
Unity decided to use the mono runtime, and to keep using an ancient version - again sacrificing performance for cheap and easy. Mono was not designed for the task unity has put to it, but it is free.

Often the problem with choosing cheap and easy solutions at the beginning is that it's neither cheap nor easy to fix them later. As we see here.

 

17 hours ago, Starman4308 said:

It's easy to say "Squad should deal with the GC collection pauses" when you're not the one who will have to go through hundreds of thousands to over a million lines of code and de-allocate every scrap of memory yourself.

Sure, now it'd be a right bear. Not so if the limitations of monos garbage collector were a prime concern from day one.

Switching to a non memory-managed language now is probably not viable, but upgrading the version of mono in use (though the benefit of the new GC algorithm is speculative) or switching to a not-mono .NET runtime is more so. One would need the Unity sources.

As Squad seem committed to Unity as it is, they (and the modding community) are going to have to put serious effort into reducing garbage generation.
There are plenty of ways to achieve this, and all of them entail work. But the alternative is a real-time game that stops dead every few seconds, and a horrible user experience.
If you're happy with a game that runs like this, cool. I am not. I've been waiting since early beta for decent performance, and yet the GC problem is only getting worse with each new feature added.

Some work has already been done on this (or so says Squad, I haven't noticed any improvement), but as the problem is still apparent it's clearly not enough.

This is not a minor irritation, it's a serious performance problem. Repeating variations of "it's too much work to fix, live with it" solves nothing.

 

59 minutes ago, Tyko said:

I'm running MemGraph with the 64-bit version and 32GB of RAM...Still getting the pauses.

You and me both.
 

59 minutes ago, Tyko said:

The definitely become more common as I have more stuff going on (ships in space, landers landed, mining going on, etc).

Indeed they do. Appears to scale with number of craft in the save.
 

59 minutes ago, Tyko said:

I'm trying to figure out if certain of the activities are more responsible for the pauses than other

I tried, and gave up in disgust. The available profiling tools are inadequate.
Memgraph is handy, but it doesn't show you what is generating garbage - for that we need better tools in the runtime.

Edited May 17, 2017 by steve_v

[HebaruSan]

On 5/13/2017 at 1:43 AM, kithylin said:

Have SQUAD Acknowledged it's existence and said they'll fix it some day at the very least?

 

[Moach]

though I'm not affiliated with squad, I do have some info from my brother (former lead dev) that explains just WHY the game suffers such issues, as well as some others

 

the main problem is:  KSP loads EVERYTHING unto memory at startup - this is also why loading takes so long with a good mod collection - and it'll have all assets up in RAM regardless of you having ever used them in any ship, deployed or not

for now, the only way around the issue, is brute force, that is, the 64bit version which simply allows you to use "moar boosters" (memory in this case) to defer the problem enough that its no longer a factor

but it will continue to take a long time to load, anyways

 

since my brother is not with squad anymore, there's no way that he can eventually get around this "optimization detail" at this point in time, so one must consider a modding alternative as the only genuine hope for a fix

 

it IS nevertheless possible to achieve this by modding - something in the lines of the popular "modulemanager" would do the trick

this mod would basically scan over all parts, and replace them with a lo-fi "surrogate" which gets loaded upon game start (storing that in place of the part itself, moving the original outside the gamedata folder for later use)

then it'd monitor the game for parts being placed in the editor, or appearing on craft which enter physics range (actually get loaded proper) - at this point, this "surrogate system" would replace the low-fi placeholders with the real parts, which could lead to a minor delay while it does this, but most likely wouldn't be a major factor much at all, since most ships use mostly a small set of the same parts multiple times (struts) a lot more than a numerous variety - this delay would be very easily accommodated, especially if asynchronous loading methods were used

 

that would effectively remove both the long load time as well as eliminate the CTD plague - so he who makes this mod shall most certainly become a hero amongst his peers

alas, I'm in no position to make it myself, due to "real life" and whatnot - so I leave it up to whomever wishes to take on the challenge and create this

"one mod to bring them all, and in sheer awesomeness bind them"

 

 

 

 

 

Edited May 17, 2017 by Moach

[Starman4308]

6 hours ago, steve_v said:

And yet that's the way it's been done since the 1970s.

Yes, and in the 1970s, main memory was less than a megabyte.

There is a reason why FORTRAN and Algol are almost nonexistent today, and why Java has largely surpassed C/C++ in popularity. Back in those days, compute time was so expensive that it was a much better tradeoff to throw programmer hours to minimize compute hours. Not so much today.

6 hours ago, steve_v said:

Often the problem with choosing cheap and easy solutions at the beginning is that it's neither cheap nor easy to fix them later. As we see here.This is not a minor irritation, it's a serious performance problem. Repeating variations of "it's too much work to fix, live with it" solves nothing.

KSP, not to mention its mods, would be far behind where it is today with manual memory management. Quite simply, you get less done working with lower-level languages, and Squad never really had the budget to consider writing in that sort of hyper-optimized style.

Simply complaining about it is even less productive than explaining what the factors were behind the choice of using Unity/Mono, which is free and has a rich collection of built-in functions, sharply reducing the amount of development time needed.

I don't think we'll ever agree on this: as I've seen things, the GC pauses are a relatively minor irritant relative to all the other stuff that can go wrong and slow down KSP, and I've had far more experience than I'd like at solving crazy issues that crop up in gigantic code bases.