Reduce RAM usage. Yes, in 1.2.2

[juanml82]

I've (probably temporarily) upgraded my ram memory from 8 to 16, mostly because my somewhat heavily modded KSP installation was getting slow at times, usually when loading a new scene and specially when quitting the game. But I also decided to monitor ram usage, and KSP (and all the mods) is using between 6-7.5 GB of RAM, which is really quite a lot for such a game.

Before 1.2 and specially before 64 bits there were several workarounds to reduce memory usage: using opengl and the mods load on demand and later Dynamic Texture Loader, but AFAIK they don't work in 1.2.2. Is there any current way to reduce RAM usage besides removing mods? 8GB seems a lot and, TBH, KSP is the only application I use which would warrant a ram upgrade, so I may prefer to reduce it's ram usage if at all possible.

[Kerbart]

I'm too lazy to find it, but I do like to point out that when everyone told me that 64 bit would be "unlimited mod" heaven my reaction was that we would run into people who'd complain that now they're not running out of memory space, they're running out of physical memory.

Can I collect my prize?

 

Don't get me wrong, KSP's strategy to load everything into memory for perceived performance gains was bound to hit a roadbump. I fully agree with you on that.

 

[jhousen]

one of my friends calls ksp computer abuse, I agree

I have 32GB of ram and am still running out, if you find a mod that fixes this, let me know.

[aRottenKomquat]

Are you actually encountering out-of-memory errors? Modern operating systems are extremely lazy about deallocating unused memory if the system is not under memory pressure. Hence, RAM usage can be artificially high because a bunch of it is not actually being used anymore but was never freed because nothing needs it.

[jhousen]

in my case, yes I'm definately running out of memory.

and I've already killed the usual hogs (glaring at you, superfetch)

total idle usage is about 2gigs with 30 free

[juanml82]

1 hour ago, aRottenKomquat said:

Are you actually encountering out-of-memory errors? Modern operating systems are extremely lazy about deallocating unused memory if the system is not under memory pressure. Hence, RAM usage can be artificially high because a bunch of it is not actually being used anymore but was never freed because nothing needs it.

Very, very occasionally, KSP would crash. But with 8GB I'm having system slowness which can be attributed to Windows having to put everything that isn't KSP in the page file

[juanml82]

Well, opengl still works. It's now using 4,5 GB in the launchpad and going into orbit

[A_name]

12 minutes ago, juanml82 said:

Well, opengl still works. It's now using 4,5 GB in the launchpad and going into orbit

How do you do opengl again?

[juanml82]

33 minutes ago, A_name said:

How do you do opengl again?

Create a shortcut to the game executable (well, the 64bits executable), right click the shortcut, go to properties, look for target in the second tab and, at the end of the line add  "-force-opengl"

It should read something like "C:\Games\SteamLibrary\SteamApps\common\Kerbal Space Program\KSP_x64.exe" "-force-opengl" depending on where you have it installed

 

Oh, and @Kerbart yep, grab your prize

[kBob]

You can do like often had to be done in the old flight simulator days go in and get rid of every process you don't need (task manager in Windows) or create a special boot configuration for that, but that seems so early 2000's late 90's.  Depending on what you have running it may free up some decent amount of RAM, kill everything you don't need it adds up including things like virus scanners (just be careful to reboot or rerun them afterwards).  Of course you'll still run out eventfully if you keep adding mods but it might free up enough room for that one extra mod you just have to have  .

Edited April 1, 2017 by kBob

[jhousen]

hmmm, unity has an "unloadunusedassets()" function...I wonder...

[Curveball Anders]

On 2017-03-31 at 11:50 PM, jhousen said:

in my case, yes I'm definately running out of memory.

and I've already killed the usual hogs (glaring at you, superfetch)

total idle usage is about 2gigs with 30 free

While an active process allocating too much memory is a problem, total allocation isn't

It's rather the other way around.

Unused ram is a wasted resource.

In a perfectly tuned system all available ram is used all the time, if not by active/user processes then by the OS to speed up the overall performance.

Background/system processes (like the glaringly obvious superfetch) should use all idle resources, the problem occurs when system designer and user don't agree on what is 'idle resources'

[Starman4308]

On 3/31/2017 at 6:16 PM, juanml82 said:

Very, very occasionally, KSP would crash. But with 8GB I'm having system slowness which can be attributed to Windows having to put everything that isn't KSP in the page file

Are you sure it's putting everything that isn't KSP into swap/page file, and whether those pieces of memory are actually being used? Slowdowns due to memory can be hard to properly diagnose; it can be hard to know whether that 95% memory usage is actually you running out of memory, or just the OS/KSP being lazy about garbage collection. Neither your OS nor KSP will bother to do garbage collection if there's plenty of memory available.

Edited April 2, 2017 by Starman4308

[juanml82]

1 hour ago, Starman4308 said:

Are you sure it's putting everything that isn't KSP into swap/page file, and whether those pieces of memory are actually being used? Slowdowns due to memory can be hard to properly diagnose; it can be hard to know whether that 95% memory usage is actually you running out of memory, or just the OS/KSP being lazy about garbage collection. Neither your OS nor KSP will bother to do garbage collection if there's plenty of memory available.

Well, not "sure, sure", but if I have 8GB of RAM and KSP is using over 7GB, then there isn't much room for everything else

[Starman4308]

2 hours ago, juanml82 said:

Well, not "sure, sure", but if I have 8GB of RAM and KSP is using over 7GB, then there isn't much room for everything else

It's entirely plausible that 7 GB of KSP plus 1 GB of everything else is a low-load condition. It is extremely hard to be certain from what simple tools like Window's resource monitor tells you, because memory management is a complicated topic, and modern programs are extraordinarily lazy about freeing up unused memory if nothing else needs that memory.

To be clear:

#1: KSP runs on Mono, which is a garbage-collected language. This means that, instead of explicitly allocating and de-allocating memory for objects and arrays like you might in C or Fortran, you just declare the object or array, and trust that Mono will de-allocate that memory when it runs a garbage collection sweep.

#2: Garbage collection sweeps are when Mono goes through, and finds all variables which can no longer be accessed by any thread. This occurs when something goes out of scope or is de-referenced. For example, in pseudocode:

int[] someIntegers = { 1, 2, 3, 4, 5};
if (rocketUnexploded) {
      List<Part> rocketParts = rocketPartTree.getParts().asList();
      doSomething(rocketParts);
} // The rocketParts list is now rootless, and subject to garbage collection, as it only lived inside the scope of the now-finished if block.
someIntegers = { 6, 7, 8, 9, 10 }; // The old someIntegers array is now rootless, and subject to garbage collection.

#3: Any decent garbage-collected language will garbage collect as infrequently as possible. This takes a fair chunk of CPU time, and simple GC implementations generally have to stop the world, halt all program execution while the GC sweeps through.

#4: As a consequence, if Mono can get away with it, it'll just allocate loads of memory to itself until the operating system cuts it off, and garbage collect as infrequently as possible.

#5: The operating system is doing that as well, refusing to clear stuff out of virtual memory or move it to the page file unless it absolutely has to.

#6: Therefore, "KSP is using 7 GB!" is a terrible metric of whether KSP is actually stressing your system's physical memory.

You might look into MemGraph to see if there is really any substantial memory problem with your installation; memory is only a problem if you have frequent garbage collection pauses.

 

[jhousen]

On ‎4‎/‎2‎/‎2017 at 3:44 AM, Curveball Anders said:

While an active process allocating too much memory is a problem, total allocation isn't

It's rather the other way around.

Unused ram is a wasted resource.

In a perfectly tuned system all available ram is used all the time, if not by active/user processes then by the OS to speed up the overall performance.

Background/system processes (like the glaringly obvious superfetch) should use all idle resources, the problem occurs when system designer and user don't agree on what is 'idle resources'

given the massive stutter I get when superfetch decides to...compress unused memory (ie. compress half of ksp, and then not hand it back when needed, causing ksp to crash), I'll leave it turned off (and wonder who had that bright idea)

not as bad as vista and 7 where superfetch would sometimes not hand memory back at all, but still...

[steve_v]

2 hours ago, jhousen said:

given the massive stutter I get when superfetch decides to...compress unused memory (ie. compress half of ksp, and then not hand it back when needed, causing ksp to crash), I'll leave it turned off (and wonder who had that bright idea)

Compressing memory sounds like as good an idea as compressing hard disk storage was in 1995... which is to say a really good way to make the machine slow to a crawl. Pretty happy I have never encountered this "superfetch" thing myself.
The whole point of RAM is that it is fast working memory, why on earth would you want to make it slower by compressing it? Many applications decompress their data into RAM, compressing it again is madness.
As for compressing only "unused" memory, how do you know what's "unused"? RAM is random access memory - any data in may be accessed any time, in any order.
If you don't have enough RAM, get more. RAM is cheap.

That said, the real problem is the way KSP loads textures. Never seen Eloo? It's terrain textures are still consuming memory, even if they're never rendered.

 

On 01/04/2017 at 1:06 PM, juanml82 said:

Well, opengl still works.

Obligatory warning: It works for some people, but it's not officially supported in the Windows player. Problems have been reported.

Edited April 5, 2017 by steve_v

[MaxwellsDemon]

2 hours ago, steve_v said:

That said, the real problem is the way KSP loads textures. Never seen Eloo? It's terrain textures are still consuming memory, even if they're never rendered.

That would explain why OPM bogs me down so much.   

[Starman4308]

6 hours ago, jhousen said:

given the massive stutter I get when superfetch decides to...compress unused memory (ie. compress half of ksp, and then not hand it back when needed, causing ksp to crash), I'll leave it turned off (and wonder who had that bright idea)

not as bad as vista and 7 where superfetch would sometimes not hand memory back at all, but still...

Superfetch should not be doing that. The point is that virtual memory should always be accessible, even if it has to go to the pagefile, and Superfetch should only be compressing seldom-used items. If your physical memory is under significant load, I wouldn't be surprised to see half of KSP compressed and/or moved to the pagefile, because the KSP devs had the bright idea to load enormous texture files straight into memory, few of which you will be using at any given moment.

4 hours ago, steve_v said:

Compressing memory sounds like as good an idea as compressing hard disk storage was in 1995... which is to say a really good way to make the machine slow to a crawl. Pretty happy I have never encountered this "superfetch" thing myself.
The whole point of RAM is that it is fast working memory, why on earth would you want to make it slower by compressing it? Many applications decompress their data into RAM, compressing it again is madness.
As for compressing only "unused" memory, how do you know what's "unused"? RAM is random access memory - any data in may be accessed any time, in any order.
If you don't have enough RAM, get more. RAM is cheap.

That said, the real problem is the way KSP loads textures. Never seen Eloo? It's terrain textures are still consuming memory, even if they're never rendered.

 

Obligatory warning: It works for some people, but it's not officially supported in the Windows player. Problems have been reported.

Compressing memory is a good idea when programs are really sloppy about what they decide to load.

Let's say you're playing KSP, and for the entire session, you never even glance at Eeloo. Your system is under memory pressure from the 127 parts mods you decided to download with max-res textures, and it needs to make some space. It makes sense to the OS to either compress the Eeloo textures or send them to the page file, because they haven't been accessed in forever, and it's unlikely they will be needed in the future, so the OS says "I'll just move these out of the way; the chance it'll actually be needed and slow down the system is minimal".

RAM is not truly randomly accessed; there are distinct usage patterns that the OS can take advant

[Kerbal101]

Play on Linux. The heaviest base system uses 0.7 GiB (kde5) instead of 1.5 GiB (w10). OpenGL loads textures on demand instead of all-at-once. You can set swappiness value to further prevent OS swap anything until ram is mostly full. It can be installed on any x86 regular machine (and designed to run on those).

Or upgrade to motherboard, which supports 64+ GiB of ram. Fact that Dx loads more into Ram helps it gain fps advantage.

Its also possible to re-compress textures to smaller size using a tool initially created for Skyrim, but I never used this. It should work, as it can manipulate any raw dds files.

[Kerbart]

On 4/2/2017 at 5:58 PM, juanml82 said:

Well, not "sure, sure", but if I have 8GB of RAM and KSP is using over 7GB, then there isn't much room for everything else

8 - 7 = 1 GB.

It all depends on your definition of "much room"

Back when I was a little punk I'd write programs that fit into 500 bytes. Barefoot. Uphill. And I liked it!! 
(It's amazing what you can do with 500 bytes of memory in Z80 assembler)

[Noobton]

has anyone tried running KSP with the new windows 10 game mode? is supposed to kill those other pesky processes.

[Kerbart]

2 minutes ago, Noobton said:

has anyone tried running KSP with the new windows 10 game mode? is supposed to kill those other pesky processes.

It is supposedly optimized for UWP ("Universal Windows Platform", App store) games and Microsoft warns that regular Win32 games as found on Steam will not benefit very much of it.

This article shows a KSP screenshot but doesn't mention how it fares under Game Mode, but it notes that there's very little benefit for other games, and mainly on computers that are already pushed to the brink. So if you're short on memory it might help.

[juanml82]

39 minutes ago, Kerbal101 said:

Play on Linux. The heaviest base system uses 0.7 GiB (kde5) instead of 1.5 GiB (w10). OpenGL loads textures on demand instead of all-at-once. You can set swappiness value to further prevent OS swap anything until ram is mostly full. It can be installed on any x86 regular machine (and designed to run on those).

But if I play it on Linux, I'd need to reboot if I want to play something else. And you can't easily keep a Windows and a Linux installation syncronized as far as savegames and mods go, if it can be done at all.

 

Quote

Or upgrade to motherboard, which supports 64+ GiB of ram. Fact that Dx loads more into Ram helps it gain fps advantage.

I didn't get this. Do you mean I should upgrade to more than 64GB or RAM to play one single game?

 

Quote

Its also possible to re-compress textures to smaller size using a tool initially created for Skyrim, but I never used this. It should work, as it can manipulate any raw dds files.

Uh, that's interesting, at least for seldom used textures like IVAs, part mods seldom used and maybe planets. What's that tool's name?

[Kerbal101]

5 minutes ago, juanml82 said:

But if I play it on Linux, I'd need to reboot if I want to play something else. And you can't easily keep a Windows and a Linux installation syncronized as far as savegames and mods go, if it can be done at all.

...

I didn't get this. Do you mean I should upgrade to more than 64GB or RAM to play one single game?

...

Uh, that's interesting, at least for seldom used textures like IVAs, part mods seldom used and maybe planets. What's that tool's name?

Usually you don't have to do it, as rarely you will want to play same game on both OSes. And in case you do, it can be done with symlinks, by symlinking the save directory location on Windows partition.

...

You can, not "should". The memory use of KSP does not come from abstract source, but from mods which you use. As such, if you use many mods, you pay with increased memory usage. Its technically possible to obtain cost-efficient hardware to cover this usage. There are no artificial limits, like which happened on x86_32 binaries/cpu modes.

...

I can't remember its name, its a standalone exe that is pretty universal and all-in-one utility.
It can easily compress 12 GiB of Skyrim mods into 4-5 GiB, although the size or quality of textures will also reduce. I also reduced Stalker(game) textures once and got rid of the out-of-memory errors, as the later is x86_32 application.
But its not an fire-and-forget/foolproof tool as some images like collective single-file UI texture, may not be compressed (in the same Stalker). So this work is manual and needs some time.

You could also walk your mods directory with some equivalent of Filelight/Baobab to determine which one drains most of space.