Dynamic memory management

[W1ntermute]

Now that 99% of KSP's core engine is practically done, it would be helpful to improve the management of memory usage with regards to mods by loading and unloading them dynamically from memory as they're being used. I feel the current system of just dumping every additional mod into ram is extremely inefficient, mods should only be loaded into memory if they are selected by the scenario that contains them or the craft you are building/flying. By streamed the mods in and out of memory, it'd free up much more ram for other mods that arent being used often, you'd reduce overall memory sizes and system resources quite dramatically

[Sirrobert]

Thing is this could result in lots of loading times during the game.

For example, if you fly a non modded craft, and randevous with a craft with lots of modded parts, the game would have to load the parts into memory before the second craft comes within physics range.

And if the combination of crafts were to suddenly be more than the max, the game would crash at that point. I'd rather have long loadtimes at the start, than random crashes during the game

[biohazard15]

Absolutely viable, and there is actually a WIP plugin that adds that functionality: http://forum.kerbalspaceprogram.com/threads/73236-WIP-Loading-textures-only-as-required

Actually, the only reason to load all parts is these small icons in VAB\SPH. Put in small PNG icons instead of actual part, and make parts load as you grab them - bingo, you saved a lot of memory. But noooo, they needed a fancy 3D graphics in ant-sized window...

[M3tal_Warrior]

Speaking as someone quite familiar with computers (administrating 4 (stacking up atm to 5) Debian root servers and quite sure more computers than you have ever had in your entire life, even though it's just a hobby) I see several issues with this suggestion, one of which was mentioned earlier.

Now to keep the memory load low is Windows thinking - it's stupid. You don't gain anything from 4 GB free RAM, they're useless. You could have 4 GB less instead, it wouldn't change a bit. Every MB that is used is of use, and that's the whole point. What you suggest - loading (seldom) and unloading (almost every time) stuff as it is used - is what Windows does automatically, and it does that too good (wasting RAM due to freeing it - this is where pagefile.sys comes in). This is the reason why you have to restart your Windows now and then to speed it up again. Why Windows does that is the factor 'responsiveness'; it expects any time to have to deal with the start of a program that's massively consuming RAM, and to lower the startup time of that program to come it's always having a considerable amount of RAM free. I don't know about you, but with me that's the last thing I'd do while playing KSP (or anything else). Linux has a different, more intelligent approach - 'swappiness' tells the kernel when to free RAM and write seldom used stuff to the swap partition (the Linux equivalent to Windows' pagefile), the value can be altered any time and usually the default in most systems is at 95% RAM usage. Disadvantage: It might not start the next big program quite as fast as Windows (compared to how fast other programs come up, that is - Linux is still almost always faster than Windows). So what you're suggesting is already done - just not by KSP.

Of course - everything that has to come from the hard drive is not immediately available to the CPU/GPU, thus creating lag when it's actually being used in real time applications. This is why database systems on Linux servers are either run entirely from RAM, or - if that isn't an option due to size or criticality of data loss due to power shortages or system errors - from massive stacks of high quality, fast responding hard drives (WD Velociraptors or Seagate Cheetahs, which run at 10k or even 15k rpm with 100-170 MB/s read rates and down to 3.4 ms seek time; while most desktop drives are at 5.6k or 7.2k rpm, with the corresponding smaller read rate & longer seek time) or even SSDs. Trust me - these drives are way out of your league, two of these babies pay you a private scale high end gaming system. Long story short: Hard drive loading is a bad thing during game, and you will see quite a lot of that if you help your operating system in what it does already.

What most people don't know: Ever heard the terms 'spin down time' or 'sleep time'? These are times of being idle (no read/write on/from the hard drive for that given amount of time) when the advertised actions are taken. So if you have 60s sdt and 300s st, the drive will spin down to a lower rpm rate (which reduces vibration/noise, heat production and power consumption) after being idle for 60 seconds and after 300 seconds spins down to zero and go to sleep (and only taking a small margin of power to respond to wake up calls from the system). These times are lower with 'green' drives, obviously, and these are increasingly common in computers nowadays (almost every 1TB and up drive is a green one). But it's a cool thing, right?

No, it's not. While it safes power and is quite good for storage drives which are used only once a day for 3 minutes or so, it is quite bad for system drives, for they will spin up quite a lot of times. Now one has to know some things about HDDs, one being the extreme correlation of temperature changes and spin up times to life expectancy. As Google found out in a test (with thousands of HDDs) it's not the type (server HDDs, desktop HDDs) determining how long the drive will stay operational without errors/breakdown, but the temperature. 20-30°C is killing drives quite fast, 30-40°C they'll have maximum life span and above the drop rate increases again. Spinning a drive up and down all the time, calling it out of sleep and such will kill the drive fastest - imagine your car motor while playing with the accelerator all the time. This is why my drives are allowed to spin down once a day (or so), while running (monitored) at 35-40°C most of the time providing data. And this is why you don't want your drives spinning up for one mod of 20 MB and afterwards go to sleep again until next mod come.

So: No dynamic memory management in KSP. Don't do that - I like my hard drives, all of them, and they like me for not torturing them. The success: While running 20 HDDs all the time for years the last to drop out was 1 year ago, and this was within a RAID where small errors drop a drive immediately - it was a single random bad sector, and it still hasn't found a friend. I almost always retire perfectly working drives (for being too small in storage size), while seeing file system problems and dying drives in other computers quite frequently.

Edited August 9, 2014 by M3tal_Warrior

[biohazard15]

*snap*

Yeah, that "green" stuff can be a constant pain in the lower back, if you didin't disabled that "green" stuff altogether (which I do for any computer - mine or not - when I'm installing OS on it).

But that alone shouldn't be the reason by KSP (or any other application) shouldn't use dynamic memory management.

[Padishar]

Now to keep the memory load low is Windows thinking - it's stupid. <snipped inaccurate, anti-windows blurb>

That is Linux fanboi thinking - it's almost totally wrong. Windows does not behave as you describe. It certainly doesn't attempt to keep a large amount of memory "free" so that it can start a new program more quickly as there is no need. There is almost always a significant chunk of memory being used for the disk read cache and this memory can be re-purposed almost instantly when needed. When I am running KSP on Windows, the "private working set" is never significantly lower than the "commit size" which means that very little of KSP's memory has been freed up by writing the contents to the pagefile (actually the difference in these numbers is almost always very close to the total size of the exe and dlls loaded by KSP).

No one has suggested that they want 4 GB of free ram, just that they don't want to have to wait while KSP loads lots of data into memory on startup that they are not going to use. E.g. I have been playing this game for about 10 months and must have started the game well over 2000 times (I sometimes restart KSP 20+ times in an hour when developing plugins) and I have only ever been outside the Kerbin SOI three times. That is 2000+ times that the game has had to load all the textures and other data for all the other planetary bodies for no purpose at all. That adds up to a significant chunk of time.

<lots of irrelevant stuff about linux database servers> Long story short: Hard drive loading is a bad thing during game, and you will see quite a lot of that if you help your operating system in what it does already.

Ever heard of the disk cache? Ever heard of pre-loading of data (shortly) before it is required? The virtual memory behaviour in Windows (and Linux) is designed to be fairly efficient for a wide variety of usage patterns and there are many programs whose memory usage pattern is far enough from the norm that it is significantly more efficient if they give the OS a helping hand. In any case, why do you think that "if you help your operating system in what it does already" will cause any more actual disk access than just letting it do it? That doesn't make sense...

<snipped patronising stuff about spin down/sleep> As Google found out in a test (with thousands of HDDs) it's not the type (server HDDs, desktop HDDs) determining how long the drive will stay operational without errors/breakdown, but the temperature. 20-30°C is killing drives quite fast, 30-40°C they'll have maximum life span and above the drop rate increases again. Spinning a drive up and down all the time, calling it out of sleep and such will kill the drive fastest - imagine your car motor while playing with the accelerator all the time. This is why my drives are allowed to spin down once a day (or so), while running (monitored) at 35-40°C most of the time providing data. And this is why you don't want your drives spinning up for one mod of 20 MB and afterwards go to sleep again until next mod come.

You first say it is the temperature but then say that it is spinning it up and down lots that kills drives. It is actually both of these things combined, but only if spun down for long enough to actually cool down significantly, i.e. it is the thermal cycling from cool->hot->cool that kills them. If you run a hard disk continually with carefully controlled temperature you will not see much difference in lifetime between one run at 10°C and one run at 70°C. Having said that, do you realise that KSP writes data to various log files and writes out auto-saves quite frequently, which requires the drive to be spun up? In comparison, reading in data on-demand is quite likely to not actually need to spin up the drive as the data may well be in the disk cache. I don't believe you can make any claims about the amount of spinning up and down that would happen without actually testing it.

So: No dynamic memory management in KSP. Don't do that - I like my hard drives, all of them, and they like me for not torturing them. The success: While running 20 HDDs all the time for years the last to drop out was 1 year ago, and this was within a RAID where small errors drop a drive immediately - it was a single random bad sector, and it still hasn't found a friend. I almost always retire perfectly working drives (for being too small in storage size), while seeing file system problems and dying drives in other computers quite frequently.

Yes, running HDs continuously is much better for them than starting and stopping them frequently. Most people turn their computer off after use and leave it off for long enough for the HD to cool to room temperature and it's this that is the main cause of early drive failure.

TL;DR: You really can't make any sort of claim that on-demand loading of rarely used data will "torture" an HD any more than loading all the data at program start without actually testing it. On the other hand, on-demand loading does make a significant difference to memory usage and startup time which is very important for people running lots of mods, running on Win32 or running with other memory hungry programs simultaneously.

[Sirrobert]

Regardless off the technicualitys, this suggestion is always only really suggested with 1 purpose

To allow people to load more mods than the memory limit currently allows. The other purpose would be to reduce initial loading times, which is not even worth the trouble.

The problem with this idea, is that now, instead of crashing on load, the game will crash when it has to load up 2 ships that, thougether, contain enough modded parts to go over the memory limit.

And crash on startup is WAY better than random crashes during the game (no lost progress, and it's way easier to find the problem this way)

[Padishar]

Regardless off the technicualitys, this suggestion is always only really suggested with 1 purpose

To allow people to load more mods than the memory limit currently allows. The other purpose would be to reduce initial loading times, which is not even worth the trouble.

Tell that to people (like me) who develop plugins and need to restart KSP 10-20 times an hour.

The problem with this idea, is that now, instead of crashing on load, the game will crash when it has to load up 2 ships that, thougether, contain enough modded parts to go over the memory limit.

And crash on startup is WAY better than random crashes during the game (no lost progress, and it's way easier to find the problem this way)

Actually, if it were implemented properly there is no reason at all for the game to crash in that situation so no reason for any loss of progress, however, if it did crash, it would not be any harder to deduce that the problem was running out of memory than it is now.

[M3tal_Warrior]

That is Linux fanboi thinking - it's almost totally wrong.

Sweetheart, I was on Win3.11, Win95, Win98SE (beginning administrating machines), WinME, WinNT4, Win2k, WinXP, Win7, still do administration for Windows machines (and did that exclusively for 8 years) and write to you using Opera on Win7 HP with 8GB RAM, which contrary to my 4 GB Debian Wheezy notebook has to be restarted at least once a month due to being damn slow. If I was less patient I'd do a restart every day, because even while only surfing about the machine slows down quite quickly with multiple programs. And the funny part is: None of these programs need a) as much space in RAM and will be any faster for a prolonged time when being restarted.

On Debian I do a sudo swapoff -a && sudo swapon -a and when this is being finished it's as fast as in the beginning, if it WOULD even need that.

So while you might talk university BS here, I'm talking hands-on knowledge. It slows down, and this is due to extremely active pagefile usage. Speaking of disk prefetch/journal cache you know what you've done when 16 GB (8 + 8 swap) isn't enough for the 4 TB HDDs (half of which are constantly sleeping data graves) in the machine, while the Debian server runs smoothly on half the 8 GB (because it's running a 2 GB MC server from /dev/shm) with more than double the (constantly active softraided & encrypted) hard disk space and no swap usage whatsoever. You can call me fanboy all night long - these are hard facts. Either Windows doesn't do effective garbage collection, is crapping in it's RAM and pagefile or - as I said - frees significant space in the RAM.

That is 2000+ times that the game has had to load all the textures and other data for all the other planetary bodies for no purpose at all. That adds up to a significant chunk of time.

Got a - for you obviously interesting new - tip: SECOND (VANILLA) INSTALL! It reduces the loading time to a bare minimum and is the platform your developed mod is thought to work with. Since KSP is designed portable (except for some screen settings nothing important is written to the registry) this will work quite easy, and those 400MB ain't no space on a modern hard drive.

In any case, why do you think that "if you help your operating system in what it does already" will cause any more actual disk access than just letting it do it?

Because it's two systems doing the same. This might be circumvented by Windows as long as enough RAM is free for doing a complete copy of every mod & KSP into RAM, buth when it's swapping time you got your two systems requesting - one the prefetch data, one the hard disk data to update the prefetch once in a while...

You first say it is the temperature but then say that it is spinning it up and down lots that kills drives.

It might have been misunderstanding, it's both, independently. Delta-T of 15K kills twice as much drives, T of 298K has a 7 times higher average failure rate as T of 318K. Engaging upspin all the time isn't helping matters, since it adds wear&tear to the parts (deltas in forces due to acceleration, adding physical stress to the components; that is the case with every technical equipment, be it airplane turbine or a bike - constant forces aren't that problematic). Read the abstract, it's quite interesting.

Having said that, do you realise that KSP writes data to various log files and writes out auto-saves quite frequently, which requires the drive to be spun up?

This actually might rescue the drive here, at least with higher threshold times for sleep/spindown. Of course this has to be tested, I just added additional thoughts here which most people easily forget (or have never heard of). AS for the patronizing: What do you want: A fully fledged scientific abstract that is understood by 5 people in here, or something that boils down information to the vast majority of users?

On the other hand, on-demand loading does make a significant difference to memory usage and startup time which is very important for people running lots of mods

With disk prefetch on i386 or running memory-hungry second software I doubt that, for either it is loaded into RAM, eating your memory, or swapped out and thus still on the drive. At least with loading times I reckon you will be waiting a lot longer (playing more than 2 hours) due to longer ingame loading screens than having all up in one go. And we've not even asked the question if unity is capable of post-start loading mods, which would be prerequisite.

[Sirrobert]

Tell that to people (like me) who develop plugins and need to restart KSP 10-20 times an hour.

Actually, if it were implemented properly there is no reason at all for the game to crash in that situation so no reason for any loss of progress, however, if it did crash, it would not be any harder to deduce that the problem was running out of memory than it is now.

And plugin makers are awesome. I love you.

If there is a way to prevent such a crash though, I have missed it. What is such a solution? Cause if there is, I'd see no other problem with this idea

[Padishar]

Sweetheart, I was on Win3.11, Win95, Win98SE (beginning administrating machines), WinME, WinNT4, Win2k, WinXP, Win7, still do administration for Windows machines (and did that exclusively for 8 years) and write to you using Opera on Win7 HP with 8GB RAM, which contrary to my 4 GB Debian Wheezy notebook has to be restarted at least once a month due to being damn slow. If I was less patient I'd do a restart every day, because even while only surfing about the machine slows down quite quickly with multiple programs. And the funny part is: None of these programs need a) as much space in RAM and will be any faster for a prolonged time when being restarted.

<snip more, not particularly relevant stuff about Linux>

First, I am not your sweetheart, but thanks for reinforcing my point about being patronising, I'm surprised you didn't add "since before you were born". Second, I take very little notice of what experience and/or qualifications people claim to have because I have personally known numerous people that have administered computers for many years (both windows and *nix, servers and user machines) but have very little understanding of what actually goes on inside the OS and make very dubious (or even downright incorrect) assertions. I judge what people say purely on what they say and how they say it. Generally, people that go on and on about how much they know tend to know a lot less than they think...

So while you might talk university BS here, I'm talking hands-on knowledge. It slows down, and this is due to extremely active pagefile usage. <snip more pointless comparison > You can call me fanboy all night long - these are hard facts. Either Windows doesn't do effective garbage collection, is crapping in it's RAM and pagefile or - as I said - frees significant space in the RAM.

I am not talking university BS, I left university over 20 years ago (before Win3.11 was even released) and have worked as a programmer (and part-time sysadmin) since. Windows machines do not get slower over time because they try to keep a large amount of memory free. This is patently obvious as they don't "try to keep a large amount of memory free" at all. Neither is it because of "extremely active pagefile usage". I can write a program that thrashes VM continuously for a week and, at the end of that time it will still be running at the same speed. There are issues that cause Windows to slow down over time but these are due to various design and implementation decisions in Windows that cause it to suffer more than it should from memory fragmentation issues. Most programmers, including many of those at Microsoft, don't bother trying to avoid these issues as the vast majority of Windows machines are rebooted often enough to make it pointless.

Got a - for you obviously interesting new - tip: SECOND (VANILLA) INSTALL! It reduces the loading time to a bare minimum and is the platform your developed mod is thought to work with. Since KSP is designed portable (except for some screen settings nothing important is written to the registry) this will work quite easy, and those 400MB ain't no space on a modern hard drive.

Ahh, more of that patronising condescension. I actually have 14 different installations of KSP of various different versions and with various different sets of mods. Oh, and each one is significantly more than 400MB, a clean install of 0.24.2 is significantly over 2GB. I'm not saying that it is a significant proportion of my HD space, just pointing out another case of you saying inaccurate things.

Because it's two systems doing the same. This might be circumvented by Windows as long as enough RAM is free for doing a complete copy of every mod & KSP into RAM, buth when it's swapping time you got your two systems requesting - one the prefetch data, one the hard disk data to update the prefetch once in a while...

It isn't two systems doing the same. If your machine is swapping while running KSP then the performance is going to suffer significantly. This is because demand paged virtual memory accesses the disk at the exact point in the code where the memory is accessed and the program has to wait until the data has been loaded. If the program doesn't initially load the large data that would otherwise be paged out and instead, loads it slightly in advance of when it is needed, the data may well be (should be if the preload is sensibly tuned) available by the time it is actually accessed.

<snip> Read the abstract, it's quite interesting.

I don't need to read the abstract (though I think I already have some time ago, it rings a bell). I am well aware that the way that most people run their machines is not good for the reliability of their HDs but you gave no evidence (let alone proof) that on-demand loading of a game's assets is actually going to affect it one way or the other.

This actually might rescue the drive here, at least with higher threshold times for sleep/spindown. Of course this has to be tested, I just added additional thoughts here which most people easily forget (or have never heard of). AS for the patronizing: What do you want: A fully fledged scientific abstract that is understood by 5 people in here, or something that boils down information to the vast majority of users?

No, I don't. I just want you to cut out the patronising tone and stick to stating actual facts (or opinions explicitly noted as being such).

With disk prefetch on i386 or running memory-hungry second software I doubt that, for either it is loaded into RAM, eating your memory, or swapped out and thus still on the drive. At least with loading times I reckon you will be waiting a lot longer (playing more than 2 hours) due to longer ingame loading screens than having all up in one go. And we've not even asked the question if unity is capable of post-start loading mods, which would be prerequisite.

When I am developing mods I generally have VS2013, a browser with 6-10 tabs open, fraps, a git shell and about a dozen explorer windows open. Even with all the other little utilities (dropbox client, video card config, audio config, virus scanner etc) my machine only uses about 3GB of physical RAM. The other 5 GB is either being used for disk cache or is temporarily free because I have just quit KSP freeing up ~1.5GB. When I do start one of my fairly standard KSP installs (i.e. one that doesn't include a lot of part mods) it takes approx. 30 seconds to get to the main menu and KSP ends up using about 1.5GB. I don't believe that the OP was implying that plugin mods should be loaded and unloaded. This is impractical for a number of reasons, most notably that it is not supported by Unity, and it wouldn't actually achieve much in the way of reducing the memory requirements. The main memory hog in KSP is part mods and, specifically, their bitmap data (textures, normal maps, etc) and these can be loaded and unloaded on demand. Also, in the stock game, there is the matter of the very large textures and height maps for all the planetary bodies, which are often not required at all during a particular run of KSP.

And plugin makers are awesome. I love you.

If there is a way to prevent such a crash though, I have missed it. What is such a solution? Cause if there is, I'd see no other problem with this idea

Thanks. I'm not saying there is a way, at present, to avoid such a crash. It would almost certainly require the on-demand loading to be done by the core game rather than in a mod. If I was implementing on-demand loading with the intention of working around a memory limit then I would try to write it in such a way that it was impossible for it to use more than that amount. This might require deliberate unloading or simplification (e.g. downsampling of textures) of data that is in use when more data needs to be loaded, or it might not be possible in extreme situations but it should still be possible to save the game state and display a nice error of some kind indicating any possible solutions to their issue (e.g. switch from Win32 to Win64 and/or use the 64bit version of the game).

Edited August 10, 2014 by Padishar

[Ruedii]

I personally prefer throwing all the mod data into an external heap, and only pulling it into the main memory when needed.

This allows the external heap to be in a separate segment of memory, beyond the 4GB barrier of the game, or even swapped out to disk. Win32 has a nasty problem with such heaps not being allowed to execute code, even that from Mono or .Net, so there may be some issues to work through there. However, it's probably best to keep all executable code in the same base segment.

Furthermore, by using page-flipping (where you change the physical address of a virtual page) and duplicate mapping (where you map in-memory pages to both the heap and the game page) you could really boost loading speeds of these segments over doing page-copies, while still allowing the OS to swap out the heap to virtual memory.

Finally, using one heap for texture memory, and one heap for everything else, would dramatically improve video performance by allowing and encouraging the video driver to map all the texture memory to single block of physical memory, if that is at all possible.

The addressing of texture memory could also be improved with the use of large page address mode, if it is available. (Linux, OS X and Windows allow this, however Windows can be difficult to configure non-administrator programs to allow this, and some Linux distros don't set this up by default to be accessible from desktop users, I am not sure of the OS X technicalities on this other than the fact that Apple says it's supported.)