So just how bad is loading all stock parts anyway? A comparison.

[sal_vager]

Sometimes players ask about running KSP on old PC's with limited resources, and the usual recommendation is to use active texture management on aggressive to compensate for Squads choice to preload all part textures.

But lately I see people posting that even ATM isn't helping, since 1.0.x all the Squad textures have been converted to the more memory efficient .dds format, but rather than mess around with ATM I thought I'd see what would happen if they were removed completely.

Bare in mind this just covers initial memory use with a stock install, any issues with PQS terrain caching, requestresource, the GUI or any addons aren't covered here.

All tests were with 32bit KSP on Linux, as 64bit programs naturally use more memory, and all memory is shown in megabytes, and this is RAM, not the size on the hard drive.

So first I needed a baseline, I unzipped a new copy of 1.0.4, opened htop, started a new game with all settings on their defaults, put a KerbalX on the pad and flew to orbit...

Javascript is disabled. View full album

Scene	Memory
Start	1194
KSC	1317
VAB	1383
Pad	1345
Flight	1365
Orbit	1373

 

Interestingly the editor used the most memory, also memory went back up to the VAB level when I reverted and dropped again when I relaunched, I tried this a few times from the beginning and the memory used was the same so things seem pretty stable.

Now to get rid of those nasty memory hogging textures!

First I wanted to know how many .dds files I was dealing with, so I used the terminal programs "find" to find every file in the KSP folder that ended in ".dds" then piped finds output to "wc" so it could count the newlines, like this:

Quote

find ~/ksp/ksp_notextures/ *.dds | wc -l

545

Hmm quite a lot! Now to get rid of them.

Quote

find ~/ksp/ksp_notextures/ *.dds -delete

find ~/ksp/ksp_notextures/ *.dds | wc -l

0

Great! Lets see how lean KSP is now...

Javascript is disabled. View full album

Scene	Memory
Start	1102
KSC	1228
VAB	1290
Pad	1248
Flight	1252
Orbit	1257

 

Well...

It's kinda funny seeing all the parts bright white, like there was a sale on white paint, but the change in memory is only about 100 megs, that's not enough to worry about so maybe the preloaded textures aren't the problem, that leaves the meshes and cfg's.

Best get rid of them, as well...

After a lot of messing around I got it down to the following:

 

Spoiler

find -name *.cfg

./Electrical/z-400Battery/z-400Battery.cfg

./Electrical/radialFlatSolarPanel/radialFlatSolarPanel.cfg

./FuelTank/fuelTankX200-16/fuelTankX200-16.cfg

./FuelTank/fuelTankX200-32/fuelTankX200-32.cfg

./FuelTank/fuelTankT400/fuelTankT400.cfg

./Aero/cones/ConeB.cfg

./Aero/wingletAV-R8/wingletAV-R8.cfg

./Command/Mk1-2Pod/mk1-2CommandPod.cfg

./CompoundParts/fuelLine/fuelLine.cfg

./CompoundParts/strutConnector/strutConnector.cfg

./Misc/PotatoRoid/part.cfg

./Engine/liquidEngineLV-T45/liquidEngineLV-T45.cfg

./Engine/liquidEngineMainsail/liquidEngineMainsail.cfg

./Engine/liquidEnginePoodle/liquidEnginePoodle.cfg

./Utility/ladderRadial/ladderRadial.cfg

./Utility/ladderTelescopic/ladderTelescopic.cfg

./Utility/launchClamp1/launchClamp1.cfg

./Utility/decouplerStack2m/decouplerStack2m.cfg

./Utility/parachuteMk16-XL/parachuteMk16-XL.cfg

./Utility/landingLegLT-2/landingLegLT-2.cfg

./Utility/decouplerRadialHDM/decouplerRadialHDM.cfg

./Utility/ladderTelescopicBay/ladderTelescopicBay.cfg

Had to keep the potato in there so it didn't cause problems, and the Mk1-2 IVA files sans textures, thanks for reminding me Stone Blue.

Okay! Now for a lean, mean and much easier-to-choose-parts-for-my-builds version of KSP...

Javascript is disabled. View full album

Scene	Memory
Start	1031
KSC	1172
VAB	1235
Pad	1195
Flight	1197
Orbit	1202

 

Well...again...but at least it loaded really fast...

So all the models and cfg's only really came to about 50 megabytes, about 150 megs for over 500 parts, Squad could double the number of stock parts and it wouldn't be a problem unless you're really pushing KSP to the 32bit memory limits, but if all parts were as light on their feet as Squads you'd need closer to 7,000 or 8,000 parts added to your install before KSP fell over.

So far I have used a new settings.cfg and save each time, but many textures aren't in convenient-to-delete .dds files, the only way I can deal with those terrain and building textures without messing with the Unity asset files is to lower the graphics settings.

So that is what I did next!

With all settings on their lowest I get...

Javascript is disabled. View full album

Scene	Memory
Start	918
KSC	1066
VAB	1114
Pad	1085
Flight	N/A
Orbit	N/A

 

Horrible menu textures! But also the lowest memory used yet at 918 megabytes, if you used a seriously lightweight OS you might be able to at least start KSP in 1 gig, though you'd need more than that to play it, but not much more!

Also seeing as the flight and orbit memory doesn't change much I skipped them this time, but added a shot of the VAB, doesn't look half as bad as I thought it would.

But that's an average reduction in the editor and the flight scene of over 100 megabytes, more than completely deleting all the part textures!

So which setting is using all the memory?

After a lot of changing of settings and restarts it turned out to be the texture quality, here's a test with an unravaged 1.0.4 install, all settings on default except texture quality, which was on the lowest setting...

Javascript is disabled. View full album

Scene	Memory
Start	1011
KSC	1145
VAB	1179
Pad	1146
Flight	1147
Orbit	N/A

 

That's a big difference from one setting change and memory used is lower than I expected it would be, so maybe the lower texture setting works better with the default SM3 shaders than it does with the SM2 and fallback shaders, also after asking around in IRC I found that all the textures in the asset files are already .dds, so I guess there's just a lot of them, they certainly have to cover a much larger area.

So what can we take away from all this?

Squads part preloading doesn't look like a problem, and ATM or even texture deletion isn't going to help much with memory use, but you could run out of memory if you use a lot of unoptimized parts.

So insist on .dds whenever possible, and convert textures yourself if you have to, there's a converter here by Lilleman, as well as free tools like Imagemagick.

But for a quick fix for high memory usage, turn down your texture quality.

[Wjolcz]

That is really nice to know. I was starting to worry about all those new parts we get every update, but that doesn't seem like a peoblem.

[monstah]

I have always been worried, too. Time to swallow the pride and play on uglier settings.

[Stone Blue]

Excellent experiment and write-up...

What about trying a fresh, default install, and now removing JUST the IVA folder/files (now Spaces folder)?... Or did you do that along with the part files and just not mention it?...

I'm thinking it also probably wont make much difference, but I know in pre-0.90, the Squad IVA folder alone was HUGE, somewhere between 250-400MB depending on KSP version...I never used the IVAs, so I always deleted the folder, and it seemed to help quite a bit...

Squad has definately improved THAT situation since 0.25, but I'm still wondering if there's any performance gain from removing the stock IVAs, and if the game is still playable without them...(I know it was pre-0.90)...

Do you have any ideas on this, sal_vager?

[sal_vager]

Yep, every .dds texture outside the asset files were nuked, not just those in Parts, and everything but the Mk1-2 IVA in the later test.

Also, folder size has virtually no correlation to the size loaded into memory

But the problem with fully removing IVA's is that if you do that you can't load any pod or cockpit that uses them.

[Stone Blue]

Yep, every .dds texture outside the asset files were nuked, not just those in Parts, and everything but the Mk1-2 IVA in the later test.

OK...that answers THAT question then...

Also, folder size has virtually no correlation to the size loaded into memory

I realise its certainly not a DIRECT correlation, but I've got to think that just having to load all the extra files alone, WILL certainly use more RAM than if they werent loaded at all, and also cutting down on loading times abit...

But the problem with fully removing IVA's is that if you do that you can't load any pod or cockpit that uses them.

Hmmm... I certainly dont recall having that problem in pre-0.90... But then again, i started using the ALCOR, RPM, and several other IVA mods in 0.90, and many re-used or reqd. the stock files, so i quit deleting the Squad files...Maybe stock now handles IVAs differently, where they ARE required to even load parts that have them??...

Oh well, still good stuff to know, thanx!

[sal_vager]

Heh, I knew I should have not done this from memory, you can get this far...

But that's it.

Poor Bob can't see!

Edited October 27, 2015 by sal_vager

[NathanKell]

If you remove Spaces and then also remove all INTERNAL nodes in part cfgs, that makes it work fine.

Great work sal!

(Quick note: DDS size on disk actually is proportional to memory usage. because there's no conversion process once the texture is imported. It's not *equivalent* to memory usage per se, because of how the shadowing bug works, but it's at least linearly proportional.)

[CaptainTurbomuffin]

If you have a closer look at the grass for example, you can see how detailed it is. Now, think about flying in the atmosphere. That same detailed texture is being used on the terrain. Even if you can see very far away, every square-meter of the landscape is covered with a rather detailed texture. More levels of texture detail should be used. More details from the suggestion in my signature. Am I wrong?

[klgraham1013]

Hmm. Time to figure out why my game runs at 4.5 gigs. I certainly don't have 7,000 to 8,000 mod textures. It seems there's more than meets the eye.

I wonder if it's as simple as converting my older texture replacer files to dds.

[nothingSpecial]

Uhm.

As someone who has a lots of trouble with memory, I'd like to say that GameData textures are not all the textures included in the game.

And Texture Quality setting affects other textures as well, starting with the main menu.

And I really suppose that I could run much better with beautiful ships if there were option to drop all the textures quality except parts ones.

[sal_vager]

More levels of texture detail should be used.

Separate quality settings for non-part textures would be nice yes.

I wonder if it's as simple as converting my older texture replacer files to dds.

Try it, it would be interesting to know how it turns out.

Uhm.

As someone who has a lots of trouble with memory, I'd like to say that GameData textures are not all the textures included in the game.

Yep, that's why I looked for .dds files in the KSP root directory, since 1.0.x all the stock textures are .dds now, but the ones in the asset folders aren't so easily deleted.

Edit:

.assets archive files I should say.

~/ksp/ksp_notextures/KSP_Data$ ls

level0 level15 level7 Resources sharedassets13.assets sharedassets3.assets sharedassets9.assets.resS

level1 level16 level8 resources.assets sharedassets14.assets sharedassets4.assets

level10 level2 level9 sharedassets0.assets sharedassets15.assets sharedassets5.assets

level11 level3 mainData sharedassets0.assets.resS sharedassets16.assets sharedassets6.assets

level12 level4 Managed sharedassets10.assets sharedassets17.assets sharedassets7.assets

level13 level5 Mono sharedassets11.assets sharedassets1.assets sharedassets8.assets

level14 level6 Plugins sharedassets12.assets sharedassets2.assets sharedassets9.assets

Edited October 27, 2015 by sal_vager

[nothingSpecial]

ah, okay then. I only now recalled that "asset" is also Unity internal designation.

[BudgetHedgehog]

the ones in the asset folders aren't so easily deleted.

And all are .dds as well anyway, IIRC.

Anyway, good writeup, it's nice to have some solid evidence about RAM usage and parts etc.

[jwbrase]

I realise its certainly not a DIRECT correlation, but I've got to think that just having to load all the extra files alone, WILL certainly use more RAM than if they werent loaded at all, and also cutting down on loading times abit...

Not necessarily, because depending on your operating system and what system calls were used to load the files, and what the program does with the files immediately after loading them, "loading" does not always mean "moving immediately into RAM". Modern operating systems will often tell a program that they've loaded a file but not actually load it until the program actually uses it (and even then, they will only load the parts of the file that the program actually uses).

[Red Iron Crown]

Not necessarily, because depending on your operating system and what system calls were used to load the files, and what the program does with the files immediately after loading them, "loading" does not always mean "moving immediately into RAM". Modern operating systems will often tell a program that they've loaded a file but not actually load it until the program actually uses it (and even then, they will only load the parts of the file that the program actually uses).

Doesn't matter, it's still using up address space which is what is limiting in 32-bit.

[jwbrase]

Doesn't matter, it's still using up address space which is what is limiting in 32-bit.

That all depends on your hardware. As I read it, the OP is discussing low memory situations in general, including cases where the amount of physical RAM is less than the 32-bit limit.

[Red Iron Crown]

That all depends on your hardware. As I read it, the OP is discussing low memory situations in general, including cases where the amount of physical RAM is less than the 32-bit limit.

Also doesn't matter, as it will still have the same address space available. As long as they haven't done something silly like disabling the pagefile it will be just as stable as it would be with more RAM, just slower.

[juanml82]

Interesting. I've just tried to install KSP to an atom based windows tablet with 2gb of ram, with low res options and stock KSP (up to the VAB) runs perfectly fine. I doubt I can make a 100+ ship and enjoy decent frame rates with that machine, but the device is below minimum system requisites.

A problem is that the VAB isn't designed for touch, so even without adding mods, it doesn't work as a VAB-lite, so to speak, to design ships on the go. And I came to rely a lot in MRS and SpaceY to make ships, so I don't 2gb of ram will be enough for my play style, even if I only use it as a VAB lite.

[r_rolo1]

Interesting I already had noticed that the textures ate a lot of RAM ( more even if you add some beautification mods ), but I haven't thought that it would make such a diference.

Anyway, what I have noticed is that KSP , while having pretty much squashed the memory leaks it had in the 0.90->1.0 transition and that it has a much reduced RAM footprint nowadays ... as long as you don't change scenes. In changing scenes, I've seen the RAM usage to spike up to the double of the current RAM base load in my system ( even when I was going stock ) and I'm pretty sure half of the issues people have with RAM nowadays are because of that ( I currently have a base load of around 2 Gigs due to some light modding, but that is enough to MAF out sometimes when loading the VAB/SPH, for a example ). It woul be niceif there was a mod for that, but AFAIK there isn't ... :/

[Jimbodiah]

I received this tip from MalevolentNinja:

With almost every mod using compressed textures natively, as well as the stock game having been converted over to compressed textures, you shouldn't be using ATM any more.

The real issue is a Unity (the game engine) bug with DirectX 9 that keeps a copy of textures in system memory in addition to offloading them to the video card. The best workarounds we have today are to force OpenGL or DirectX 11 via the command line switches -force-opengl or -force-d3d11 (use one or the other, not both). They both have their drawbacks (for example no shadows in OpenGL, messed up fonts with DirectX 11) but they both work well at reducing the memory footprint to a reasonable level, even with a lot of mods installed. I recommend you try both and see which works best on your system.

It reduced the memory load about 25-30% for me.

And about running in low texture quality... yeah, so 1980s

[Talavar]

Actually, its not really the game assets causing the problem.. it's either a lack of memory scrubbing, or a memory leak is hiding in there... somewhere...... With the Kraken... The thing is, the longer you play, the larger the games memory footprint becomes. This generally means that entities aren't being zero'd out of memory correctly.. It could be a problem with unity storign the textures, but looks more like a leak to me, since the memory usage keeps increasing. I use the 64 bit hack with only mechjeb installed (Coffee game setup), and I sometimes get to 6 Gigs usage before restarting to lower it.

Edited November 14, 2015 by Talavar