Jump to content

What is this memory consumption issue in KSP?


Recommended Posts

Hi, I see it over and over, many texture reduction packs and a mod (ATM) to do so too.

What is this thing with KSP x86 memory that x64 should not suffer from?

How is texture size reduction supposed to help? Honestly textures are supposed to be loaded on the GPU VRAM when needed not in RAM.

So lets say I have 100 mods with crazy 8K textures, only those that are needed will load, not all 1000 parts with all their 8K textures right? But only those 10 parts that are actually used.

Or did developers did not get this and somehow textures are loaded all all the time even when not needed? Why all the loading scenes then?

And the RAM is not maxing out because of textures but because of game objects or other game mechanics?

I'm about to install many mods so I would like to know what the limits of KSP are and why.

Most games are fine even with huge textures with 3GB VRAM, is KSP kicking above that with mods?

x86 version is limited to roughly 4GB RAM because of ancient times but I doubt those parts descriptions and game mechanics are eating that much RAM.

Why are most if not all mod textures in PNG? Aren't textures better done compressed to a format the GPUs support like DDS DXT5 with mipmaps?

Unity/KSP has to convert the textures every time the game loads? *faint*

Ok, looking at SQAUD textures, MBM, looks to be PBM, you gotta be kidding. Yes maybe "cross-platform" but uncompressed and all, basically RAW. No wonder it takes so long for even stock KSP to load, they must be converting all the textures...

Does SQUAD plan to move to use more sensible texture formats and management?

I don't mean to ditch it but this is the worst texture management I have seen so far probably *shrug*

Edited by JackCY
Link to comment
Share on other sites

Indeed KSP does seem to load basically everything all the time. For example in the VAB/SPH, when you see the little icons and rotating views of the parts in the parts list, that's not a separate image, it's the actual model loaded and rendered there.

I expect Squad will look at these things at some point, but don't hold your breath. After the Unity 5 port with any luck; optimising before that's out would probably be wasted effort, especially as for stock players there's not really any negative impact.

And I don't know how much of it is Unity's fault and how much is Squad's.

Link to comment
Share on other sites

Yeah I've heard the 64bit can be buggy and that there might be some clash with mods that are not 64bit? Who knows how well they will support it.

I'm more interested in the rest, why is it RAM limited when it rather should be VRAM limited. I guess they dumbly convert and load all textures of all objects on game start up into RAM? That is just insane.

Link to comment
Share on other sites

Offcourse the 64bit version will become suported by mods, once Squad manages to get it stable. Noone wants a stable 64bit version more than mod users and builders. But so far, yes, you're better off using 32 bit with a little les mods

About the Ram: This is the way they deceided.

The alternative is that the textures would only load when you need them, which would mean a delay every time you click on a part in the VAB (rather than the streamlined thing we have now) and, even worse, a much larger loading time each and every time you switch to a different craft (and a loading time when you come close to another craft).

And if you'd suddenly overshoot the limit that way, your game would simply crash. I'd rather just have the game tell me that it can't start, than crash when I try to dock

Link to comment
Share on other sites

I'm more interested in the rest, why is it RAM limited when it rather should be VRAM limited. I guess they dumbly convert and load all textures of all objects on game start up into RAM? That is just insane.

Because if all the models and textures are not preloaded and hold in "RAM" it will be a pretty big lagfest when you're trying to build something in SPH / VAB, because everything would be loaded on demand from your hard disk, which is like 30-50 times slower than your RAM, even if you got an SSD.

Link to comment
Share on other sites

Indeed KSP does seem to load basically everything all the time. For example in the VAB/SPH, when you see the little icons and rotating views of the parts in the parts list, that's not a separate image, it's the actual model loaded and rendered there.

I expect Squad will look at these things at some point, but don't hold your breath. After the Unity 5 port with any luck; optimising before that's out would probably be wasted effort, especially as for stock players there's not really any negative impact.

And I don't know how much of it is Unity's fault and how much is Squad's.

Many pc games does this, pretty noticeable in Sims2 and 3, if you have plenty of mods. Might be part of the reason why the areas in sims4 is so small (simplest way and its an EA game)

For KSP its part squad fault and part unity, unity might load all by default and ksp do this.

Note that optimization is usually done late in development as so major changes tend to wreck it. Also an good chance unity 5 will have better memory optimization build in.

Most other games are cross platform and ps3 and 360 only have 512 mb total so they have to put memory optimizing as an very high priority, this make running them on a pc a joke even with better textures.

Link to comment
Share on other sites

In short, the way textures are handled in KSP right now is totally terrible. Everything gets loaded into RAM at startup, meaning you run out of it quickly and loading takes forever. I use LoadOnDemand to avoid those issues. My load times are much faster as a result and I don't hit the memory limit pretty much ever. Occasionally there's a bit of a "slowdown" when tabbing quickly through pages in the editor, but not enough that it's annoying. It does take a moment for the full textures to load, but until then there's lower res textures in place and I don't mind waiting a second for everything to be "pretty" when I can still fully interact with the parts.

If a mod can do it without access to the actual source, I see no reason why Squad couldn't do it in the future. I'm sure they will, but for now we're stuck with a stupid but simple method of loading textures.

Edit: I'd also like to point out that there's no additional delay when loading ships. Physics and parts get loaded as normal, the textures load as quickly as they can and apply, but everything is functional even without fully loaded textures.

Edited by magico13
Link to comment
Share on other sites

I'd argue that it's probably better than a delay when playing when the craft you're piloting approaches one whose textures aren't loaded yet. I'd also point out that it's version 0.24. :)

This is a misconception mostly, you can preload textures and objects in advance based on distance from you. Very simple, very effective. Loads in background in parallel, you won't even notice something is loading.

The alternative is that the textures would only load when you need them, which would mean a delay every time you click on a part in the VAB (rather than the streamlined thing we have now) and, even worse, a much larger loading time each and every time you switch to a different craft (and a loading time when you come close to another craft).

And if you'd suddenly overshoot the limit that way, your game would simply crash. I'd rather just have the game tell me that it can't start, than crash when I try to dock

There's already a delay when another vessel comes into physics range, sometimes a noticeable one. But the VAB/SPH is the bigger issue - there, you need all the parts "on hand".
Because if all the models and textures are not preloaded and hold in "RAM" it will be a pretty big lagfest when you're trying to build something in SPH / VAB, because everything would be loaded on demand from your hard disk, which is like 30-50 times slower than your RAM, even if you got an SSD.

I've never had a game crash because it ran out of VRAM honestly. And I've had so far GPUs with very little RAM and power, true I didn't play heavily moded Skyrim or something.

There doesn't have to be a delay in VAB/SPH, there is probably still a delay when clicking between pages so the textures can preload, simple as that, no delay. Either hold only current page or preload previous and next possible pages etc. there are ways to circumvent the delays of loading and loading everything on start is the simplest and dumbest way to do it.

Vesels can be preloaded too.

Even if it would preload all your vessels or something you probably don't use all 1000 textures but maybe 100 hundred.

In short, the way textures are handled in KSP right now is totally terrible. Everything gets loaded into RAM at startup, meaning you run out of it quickly and loading takes forever. I use LoadOnDemand to avoid those issues. My load times are much faster as a result and I don't hit the memory limit pretty much ever. Occasionally there's a bit of a "slowdown" when tabbing quickly through pages in the editor, but not enough that it's annoying. It does take a moment for the full textures to load, but until then there's lower res textures in place and I don't mind waiting a second for everything to be "pretty" when I can still fully interact with the parts.

If a mod can do it without access to the actual source, I see no reason why Squad couldn't do it in the future. I'm sure they will, but for now we're stuck with a stupid but simple method of loading textures.

Edit: I'd also like to point out that there's no additional delay when loading ships. Physics and parts get loaded as normal, the textures load as quickly as they can and apply, but everything is functional even without fully loaded textures.

Thanks, I will look at this mod :)

Too bad there is not a better explanation of Active Texture Management and what it truly does with the textures and what it supposedly manages, is it only a convertor that will convert the loaded textures to DXT? Or does it manage the textures and loads on demand? There is some talk of caching and preloading.

The LoadOnDemand is from someone who advised on preloading for ATM.

Can these two plugins work together?

If this can be fixed via mods that would be sufficient for now I think. I'm surprised though that they let mods intercept texture management of the game.

I'm tempted to convert all textures to DXT, not on load, on the drive, change the model's links to these textures and have them load on demand :D

PBM for textures *shakes head* I still can't believe it.

Edited by JackCY
Link to comment
Share on other sites

there are ways to circumvent the delays of loading and loading everything on start is the simplest and dumbest way to do it.

Vesels can be preloaded too.

Therein lies the rub. The current solution was simple to code and is good enough. It doesn't affect the stock game beyond a mildly annoying startup time, only causing problems for combinations of certain mods. As such while there's no shortage of cleverer ways things could be done to reduce RAM usage, coding and testing one of those cleverer ways rightly ranks as low priority for Squad. Fixing bugs that directly affect the gameplay and adding features to keep old customers happy and drive new sales are more important.
Link to comment
Share on other sites

Therein lies the rub. The current solution was simple to code and is good enough. It doesn't affect the stock game beyond a mildly annoying startup time, only causing problems for combinations of certain mods. As such while there's no shortage of cleverer ways things could be done to reduce RAM usage, coding and testing one of those cleverer ways rightly ranks as low priority for Squad. Fixing bugs that directly affect the gameplay and adding features to keep old customers happy and drive new sales are more important.

That would be great, if it wasn't for the fact that moddevs are adding content and features at about 10 times Squad rate, and their support is almost as good, if not at times better. That's why "reveals" are now centered around showing you 3 or 4 parts that have essentially been in the game since last year in the form of a mod which was just as good on it's own without "official" integration.

First contract was great, but nothing we hadn't seen before, mission control extended did this long before squad did it, though admittedly the squad integration was a bit "cleaner". So far the only truly innovating upgrade has been the 64 bit release, and we all know what a PITA that was and continues to be.

I'm not trying to take a dump on Squad here, what i'm saying is that mods are ALREADY adding new features and content to keep old customers happy... If only they could catch a frickin break from horrible texture management making the game crash to a miserable heap of error logs every time more than 4 parts are loaded at once. The reason why stock parts work in the first place is because they look like something that you typically find in a dumpster. No offence, but the stock art is just begging for an overhaul, and to absolutely noone's surprise that's exactly what it's getting.

I'd say Squad would be better served making sure that the modding community are getting all the support they need to keep pumping out content and features for KSP, because really, mods are doing it faster and better than Squad is.

Bottom line: the 32 bit memory limit is a painfully idiotic problem to have! Now OR in the future...

Edited by Blowout
Link to comment
Share on other sites

No offence, but the stock art is just begging for an overhaul, and to absolutely noone's surprise that's exactly what it's getting.

I'd say Squad would be better served making sure that the modding community are getting all the support they need to keep pumping out content and features for KSP, because really, mods are doing it faster and better than Squad is.

It seems that way. I don't get it either why many games do not buy the mods and integrate them into the game. It encourages modders to create quality content too and possibly get rewarded, hired, etc.

The stock KSP feels like an alpha release. Nothing seems finished apart from sandbox mode. The career modes change with updates and start to get built now.

Stock parts are limited and low resolution textures are just laziness, one can always setup how much detail to load from high res. textures so your machine can handle it, use mipmaps Squad.

I'll see how well ATM works and if the "Loading textures only as required" plugin will work.

At least KSP is open. A different game I made user stuff for is completely closed and you won't add a thing to it so most extra features are external apps and all game modding content is practically booted out of official forum since it's viewed as illegal.

Might see how this texture memory issue goes and if it could be fixed in KSP before Squad does it some years later. It's surprising that someone actually does such things in production.

Link to comment
Share on other sites

Bottom line: the 32 bit memory limit is a painfully idiotic problem to have! Now OR in the future...

Heres the thing. You are mistaken. This limit is NOT Squads fault. It is an issue with Unity.

Link to comment
Share on other sites

Heres the thing. You are mistaken. This limit is NOT Squads fault. It is an issue with Unity.

It's not even an issue with Unity, its a base-level operating system limitation. 32-bit operating systems are simply unable to address more than 4GB of RAM. Period. So don't blame squad, don't blame unity, and don't even blame the OS developers. It's just the limitation of the 32-bit architecture itself.

Link to comment
Share on other sites

It's not even an issue with Unity, its a base-level operating system limitation. 32-bit operating systems are simply unable to address more than 4GB of RAM. Period. So don't blame squad, don't blame unity, and don't even blame the OS developers. It's just the limitation of the 32-bit architecture itself.

I will concede you that point on 1 caveat and that is, he was calling it a stupid limitation to have. To me this says he finds fault with them for still having a 32 bit limit. I counter with, it's because of unity being used. I follow this with the fact win 64 bit, last I heard isn't stable, or as stable as 32 bit is. This is my conclusion only.

Link to comment
Share on other sites

I will concede you that point on 1 caveat and that is, he was calling it a stupid limitation to have. To me this says he finds fault with them for still having a 32 bit limit. I counter with, it's because of unity being used. I follow this with the fact win 64 bit, last I heard isn't stable, or as stable as 32 bit is. This is my conclusion only.

ah, I understand better now the point you were trying to make, and agree.

Link to comment
Share on other sites

Mods, you can close this so we don't continue complaining about something we all know now.

It is alpha it has many issues and despite it's wide release and sells it is still an alpha version that needs a lot of polishing even on core things.

For a game to need more than 4GB RAM, that's ridiculous. There are often 2-4GB VRAM, use it Squad. Unity probably didn't care as much to move to 64bit because most games don't need that much RAM in general, it's when you code silly that you need so much RAM.

Edited by JackCY
Link to comment
Share on other sites

It seems that way. I don't get it either why many games do not buy the mods and integrate them into the game. It encourages modders to create quality content too and possibly get rewarded, hired, etc.
Copyright and licensing issues mainly, coupled with a dose of "not invented here" syndrome. Squad themselves have favoured hiring/contracting to modders; the SP+ parts in .25 are changed a fair bit from those in the mod.
It's not even an issue with Unity, its a base-level operating system limitation. 32-bit operating systems are simply unable to address more than 4GB of RAM. Period.
32-bit desktop Windows can't use more than 4GB, a deliberate decision taken by Microsoft. 32-bit Linux, Windows server, and other OSes have supported something like 128 GB 64 GB since around the year 2000. It doesn't help KSP or any single program since there's still a per-process 4GB limit, but "32-bit can't use more than 4 GB" is a lie propagated by Microsoft* and it really hacks me off when I see it repeated. /rant

* I have MS-endorsed study materials that state it.

Edited by cantab
Link to comment
Share on other sites

32-bit desktop Windows can't use more than 4GB, a deliberate decision taken by Microsoft. 32-bit Linux, Windows server, and other OSes have supported something like 128 GB since around the year 2000. It doesn't help KSP or any single program since there's still a per-process 4GB limit, but "32-bit can't use more than 4 GB" is a lie propagated by Microsoft* and it really hacks me off when I see it repeated. /rant]

Indeed, even early versions of Windows XP (before SP2) were able to address more than 4GB of RAM, but it was disabled in later updates due to driver licensing issues and software incompatibilities. It's pretty trivial to design an instruction set that can get to areas of RAM that have addresses larger than the bit-width of the processor, and in fact that capability has been built into various home-market CPUs since 1995.

Link to comment
Share on other sites

Copyright and licensing issues mainly, coupled with a dose of "not invented here" syndrome. Squad themselves have favoured hiring/contracting to modders; the SP+ parts in .25 are changed a fair bit from those in the mod.

32-bit desktop Windows can't use more than 4GB, a deliberate decision taken by Microsoft. 32-bit Linux, Windows server, and other OSes have supported something like 128 GB since around the year 2000. It doesn't help KSP or any single program since there's still a per-process 4GB limit, but "32-bit can't use more than 4 GB" is a lie propagated by Microsoft* and it really hacks me off when I see it repeated. /rant

* I have MS-endorsed study materials that state it.

Then you should learn some architecture and then you will know why 32 bit can't use more that 4 GB. A RAM has address and data lines , if a processor ( 32 bit architecture ) have only address lines only for 4GB storage position , how you would expect to handle 6GB ram , where the processor has only x address lines and 6GB ram need X+y lines ?.

There is no point in force a 32 bit CPU to work with more that 4 GB ram with hard coded command set's , it will never get a 64 arh. build CPU power.

For that is called 64bit because those processors have more address / data lines. And even with 64bit you cant use 128GB ram i don't get from where you take that in 2000 it was able to handle 128GB RAM.

Link to comment
Share on other sites

This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...