Someone please explain this RAM limitation to me.

[Dafni]

Okay gang, I am no big computer wizard, but it's pretty clear to me that I must be missing something here.

I keep hearing all this Unity5 and 32/64bit talk, but it doesn't add up for me. I was never into PC games before KSP, and at first I used my sons Steam and computer to play it. Then I went out and got myself a pretty decent gamer laptop, reading up on all the stats, requirements etc... waiting for the full 1.0 release to make sure most bugs are gone and it is optimized.

Now it seems KSP is only using a fraction of my resources and still crashes after app 90mins. I learned a bit about the reason for this on here but it is still a mystery to me WHY this needs to be like this.

My son plays games that look much more graphics heavy, much quicker scene changes, live online against people on different continents and all that, and still he never gets crashes or FPS drops like I do. His computer is considerably slower too. So what's the story here?

If it really is a well known and transparent problem, why wasn't it fixed in all those months of development? I even hear people saying in 0.9 they were able to launch bigger rockets, and I believe this. I seem to get low FPS on anything with more than 180 or 200 parts, add a service bay and it's more like 50 parts.

I love the game and I will keep playing it, of course, but I got to say those crashes to desktop spoil the fun for me sometimes, it just breaks the immersion. I would just like to know why it must be like that and why nothing has been done about these infamous memory leaks.

 

Thanks for reading

-Daf

Edited December 3, 2015 by Dafni

[Mischief]

Simple version -

 

32bit only lets you use 4gb of ram on your computer. As more stuff gets added to the base game it uses more of that 4gb. More than a handful of serious mods (some are big enough to chew the rest of that up alone) will chew up the rest of that and give you out of memory errors. Take me for example. I've effectively quit playing until the 1.1 update is out because with the mods I want to use I can't stay under 4gb and still play without crashing.

 

64bit means you don't have the 4gb limit. It means you can load KSP, new texture files, one of the expanded system mods, the whole USI suite of mods, Infernal Robotics and whatever else you want and if you've got a 8-16gb system you're going to be fine. On your system you're likely running the graphics on the motherboard; that means that the graphics are chewing up that 4gb limit as well (it's a bit more complex than that but that's close enough) where as someone with an independent graphics card may be running 2-4gb of graphics content offloaded to their video card. That's the super-simple version. Also, physics calculations. The more you're doing the more memory it uses.

 

It's the biggest deal for the game. More important than anything else coming with the new Unity because it fully unlocks the massive list of mods you can (and probably should) be playing the game with, freeing up players to really play this massive sandbox however they want.

Edited December 3, 2015 by Mischief

[Kobymaru]

36 minutes ago, Dafni said:

Okay gang, I am no big computer wizard, but it's pretty clear to me that I must be missing something here.

I keep hearing all this Unity5 and 32/64bit talk, but it doesn't add up for me. I was never into PC games before KSP, and at first I used my sons Steam and computer to play it. Then I went out and got myself a pretty decent gamer laptop, reading up on all the stats, requirements etc... waiting for the full 1.0 release to make sure most bugs are gone and it is optimized.

Now it seems KSP is only using a fraction of my resources and still crashes after app 90mins. I learned a bit about the reason for this on here but it is still a mystery to me WHY this needs to be like this.

About RAM:
Older computers had Processors with registers that were 32 bits wide. That also means, that all addresses can have 32 bits at most. 2^32 = 4294967295 bytes, which are 4 Gigabytes exactly. So 32-bit programs can have at most 4GB of memory. In windows, the OS shaves off another Gigabyte for the OS address space, so you are left with 3GB only.

64-bit programs can handle much more. I'm not going to do the math, but let's just say it will be enough for the ram of a few decades to come.

Quote

My son plays games that look much more graphics heavy, much quicker scene changes, live online against people on different continents and all that, and still he never gets crashes or FPS drops like I do. His computer is considerably slower too. So what's the story here?

The crashes come from the memory limit, see above.
Now why your sons slower computer seems faster is because those games use the graphics card a lot, because the "hardest" thing that they do is the graphics. Usually, their physics aren't that pronounced or not there at all.

 

Quote

If it really is a well known and transparent problem, why wasn't it fixed in all those months of development? I even hear people saying in 0.9 they were able to launch bigger rockets, and I believe this. I seem to get low FPS on anything with more than 180 or 200 parts, add a service bay and it's more like 50 parts.

That's because of the way how those parts are handled. In your sons game, you might have "a few hundred" things on the screen, but they are not fully-fledged physics objects, and they (mostly) don't interact.

In a 200 part ship, you have many, many interactions between those parts. They can collide, they convay torque, they can convey force. This is all done on *one* core only, because it's hard to program for multiple cores. Unity 5 uses a new PhysX version, which a) performs better in general and b ) can use multiple cores in some situations. That should bring *some* alleviation, but you can't get rid of the principal problem of the many interactions of 200 parts.

 

Quote

I love the game and I will keep playing it, of course, but I got to say those crashes to desktop spoil the fun for me sometimes, it just breaks the immersion. I would just like to know why it must be like that and why nothing has been done about these infamous memory leaks.

I fully understand that, and I agree. I hope they fix it as well. But I know some software development, and I know these things are hard, because:

- You can't consider mods, because then you would have to deal with code you don't have control over that's often not well written or very different from your style. Since most memory problems get worse only with mods, the devs probably don't even SEE the crashes, because on their systems, in stock, it works fine most of the time. Bugs where the program works "most of the time" are the absolute worst, because it's often super hard to reproduce them.

- KSP is written in C#, which is a "managed" language, that means that all Objects that you allocate get Garbage Collected. *IN THEORY*. In practice, the previous sentence is not much more than a marketing gag, and what it actually does is make your memory management really really hard to debug, because you might end up with dangling references left and right. That's why debugging memory problems is really, really hard.

- People care about content. Eventually, they get pissed if the game crashes, but if they can keep playing most of the time they are happy. The devs know that, so they prioritize new Content (more or less linear development, with little surprises) over heavy bugfixing (very hard, time consuming and might even turn out to be impossible in some cases).
Also, they have known that they will revamp some parts of the game (like the UI, the orbit lines...) for quite some time. It makes no sense to put in efforts into something that might "fix itself" in a few months.

Now that should not be an excuse. I still hope that with the added magicians such as NathanKell and the UI-Rewrite, they can fix up some of the longstanding bugs.

Edited December 3, 2015 by Kobymaru

[Daid]

I also recommend running: http://www.memtest.org/ I've seen a few times when people where complaining hard time that software was buggy as [SUNNY DAYS]. And in the end it was that their hardware (RAM) was broken, and thus was causing all the errors.

 

 

And, as a word on the "3GB" limit. KSP will crash before it will show 3GB usage in Windows. Windows memory usage display is always a bit odd. And the 3GB is a "virtual address space" limitation. But tough to explain, but it is running out of locations where memory "could be". Does not mean there is memory everywhere, just that it's reserved to be there. Add into the mix that files can also be "mapped", which causes them to take up space in the "virtual address space" the whole thing starts to become more complex. (And I think, not sure, that 3D drivers are also great at using up address space without telling you).

But note that I never have issues on my machine. I'm running all graphic settings at the lower possible values, so that might be helping me. (PC is fine, just the Intel graphics card that is slow)

Edited December 3, 2015 by sal_vager
Profanity.

[Dafni]

Thanks for the replies so far, I will carefully go through them when my brain is fully awake.

Just for completeness, I run a Lenovo Y50 with 16GB of RAM and a Nvidia GeForce GTX 860M GPU with 4GB of VRAM. The SSD is in good shape too, nothing but KSP installed (well, next to the obvious browser and Win8 stuff)

Thanks again, I'll get back to you later. I appreciate your time

[Kobymaru]

15 minutes ago, Dafni said:

Just for completeness, I run a Lenovo Y50 with 16GB of RAM and a Nvidia GeForce GTX 860M GPU with 4GB of VRAM. The SSD is in good shape too, nothing but KSP installed (well, next to the obvious browser and Win8 stuff)

Yeah, it's definitely not your machine. And at the moment, there is nothing you could do on your end to improve KSP performance, short of overclocking your CPU to 7 GHz and cool with liquid nitrogen.

Edited December 3, 2015 by Kobymaru

[nuclearping]

You can check how much RAM your KSP consumes when opening your Windows Task Manager (Ctrl+Shift+Esc), then switch to "Details" tab and search "KSP.exe". If you don't have any mods installed, it shouldn't be more than 1,5-2GB.

Personally, until they release a proper 64bit version, I ended up installing Ubuntu Linux and reboot my PC if I want to play KSP.

Linux doesn't have the 32bit / 64bit problems like Windows does. So in Linux KSP naturally runs in 64bit mode and therefore can address more than 4GB RAM.

However, it still crashes in Linux now and then, especially when doing heavy scene-switching. Like when you test new ships, reverting back and forth to VAB / Launch, switching ships, etc. But this doesn't have anything to do with memory limits.

A version "1.0" also doesn't mean it is bug-free.

[Dafni]

6 minutes ago, Kobymaru said:

Yeah, it's definitely not your machine. And at the moment, there is nothing you could do on your end to improve KSP performance, short of overclocking your CPU to 7 GHz and cool with liquid nitrogen.

Thank you. Maybe I can squeeze some more playtime out of it when I go back to complete stock. Running EVE and Planetary Bases at the moment.

Hey, my job gives me not only access to liquid N2, but liquid Helium too!!.... ooohhh, the possibilities

thanks

Edited December 3, 2015 by Dafni

[Hannu2]

Squad is a small developing team. They do not have resources of large game companies. It seems also that KSP has become more complicated and more popular than they expected. Tools they selected at beginning are not very well suitable for project of this size but they do not have resources to program a new engine and port everything to it. In spite of that I would say that quality of KSP is on the same level which is typical for popular products of large companies. They can also been buggy and unfinished before first updates.

[selfish_meme]

If you dual booted that machine with Linux you could run a very decent 64bit game with all the visual and parts mods you like. Currently the linux version (using intel & nvidia hardware) is what 1.1 will be like for Windows users, relatively crash free and heavily moddable.

[Dafni]

48 minutes ago, selfish_meme said:

If you dual booted that machine with Linux you could run a very decent 64bit game with all the visual and parts mods you like. Currently the linux version (using intel & nvidia hardware) is what 1.1 will be like for Windows users, relatively crash free and heavily moddable.

Hmm, good point. Now you have me thinking. I was an Ubuntu user for a while and really loved it. Now I got that laptop with Win8.1 pre-installed and I thought well, for as long as it runs smooth I keep it. But this sounds actually very tempting now. Thank you for this information.

[theend3r]

Another reason why other games run better even when RAM heavy is that they don't keep everything in their memory but just the things they need.

For some incomprehensible reason, KSP keeps the textures and models of all the parts in its memory even when you only use 10% of them in your flight scene. There was a mod that partially soved this but it's a problem that needs to be solved by Squad for it to work well.

[wizzlebippi]

For the record, a win32 application by default can only utilize 2GB of RAM.  A long mode enabled application can use 3GB.  

[Padishar]

15 minutes ago, wizzlebippi said:

For the record, a win32 application by default can only utilize 2GB of RAM.  A long mode enabled application can use 3GB.  

And a "large address aware" Win32 process running on Win64 can use just short of 4GB.  However, all these quoted amounts also include all the exe and dll files (including windows dlls) used by the process (they have to be mapped into the address space to be runnable) so the amount of memory the program can allocate is considerably less than the address space limit.

Edited December 3, 2015 by Padishar

[radonek]

49 minutes ago, theend3r said:

For some incomprehensible reason, KSP keeps the textures and models of all the parts in its memory even when you only use 10% of them in your flight scene. 

Quite comprehensible to me - you'd need to fetch all of them every time you enter VAB/SPH. That wouldn't be very smart. Shooter games can do such tricks because their scene rendering is limited by what player can see and they can (and have to) design scenes to keep number of texures down. As for vehicle parts, complete preloading is a good strategy. Now, if Squad guys were to implement something visibility-limited, like more detailed planet surfaces, then they could use this technique. And event that only to a degree, since you'd still have pretty large scenes while flying above surface. This is why open-world games are less detailed then corridor shooters.

[theend3r]

3 minutes ago, radonek said:

Quite comprehensible to me - you'd need to fetch all of them every time you enter VAB/SPH. That wouldn't be very smart. Shooter games can do such tricks because their scene rendering is limited by what player can see and they can (and have to) design scenes to keep number of texures down. As for vehicle parts, complete preloading is a good strategy. Now, if Squad guys were to implement something visibility-limited, like more detailed planet surfaces, then they could use this technique. And event that only to a degree, since you'd still have pretty large scenes while flying above surface. This is why open-world games are less detailed then corridor shooters.

You could just load low-poly models and texture thumbnails for VAB and load the real ones on demand. Loading a single model+texture would take like 1s.

[Engineer of Stuff]

This topic will be good to anyone else who has this problem.

[Robotengineer]

2 hours ago, Dafni said:

Hmm, good point. Now you have me thinking. I was an Ubuntu user for a while and really loved it. Now I got that laptop with Win8.1 pre-installed and I thought well, for as long as it runs smooth I keep it. But this sounds actually very tempting now. Thank you for this information.

I highly recommend Linux Mint if you are going to dual boot. I ran a Linux Mint partition for KSP x64 for about a year before I lost it to hard drive partition issues (thanks Apple!). With all the mods your RAM can take KSP almost becomes a completely different game.

[Death Engineering]

I run a modest list of mods but have resolved the memory issue/crashing with the -force-opengl switch on the shortcut. Stock KSP went from about 1.5GB to just over 1.0GB. To preserve graphic quality, I had to override KSP.EXE in the video driver to have anti-aliasing (looked awful without that).

My Lenovo is very similar to your Lenovo.

[Foxster]

Basically, it is lazy programming. 

4GB of memory is huge! Having 100s of GB of disk space is huge too. But having all these resources available makes game engine designers and game programmers lazy. Instead of optimizing their code like we used to do when there was 1MB of memory available on a PC, they just ignore resource limitations, pull everything into memory and optimize nothing. 

 

[radonek]

17 minutes ago, theend3r said:

You could just load low-poly models and texture thumbnails for VAB and load the real ones on demand. Loading a single model+texture would take like 1s.

That is most basic LOD work and wont do. For example, you build you ship (cursing texture-loading lag all the time) and as you add parts, you eat the memory again (plus memory for low-lod models) until editor wont let you add any more. So you design ship carefully to keep texture memory down, launch it it, all is fine. Then you design another, add some parts, remove others to keep memory down. What will happen when those two ships meet? OOM kaboom.  You'd be limited not only by your current design, but by parts on all ships in game.  So, you end with a game that lets you add many mods, but wont let you actually use them anyway - not that much different from what we have now. 

[jm764]

41 minutes ago, Foxster said:

5 hours ago, Dafni said:

Thanks for the replies so far, I will carefully go through them when my brain is fully awake.

Just for completeness, I run a Lenovo Y50 with 16GB of RAM and a Nvidia GeForce GTX 860M GPU with 4GB of VRAM. The SSD is in good shape too, nothing but KSP installed (well, next to the obvious browser and Win8 stuff)

Thanks again, I'll get back to you later. I appreciate your time

Basically, it is lazy programming. 

4GB of memory is huge! Having 100s of GB of disk space is huge too. But having all these resources available makes game engine designers and game programmers lazy. Instead of optimizing their code like we used to do when there was 1MB of memory available on a PC, they just ignore resource limitations, pull everything into memory and optimize nothing. 

 

@FoxsterActually KSP is very well optimized for the base game. The reason why it uses soo much ram is for the fact that the ENTIRE game is loaded into ram to reduce loading times and make the game render "seamlessly" without more loading screens between large loading changes.

 

@DafniThe reason STOCK KSP tends to crash is mostly due to things you add to it like mods, old hardware, or driver issues. While the E.V.E. and Planetary Bases mods use a fair amount of ram, they're normally not enough to crash the games them selves.

 

For the record I use ALOT of mods on my KSP install and run/play on my own KSP Multiplayer Server and it runs very well on all my PCs. I've even gotten the modded version of my game to run on an old AMD Athlon 64x2 with only 4gb of ram at a decent rate on lower settings.

Edited December 3, 2015 by jm764
Pressed post by accident.

[Foxster]

1 minute ago, jm764 said:

@FoxsterActually KSP is very well optimized for the base game.

No. It simply is not. If you write a programme that takes 2 or 3GB of memory then you are not optimizing at all. 

Do you code professionally? I do. We make systems that will work on minimum spec hardware and so have to work hard at optimizing. If one of our products (which are not less complicated than KSP) took 2GB of RAM then we simply would not have a product that could be sold. 

 

[Kerbart]

34 minutes ago, radonek said:

That is most basic LOD work and wont do. For example, you build you ship (cursing texture-loading lag all the time) and as you add parts, you eat the memory again (plus memory for low-lod models) until editor wont let you add any more. So you design ship carefully to keep texture memory down, launch it it, all is fine. Then you design another, add some parts, remove others to keep memory down. What will happen when those two ships meet? OOM kaboom.  You'd be limited not only by your current design, but by parts on all ships in game.  So, you end with a game that lets you add many mods, but wont let you actually use them anyway - not that much different from what we have now. 

I'm not sure if it's that easy to dismiss. First of all there's a solid chunk of textures in use for the planets. Now obviously there's only one body you can orbit at the same time (yes, it could be a moon and you'd need a texture for the planet of that moon, but that can be a microtexture again) so that'd leave out a good chunk of textures, especially when people replace the stock textures with their own hi-res ones.

Second, yes, theoretically that scenario would arise, but only if you're using practically every part available between the two ships. The memory problems arise because people are using dozens of part mods, each with textures that eat up precious memory. I doubt you'd be using practically every part from every mod. Proper texture management would mitigate a lot of these cases though. Not sure if you would build ships that would cause these problems that often.

The current approach is great when you're only using stock parts (I am, thus I have little problems), as there is no stuttering in the game due to loading textures. And it's understandable as the underlying architecture was probably designed when this wasn't an issue. But it is now, for a lot of players, and Squad needs to address it at one point. 64 bit is only a temporary solution as people will now invariably load hundreds of mods and simply run out of real memory (not just addressable memory).

 

[theend3r]

52 minutes ago, radonek said:

That is most basic LOD work and wont do. For example, you build you ship (cursing texture-loading lag all the time) and as you add parts, you eat the memory again (plus memory for low-lod models) until editor wont let you add any more. So you design ship carefully to keep texture memory down, launch it it, all is fine. Then you design another, add some parts, remove others to keep memory down. What will happen when those two ships meet? OOM kaboom.  You'd be limited not only by your current design, but by parts on all ships in game.  So, you end with a game that lets you add many mods, but wont let you actually use them anyway - not that much different from what we have now. 

There'd be no lag while building, the model would start out in its reduced form and load while you place it.

When two ships meet, their parts load. Increasing the 3 second lag you get when another ship loads to 5 seconds is something I'd frefer over a crazy memory usage.