Memory Swapping

[Daveroski]

Way too many years ago I used to be a programmer on z80 and 6502. Now I'm at the age where I probably have forgotton more than most people know. (Including where I put my glasses)

Anyway before I start to ramble about dead relatives and the weather, I'd like to pose a suggestion.

When we were programming, we had the age old problem of not enough RAM.

So we would swap memory in and out from the hard drive.

Imagine if you will a KSP which doesn't load any parts at all. Except for the catalogue of parts.

The catalogue will show you all the parts you currently have access to. The ones you have researched. It will even tell you if, and how many of a given part you have in memory.

These parts are not held in the computers memory until you call for one of them and put it on a ship.

Once it is in the real world (as virtual as it may be) then it is kept in memory. Until such a time as that part no longer exists as a part in in real world. Then its memory is usable by another part.

No matter how many of an individual part you would have in the world, you would only need one part file/texture etc. in memory at any one time. You wouldn't need to have every item from every installed mod in memory at once.

You may have to have an array for tweakables and current state of each occurrence of the part but the rest of your hundreds of parts are not loaded until needed.

When you install a mod, you would add it to the catalogue. The catalogue could be HUGE and accessible in game. You wouldn't even need to keep the whole catalogue in memory at the same time. Just the page you are looking at. This would open up options of really checking out a part before adding it to a ship. Full spec. sheet, 3D rotatable zoomable preview. Or just click and grab it and fit it as you do now. Nice to have options.

Now I'm curious... You see memory has been a problem since modding for the game began. (well, shortly thereafter) what I'm curious about is whether it's 8 bit, 16,32, or even 64 bit. the same techniques are still good.

Why haven't they been implemented?

[coolitic]

Dude, do you know how horrible gameplay would be? We would get major hitching and stuttering every time new parts come in and go.

The only solution is 64 bit, which will supposedly come with unity 5.

[sal_vager]

Much of KSP is actually Unity code, so the Squad developers are having to put up with the optimisations or lack thereof in that code, things such as graphics, I/O, sound, even the physics.

Also game assets are much larger than they were in the days of the Spectrum or Commodore, though I seem to remember many early home computers used cassette tape, small capacity floppy disks and if you were very lucky, erasable programmable ROMS.

This means that they had to hold everything in memory, and aggressive optimisation was a requirement not a luxury.

This doesn't mean that such methods are defunct today, but they are very different, in the past we'd use assembler, low level languages and many tricks to get everything into RAM, though the simplicity of the games back then helped, even Frontier, possibly one of the most ambitious games of the early IBM compatible PC's came on two floppy disks.

These days we'd have to use different methods such as generating game assets on the fly, CPU's are powerful enough now to allow this, we could load assets from the hard disk when needed but this would be and is slow for many games, which is why we see loading screens in other titles.

Everything has gotten bigger, this is the real problem, graphics are at higher resolutions, sound at higher bitrates, code is more complex and the hardware that runs it is so powerful that old school optimisations aren't needed as much.

[Kasuha]

Memory mapped files are feature which allows a 32-bit program to allocate/access way more storage than what fits into 32 bits address. If the system has enough RAM, it can even hold these files in file cache, enabling reasonably fast access to data stored this way. The problem is, it is a lot of work to do it well and if there is promise that Unity will become 64-bit eventually, it may be not worth the effort.

[Daveroski]

Parts would only change in memory in the Design section of the game. In that particular area the slowest part of the computer is the user.

In space all the parts for a ship would be in memory from launch and until such parts were no longer required in memory.

There would be no difference to the amount of hitching and stuttering than there is right now. In fact the opposite might well be true.

There is no such promise and memory swapping would solve the problem now.

[Daveroski]

Once the hard disk was in use, memory swapping came to the fore. Random access filing.

As for loading screens that wouldn't be a problem as parts would be loaded on a click by click basis and the user would barely be aware of the loading process.

I spent a lot of my time on computers a little bigger than the Spectrum. Such as the Kray xmp. We still used memory swapping in many applications.

Files are bigger. Storage is bigger computers are faster. It all evens out in the end.

[Nuke]

you could probibly stream in and out the higher resolution mip levels based on texture usage (preferably in another thread). after all textures are the lions share of memory usage, the rest of the data is negligible. you do have to keep everything in memory, so if you suddenly need to draw something, you can without needing to wait. but you dont need to keep it all in memory, just what you need when you have an unexpected need to draw something.

for example your ship might only use a tiny fraction of the parts available to you. so you can reduce the maps for those unused parts down to a really low mip level, like 64x64 so that you have memory for other tasks. now lets say you rendezvous with another ship with completely different set of parts. you dont have to wait for the engine to load any textures, just use the low res textures instead, and load the high res versions during approach, and once loaded you can start drawing those textures in full detail (which in most cases is going to be while the ship is still smaller than the target box) without a hickup while this happens.

[Daveroski]

Oh , by the way... "Kray" with a K was a pet name. Sometimes we called it the Kraken.

It only had 16 megs of memory but it was effectively 32 bit.

[Daveroski]

You clearly either didn't read what I wrote or didn't understand it.

[plausse]

My understanding of programming is limited. So am my walking skills, but my stumbling on the other hand... Is impeccable. So I happen to stumble across this:

http://forum.kerbalspaceprogram.com/threads/73236-WIP-Loading-textures-only-as-required

[Daveroski]

That's on the right track. But imagine if you will, using for example the B9 Mod or KW or many others with lots of parts.

Have you ever played a game and used ALL of the parts from a pack so that somewhere in kerbal space there was an instance of one of those parts?

Do you cherry pick parts from mods because of how much memory the parts that you will never or hardly use will take up?

Using the system I have in mind. Every part made will be accessible. Of course it will still be possible to use up all of the available memory by building lots of part intensive ships but you will have had so much more fun by the time that happens.

[Nuke]

orders of magnitude.

cfgs (a few k)

models (tens to hundreds of k)

textures (a few megs)

thing about textures is you can down sample them fairly quickly (or preoptimize them in a compressed format with pre-generated mip maps but i wont go there for simplicity, thats another discussion entirely). the formula for texture memory usage is w*h*bpp. for technical reasons power of 2 textures are preferred. 24bpp is common for 3 channel uncompressed bitmaps (dxt1 can do 4bpp, but again thats a different discussion). a 24-bit 1024^2 texture has mip levels of the following sizes:

1024 - 3072k

512 - 768k

256 - 192k

128 - 48k

64 - 12k

32 - 3k

...

as you can see just dropping a couple mip levels shaves a lot off of your texture size. so if the textures to all your unused parts dropped down to the 32^2 level, you free up a lot of memory. if you track your texture usage stats internally, load higher resolution versions of textures that are used a lot, unload high resolution content that is not used at all. since texture usage changes based on gameplay, you need to keep on the ball to optimise texture quality vs usage while meeting a memory usage target. you can do this in another thread for best results.

textures are usually passed by reference, because there is too much data to move around. thing about references, they can be changed quickly. so you can load new data->change the reference->unload old data. to do this on another thread you just need spin locks on those references, so you can only change them when they aren't being used (if they are buffer em up and try again on the next loop of the texture management thread).

[Daveroski]

Used in conjunction with memory swapping parts in used in the universe, I can see how this would enable the user to have considerably more unique parts available for use in the universe than memory swapping could alone.

As a standalone feature, it would greatly increase the number of parts available in the universe at one time. But you would still have possibly many hundreds of parts in memory that are not being used anywhere in the universe at all.

Having to access the drive while in an active scene in the universe is something we would strive to avoid. Loading textures for a part-heavy scene in the middle of a maneuver would likely cause an unwanted jerk in the scene as the larger texture files load.

[Nuke]

thats why you do it on a different thread. if you did it on the same thread you would need to spend a lot of time waiting for data. do it right and the game wont even know that there is another thread moving pixels around. you might get some bus contention on the pci-e interface, since textures being streamed in need to be moved to the video card pretty quickly. the best time to do this is while physics is being handled, when rendering is not being done pci-e traffic should be virtually non-existant. whenever you have a race condition the rule of thumb would be the main thread goes first. the texture loader doesn't need to operate in near real time like the main thread.

the other big rule is that you must always have something to draw when something unexpected happens. last thing you want is to have to load a gig of textures when two spacecraft with different sets of parts meet. so a bare minimum of data needs to stay in memory so that the object can be drawn when it needs to. higher definition textures can load over the course of several frames, which is a blink of the eye for the player. but you can still draw with the low res data until that happens. that way you never have to wait for the hard drive (in the main thread).

as for the memory cost: most squad models are under 100k and a 64^2 (a good minimum) texture would be 12k. thats certainly much less compared to the 512^2 textures @ 768k by themselves (times hundreds of parts). you are not going to save all the memory, but you stand to reduce your memory usage a lot.

Edited April 12, 2014 by Nuke

[John FX]

It could work the way Mun does now. You get closer to Mun in warp and come out of warp at periapsis and while rendering, Mun is procedurally generated until it gets to full beauty.

Do something like that with textures so you don`t get the big, system stopping texture loading session. Maybe a tiny placeholder texture is loaded first quickly so there is `something to render` then the game engine carries on, the player plays the game, and the rest of the textures are loaded in the background. The tiny placeholders could even be used if lack of memory is about to cause a crash...

EDIT : It would be nice for something like that to happen now. A prediction that an object will be closer than 2.5km in a few real world seconds triggers a preload of everything that will be needed when the craft has to be rendered and physics applied in a way that does not impact performance any more than rendering the object. Obviously the player could cause this to not happen but that will be the less common situation by a few orders and you could just unload.

Edited April 12, 2014 by John FX

[Slouch]

Of course it will still be possible to use up all of the available memory by building lots of part intensive ships but you will have had so much more fun by the time that happens.

Now you've handed memory management to the user. One click too many and their game crashes.

It's one thing to expect a user to know: Too many mods -> Game doesn't work.

Its another expect them to know: Too many mods -> Game doesn't work if I select an arbitrary set of parts. I.E *add part*, *add part*, *add part*, *add part*, *Crash*

OR

*add part*, *add part*, *add part*, *add part*,*add part*, *add part*, *add part*, *add part*,*add part*, *add part*, *add part*, *add part*, *Crash*.

The unless the user works out which mods they're currently using and the memory requirements of each of those mods is they won't know when their game will fail.

Also on the fly memory management is low level and a ball-ache. It's likely to cause plenty of bugs and shift error prevention to the user at run time.

[John FX]

Now you've handed memory management to the user. One click too many and their game crashes.

It's one thing to expect a user to know: Too many mods -> Game doesn't work.

Its another expect them to know: Too many mods -> Game doesn't work if I select an arbitrary set of parts. I.E *add part*, *add part*, *add part*, *add part*, *Crash*

OR

*add part*, *add part*, *add part*, *add part*,*add part*, *add part*, *add part*, *add part*,*add part*, *add part*, *add part*, *add part*, *Crash*.

The unless the user works out which mods they're currently using and the memory requirements of each of those mods is they won't know when their game will fail.

Also on the fly memory management is low level and a ball-ache. It's likely to cause plenty of bugs and shift error prevention to the user at run time.

Which is exactly why I suggested small fast placeholder textures which are only replaced by larger ones if there is enough RAM, avoiding making the user handle memory management. If they want too many parts and textures the quality of the textures is adjusted to match the available memory. Texture swapping is much easier than low level memory management and let`s face it, textures are the biggest memory hog AFAIK.

Then your available memory is not a fixed amount of parts or textures.

[Guest]

The unless the user works out which mods they're currently using and the memory requirements of each of those mods is they won't know when their game will fail.

How is this different from what we have now? Someone posts " Game crashes on startup." almost every single day.

[Slouch]

The difference is the game will crash if too many mods are installed and the game may crash (on an arbitrary click).

@JohnFX Texture management sounds better but swapping out textures biased on available RAM sounds like it could lead to plenty of errors down the line. The devs are great and all, but they are only human (AFAIK). Also how would texture swapping work if you played on the low graphics settings?

Although if I saw an example of it being done well elsewhere then that sounds awesome.