32bit vrs 64bit???

[M5000]

Look, here, I'll make this as simple as possible.

Say you have 15 tons of sand.

Say you need to get it somewhere else 100 yards away. You have to use a conveyor belt that runs at a fixed speed.

You can choose between a 3 foot wide and a 6 foot wide conveyor belt.

The 6 foot wide belt can move more sand, even though both belts are going the same speed, because it is WIDER and has more CAPACITY.

The pile of sand is data in the computer's memory.

The speed at which the conveyors run would be your processor's clock rate.

The "somewhere else" is the computer's processor and other things after that.

The conveyor belt is the data bus for the memory and processor.

The wider you make the data bus, the more data you can move.

The comparison from 32 to 64 bit is actually less like a 3 foot and 6 foot wide conveyor, and more analogous to comparing a 3 foot and a 200-yard wide conveyor. (It's exponentially scaled, not linear.)

The problem with the 200-yard wide conveyor is that the structure may get unstable at such large sizes.

That's why 64 bit is better in every conceivable way, but since it's a bit unstable on Windows right now, it's hard to implement.

That's as simply as I can explain 32 vs 64 bit hardware/software.

Motokid600: Yes, you could install and run as many mods as you wanted, as long as it doesn't use up all the RAM you have available on your PC itself. It would allow your computer to, instead of addressing the 3.5gigs-ish it can with 32 bit, allow it to address up to exobytes, which is more data than you'll ever use on your computer ever. Limited by Windows, it's serveral hundred Terabytes, which is probably still more memory than all the existing, previous, and future installs of KSP and all past, current, and future mods take up combined..

So yeah, it will take the limitation off of the software, and place it on the hardware. If you exceed your hardware's limitations, you may experience crashes just like you do now.

Edited September 24, 2013 by M5000

[asmi]

The comparison from 32 to 64 bit is actually less like a 3 foot and 6 foot wide conveyor, and more analogous to comparing a 3 foot and a 200-yard wide conveyor. (It's exponentially scaled, not linear.)

This is not correct. RAM bus width is 64 bit times number of channels (128 bit for dual channel, 256 bit for quad channel) and CPU utilizes the full width regardless of the mode it's operating (be it 16 bit mode, 32 bit mode or 64 bit mode). What is referred to as 64 bit mode is the width of pointer to memory location (it's 32 bits for 32bit mode and 64 bits for 64bit mode). I have to notice that even in 32bit mode it's possible to address more than 4 GB of memory (CPU's physical address bus is wider than 32 bits for quite some time) via mechanism known as PAE (physical address extension). I won't go into technical details as to how exactly this works, just want to point out that 32bit Windows OS since Windows 2000 were able to use more than 4GB of memory.

P.S. There are so many myths and misconceptions regarding this subject, sounds like an article is in order...

[magnemoe]

The devs mentioned that Friday on their livestream, that they wanted people to focus on the problems, not what they saw as the easiest solution, because while a 64 bit client would ease memory problems for those players that had more than 4GB of ram and a 64-bit OS, it wouldn't help anyone that didn't have both, but memory optimizations within the game or engine would benefit everyone.

How many players is still using 32 bit?

How many will still be using 32 bit windows then KSP is released?

Note that the pc version of next gen console games will require 64 bit as the consoles has 8 GB.

The reason why 64 bit windows is not required yet is that office software manages with less than 4Gb ram and most games are also able to run on 360 and ps3 with 512 MB ram.

[DAVIDESCOTT]

How many players is still using 32 bit?

How many will still be using 32 bit windows then KSP is released?

The answer to both questions is "it doesn't matter". You clearly don't understand that having a 64bit address space is not important to most games and certainly not to a game like KSP. The only time a game would benefit from a 64bit address space is if it consumed >4GB of memory, and consuming that much memory is a bad thing. There is no reason for KSP to ever consume remotely close to 4GB of RAM. It is simply not a texture or object heavy game, it is not GTA V.

Furthermore those 64bit wide pointers come at a performance cost. You can only ship 1 pointer across in a single machine word where you could be shipping 2. That is why there is an effort to introduce an x32 ABI for linux and make it the default for all applications. In x32 the applications see a 32bit memory space, but get to work with the larger 64bit integers (which is useful for lots of things including say solving the Year 2038 problem). Since KSP (like most games) is constrained by fp issues not by integer arithmetic, even x32 does not really benefit KSP.

If you want to push for a 64bit version of KSP at least push for it for valid reasons. The only valid reasons are additional registers and additions SIMD instructions. There are so many other things that the developers can do which will improve the game dramatically more than using AMD64 every could that its ridiculous to suggest that they waste their time with 64bit.

Honestly the more people complain about stuff like this in the forums the less time they will have to make real improvements that would actually benefit the game.

Edited September 25, 2013 by DAVIDESCOTT

[Payload]

This is not correct. RAM bus width is 64 bit times number of channels (128 bit for dual channel, 256 bit for quad channel) and CPU utilizes the full width regardless of the mode it's operating (be it 16 bit mode, 32 bit mode or 64 bit mode). What is referred to as 64 bit mode is the width of pointer to memory location (it's 32 bits for 32bit mode and 64 bits for 64bit mode). I have to notice that even in 32bit mode it's possible to address more than 4 GB of memory (CPU's physical address bus is wider than 32 bits for quite some time) via mechanism known as PAE (physical address extension). I won't go into technical details as to how exactly this works, just want to point out that 32bit Windows OS since Windows 2000 were able to use more than 4GB of memory.

P.S. There are so many myths and misconceptions regarding this subject, sounds like an article is in order...

I think you should. Some people don't really have a grasp of what the difference really is. The real difference is addressable memory space and increased precision speed. The level of precision doesn't change, the speed of it's calculation gets faster.

Simply referring to programmable hardware, since that is what I have the most experience with. The 8 bit processors that I commonly use have no problems preforming 32 and 64 bit floating point. They would be invoked in any number of ways such as declaring the variables as a float32 or a float64. The problem is it takes more time to calculate because the processor is only 8bit. There is a minor speed boost to be had but it isn't huge and most programs don't require the use of quad floats.

The main difference is the addressable bit space. IMO that wont be nearly as big of an issue once the game gets a little optimization. To be honest, the vanilla game isn't close to blowing the limit yet as far as I can tell. It's the fact that we all like mods and there are so many good ones. The main issue is texture compression. Or rather KSP's lack of it. Solve that before we worry about addressable space.

EDIT: about PAE. I would assume that KSP uses that, but I have 8 gigs of memory and KSP crashes at 3.4 gigs every time.

Edited September 25, 2013 by Payload

[RuBisCO]

The OP asked about how 64 bits allow for far more RAM than 32 bits, I will now demonstrate this in a really long round about way:

Back in the early days of the unix operating system (early 1970's) some engineer came up with the idea of "Unix Time" for computers, that being counting the seconds from a specific date, for example right now in my linux operating system I typed "date +%s" into a terminal and out it spits the number "1380065956" this is the Unix Time Stamp or how many seconds since midnight, January 1st, 1970, in London (roughly)! Now that a BIG number, in binary that number is "1010010010000100010001010100100" or 31 bits long! Now there is a minor problem with Unix time: its limited to 32 bit numbers, once the Unix Time reaches "2147483648", in binary as "10000000000000000000000000000000", or 03:14:08 on 19th of January, 2038 it won't be able to count to the next second because that would require a 33 bit long number (or more exactly switching that final 0 to 1 and counting negative numbered time!), this will lead to an "overflow" error and Y2K like shenanigans will follow (like negative dates or time restarting in 1901 or 1970 and going backwards!).

Thankfully there is a fix: Just use 64-bit long Unix Time, now you might ask "but wait is that not just pushing the overflow problem out to a further date?" Yes in fact on Sunday at 3:30:08 pm, 4 December in the year 292 billion, 277 million, 26 thousand and 596: Unix time in 64-bit will reach its end! If the universe still exists then someone is going to have to make sure everything is switch to 128 bit time stamps or there is going to be blood I tell you (or what ever pumps though veins in that time)!

What does this have to do with RAM size? Same problem! When a computer pulls a value from memory, from Random Access Memory, it needs to know the exact location of that value: 32-bit memory addressing can count out to the number 4294967296 or access every byte of memory that is 4 billion, 294 million, 967 thousand and 296 bytes large and no larger. Since the computer stores values as 8 bit long numbers called "bytes" that is where the 4 gigabyte or "4GB" memory limit comes from. Switching over to 64 bits solves this problem for in theory the computer could recall any byte out of 18 quintillion bytes (QB)! Sadly x86-64 CPUs only use 48 bits out of the 64 for memory addressing limiting us to *only* 256 Trillion Byte (TB) large RAM (crying shame I tell you!). They do this because using the full address bus when no present day computer, server or gigantic supercomputer mainframe could even come close to using 18 quintillion byte large memory is silly. Thankfully they could change this someday when needed with ease and still be backwards compatible.

Someday when harddrives and RAM become one, using MRAM or spintronic memory or god only knows what, we might have terrabytes of RAM and people might complain about how the 18QB limitation is stopping their household robot from figuring out how to wash the grime off of cast iron pots:

"I keep telling you, you stupid droid, DON'T USE SOAP!"

"MEMORY ERROR! MEMORY ERROR!"

"Fine, fine, just walk the ****ing dogs, jeez, I'll handle this, just go, yes, yes, I still love you."

For now though we complain about 4GB limitation when using 32-bit computer programs because all the extra RAM space we could use on our 64-bit CPU's would come in really handy say when playing KSP and using a huge number of mods and parts and crafts all over the kolar system... you get the idea by now right? If you want this, get linux, buy 8, no 16 GB of RAM, play KSP, problem solved.

Edited September 25, 2013 by RuBisCO

[asmi]

I think you should. Some people don't really have a grasp of what the difference really is. The real difference is addressable memory space and increased precision speed. The level of precision doesn't change, the speed of it's calculation gets faster.

I will look into it once I'm done with Newton physics mod. Right now I'm working on optimizing algorithm's performance.

Simply referring to programmable hardware, since that is what I have the most experience with. The 8 bit processors that I commonly use have no problems preforming 32 and 64 bit floating point. They would be invoked in any number of ways such as declaring the variables as a float32 or a float64. The problem is it takes more time to calculate because the processor is only 8bit. There is a minor speed boost to be had but it isn't huge and most programs don't require the use of quad floats.

This is different - in most microcontrollers FP operations are emulated by int-core, while in x86-compatible CPUs all FP operations are performed by separate co-processor that is on the same die with main CPU, but it still is separated, even to the point that it can work asyncronously with main CPU unit (so CPU and FP/SSE command can be executed simultaneously), and that co-processor have FP operations implemented in hardware. This is why FP commands use separate from main int-core CPU registers (XMM). And that co-processor (commonly called FPPU or FPU) doesn't care about CPU's operating mode, and as such, all FP commands work the same way regardless of weither main CPU works in 32bit mode or 64bit. I don't feel like going into more technical details of this, but if you want to learn more, I'd highly recommend you to take a look at Intel developer manuals - they explain these things in great details.

The main difference is the addressable bit space. IMO that wont be nearly as big of an issue once the game gets a little optimization. To be honest, the vanilla game isn't close to blowing the limit yet as far as I can tell. It's the fact that we all like mods and there are so many good ones. The main issue is texture compression. Or rather KSP's lack of it. Solve that before we worry about addressable space.

As a developer, you most likely know that optimization is hard. There are many reasons as to why it's so, but the problem is that in our world hard equals expensive. And Squad have just so many developers AND those devs are busy doing other great stuff.

So from a practical standpoint, getting customer to buy additional memory is way more efficient in terms of effort/benefit. Unfortunately it's a trend in a real life that most projects don't give high priority to optimizations, and as concequence vast majority of software in alpha stage are memory- and CPU time hogs. So if they'll find a way to make 64bit build of KSP, they'll solve this problem for the time being, as they can always tell to complainers to get more memory, and so there at least would be some way to work around the problem until they will have more time to deal with optimizations.

EDIT: about PAE. I would assume that KSP uses that, but I have 8 gigs of memory and KSP crashes at 3.4 gigs every time.

[DAVIDESCOTT]

EDIT: about PAE. I would assume that KSP uses that, but I have 8 gigs of memory and KSP crashes at 3.4 gigs every time.

Userspace programs never used PAE. It was a way for the kernel to provide multiple independent 4GB chunks to different userspace programs. You had to be in Ring 0 to even take advantage of it and it was available across different page tables which makes it hard and slow to use. PAE does not increase the virtual address space. If you really needed to use >4GB in a single program then you bought an Alpha or other 64bit architecture.

In any case as you indicated there are other ways KSP can reduce memory usage (such as texture compression). Given how few objects and textures there are in the game there is no reason for it to ever use remotely close to 4GB of RAM. In your case the bug is that it uses 3.4GB not that it crashes. The crash is just a side-effect.

Edited September 25, 2013 by DAVIDESCOTT

[asmi]

Userspace programs never used PAE. It was a way for the kernel to provide multiple independent 4GB chunks to different userspace programs. You had to be in Ring 0 to even take advantage of it and it was available across different page tables which makes it hard and slow to use. PAE does not increase the virtual address space. If you really needed to use >4GB in a single program then you bought an Alpha or other 64bit architecture.

Actually it's not entirely true as there is a way for Ring3 mode process to access over 4GB memory if OS have such API. For example, here is Windows AWE API for that: http://msdn.microsoft.com/en-us/library/windows/desktop/aa366527(v=vs.85).aspx. Right now there is not much point to use this API since transistioning to 64bit is so much easier, but remember that PAE first appeared in Pentium Pro CPU!!

[Payload]

Userspace programs never used PAE. It was a way for the kernel to provide multiple independent 4GB chunks to different userspace programs. You had to be in Ring 0 to even take advantage of it and it was available across different page tables which makes it hard and slow to use. PAE does not increase the virtual address space. If you really needed to use >4GB in a single program then you bought an Alpha or other 64bit architecture.

In any case as you indicated there are other ways KSP can reduce memory usage (such as texture compression). Given how few objects and textures there are in the game there is no reason for it to ever use remotely close to 4GB of RAM. In your case the bug is that it uses 3.4GB not that it crashes. The crash is just a side-effect.

I don't know that it's a bug causing it to use 3.4gb of memory since I caused it purposefully to see what it would take. I just loaded up my parts library to see what it would take. That was it, 3.4Gb of ram. I do know that the game SkyRim had no support for more than 4Gb of memory at first and peoples mods where way over that. So someone made an unofficial patch that allowed it to use more. So utilizing PAE in a user space program is done. If I am not mistaken, it was as simple as a compiler flag. The game developers subsequently compiled the newer patch with the flag and we didn't have to use the unofficial exe any more.

I would like to express that even with the mods that I have, I have not had the game crash from using too much memory. I had to do that on purpose.

[DAVIDESCOTT]

Actually it's not entirely true as there is a way for Ring3 mode process to access over 4GB memory if OS have such API. For example, here is Windows AWE API for that: http://msdn.microsoft.com/en-us/library/windows/desktop/aa366527(v=vs.85).aspx. Right now there is not much point to use this API since transistioning to 64bit is so much easier, but remember that PAE first appeared in Pentium Pro CPU!!

But then they are using the AWE API not PAE. There is an API on Unix that has allowed you to store and retrieve terabytes of data for decades its called "fopen" ;-)

In any case without special support and some unpleasant modifications to your program structure a single process could not access 4+GB even with PAE.

[DAVIDESCOTT]

I don't know that it's a bug causing it to use 3.4gb of memory since I caused it purposefully to see what it would take.

Its a bug that the parts loader is so inefficient that it takes that much memory to load all your mods. The whole point of not doing a 64bit transition is to fix the issues present in the parts loader and "do it better" so that its not taking 4GB of RAM to load all the mods. Adding things like texture compression, being careful to lazy load parts, searching for memory leaks, etc.

[asmi]

But then they are using the AWE API not PAE. There is an API on Unix that has allowed you to store and retrieve terabytes of data for decades its called "fopen" ;-)

AWE API uses PAE internally.

In any case without special support and some unpleasant modifications to your program structure a single process could not access 4+GB even with PAE.

I agree, but your statement was absolute. I can name at least one user-mode application that uses AWE (and so PAE) - 32bit builds of MS SQL Server.

[egreSS]

This is not correct. RAM bus width is 64 bit times number of channels (128 bit for dual channel, 256 bit for quad channel) and CPU utilizes the full width regardless of the mode it's operating (be it 16 bit mode, 32 bit mode or 64 bit mode). What is referred to as 64 bit mode is the width of pointer to memory location (it's 32 bits for 32bit mode and 64 bits for 64bit mode). I have to notice that even in 32bit mode it's possible to address more than 4 GB of memory (CPU's physical address bus is wider than 32 bits for quite some time) via mechanism known as PAE (physical address extension). I won't go into technical details as to how exactly this works, just want to point out that 32bit Windows OS since Windows 2000 were able to use more than 4GB of memory.

P.S. There are so many myths and misconceptions regarding this subject, sounds like an article is in order...

Actually he was right and you are incorrect. M5000 said data bus, not address bus because we're talking about the CPU here. That's like comparing HyperTransport to PCI Express - All of them are controlled by the northbridge but they are totally different.

The analogy with the conveyor belt, while crude, was actually quite accurate in that you can move more data per clock(again, CPU not memory) cycle. His statement about 64bit scaling exponentially not linearly to 32bit(32bit = 2^32, 64bit = 2^64) and the statement about Windows OS being hard limited to 4GB was also correct.

Yes Microsoft has known about PAE since Windows 2000 but every single desktop kernel to date only uses allows programs access to virtual memory. Even 32bit editions of Windows 8 Pro are hard limited to 4GB. The only way around that would be to create a new windows kernel with a custom memory management solution. There are hacks available for Windows 7 to patch in Datacenter's PAE support but according to M$ it is disabled in all but server editions due to "licensing".

http://en.wikipedia.org/wiki/Physical_Address_Extension#Microsoft_Windows

[sjwt]

And its time to let this thread die...

RIP.