Will KSP 2 be optimised for NVMe M.2 on PC ?

[Infinity and Beyond]

Will KSP 2 be optimised for NVMe M.2 on PC?

Edited January 11, 2021 by Infinity and Beyond
Device not specified in original post

[RealKerbal3x]

It's unlikely the game will be optimised for one specific component.

We do know KSP2 will have a large improvement in performance over KSP, though.

[Kerminator K-100]

I would be more worried about optimization on CPUs and GPUs not hard drives.

[Incarnation of Chaos]

I think he's more thinking about the Xbox Series X and PS5s tech.

I saw those in action, in actual games the loading times are still there. And around 18 seconds, so as I suspected all of it was nothing but marketing nonsense and the increase in loading times was soley due to the massive increase in CPU IPC over the absolute garbage kavari cores in the previous gen.

To give you perspective....18 second loading times are easily achieved by my computer with games running off a 7200 RPM HDD. And yes, even the "Complex" 100GB monsters of today xD

32GB of RAM and 8GB VRAM does wonders...

But that's semi OT, to actually address your question there's no indication of KSP2 supporting any rapid storage or similar technology at the moment.

Likely will have more info whenever they get around to confirming the console versions.

[Infinity and Beyond]

It actually does improve speed significantly..... I had a SATA SSD before, it took atleast 10 seconds to boot Windows 10 but now it takes no time at all with NVMe M.2 drive. If the game is optimised to use the drive then the loading time will significantly be lowered on SSDs and NVMe M.2 drives.

[shdwlrd]

12 minutes ago, Infinity and Beyond said:

It actually does improve speed significantly..... I had a SATA SSD before, it took atleast 10 seconds to boot Windows 10 but now it takes no time at all with NVMe M.2 drive. If the game is optimised to use the drive then the loading time will significantly be lowered on SSDs and NVMe M.2 drives.

That's because instead of using the slower SATA bus, and SATA controller for the SSD, the NVMe is using the faster PCI bus with a direct connection to the  CPU. 

That's the main reason why NVMe cards are faster than standard SATA drives.

Edited January 12, 2021 by shdwlrd

[Infinity and Beyond]

16 minutes ago, shdwlrd said:

That's because instead of using the slower SATA bus, and SATA controller for the SSD, the NVMe is using the faster PCI bus with a direct connection to the  CPU. 

That's the main reason why NVMe card are faster than standard SATA drives.

Games not optimised for SSDs don't take full advantage of them, thats why the loading time is not very fast with any SSD.

[NoMrBond]

I thought KSP's loading was slow because some part of it was only ticking over on a per-frame basis (I forget how asset ingest is tied to Unity frame/ticks), plus nothing happens in parallel with the loading, and you have to load almost everything right at the start.

So even with the fastest HD/SSD/NVME drives in the world, you stop seeing any loading speed improvements as soon as your drive can supply info faster than KSP can ingest it

I highly doubt KSP2 will use the same kind of loading (probably move to Unity asset bundles), but will it use DirectStorageAPI (the PC equivalent of XBX's Velocity architecture or PS5's Kraken-12) for like really taking advantage of storage access speeds, who knows (it's not out yet)

[runner78]

Many games only compile the shaders when the game starts. And when loading a level, much more happens than just reading from the hard drive. The assets have to be unpacked, loaded into the scene, and graphic assets also sent to the GPU. Start scripts run to prepare the scene, in KSP e.g. when starting a vehicle it has to be reassembled from the save / craft file.
Unity DOTS has the concept of subscenes, which was designed to save the data of the subscene exactly as it is in the RAM and then read it 1:1 into the RAM. But this is in develeopment.

[Incarnation of Chaos]

On 1/11/2021 at 8:19 PM, Infinity and Beyond said:

It actually does improve speed significantly..... I had a SATA SSD before, it took atleast 10 seconds to boot Windows 10 but now it takes no time at all with NVMe M.2 drive. If the game is optimised to use the drive then the loading time will significantly be lowered on SSDs and NVMe M.2 drives.

I use a SATA SSD as my boot drive

The difference between a 4 second boot and 12 second boot is a couple of processes i disable on startup.

I wasn't saying NVME isn't faster, objectively it will always be faster than a SATA SSD. But this entire idea of loading times and drive performance being this tightly coupled?

That's the part i highly disagree with, IO is always the bottleneck. It's why we always get assets into RAM and then VRAM ASAP, and why you can approach these performance levels with spinning rust even today.

And I'd rather them not waste months of development time for a feature that allows me to load games....exactly as fast  as i do now. At least not until they have literally nothing else to do.

[steve_v]

On 1/12/2021 at 2:31 AM, shdwlrd said:

slower SATA bus

600MB/s (SATA 3) is fine for most tasks, since most tasks care about latency and IOPS far more than raw bandwidth.

KSP with both expansions is ~4.8GB on disk (or 2.8GB in my case, thanks to ZFS/LZ4 transparent compression), so at SATA speeds loading should take less than 10 seconds if IO bandwidth is the bottleneck.
That said, the PCIE bus is absolutely the way to go for local storage. It kinda annoys me that it's in the form of a special connector on the motherboard rather than an actual PCIE card, but whatever. I'm in no hurry to upgrade.

I fail to see how one would go about optimising a game for SSDs anyway TBH, unless you're working at the block level and you're almost certainly not going to bother with that krakens for a game.
On any mainstream OS almost all applications (the exceptions are largely specialised forensic or data recovery tools), including games, have no idea where or how their data is stored on the hardware. That's the kernel/IO scheduler/filesystems  job.
 

[Incarnation of Chaos]

6 hours ago, steve_v said:

600MB/s (SATA 3) is fine for most tasks, since most tasks care about latency and IOPS far more than raw bandwidth.

KSP with both expansions is ~4.8GB on disk (or 2.8GB in my case, thanks to ZFS/LZ4 transparent compression), so at SATA speeds loading should take less than 10 seconds if IO bandwidth is the bottleneck.
That said, the PCIE bus is absolutely the way to go for local storage. It kinda annoys me that it's in the form of a special connector on the motherboard rather than an actual PCIE card, but whatever. I'm in no hurry to upgrade.

I fail to see how one would go about optimising a game for SSDs anyway TBH, unless you're working at the block level and you're almost certainly not going to bother with that krakens for a game.
On any mainstream OS almost all applications (the exceptions are largely specialised forensic or data recovery tools), including games, have no idea where or how their data is stored on the hardware. That's the kernel/IO scheduler/filesystems  job.
 

HUH

You can get cards like this that basically just split a x16 lane into multiple x4 (Might have issues with supporting bifurcation depending on mobo)

https://www.newegg.com/asus-hyper-m-2-x16-gen-4-card-pci-express-to-m-2-card/p/N82E16815293045?Description=PCI ssd&cm_re=PCI_ssd-_-15-293-045-_-Product

Or actual PCI SSDs (Mind you these are some of the more expensive ones)

https://www.newegg.com/p/pl?d=PCI+ssd&N=100011693 600521290&isdeptsrh=1

 

 

[steve_v]

On 1/13/2021 at 2:33 PM, Incarnation of Chaos said:

HUH

I should probably clarify that: What I'd really prefer is an additional PCIE slot or two on the mobo rather than a dedicated NVME slot. If the majority of NVME drives were PCIE cards that's what we would have got.

It's less about the NVME form factor itself, more about the continual erosion of the original role of a motherboard as a dumb backplane and incessant addition of extra (and rapidly obsolete) crap to it.  I don't want motherboard audio, video, wifi, or idiotic RGB lighting either TBH, just give me standard expansion slots so I can add the things I actually do want... Like a nice SAS or FC HBA and a DAC that doesn't sound like a rat in a cake tin.
I remember when a motherboard didn't have any storage interfaces built-in at all, and you added the IDE/FDD controller as an ISA card. Hell, I've owned machines where the CPU(s) are on expansion cards.

One or two NVME slots on the board gets me one or two drives, in a ridiculously difficult to get to location. One PCIEx16 slot gets me somewhere to put an HBA that can serve 8 or 16 drives that I can then mount in a hotswap cage where they belong. Guess which one I'd prefer.

Enough off-topic though, back to the scheduled broadcast.

 

Edited January 15, 2021 by steve_v

[Incarnation of Chaos]

16 minutes ago, steve_v said:

I should probably clarify that: What I'd really prefer is an additional PCIE slot or two on the mobo rather than a dedicated NVME slot. If the majority of NVME drives were PCIE cards that's what we would have got.

It's less about the NVME form factor itself, more about the continual erosion of the original role of a motherboard as a dumb backplane and incessant addition of extra (and rapidly obsolete) crap to it.  I don't want motherboard audio, video, wifi, or idiotic RGB lighting either TBH, just give me standard expansion slots so I can add the things I actually do want... Like a nice SAS or FC HBA and a DAC that doesn't sound like a rat in a cake tin.
I remember when a motherboard didn't have any storage interfaces built-in at all, and you added the IDE/FDD controller as an ISA card. Hell, I've owned machines where the CPU(s) are on expansion cards.

One or two NVME slots on the board gets me one or two drives, in a ridiculously difficult to get to location. One PCIEx16 slot gets me somewhere to put an HBA that can serve 8 or 16 drives that I can then mount in a hotswap cage where they belong. Guess which one I'd prefer.

Enough off-topic though, back to the scheduled broadcast.

 

Ah, got it.

Spoiler

Blame intel for enforcing segmentation by reducing the PCI lanes on their CPU's.

 

[paul_c]

6 hours ago, steve_v said:

I should probably clarify that: What I'd really prefer is an additional PCIE slot or two on the mobo rather than a dedicated NVME slot. If the majority of NVME drives were PCIE cards that's what we would have got.


 

Interesting, I am guessing you have an ATX sized motherboard.

I have a mini-ATX and it has 3 PCIe slots. The first is the fast one. Once a double-width graphics card is fitted it covers the little one. Leaving the 3rd one, which if a card were to be fitted, would block the fan of the graphics card! So really, a mini-ATX with a fat graphics card is "1 slot" with a tease of more. Thus, I'm glad of all the integration onto the motherboard. 

In particular, NVMe drives are a fair bit smaller than the old 2.5" standard, and require 2 fewer cables too - better for neatness, cooling (maybe) etc IMHO of course.

[fragtzack]

I built a new PC few months back with X2 NVMe M.2 . 

I noticed a huge improvement in KSP 1 loading times and VAB performance when large number of parts.

Whether the performance gain was solely related to NVMe M.2  is debatable because also upgraded from 16GB RM to 32GB higher performance RAM + better CPU cores via AMD.

[K^2]

Fast SSD is a big feature of next gen consoles. I suspect, a lot of optimization will go into loading speed both on the Unity end and at intercept. I don't know if that will make a huge impact on NVMe speeds, but it should help.

[steve_v]

7 hours ago, paul_c said:

Interesting, I am guessing you have an ATX sized motherboard.

There are 2 machines on this desk right now, standard ATX and E-ATX respectively.
 

7 hours ago, paul_c said:

I have a mini-ATX

Well if you want a tiny board, compromises are going to be made.

 

7 hours ago, paul_c said:

2 fewer cables too - better for neatness, cooling (maybe) etc IMHO

I really don't get why anyone would care how "neat" the inside of a PC case is, once the covers are on nobody sees it.

As for cooling, if a couple of dinky little SATA cables create an airflow problem there's something seriously wrong with your cooling solution.
This old thing (since upgraded) has 2 CPUs, a non-modular PSU, no fancy cable-hiding compartments, 13 SATA cables... And zero cooling problems.

I've also upgraded the appallingly horrible camera that took that.

[paul_c]

5 minutes ago, steve_v said:

There are 2 machines on this desk right now, standard ATX and E-ATX respectively.
 

Well if you want a tiny board, compromises are going to be made.

 

I really don't get why anyone would care how "neat" the inside of a PC case is, once the covers are on nobody sees it.

As for cooling, if a couple of dinky little SATA cables create an airflow problem there's something seriously wrong with your cooling solution.
This old thing (since upgraded) has 2 CPUs, a non-modular PSU, no fancy cable-hiding compartments, 13 SATA cables... And zero cooling problems.

I've also upgraded the appallingly horrible camera that took that.

PCs are needlessly big these days, they've grown and grown in recent times, what with the advent of cable management backside and space for multiple 360mm radiators and fans etc..... Coming from a laptop, I went for a smallish case, I figured I'd never actually need to add in any expansion card(s) so m-ATX suited my needs fine. 

Obviously there's a use case for lots of drives in a computer - not for my desktop though, it will do with 3 (1x NVMe and 2x 2.5" SATA) I never followed the fashion for glass side and  internal neatness/OCD to the point of it being a work of art, although I like to keep it tidy in there. 

My Dad's new PC fits inside a 9cm x 30cm x 30cm case, its an ITX motherboard and in there is a CD drive, old 3.5" hard disk and media card reader. NVMe drive is fitted on the underside of the motherboard, so that's a definite bonus/use case. 

[K^2]

On 1/13/2021 at 12:05 AM, steve_v said:

I fail to see how one would go about optimising a game for SSDs anyway TBH, unless you're working at the block level and you're almost certainly not going to bother with that krakens for a game.

Sorry, I'm replying out of order here a bit, but the optimization is read-speed vs processing speed, generally. For example, it makes sense to leave more resources uncompressed if you are going to be reading them from SSD rather than HDD. At a slower read speed of HDD, you often get an asset in memory faster if you are reading a compressed stream and decoding it as blocks are read. On the other hand, with NVMe M.2 SSD, it can take longer to decode compressed data than to read uncompressed data in directly if the CPU is busy doing other loading tasks at the same time. Of course, this also impacts disk size, so compression still has its place, but optimal balance between compressed and uncompressed assets is different for fast SSD vs HDD. (Note, this doesn't apply to next consoles, as they have dedicated decompression hardware that's fast enough to decode things as they are being read, so you get both benefits.) Another good example is animation data. There are a whole bunch of compression tricks used to reduce the size of animation tracks on disk, and it's mostly done to improve streaming performance rather than for the sake of disk space. Leaving animation tracks uncompressed on disk would make streaming and loading way more efficient on a fast SSD.

There are also examples of optimization that are done for sake of seek time on HDD that hurt other performance and can actually increase size of the game on disk. For example, it's not uncommon to pack some smaller textures along with a model so that when you need to load model and textures, you are more likely to read a continuous set of blocks on the disk reducing read time. Unfortunately, it means that if a texture is re-used it's stored in several places on the disk. On modern SSDs, there is absolutely no difference if you are reading blocks that are continuous or not because of how SSD memory management works. (There's usually a dedicated chip that's there to map disk address to physical location on memory chips, kind of like virtual addressing in RAM.) So when optimizing for SSD, you would usually avoid this technique and always store textures as separate files to reduce duplication.

Long story short, there are definitely optimizations we put into games that address certain features of storage, and while for consoles we just target whatever's in the box, for PCs we usually target whatever hardware is more common. With next gen of console games being built around the SSD storage and with more and more gaming PCs having either a SATA or NVMe SSD, a lot of games are likely to start treating that as default on PCs.

[steve_v]

48 minutes ago, K^2 said:

leave more resources uncompressed if you are going to be reading them from SSD rather than HDD.

I see. We're lust talking about asset compression and the like, not actual block-level optimisation. Sounds like a fairly trivial change to me, though frankly I could do without the storage bloat uncompressed assets will cause.
 

1 hour ago, K^2 said:

a lot of games are likely to start treating that as default on PCs.

Recon I'll take slightly slower loading over twice the footprint anyday, and just keep the filesystem-level LZ4 compression I'm already running.

[K^2]

1 hour ago, steve_v said:

Recon I'll take slightly slower loading over twice the footprint anyday, and just keep the filesystem-level LZ4 compression I'm already running.

Yeah, especially, if you have some striping across disks to increase read speeds, I don't think it'll make that much difference until we do start getting the same kind of in-line decompression and DMA that we see on PS5/XBSX. So a few years, at least, and even longer until enough games really make use of it. Even then, it'll probably only make sense for games. Personally, I use a SATA SSD for OS, because it's not that much space and it does improve boot times, an M.2 for the Steam library to get the games spinning up fast, and everything else goes on conventional mechanical disks.

At work we have similar setups. All of the raw assets and code syncs go to mechanical HDDs, since they're still better for working in volume, but then the game gets built to an SSD to speed up deployment to consoles or to run PC version directly.

[steve_v]

1 hour ago, K^2 said:

Personally, I use a SATA SSD for OS, because it's not that much space and it does improve boot times, an M.2 for the Steam library to get the games spinning up fast, and everything else goes on conventional mechanical disks.

My desktop is running two SATA SSDs in a striped vdev (roughly equivalent to RAID0), and pretty much everything else is on a hybrid pool (24TB mechanical RAIDZ2 + 240GB SSD adaptive read cache) over NFS... Including regular snapshots of that fast-but-fragile RAID0.
I've just about completely transitioned to ZFS as far as filesystems go TBH, nothing else comes close to the featureset or flexibility. One day I'll upgrade the desktop to something that actually has M.2 support, and no doubt that'll be ZFS as well.

As for boot times... Err, what boot times? I think the last time I rebooted anything was well over a month ago. :P

[kspnerd122]

Again, I feel like we should first off, do what MC did recently, there will be a cutoff where you will not get updates anymore, for example, for older consoles, 1.13 was their last update, do it again, but to current gen consoles, they can play the game, but after a certain point, no more updates(I feel like a way for these players to keep playing long term is to add moding API in the last update(basic ingame part/planet editor, can share the mod to anyone, anyone in that update+ can use the mod(a version would be kept of each version since the mod release, so console players could just install some mod and play it, or make their own.(maybe as a form of DLC))