Loading. Just loading. How does one speed it up?

[weissel]

On 25/03/2018 at 3:41 PM, Linventor said:

Um... I no comprendo senòr? I may be a bit of a computer nerd, but I'm not that much of one. Mind dumbing down your explanation?

I’ll try. It’s hard to take out the technical details because then you have to “simply believe me” instead of being able to reason — and tell me where I have gone wrong. 

In very short: SSDs are very different to HDDs internally but are talked to as if they were HDDs, and despite a lot of smarts used internally by the SSD to keep stuff running (and some help by the OS[1]), heavy use can slow down an SSD a lot, especially older ones and nearly full ones.

The only “cure”, if that becomes too bad, is to basically fully reset and empty out the SSD via “secure erase”.  After which you get to find out if your backup works. And with the same lifestyle, I mean usage patterns you get similar results, and the SSD is not getting younger.

But no cure before diagnosis: do get yourself an (SSD) speed test program — there should be plenty around for free — and see how fast random 4kB and sequential reads are, and compare that to what the manufacturer stated.

You might also want to track the queue length (how many commands on average are waiting for the drive to have a slice of time for them) and transfer rate (how fast reads/writes are and %idle time (if the drive is all running flat out or if it has some space in-between) while starting up KSP. This will help to find out if the problem is mostly the drive or not in first place  

 

Any clarity left over? Just ask and I’ll muddy it for you!

 

[1] TRIM can tell e.g. an SSD that these and these sectors are only holding junk data now and do not be copied or preserved. Since copying stuff around is necessary, not needing to copy junk data is a boon.

[DStaal]

Still a good explanation, if you want to really get into it:

https://arstechnica.com/information-technology/2012/06/inside-the-ssd-revolution-how-solid-state-disks-really-work/

Note that in the past ~6 years, a lot of work has been done to mitigate some of the issues, and to make things work better.  If you're using a modern OS, you can probably ignore the issues - TRIM is almost certainly supported on your OS, and the SSDs are better at balancing things out.  (Though deletes/changes will still take longer than writes, and cause issues over time.)  You can definitely check out your setup for yourself, and fix issues - but for KSP loading, raw access speed will help a lot, and even the worst fragmented SSD is still better than a spinning disk.  (Though multiple high-end disks might get close.)

[Kerbital]

There is no magic mod for this As others said: fewer mods. Any mods that introduces new parts and/or patches will lengthen your startup times. I have around 60 mods only and a PCI SSD and it takes couple of minutes to load.

Edited March 30, 2018 by Kerbital

[weissel]

Yet another possibility is to use a better SSD-cache and/or RAM-cache system that learns what is accessed when and can thus pre-cache and also return data that has been read recently. 

This is most important for small files as the access time on spinning rust (HDDs) is vastly longer than the actual read time. And here the very fast access times of SSDs come in handy. Or even the (again) much much faster access times of RAM. 

On the other hand, HDDs are best when reading large chunks sequentially — movies, large resource files for games … and if they have to seek to a new fragment every 500 MB, that is not that bad (compared to every 4 KB) and therefore needs caching least. And it would take the space of hundreds of small files, too, so the cost of caching is highest and the gain is smalllest — which is a happy coincidence.

 

Theoretically your OS should already do proper caching with all of the currently unused RAM, but with Windows … let us just say the number of 3rd party caching solutions out there is telling.

[i_like_kerbals]

On 3/23/2018 at 5:11 AM, Linventor said:

I don't hoard in IRL, just sayin'.

I mean, updating to 1.4 would be a good way of cleaning up my modlist, but the problem is the mod that I literally NEED isn't updated to 1.4, namely Kopernicus. And To Boldly Go.

Just because a mod isn't updated doesnt mean it wont work with the latest ksp... i have module manager always warning me about out of date mods that function fine... i keep ksp updated, like the latest update made my tacs life support mod go out of date yet it works perfectly fine. Do you know for a fact they dont work on 1.4?

[Linventor]

22 hours ago, i_like_kerbals said:

Just because a mod isn't updated doesnt mean it wont work with the latest ksp... i have module manager always warning me about out of date mods that function fine... i keep ksp updated, like the latest update made my tacs life support mod go out of date yet it works perfectly fine. Do you know for a fact they dont work on 1.4?

well, Kopericus updated to 1.4.2 a few days ago, so I'll see if there's a possibility of me getting this working in 1.4. Wish me luck! (I'll need it)

EDIT: well that was fast. I found out within less than a few minutes that it wasn't going to work for me. Extraplanetary Launchpads, Infernal Robotics, all the Near Future mods, KER, and a bunch of others that are quite important. I'll have to wait.

Edited April 4, 2018 by Linventor

[Wolf Baginski]

On 22/03/2018 at 9:44 PM, MaximumThrust said:

I use more than 100 mods, and reduced the load times by almost half reducing the size of 4k textures, and other textures that were too big. I used a program called Ordenator: https://www.nexusmods.com/skyrim/mods/12801/

It's a bit more work, but if you're using Linux you can use the "find" command to get a list of files above a certain size. Might work on a Mac command line too

find $HOME/KSP_Linux* -size +9M

There are a few mods which have used really huge texture files, but a 2k-resolution file comes out at about 5.6 Megabytes Quite a few seem to be .mbm files, which are an older graphics format that KSP was using, different to the more usual .dds format.

Since KSP has an option in Settings, under Graphics, to use lower-res graphics, and the default is half-res, according to the Wiki, downscaling the graphics could pay off. The on-disk file size might be smaller than what is stored in RAM, and if the default half-res looks good enough for you, you might save as much as 80% of the RAM used for .dds files by downscaling and switching to full-res. Some models use .tga and .png still, which is easy to downscale.

There are instructions here for getting an .mbm file into editable format, and it scares me. Yes, I know I use Linux. It still scares me. There are scattered mentions of tools which do the conversion but all the links I find are dead-ending. 

I hope I can find something for The GIMP but either .mbm is a rather old format used by some mobile phones, or KSP used something else. It does look rather as if the programming team are as bewildered as I am, since it is hanging around on a few old parts.

[Linventor]

1 hour ago, Wolf Baginski said:

It's a bit more work, but if you're using Linux you can use the "find" command to get a list of files above a certain size. Might work on a Mac command line too

I'm not using Linux. Thanks though.

[Wolf Baginski]

19 minutes ago, Linventor said:

I'm not using Linux. Thanks though.

I did check on this. Linux and Mac are both Unix-style under the graphics. It's not for everyone, but I'll put this link here that explains things. I don't use the shell much, but it's there.

Using the Mac terminal

[StahnAileron]

And SSD would alleviate the transfer of the game data to RAM/main memory. However, KSP and the mods for it are Just-In-Time compiled on load. A faster processor would (in theory) help as well when Module Manager has to actually process the config files and compile the parts.

The only other thing that might help is to get a RAM drive and enough RAM so that an install (or more) of KSP can reside on it. Problem there is cost and the fact you'll either be chewing up power keeping the drive alive or having to re-copy the content during each power cycle or reboot. (You'll be bottlenecked by the transfer speed of the interface though, like any other data transfer-related problem.) Some RAM drive have a battery back-up system. Still, RAM drive are fairly power hungry last I looked into it.

As noted before, smaller textures would help since you would literally be reducing the amount of data you're transferring each time on game start-up. The question is if you're willing to live with the lower quality visuals. (This is dependent on the screen resolution you play at as well.) You don't need 4k textures when you play at 1280x720 all the time, for example.

BTW, the issue with SSDs mentioned earlier is mainly on the WRITE side. READ operations are generally not a problem. However, in KSP's case, I can't be sure as I know KSP keeps quite a log going and sometimes a LOT can happen in a game session to bloat that log, especially with lots of mods. I'd get a second SSD just for KSP/games in general rather than having it alongside your OS install. (I'm using a hybrid system via Intel RST for my game drive. OS is a dedicated SSD-only drive.) At the very least, it should be more consistent performance without OS disk operations getting in the way of KSP.

[Linventor]

On 4/5/2018 at 6:10 PM, StahnAileron said:

And SSD would alleviate the transfer of the game data to RAM/main memory. However, KSP and the mods for it are Just-In-Time compiled on load. A faster processor would (in theory) help as well when Module Manager has to actually process the config files and compile the parts.

The only other thing that might help is to get a RAM drive and enough RAM so that an install (or more) of KSP can reside on it. Problem there is cost and the fact you'll either be chewing up power keeping the drive alive or having to re-copy the content during each power cycle or reboot. (You'll be bottlenecked by the transfer speed of the interface though, like any other data transfer-related problem.) Some RAM drive have a battery back-up system. Still, RAM drive are fairly power hungry last I looked into it.

As noted before, smaller textures would help since you would literally be reducing the amount of data you're transferring each time on game start-up. The question is if you're willing to live with the lower quality visuals. (This is dependent on the screen resolution you play at as well.) You don't need 4k textures when you play at 1280x720 all the time, for example.

BTW, the issue with SSDs mentioned earlier is mainly on the WRITE side. READ operations are generally not a problem. However, in KSP's case, I can't be sure as I know KSP keeps quite a log going and sometimes a LOT can happen in a game session to bloat that log, especially with lots of mods. I'd get a second SSD just for KSP/games in general rather than having it alongside your OS install. (I'm using a hybrid system via Intel RST for my game drive. OS is a dedicated SSD-only drive.) At the very least, it should be more consistent performance without OS disk operations getting in the way of KSP.

I already did the texture reduction, despite my 4k resolution screen. (I'm willing to go with lower resolutions if I can speed up loading) Unfortunately, I'm using a laptop and I move it around frequently (twice a day on weekdays), and having a RAM drive probably wouldn't be very good for me to have, considering my lack of available money.

[Lt. Alex Kerman]

idk how to make loading fast

i know how to make it slow tho,

[Linventor]

3 minutes ago, Lt. Alex Kerman said:

idk how to make loading fast

i know how to make it slow tho,

Pretty sure everyone knows how.

[sarbian]

I want to point out again that the mod I posted earlier in this thread works well for HDD users 

[Linventor]

Just now, sarbian said:

I want to point out again that the mod I posted earlier in this thread works well for HDD users 

Fair enough.

[weissel]

On 05/04/2018 at 9:03 PM, Wolf Baginski said:

It's a bit more work, but if you're using Linux you can use the "find" command to get a list of files above a certain size. Might work on a Mac command line too

find $HOME/KSP_Linux* -size +9M

 

Works on Windows at well (if you install cygwin, which I recommend for everyone who occasionally need the power of (or is comfortable with) a real, sane command line )

 

On 05/04/2018 at 11:10 PM, StahnAileron said:

Problem there is cost and the fact you'll either be chewing up power keeping the drive alive or having to re-copy the content during each power cycle or reboot. (You'll be bottlenecked by the transfer speed of the interface though, like any other data transfer-related problem.) Some RAM drive have a battery back-up system. Still, RAM drive are fairly power hungry last I looked into it.

Or you can simply use your main RAM as the RAM disk (if you have enough RAM).  Then the speed of reading and writing is basically the same speed as your RAM, vastly faster than an SSD.  And then the drive is just a s power hungry as your other RAM[1].

And sequential reading from a HDD does need minimal seek time, so if you, say, save an image of the RAM disk (or otherwise structure the data and file layout to be mostly sequential), the loading of 8 or 16 GB should be on the order of 40 - 110 s (3/4th of a minute to just less than 2 minutes), assuming 150-200 MB/s from the HDD and uninterrupted bulk loading (no other disk activity going on).

[1] that's not totally true ... Laptops in sleep mode often purge all the RAM they can, i.e. everything that can be recreated from e.g. the disk(s), which means amongst others any cached disk data by the OS (with dirty pages being written out first, of course) so they can shift what they really have to keep into the lowest number of DRAM chips,.  Since DRAM needs power (the cells have to be refreshed IIRC 2-3 times per second), less chips to refresh means less power used, longer battery life, ...

[traisjames]

On 3/22/2018 at 4:44 PM, MaximumThrust said:

I use more than 100 mods, and reduced the load times by almost half reducing the size of 4k textures, and other textures that were too big. I used a program called Ordenator: https://www.nexusmods.com/skyrim/mods/12801/

Ohh...need a mac version of that for me