Loading. Just loading. How does one speed it up?

[Linventor]

I've got a copy of KSP with a ton of mods on it. (along with 3 others, namely my semi-stock, RO/RSS, and Kerbal Kommander ones) And it takes a reeeeeally long time to load. I've got over 150 mods on it, all of which aren't exactly disposable. So, my question is, what mods are there that'll speed up loading times? And no, removing the big mods is not an option here.

[Tex_NL]

You can not squeeze 10 pounds of poop in a 5 pound bag. Loose mods or be patient. Pick one.

Edited March 22, 2018 by Tex_NL
typo

[DStaal]

The only mod I know which is likely to help at all is Janitor's Closet:

Use it to 'permaprune' any parts you don't see needing, and you'll speed up your load time by a tiny bit.

More important I found is disk speed.  Put KSP on a fast SSD and that will help more than anything else, for this issue.

 

[Linventor]

1 minute ago, DStaal said:

The only mod I know which is likely to help at all is Janitor's Closet:

Use it to 'permaprune' any parts you don't see needing, and you'll speed up your load time by a tiny bit.

More important I found is disk speed.  Put KSP on a fast SSD and that will help more than anything else, for this issue.

 

I've already got janitor's closet installed, but there's not exactly anything I won't need. I mean, maybe the legacy parts, but there's probably not that many in there. (cue me finding a gigabyte's worth of legacy parts) As for the SSD suggestion, my KSP directory is close to 9 gigabytes now. I have my personal data/games/programs/memes stored on my data directory, and my OS and stuff on my OS directory. The OS is an SSD, data's hard disk. I don't really have that much spare cash, as I've got other money-suckers requiring my income. Fortunately I don't pay rent. Or bills. Ah, the joys of being 14.

[MaximumThrust]

I use more than 100 mods, and reduced the load times by almost half reducing the size of 4k textures, and other textures that were too big. I used a program called Ordenator: https://www.nexusmods.com/skyrim/mods/12801/

[Linventor]

Just now, MaximumThrust said:

I use more than 100 mods, and reduced the load times by almost half reducing the size of 4k textures, and other textures that were too big. I used a program called Ordenator: https://www.nexusmods.com/skyrim/mods/12801/

so, it just reduces the size of textures?

[MaximumThrust]

Just now, Linventor said:

so, it just reduces the size of textures?

The program have a menu with some options. It can also change the formats to save space in some textures (the ones this won't cause quality loses). Textures that don't have transparencies can use lighter formats, for example.

Some 4k textures where at ~25 mb each. Changing to 2k reduces to ~5 mb, and I can barely tell the difference. I simple used it to reduce by half all the textures in some mods that uses files that are unnecessarily big, in my opinion/taste. Is possible to see the parts loading much faster when starting up the game.

You can easily identifies this mods searching by ".dds" in your gamedata folder, and sorting them by size.

[Linventor]

Just now, MaximumThrust said:

The program have a menu with some options. It can also change the formats to save space in some textures (the ones this won't cause quality loses). Textures that don't have transparencies can use lighter formats, for example.

Some 4k textures where at ~25 mb each. Changing to 2k reduces to ~5 mb, and I can barely tell the difference. I simple used it to reduce by half all the textures in some mods that uses files that are unnecessarily big, in my opinion/taste. Is possible to see the parts loading much faster when starting up the game.

You can easily identifies this mods searching by ".dds" in your gamedata folder, and sorting them by size.

I'll look into it, thanks.

[Gargamel]

16 hours ago, Linventor said:

I've already got janitor's closet installed, but there's not exactly anything I won't need.

You might want to try this link:  http://www.hoardersanonymous.org/

 

Edited March 23, 2018 by Gargamel

[Linventor]

5 hours ago, Gargamel said:

You might want to try this link:  http://www.hoardersanonymous.org/

 

I don't hoard in IRL, just sayin'.

I mean, updating to 1.4 would be a good way of cleaning up my modlist, but the problem is the mod that I literally NEED isn't updated to 1.4, namely Kopernicus. And To Boldly Go.

Edited March 23, 2018 by Linventor

[Wolf Baginski]

I have noticed, several times, some huge textures in mods, and it can make a big difference. There's also a reduced-size option in settings, putting less load on your graphics card. I doubt using that changes loading speed, but it would give you a way of checking what the visual effects would be.

Since I am a Linux user, a Windows utility is of little use to me, but there is a .dds plugin for The GIMP. I had to use a different version of the program for the plugin to work, but it's all standard for the Linux version I use, in the Repository.

While it's easy enough to delete the transparency map, the alpha channel, I would want to check what it does before I did that. It may, for instance, affect the glossiness of a surface.

The way the file sizes change, shrinking those 20MB files can soon save a few hundred MB of disk space.

I rarely bother with using an IVA, and these have been some of the biggest users of large files. Some mod creators explicitly say they can be deleted. Some I have seen are really not optimised, huge files with such things are detailed labels for the content of a storage container, detail I have never been able to get a camera close enough to see.

 

 

[CobaltWolf]

21 hours ago, MaximumThrust said:

The program have a menu with some options. It can also change the formats to save space in some textures (the ones this won't cause quality loses). Textures that don't have transparencies can use lighter formats, for example.

Some 4k textures where at ~25 mb each. Changing to 2k reduces to ~5 mb, and I can barely tell the difference. I simple used it to reduce by half all the textures in some mods that uses files that are unnecessarily big, in my opinion/taste. Is possible to see the parts loading much faster when starting up the game.

You can easily identifies this mods searching by ".dds" in your gamedata folder, and sorting them by size.

I will say, that is VERY dependent on the parts in question. For example, many stock parts use horribly inefficient textures - no reuse, horribly high amounts of pixels used for a given surface area. However, many mod authors like myself (and I think some of the newer stock parts) use large textures as atlases that can have numerous parts which have reasonably pixel/surface ratios. In that case they are most likely desperate for every single pixel of those sheets, and won't respond well to being crunched down.

With all that said, you sound like you know which ones needed the treatment - when I first read the post I was afraid it was just batch converting every 4K texture in your GameData but after re-reading I don't think that's the case. And at any rate, it is of course up to the user to decide the balance between performance and quality

[adsii1970]

Least helpful answer but probably by far the most accurate: LESS MODS.

[MaximumThrust]

4 minutes ago, CobaltWolf said:

I will say, that is VERY dependent on the parts in question. For example, many stock parts use horribly inefficient textures - no reuse, horribly high amounts of pixels used for a given surface area. However, many mod authors like myself (and I think some of the newer stock parts) use large textures as atlases that can have numerous parts which have reasonably pixel/surface ratios. In that case they are most likely desperate for every single pixel of those sheets, and won't respond well to being crunched down.

With all that said, you sound like you know which ones needed the treatment - when I first read the post I was afraid it was just batch converting every 4K texture in your GameData but after re-reading I don't think that's the case. And at any rate, it is of course up to the user to decide the balance between performance and quality

Yes, I take a look at the textures first with PS, before deciding what I'll change or not. OPT, for example, uses 4k textures shared between a lot of parts, but reducing it to 2k don't cause any major quality loses (in my preference), and cut the size by ~75%. Some mods uses 1024 or 2048 textures for very small parts, I also reduce this ones. I guess the most appropriated term would be "textures that try to archive 4k quality", or something like that.

I like very much this approach from modders, that shares textures when possible, or even use the stock ones.

Knowing that some parts uses shared textures is also good to know when pruning parts. There's almost no benefit in pruning a part if you'll need keep another one that uses the same texture.

[sarbian]

You can try some new magic

 

[Linventor]

1 minute ago, sarbian said:

You can try some new magic

 

*sigh* I'm using KSP 1.3.1.

[sarbian]

Just now, Linventor said:

*sigh* I'm using KSP 1.3.1.

It should work on 1.3.x

Edited March 23, 2018 by sarbian

[0something0]

If you don't have an SSD and can afford to do so, then get an SSD.

[goldenpsp]

1 hour ago, 0something0 said:

If you don't have an SSD and can afford to do so, then get an SSD.

He already addressed that.

[artwhaley]

On 3/22/2018 at 10:39 AM, Linventor said:

I have my personal data/games/programs/memes stored on my data directory, and my OS and stuff on my OS directory. The OS is an SSD, data's hard disk

Moving KSP to the OS SSD would make a big difference, if you can clear up the space on that drive.

Moving all those memes to the SSD would also make it faster to access them.   Gotta remember what's important.    

 

[Linventor]

7 hours ago, artwhaley said:

Moving KSP to the OS SSD would make a big difference, if you can clear up the space on that drive.

Moving all those memes to the SSD would also make it faster to access them.   Gotta remember what's important.    

 

well, I moved KSP to it, like you suggested, and I have no idea if there's a difference or not, but just in case, I'm keeping it on there.

 

14 hours ago, sarbian said:

It should work on 1.3.x

That's good, cuz I am fully unable to use 1.4.x until kopernikus gets updated, along with To Boldly Go. (I'm doing an old challenge that I'm planning on reviving under a new thread)

[weissel]

On 22.3.2018 at 3:24 PM, DStaal said:

More important I found is disk speed.  Put KSP on a fast SSD and that will help more than anything else, for this issue.

1. If you have the RAM (24 GB minimum, I'd say), pre-load KSP into a RAM disk.  RAM is many times faster than SSDs.
2. If you use SSDs, the only way they know the computer has "thrown away" data on the HDD they emulate is when the computer actually overwrites a block --- or when it is being told by the computer.  The TRIM command tells the SSD (ranges of) blocks that carry no useful data anymore, so they need not be copied when the data has to be moved to a new erase block (128 kB usually) --- you cannot overwrite on flash memory, you need to wipe a whole erase block, which means 
   1. find which erase block to use next (wear levelling is very important for a usable life time)
   2. copy all the information not known to be useless ('overwritten in the logical 'HDD' emulation or named via TRIM) to a reserve erase block.
   3. wipe the erase block and put it in the "ready to reuse" queue.
3. This causes a more and more "fragmented" SSD, with many blocks partially full, despite the garbage collection the SSD runs in the background. 
4. Erasing is slow.
5. Writing to a flash cell where the logic does not only expect 0% and 100%, but, say 0%, 25%, 50% 75% and 100% is *really* slow as you have to poke the cell that it only holds the charge you need it to hold.  This means usually several pokes and reading the results after every poke.  But as it allows you to cram more  storage into the same flash cell(s), consumer SSDs tend to have that.  (Physically the cells are identical, and it is not unheard of an SSD that uses on/off for critical internal data, like where in the physical flash the data for the virtual block (that makes the computer see a HDD) is, or even as cache during write operations, speeding up writes (until the cache is full) and committing them to the many-values-cells in the background.
6. Many consumer SSDs are not graceful when TRIMs are sent for each block, as they are invalidated.  However, most are amenable to a, say weekly run, program that marks whole ranges as "free for reuse" via TRIM.
7. If your SSD does not do TRIM, consider a backup, a total wipe and reset to factory conditions for the SSD, and then restore what you need onto the SSD.
   This cleans up any internal fragmentation, when 128 kB erase pages are half filled and a lot of extra erasing and writing is needed to keep "data" the file system has abandoned.  (-> Write Amplification).

There are also 'prefetch' programs for Windows and Linux (and I'd be surprised if they didn't exist on OS X), which intelligently try to pre-load and cache what likely is asked for next --- they partially overlap with RAM caches which keep or toss data in RAM as the algorithm says it's worth it or not.

[Linventor]

15 hours ago, weissel said:

1. If you have the RAM (24 GB minimum, I'd say), pre-load KSP into a RAM disk.  RAM is many times faster than SSDs.
2. If you use SSDs, the only way they know the computer has "thrown away" data on the HDD they emulate is when the computer actually overwrites a block --- or when it is being told by the computer.  The TRIM command tells the SSD (ranges of) blocks that carry no useful data anymore, so they need not be copied when the data has to be moved to a new erase block (128 kB usually) --- you cannot overwrite on flash memory, you need to wipe a whole erase block, which means 
   1. find which erase block to use next (wear levelling is very important for a usable life time)
   2. copy all the information not known to be useless ('overwritten in the logical 'HDD' emulation or named via TRIM) to a reserve erase block.
   3. wipe the erase block and put it in the "ready to reuse" queue.
3. This causes a more and more "fragmented" SSD, with many blocks partially full, despite the garbage collection the SSD runs in the background. 
4. Erasing is slow.
5. Writing to a flash cell where the logic does not only expect 0% and 100%, but, say 0%, 25%, 50% 75% and 100% is *really* slow as you have to poke the cell that it only holds the charge you need it to hold.  This means usually several pokes and reading the results after every poke.  But as it allows you to cram more  storage into the same flash cell(s), consumer SSDs tend to have that.  (Physically the cells are identical, and it is not unheard of an SSD that uses on/off for critical internal data, like where in the physical flash the data for the virtual block (that makes the computer see a HDD) is, or even as cache during write operations, speeding up writes (until the cache is full) and committing them to the many-values-cells in the background.
6. Many consumer SSDs are not graceful when TRIMs are sent for each block, as they are invalidated.  However, most are amenable to a, say weekly run, program that marks whole ranges as "free for reuse" via TRIM.
7. If your SSD does not do TRIM, consider a backup, a total wipe and reset to factory conditions for the SSD, and then restore what you need onto the SSD.
   This cleans up any internal fragmentation, when 128 kB erase pages are half filled and a lot of extra erasing and writing is needed to keep "data" the file system has abandoned.  (-> Write Amplification).

There are also 'prefetch' programs for Windows and Linux (and I'd be surprised if they didn't exist on OS X), which intelligently try to pre-load and cache what likely is asked for next --- they partially overlap with RAM caches which keep or toss data in RAM as the algorithm says it's worth it or not.

Um... I no comprendo senòr? I may be a bit of a computer nerd, but I'm not that much of one. Mind dumbing down your explanation?

[Pleb]

21 hours ago, weissel said:

If you have the RAM (24 GB minimum, I'd say), pre-load KSP into a RAM disk.  RAM is many times faster than SSDs.

I hadn't thought of this before but I might try this! I have 32GB of RAM so I might give this a go as it takes over 5 minutes to load KSP and I have an SSD and a ridiculous number of mods.

[Linventor]

9 minutes ago, Pleb said:

I hadn't thought of this before but I might try this! I have 32GB of RAM so I might give this a go as it takes over 5 minutes to load KSP and I have an SSD and a ridiculous number of mods.

well, I'm glad my question helped someone else out too.