Jump to content

[1.4+ & 1.8+] Hyperspace - Load KSP faster on HDD (or not)

sarbian

By sarbian
in KSP1 Mod Releases

 Share

Recommended Posts

Stone Blue

Stone Blue

Posted
Posted
  On 1/24/2019 at 11:24 PM, SquaredSpekz said:

How do you install this? drop the .dll in GameData or the whole zip? Could you please update the folder structure within your download so it's easier to figure where to drop the mod. 

Expand  

KSP currently doesnt read from .zips
Always extract the contents, which generally have their own folder structure that gets dropped into GameData... ModuleManager is just a dll and its subsequently generated cache files... to install, yes, the .dll just goes straight into /GameData

Link to comment
Share on other sites
  • 3 weeks later...
Lisias

Lisias

Posted
Posted

There's a curious side effect when using HyperSpace: the KSP.log is updated by large chunks of text. It's an annoyance when you live monitor the KSP.log as that last message that you are expecting to check is stuck on the buffer, waiting it to be flushed. But it also ends speeding up the KSP itself, even than just a bit. Writing large chunks of text to the log is faster than writing the same amount of data in small chunks.

So, if you need to live monitor your KSP.log, don't use this and take the hit on the loading time. Other than that, by all means, use this. :) 

  On 1/29/2019 at 3:56 PM, Gapone said:

Can it do something with my Win7 64bit 4gb RAM?

Expand  

Yes. It uses a small extra bit of RAM for each file KSP opens to make reading (and writing) slightly faster. Just a bit faster, but once you have a huge basket of files to be read, that savings pile up.

Go for it.

Link to comment
Share on other sites
  • 1 month later...
Joker58th

Joker58th

Posted
Posted

Here are my results.  i7-6820HQ, SSD, 16GB RAM, 81 mods.

[LOG 13:24:40.853] Loading Systems: Elapsed time is 101.4828s
[LOG 13:26:35.238] Loading Systems: Elapsed time is 70.92377s
[LOG 13:28:41.921] Loading Systems: Elapsed time is 69.332s
Hyperspace added
[LOG 13:31:55.804] Loading Systems: Elapsed time is 90.33533s
[LOG 13:33:33.261] Loading Systems: Elapsed time is 67.10857s
[LOG 13:35:11.984] Loading Systems: Elapsed time is 68.48885s

Link to comment
Share on other sites
  • 4 months later...
LupinoArts

LupinoArts

Posted
Posted

Here's my test results: 

Plain
[LOG 10:39:22.929] Loading Systems: Elapsed time is 607.2117s
[LOG 10:57:09.827] Loading Systems: Elapsed time is 632.0426s
[LOG 11:11:47.336] Loading Systems: Elapsed time is 472.6618s

After installing Hyperspace
[LOG 11:32:34.623] Loading Systems: Elapsed time is 641.9018s
[LOG 11:48:28.292] Loading Systems: Elapsed time is 471.1871s
[LOG 12:03:03.552] Loading Systems: Elapsed time is 472.1698s

Rig: i7-4770K 3,5GHz, 16GB RAM, Win 8.1 64bit, KSP installed on a Samsung 860 EVO 1TB SSD

CKAN lists 182 Mods installed, plus some additional 30 or so non-ckanned mods and the whole WildBlue Suite.

Link to comment
Share on other sites
sarbian

sarbian

Posted
  • Author
Posted
  On 8/17/2019 at 8:10 PM, jeancallisti said:

So you've swapped the file reader constructor... Have you considered also implementing a way to swap KSP's pseudo-JSON parser?

In combination with GameData Switcher, the game could load compressed files.

 

 

Expand  

Everything is doable but when you do larger change you need increase the risk of needing a large update after each patch. I would rather keep it simple. Compressed file would not really save much loading time. The slowdown is actually after the data is loaded (as shown by the limited impact of this mod)

Some part of the stock loading could be made faster but it s something best done on Squad side... 

 

I think @blowfishlooked at zip loading but I don't remember the results.

 

  On 8/18/2019 at 10:11 AM, LupinoArts said:

Rig: i7-4770K 3,5GHz, 16GB RAM, Win 8.1 64bit, KSP installed on a Samsung 860 EVO 1TB SSD

CKAN lists 182 Mods installed, plus some additional 30 or so non-ckanned mods and the whole WildBlue Suite.

Expand  

Yeah, the result varies wildly with the hardware. I guess you have enough memory for the whole game to be quickly cached.

Link to comment
Share on other sites
blowfish

blowfish

Posted
Posted
  On 8/17/2019 at 8:10 PM, jeancallisti said:

So you've swapped the file reader constructor... Have you considered also implementing a way to swap KSP's pseudo-JSON parser?

In combination with GameData Switcher, the game could load compressed files.

Expand  

 

  On 8/18/2019 at 7:08 PM, sarbian said:

Everything is doable but when you do larger change you need increase the risk of needing a large update after each patch. I would rather keep it simple. Compressed file would not really save much loading time. The slowdown is actually after the data is loaded (as shown by the limited impact of this mod)

Some part of the stock loading could be made faster but it s something best done on Squad side... 

I think @blowfishlooked at zip loading but I don't remember the results.

Expand  

I did.  It ends up being pretty complicated, although I got a working prototype.  It requires reaching deep into KSP's/Unity's/.NET's internals in a way that I'm still not sure I feel comfortable with.  Also some mod plugins do things that are straight up not compatible with it.

Link to comment
Share on other sites
jeancallisti

jeancallisti

Posted
Posted
  On 8/18/2019 at 7:08 PM, sarbian said:

The slowdown is actually after the data is loaded (as shown by the limited impact of this mod)

Expand  

With the vanilla game, yes, but I also suspect that there's a slowdown when there's a gazillion small files been loaded sequentially, as seen with heavily modded games. Storing those files together in blobs could help speeding up things?

Link to comment
Share on other sites
sarbian

sarbian

Posted
  • Author
Posted
  On 8/19/2019 at 4:00 PM, jeancallisti said:

With the vanilla game, yes, but I also suspect that there's a slowdown when there's a gazillion small files been loaded sequentially, as seen with heavily modded games. Storing those files together in blobs could help speeding up things?

Expand  

Caveat for this post: I did a lot of testing when I wrote this mod 15 months ago but I do not remember every results. 

The loading (reading from disk and parsing) of the cfg themselves is actually really fast. The slow part is how KSP reads models (and texture?) and build the part DB. The pure disk IO part is not really a big part of it (unless of course if you load the game from a fragmented HDD but not much can help in that case). One of the slow part is how the data stream is divided in byte/long/float/etc 

If you want to understand some of the underlying problem then I refer you to this article that explains what is actually slow (not in a KSP specific context but it is how KSP does it)  

There are some solution in this article than may help with KSP loading but as I said earlier I do not think it is a good idea for a mod to rewrite such a large part of the stock game code. It is something that should be done on Squad side but it takes times away from bugs/features and may not even be a lot faster for everyone...

Link to comment
Share on other sites
jeancallisti

jeancallisti

Posted
Posted
  On 8/19/2019 at 5:10 PM, sarbian said:

The loading (reading from disk and parsing) of the cfg themselves is actually really fast. The pure disk IO part is not really a big part of it. One of the slow part is how the data stream is divided in byte/long/float/etc 

If you want to understand some of the underlying problem then I refer you to this article that explains what is actually slow (not in a KSP specific context but it is how KSP does it) 

Expand  

Fair enough. Thanks for considering and answering!

Link to comment
Share on other sites
  • 4 weeks later...
  • 4 weeks later...
  • 3 months later...
  • 4 weeks later...
  • 5 weeks later...
robinxb

robinxb

Posted
Posted (edited)
  On 4/4/2020 at 5:06 PM, luizsilveira said:

I keep wondering, why isn't something like this in stock by now? Any real down sides?

Expand  

Because it's NOT ABSOLUTELY improving your performance.

In general, this SHOULD work, but may less stable for all users.

Actually there's some more aggressive method to improving loading speed, but more complex and much more unsafe. I'm currently trying it.

Edited by robinxb
Link to comment
Share on other sites
  • 2 months later...
Bellabong

Bellabong

Posted
Posted (edited)

Before:

Loading Systems: Elapsed time is 883.7227s

 

After:

Loading Systems: Elapsed time is 602.8457s

 

Intel 5820k@4.4Ghz, 40GB RAM, 2.5GB/s SSD. 

Can rule out caching. Thanks!

 

Modlist:

 

  Reveal hidden contents
Edited by Bellabong
Link to comment
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...