[WIP] Loading textures only as required

[Faark]

Hey guys,

this mods loads textures as required instead of on start. It is still far from done (thus not being under releases... loaded textures are not even DXT compressed), but it is time find out about potential compatibility issues and collect a bunch of helpful crash reports

Download Plugin Version 3.3

(for KSP 0.25; Different versions; Source available here or via ILSpy)

It is an experimental release, so don't expect it to not break anything!

Key features

- Part textures won't be loaded on startup (cuts initial loading times pretty much in half).

- Starts to background-load a low-res 32x32 thumb for any texture managed by this mod once the game finished loading.

- Once a texture is required (used in flight or editor), the mod will replace that thumb with the high resolution texture.

- There should barely be any performance impact at all caused by this mod loading textures.

Installation

- Make sure you satisfy all requirements (see below).

- Download zip archive and drop the containing .DLL into GameData.

Requirements

- Windows & DirectX9 only, for now.

- Latest .NET Framework (4.0) installed.

- Visual C++ Redistributable Packages for Visual Studio 2013 (vcredist_x86.exe) installed.

License

All rights reserved for now, until i've decided what real license to use. You will likely get approval for *insert stuff*, so just aks.

Do whatever you want as long as you mention that your stuff is based on this and ofc do whatever is inferred by licenses of 3rd party stuff used by this project.

Known Issues (at least the more important ones)

- Not really a problem, but even compared to stock KSP the current memory savings might not be as huge as expected, since this mod currently doesn't compress loaded textures.

- The current versions aims to crash KSP in case of unexpected situations. Please report those.

- Textures used on Parts might be up/down-scaled depending on part-lifetime, regardless of potential other usages.

- The current memory management is rather underdeveloped and will happily attempt to load more textures than memory available, resulting in a crash (e.g. when quickly browsing parts in Editor).

Mod Compatibility

- Active Texture Management: Compatible since ATM 3.0. LOD will not work with older versions of ATM

- TextureReplacer: You have to disabled texture compression and cannot use it to replace part textures.

-

Other mods might depend on LOD or like ATM would take control of textures thus preventing LOD to do the same. Thus LOD usually has to be loaded before those are.

KSP loads mods as it finds them, and it recursively searching for them in GameData. Files are processed before directory, each sorted by name. The the following mods would be loaded in this order:

GameData\ModA.dll

GameData\ModC.dll

GameData\ModB\ModCode.dll

GameData\ModD\BlaStuff.dll

The easiest way to make sure LOD is loaded first is to drop the DLL file directly into GameData, without creating a subdirectory. Feel free to rename the DLL, in case you want stuff like version info. You can also place it into a subdirectory, but you better name it sth like "aaLoadOnDemand", so it is loaded before ATM. Regardless of what you do, LOD will create GameData\LoadOnDemand for its config and temporary files.

- Please report any problems you occur when using LOD and please be detailed as possible.

* Logs: Beside KSP's usual detailed log at KSP_Data\output_log.txt LoD also has its own file at GameData\LoadOnDemand\LoadOnDemand.log. Both should usually be included when reporting an issue.

* RAM: LoD will often be used when available memory is rare, thus a lot of problems are memory related. Task manager isn't particular great for that job as well, especially in its default configuration. Microsoft's VMMap is a great little tool to visualize the actual memory usage of a process. Use it e.g. to verify there actually is enough memory available, and saving & including one of its mmp-files can't hurt as well.

Thanks for your interest & testing. I hope this project will be useful to you and any form of feedback is welcome!

Edited October 13, 2014 by Faark
updated dl link to v3.3 for KSP 0.25

[blackheart612]

It doesn't slow down if it loads the high res texture when clicking a part?

[Supergamervictor]

Yet to fix my crashes.I see a good potential in this mod

[Bloodbunny]

Hmm, this could be a godsend if properly developed. Is it possible to make it UNload a texture from RAM once it is no longer in editor or flight mode?

[acc]

good idea, I'll keep an eye on this project

[MK3424]

I felt a significant difference in loading times and FPS.

And haven't came across a problem when closing KSP.

One question remains: how do you close LOD's window ingame?

[Faark]

It doesn't slow down if it loads the high res texture when clicking a part?

It shouldn't be noticeable, yes.

Is it possible to make it UNload a texture from RAM once it is no longer in editor or flight mode?

Thats kind of the point and the mod already does so (though there might be a delay... thus the current version might crash on very fast VAB browsing).

I felt a significant difference in loading times and FPS.

For the better, i hope (Though FPS shouldn't really change)

And haven't came across a problem when closing KSP.

Interesting... scary. Through it would/should be gone with next version anyway.

One question remains: how do you close LOD's window ingame?

It will be gone once the next version is out, hopefully tomorrow. For now just drag it somewhere in the bottom where its pretty much outside of the game window and doesn't distract you.

Yet to fix my crashes.I see a good potential in this mod

Still crashes? It shouldn't, unless there are some nasty mods that spend lots of RAM on part-unrelated stuff... Kerbin City really took a lot in my test environment.

Have you properly installed this mod and its dependencies? In that case log files would be nice, both output_log.txt and GameData\LoadOnDemand\LoadOnDemand.log

[MK3424]

I think i have a issue:

I installed all the required programs but i get this in the debug menu DllNotFoundException: NetWrapper.native

And the LOD's window is blank.

What did i do wrong?

[6.forty]

What about this is limiting it to Windows only and how many penguins would I need to throw at it to try this out? Or is it using DirectX voodoo of some sort?

[Bloodbunny]

Hmm, I'll have to give this a shot.. it has potential to make the texture compression mod mostly obsolete.

[BananaDealer]

Hmmm... Interesting....

[Bloodbunny]

Well...tried it, but not sure if its working or not.. I see a small window pop up, but it never displays any info; not in flight, not in the vab/sph. Memory usage at main menu is around .4 GB higher than what I got with the Active Texture Management mod. Will continue trying it

[Faark]

it has potential to make the texture compression mod mostly obsolete.

That is definitively the intention. But it looks like i have missed a dependency that was present on the computers i developed & tested on. Hope to have more information later today.

Update: I think it is Visual C++ Redistributable Packages for Visual Studio 2013. Microsoft offers versions for 32bit, 64bit and ARM (nothing to do with asteroids in that case^^) devices. This mod does require the one for 32bit, filename "vcredist_x86.exe". I haven't found a direct link to that file and worse, the old dl link defaulted to a wrong version. Please try this one, it will at least force you to choose.

Edited March 24, 2014 by Faark

[MK3424]

It works!

Fault was due to wrong C++ pack (had installed x64, should be x86)

only info button needs rework: keeps closing info window.

[Bloodbunny]

OK! Getting vcredist x86 seemed to do the trick. The game loaded faster than I have ever seen, unfortunately, the game now consistently crashes just as it tries to go to the main menu... mod conflict? Probably... Will make a copy of the KSP folder and delete all mods, adding them in one by one until the culprit is found.. ah bug hunting...

[Faark]

Wait, crashes are great. This build is supposed to crash on any form of texture loading issue, so we will actually notice, can identify and resolve issues. Can you send me the logs, pls? KSP.log, KSP_Data\output_log.txt and especially GameData\LoadOnDemand\LoadOnDemand.log

But yes, ActiveTextureReduction might be a problem. It might attempt to compress textures, what this mod doesn't yet support and thus "quits" once it encounters a compressed texture.

Edited March 24, 2014 by Faark

[Bloodbunny]

I took out Active Texture Management so that isn't an issue. I went back to a stock only folder and the mod works/loads the game fine. I have the files in my dropbox to share if you have one as well

EDIT: Yeah, I'm pretty sure this has the potential to be the best mod ever. ATM is good, but it takes the length of a presidential term to load the game, and the texture reductions make KSP look like crap. This thing lets you run full res textures, and the game loads faster than a greased jackrabbit on crack

EDIT (part 2): Ok, TextureReplacer is absolutely incompatible, using it WILL cause a crash 100% of the time. This is big because even stock only players like to replace the hideously ugly default skybox. Also, the texture swapping feature for StretchySRB does not work. Oddly enough, the texture swapper used by FireSpitter works (sort of).

Edited March 24, 2014 by Bloodbunny

[MK3424]

I still prefer this over active texture reduction (ugly parts in VAB/SPH and inflight).

Even with it hiccups the game runs smoother than before ( tested with continuing building space station with KSO shuttle).

[rbray89]

Hey guys,

this mods loads textures as required instead of on start. It is still far from done (thus not being under releases... loaded textures are not even DXT compressed), but it is time find out about potential compatibility issues and collect a bunch of helpful crash reports

Download Plugin

(Different versions; Source available here or via ILSpy)

http://i.imgur.com/hmJPU97l.png

It is an experimental release, so don't expect it to not break anything! In fact, it does make KSP go haywire when you try to quit the game. I kinda know the cause, but fixing it is rather difficult...

Key features

- Part textures won't be loaded on startup (cuts initial loading times pretty much in half).

- Starts to background-load a low-res 32x32 thumb for any texture managed by this mod once the game finished loading.

- Once a texture is required (used in flight or editor), the mod will replace that thumb with the high resolution texture.

- There should barely be any performance impact at all caused by this mod loading textures.

Installation

- Make sure you satisfy all requirements (see below).

- Download zip archive and drop the containing .DLL into GameData.

Requirements

- Windows & DirectX9 only, for now.

- Latest .NET Framework (4.5.1) installed.

- Visual C++ Redistributable Packages for Visual Studio 2013 (vcredist_x86.exe) installed.

License

All rights reserved for now, until i've decided what real license to use.

Known Issues (at least the more important ones)

- Yes, this mod makes KSP freeze when you attempt to quit the game, as mentioned earlier.

- Not really a problem, but even compared to stock KSP the current memory savings might not be as huge as expected, since this mod currently doesn't compress textures.

- The current versions aims to crash KSP in case of unexpected situations. Please report those.

- Probably not compatible with other texture manipulating / compression mods.

- Textures used for Parts and other something else will be up/down-scaled depending on part-lifetime, regardless of potential other usages.

- The current memory management is rather underdeveloped and will happily attempt to load more textures than memory available, resulting in a crash (e.g. when quickly browsing parts in Editor).

Thanks for your interest & testing. I hope this project will be useful to you and any form of feedback is welcome, including suggestions on how to name this mod!

I had considered doing something like this, but I don't like the idea of increased loading times between vessel swaps, and the mod incompatibility would drive me CRAZY. I haven't looked at your source, but if you are doing what I think you are doing, this would mess up any "partless" mods (eg. TextureReplacer, RasterPropMonitor, VisualEnhancements). This would also be incompatible with the "Distant Orbital Objects" mod as well. (it creates a "dummy" model of un-loaded vessels to look at when too far away)

[sidfu]

the best way for tis mod to do is to try to have where u can say these folders are the ones to do and dont touch ther others type of setup. so say for rt2 folder for example u would say only to touch the parts folder but dont touch the part with the .dll. maybe that would help somehwhat with the mod incompatiblites

[Faark]

Hm, TextureReplacer doesn't seem to replace part textures, so that shouldn't be a problem. But it does try to compress all textures... likely including the ones this mod wants to handle. You might want to try turning it of in your TextureReplacer.tcfg (isCompressionEnabled). StretchySRB is probably an easy fix (next version), looks like i missed to set the textures name. Can't ofc tell how it will behave once the names are like in the stock game. What do you mean with "sort of"? Low-res-version?

Even with it hiccups the game runs smoother than before.

Does that mean you do notice this mod loading stuff? How bad is it?

I had considered doing something like this, but I don't like the idea of increased loading times between vessel swaps, and the mod incompatibility would drive me CRAZY. I haven't looked at your source, but if you are doing what I think you are doing, this would mess up any "partless" mods (eg. TextureReplacer, RasterPropMonitor, VisualEnhancements). This would also be incompatible with the "Distant Orbital Objects" mod as well. (it creates a "dummy" model of un-loaded vessels to look at when too far away)

There will be mod incompatibilities. But this mod works on a white-list (i kinda create my own database of textures that probably belong to parts) and only handle those textures, so "partless" mods should be fine (though they clutter up memory, of). If it gets established i hope to provide an easy-to-use API in case someone want to integrate optional support into his mod. Haven't looked at Distant Orbital Objects, yet.

[rbray89]

Hm, TextureReplacer doesn't seem to replace part textures, so that shouldn't be a problem. But it does try to compress all textures... likely including the ones this mod wants to handle. You might want to try turning it of in your TextureReplacer.tcfg (isCompressionEnabled). StretchySRB is probably an easy fix (next version), looks like i missed to set the textures name. Can't ofc tell how it will behave once the names are like in the stock game. What do you mean with "sort of"? Low-res-version?

Does that mean you do notice this mod loading stuff? How bad is it?

There will be mod incompatibilities. But this mod works on a white-list (i kinda create my own database of textures that probably belong to parts) and only handle those textures, so "partless" mods should be fine (though they clutter up memory, of). If it gets established i hope to provide an easy-to-use API in case someone want to integrate optional support into his mod. Haven't looked at Distant Orbital Objects, yet.

Just took a look at how you are doing it. WOW. You are not doing at all what I thought you were doing. I didn't know you could hook into DirectX calls like that. Clever!

The way I was thinking of implementing it would be more system agnostic. Using KSP callbacks for loading and managing textures, replacing them with the "placeholder" thumbnail when not in use. I was also playing around with the idea of using a MEMORY cache in an external process using RPC or such to fetch the texture to get around the single-process restriction. Don't know if unity can launch separate processes though

BTW: If you don't mind the MIT license, ATM handles JPG, PNG, Targa, and MBM texture loading with correct NormalMap conversion. https://github.com/waka324/TextureCompressor/blob/master/TextureCompressor/TextureConverter.cs

[MK3424]

Does that mean you do notice this mod loading stuff? How bad is it?

I meant the gui crashing and the LOD info panel not keeping open.

Still a awesome plugin.

[Bloodbunny]

TextureReplacer works fine if you disable texture compression (thanks for the tip!) RasterPropMonitor and VisualEnhancement both seem to work just fine. Distant Objects I am not sure of.. The distant planets work just fine, but I don't have a current save in a situation to test out the distant vessels, someone else should test this

[sidfu]

TextureReplacer works fine if you disable texture compression (thanks for the tip!) RasterPropMonitor and VisualEnhancement both seem to work just fine. Distant Objects I am not sure of.. The distant planets work just fine, but I don't have a current save in a situation to test out the distant vessels, someone else should test this

if uw ant to test the distant vessel just try to do a redev but untarget the other vessel once set up the node then watch toward the target and see if u see it once u get so close u should see the moving star of it