rbray89

[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

Recommended Posts

Some of the Interstellar cloud textures are currently being converted at 1:1 scaling (my settings) and are well over 70MB a peice...is this normal? Is that because the mipmaps are combined into one file?

Share this post


Link to post
Share on other sites

So, it was actually the CityLights folder that I needed to delete and re-cache to get the new EVE working (I figured I'd try clearing Terrain after Clouds and Atmosphere didn't help, and then I tried CityLights after that). Here's my hypothesis as to why: EVE 7-4 has main.tga, and Overhaul has main.png, but ATM only stores one "main" cache, so ATM might have seen that it already had a "main" and didn't bother re-caching. After deleting that folder, EVE Overhaul asks to load main.png, ATM intervenes and caches the correct file, and they all go home happily ever after.

Share this post


Link to post
Share on other sites
So, it was actually the CityLights folder that I needed to delete and re-cache to get the new EVE working (I figured I'd try clearing Terrain after Clouds and Atmosphere didn't help, and then I tried CityLights after that). Here's my hypothesis as to why: EVE 7-4 has main.tga, and Overhaul has main.png, but ATM only stores one "main" cache, so ATM might have seen that it already had a "main" and didn't bother re-caching. After deleting that folder, EVE Overhaul asks to load main.png, ATM intervenes and caches the correct file, and they all go home happily ever after.

This makes sense.

Share this post


Link to post
Share on other sites
Any way of pairing this with DDS4KSP? The first conversion on ATM takes an ungodly amount of time - DDS4KSP does the whole GameData folder in under a minute...

It cheats :) The problem is that it links Microsoft DirectX libraries and uses conversion mechanisms from it (likely optimized, machine-code or graphics driver conversion), and I want all my mods to be platform agnostic.

Share this post


Link to post
Share on other sites
It cheats :) The problem is that it links Microsoft DirectX libraries and uses conversion mechanisms from it (likely optimized, machine-code or graphics driver conversion), and I want all my mods to be platform agnostic.

Hah, that sneaky son of a... still though, the difference is astronomical; surely whatever dark magic DirectX stuff is working isn't (almost literally) 50x or 100x faster than what you're doing...? Or is it?!

Share this post


Link to post
Share on other sites
Hah, that sneaky son of a... still though, the difference is astronomical; surely whatever dark magic DirectX stuff is working isn't (almost literally) 50x or 100x faster than what you're doing...? Or is it?!

Probably is actually... Heck, if I were able to make the implementation a compute shader, or hook into the vector processing power that the graphics hardware has the conversion could be done in minutes EASILY. This is my guess what the DX stuff does.

Share this post


Link to post
Share on other sites

Hi I'm having trouble first time loading the "Kerbin1" texture from Astronomer's Visual Pack. Tried twice each for about 20 minutes on "kerbin1". Is it broken or is it just taking really long?

Share this post


Link to post
Share on other sites

rbray, I just updated ATM (x86 Aggressive) and Nertea's MkIV parts, and found that any new parts that reference another parts texture in the asset definition aren't showing up in game. I've been through the error log, and my barely trained eye didn't spot much. Could this be an ATM thing?

Edit: this is on the second startup; the game had already loaded to completion once, so everything was cached already

Edit: posted error log in the MkIV thread, thought it belonged here too: https://www.dropbox.com/s/23fj7b8lfq5lk5l/KSPLog.rtf?dl=0

Edited by Starbuckminsterfullerton

Share this post


Link to post
Share on other sites
rbray, I just updated ATM (x86 Aggressive) and Nertea's MkIV parts, and found that any new parts that reference another parts texture in the asset definition aren't showing up in game. I've been through the error log, and my barely trained eye didn't spot much. Could this be an ATM thing?

Edit: this is on the second startup; the game had already loaded to completion once, so everything was cached already

Edit: posted error log in the MkIV thread, thought it belonged here too: https://www.dropbox.com/s/23fj7b8lfq5lk5l/KSPLog.rtf?dl=0

This was a silly omission on my part. Should be fixed in the next version.

Also working on trying out the Nvidia Texture Tools converter. Seems faster, but not sure. If you want to try it out, you can try it here: https://github.com/rbray89/ActiveTextureManagement/tree/71786a492b7d5fe0b6a0a56b2081ba748fd64fa0 (look at the file list and choose a release zip file). Then you'll need to edit the main config (boulderCo/ActiveTextureManagement.cfg) and add an item like "USE_NVIDIA=true"

Share this post


Link to post
Share on other sites

so dont use newest version till u fix it not allowing asset type parts not showing?

Share this post


Link to post
Share on other sites

The main way I see to create the cache faster would be to convert multiple image at the same time in multiple threads, but that would mean breaking the loading system. Unless one of the converter (Nvidia) can use threading.

Share this post


Link to post
Share on other sites
ACTIVE_TEXTURE_MANAGER_CONFIG

{

folder = ThemodIwantToSkip

enabled = false

}

just put this kind of file along with the others .cfg in boulderCo/activetexturemanagertconfig

Definitely not working, still does the caching of the BouldCo folder after editing the cfg file included to false and/or deleting the OVERRIDE code included.

Share this post


Link to post
Share on other sites
Is it normal to load for 12 hours?

I don't know about 12 hours, but depending on your computer and how many mods you have (and also if you keep alt-tabbing away from KSP when it's using DirectX, like I did...), multiple hours is definitely not unreasonable.

Share this post


Link to post
Share on other sites

No clouds with ATM + Interstellar. Tried taking out the main and detail tga's and recacheing the CityLights folder as suggested and it did not work. Any suggestions rbray?

Share this post


Link to post
Share on other sites

First up, many thanks rbray89 for the update. As I'm back playing KSP I'll be sure to try it out at the weekend :)

The main way I see to create the cache faster would be to convert multiple image at the same time in multiple threads, but that would mean breaking the loading system. Unless one of the converter (Nvidia) can use threading.

Way back when I was playing with the cache demo I also spent some time experimenting with multi-threading the initial conversion process. Great gains are there to be had, though at that time the conversion was simple enough to be split into two ( eg thread1 did the top half, thread two did the bottom half ) or four regions. Not looking at the latest code, and since I have no knowledge of texture formats I cannot comment on the ease of multi-threading a single texture.

Does the loading system process textures one at a time or is there some kind of 'list of textures to be loaded' that could be threaded and delivered in the expected order?

On the very first texture load, could rbray89 scan the cache vs textures requiring compression/caching and process then all en mass in some threaded code before resuming the current loading sequence?

Share this post


Link to post
Share on other sites

That's a different mod causing that, I use KSPI and I'm not seeing this behavior. What other mods are you running, that would help a lot in finding the problem.

Share this post


Link to post
Share on other sites

This would be a great mod if other mods had alternate DXT versions in their textures so there was no slow load in the initial run, nearly 3hrs for me only to find my clouds disappeared and no planetshine either. Reverting back to ATM 3-1 till further notice

Share this post


Link to post
Share on other sites
Does the loading system process textures one at a time or is there some kind of 'list of textures to be loaded' that could be threaded and delivered in the expected order?

One at a time

On the very first texture load, could rbray89 scan the cache vs textures requiring compression/caching and process then all en mass in some threaded code before resuming the current loading sequence?

He could but the lack of use feedback would be bad :(

Share this post


Link to post
Share on other sites
...and I want all my mods to be platform agnostic.

Maybe I'm confused--but by platform you mean "graphics APIs," right? As in, not dependent on DirectX or OpenGL?

Someone on Google+ asked if ATM only worked on Windows, and I replied yes since DLL files are Windows resources, but your comment has me wondering if there's something I missed.

Share this post


Link to post
Share on other sites
Definitely not working, still does the caching of the BouldCo folder after editing the cfg file included to false and/or deleting the OVERRIDE code included.

Yep, not working.

- - - Updated - - -

Definitely not working, still does the caching of the BouldCo folder after editing the cfg file included to false and/or deleting the OVERRIDE code included.

Yep, not working.

Share this post


Link to post
Share on other sites
Maybe I'm confused--but by platform you mean "graphics APIs," right? As in, not dependent on DirectX or OpenGL?

Someone on Google+ asked if ATM only worked on Windows, and I replied yes since DLL files are Windows resources, but your comment has me wondering if there's something I missed.

You are wrong. DLL used by KSP plugins work on OSX and Linux without any change. They are .Net compiled code and those are platform agnostic.

Share this post


Link to post
Share on other sites
One at a time

He could but the lack of use feedback would be bad :(

Then do both?

The main thread does exactly what it does right now and one or more background threads speculatively run around the cache and the uncompressed textures processing as necessary. Would need to arrange things so no threads try to process the same texture, including the temporary blocking of the main thread should it's target be one currently being worked on in background, but it looks doable?

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.