[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

[rbray89]

This is a simple mod to reduce texture memory use by compressing/resizing them to DXT1 & DXT5. This performs the compression to as many textures as possible on game load. This will include ALL textures, including mods.

Credit to Sarbian for demonstrating how to go about compressing/modifying textures as they are being loaded.

MORE credit to Sarbian for demonstrating DXT compressed loading.

Even MORE credit to Sarbian for implementing the multithreaded portion of DXT compression!

Credit to ecat for demonstrating/suggesting a caching mechanism for faster loading.

MORE credit to ecat for suggesting multithreaded compression.

Credit to Faark for the idea of pre-loading textures.

Credit to Green Skull for the wonderful donation of additional configs.

Credit to Kerbas_ad_astra for the donation of additional configs.

This mod allows you to save LOTS (hundreds of MB to a couple GB!) of memory on textures!

The first load will take a VERY long time (can be upwards of an hour) and it might take multiple runs if you are close to the memory limit. So sit back and let it load, if it crashes, just bring it up again. Eventually it will load fully. Exit, then load again. It will then load VERY fast, and fully utilize DXT compression/loading.

If you have any issues, we will need your output_log.txt file from KSP_Data folder. I will also need KSP version, window/linux/mac version and architecture, and memory size. Please don't post an issue without it. I will act as though you don't exist if you don't post this info.

What Basic does:

• Compresses all textures to DXT format
  
• Generates mipmaps for all textures.
  
• Only resizes normal maps to 1/2 size, with a max of 512x512.
  

What Aggressive does:

• Compresses all textures to DXT format
  
• Generates mipmaps for all non-normal map textures.
  
• Rresizes all textures to 1/2 size, with a max of 256x256 for normal maps.
  

If you use the Aggressive version, you should have no need for texture reduction packs on any mods! If you do use texture reduction packs, you may have to edit configs to specify normal maps.

BoulderCo/textureCompressorConfigs/textureCompressor.tcfg to alter the paremeters. Documented within.

5.0:
Updated for KSP 1.0
4-3:
More configs (thanks to Kerbas_ad_astra)
Folders use regex matching for advanced folder handling
Everything under ActiveTextureManagement
Misc. fixes
4-2:
Multithreading, Uncompressed, & Fixes for huge textures
4-1:
Fixed weird bug with EVE and other mods. DOES NOT need to re-cache textures.
4-0:
DXT caching instead of PNG.
3-8:
Duplication fix, min size, & extension based loader disable.
3-7: 
fixed undefined folders issue
3-6:
Sarbian figured out how to override KSP's texture loading altogether.
3-5:
.25 Update
KSP broke the previous loading mechanism, so now loading is forced to be considerably slower.
3-4:
License, B9, Squad Fix
3-3:
.24 fix
3-2:
Fixed configs for EVE.
3-1:
Added more configs from Green Skull.
3-0:
Pre-loads textures at startup to save time/memory.
Will speed up loading times considerably.
Re-named.
Config alterations.
2-15:
2-15-1: Added KSO config.
REALLY fixed normalmaps. For real this time guys.
Now using a dynamic cache mechanism (as suggested by ecat on KSP forums).
.1: Added PlanetFactory config.

2-14:
Fixed mipmap setting for normalmaps.
Added enable option in configs.
Moved to regex config parsing.
Due to regex parsing, merged folder and specific texture parsing.
Updated config files to match.
Renamed config files to not interfere with .cfg loading.

2-13:
Edited exceptions for warpplugin(interstellar) and JSI.
Added GC code to clean while loading.

2-12:
Added tons of new configs.
Updated the config loading to be less sensitive to incomplete configs.

2-11:
Added more advanced folder control

2-10: 
Fixed issue with textures that aren't managed (but still compressed) by plugin.
Added KSPX config 
Added AIES_Aerospace config
Added documentation to BoulderCo/textureCompressorConfigs/textureCompressor.cfg

2-9: AVOID USING, WAIT FOR UPDATE ([B]requires a game load and restart if coming from 2.0-2.4[/B])
UPDATE: A glitch has been discovered that can cause parts that do not have configs to not be loaded.
Fixed TGA reading.
Added PorkWorks config (habitat pack)

2-8: ([B]requires a game load and restart if coming from 2.0-2.4[/B])
Moved to new config mechanism
Fixed minor *_NRM.tga file issue.

2-7: ([B]requires a game load and restart if coming from 2.0-2.4[/B])
Changes to handle normal maps properly.

2-6: ([B]requires a game load and restart if coming from 2.0-2.4[/B])
Added tiny texture protection from compression/resize.

2-5: ([B]requires a game load and restart if coming from 2.0-2.4[/B])
Moved to a method that does NOT require disk changes to be made. Will automatically revert textures back.
Created two different profiles for re-sizing/compression (basic/aggressive)

With the aggressive settings, when running with KW and B9 I'm running at just over 1.5Gb without texture reduction packs.

2-4:
Remove alpha dropping. Messes up normal maps.

2-3:
Added explicit folder selection, to add a mod, the path has to be added to the folder list in the config. B9 and KW are added as defaults.
To convert textures back to mbm, simply remove the folder from the list.

2-2:
Hotfix to remove normalmap converion.
Hotfix to actually apply texture resizing.
Allows users to manage all aspects of texture loading. 
Increases start-up considerably in some cases, but allows LOTS of memory to be saved in the process. 
Config allows many texture parameters to be controlled. 
Log shows the end-state of textures and how much memory is saved.

1-1 Changes:
Derived from github user sarbian's commit, textures are now compressed as they are loaded, so you can theoretically cram even more into your game!
Now shows in the log approximately how much memory is saved.

Download:

[sTABLE] https://github.com/rbray89/ActiveTextureManagement/releases/tag/5-0

This is done in my spare time, and any contribution helps!

Source:

https://github.com/rbray89/ActiveTextureManagement

License (MIT):

https://github.com/rbray89/ActiveTextureManagement/blob/master/README.md

Edited April 28, 2015 by rbray89

[Bobhendly]

Edit: There we go.

Will this break any mods of any kind? I may consider getting it.. Im starting to get RAM Crashes

[rbray89]

Edit: There we go.

Will this break any mods of any kind? I may consider getting it.. Im starting to get RAM Crashes

I don't expect it to. The only side effects I can think of would be that some textures might not look as good, Normal/Heightmaps might be a little different.

Install/removal is VERY easy. Just one plugin file.

[localSol]

I'm using about 10 of the larger texture-including mods and I already sized down most of them by editing mbms to half or quarter dimensioned tgas. This makes my install even better! I got this just now at the main menu after loading :

KSP private working set with lotsa mods sized-down and squad texture reduction pack : 2,527,784K

Same install with Active Memory Reduction Mod : (-) 1,895,684K

(~) 632,100K saved (!)

All right!!

[rbray89]

Nice. This is pretty much what I expected to happen. I kept getting the excuse that people didn't have room for my Cloud and City Lights mod. Shouldn't be as much of an issue now.

[Storywalker4]

You sir... are amazing. NOW I CAN ACTUALLY TURN MY GRAPHICS UP (if only a little )

[DasBananenbrot]

Nice. This is pretty much what I expected to happen. I kept getting the excuse that people didn't have room for my Cloud and City Lights mod. Shouldn't be as much of an issue now.

Well I didn't used it as a excuse because I really wanted to use this mod and it felt to not use it. But now there seems to be salvation xD

[DaZombieKilla99]

Between this and ferram's Joint Reinforcement mod, I hope to see more "utility"-esque mods to help out KSP in general that doesn't directly add content but helps the game. Keep it going guys!

[Thourion]

...(~) 632,100K saved (!)...

Are you telling me that you got a 600mb reduction with 1 dll ?

rbray! quick add 1 dll more:)

Jokes aside, Im gonna try it as well, if this works we need to make you a statue.

[wasmic]

Nice. This is pretty much what I expected to happen. I kept getting the excuse that people didn't have room for my Cloud and City Lights mod. Shouldn't be as much of an issue now.

I kept getting the excuse that people didn't have room for my Cloud and City Lights mod.

I kept getting the excuse.

You kept getting an excuse, and then went out of your way to remove that excuse??? You, good sir, rock.

[BoZo]

So is this the same thing as: http://forum.kerbalspaceprogram.com/threads/49011-Testing-Compressing-Textures-in-Memory

[rbray89]

So is this the same thing as: http://forum.kerbalspaceprogram.com/threads/49011-Testing-Compressing-Textures-in-Memory

Haha, very similar, but this won't fill your log with errors, and will only load once.

[BoZo]

Haha, very similar, but this won't fill your log with errors, and will only load once.

Sounds good, the other one reduced KSP's RAM usage by 900-1000 MBs for me.

[localSol]

Are you telling me that you got a 600mb reduction with 1 dll ?

rbray! quick add 1 dll more:)

Yep! Though I think the fact that I made several mods' textures smaller probably helps compression too, though only a guess, I don't know the specifics of how Dxt works, I would guess that there are fewer ways to represent the data in a smaller texture than a larger one so that may help compression efficiency? Also definitely helps on textures with lots of repeated same-color or normal level, like in stretchySRB tanks, SMA's ducted fan textures(this needed a huge manual reduction though), several b9 adapters. If you edit and resize kethane's textures to use TGAs, you'll get odd looking textures on the external fuel tanks and a few non-green areas on others because some sort of keying to the green coloring is lost, but you can get it down to a small fraction of its normal size with not much quality loss. IIRC the large fuel tanks look best with at least 512x512 though I decided to reduce the normals more. Edit : That is, at least 512 in the larger dimension, if applicable. Also IIRC needed to flip them vertically.

Edit : Thank you rbray89!

Edited November 20, 2013 by localSol
edit : more notes

[Thourion]

Oh boy, i grabbed it and made 3, quick game load-do some stuff-and exit, in turns. Same game installation, same save, same loading of ships.

A Load. Just my heavy moded installation. Runs around 3 to 3.4 gigs when using max res. in the texture settings, and 2.7 abouts at half res.

B Load. Added your plugin as well, and i got 2.4 to 2.6 with full rez. oO, at half rez i was down to 1.6-1.8 gigs

C Load. I decided i wanted to experiment a bit more, so i removed your plugin, loaded up the A again, and went to match your ~1.6 range in ram.

I could only do it, in lower than quarter rez, where the graphics really start to crack up.

So all and all its good. Only 2 problems i could see in the short play were:

1) If anyone is using a tex reduction pack, B9's for example, you can expect to notice the less detail right away. Just go at Full rez settings you still gonna get more than 500mb of ram free. Or try to experiment, and use that mods full/normal tex pack instead.

2) The other thing of course is the longer initial game load time, since your machine needs to do extra compression? (not sure about that last part, im just saying). But at least its only at the game start, it doesnt affect later loads, like your saves or switching ships/planets etc.

Well that was my quick an dirty run, and the numbers are not measured with a "ruller", but yeah i believe anyone who gots hips of mods and cant breath (and doesnt want to go lower than half rez, i dont), deffinetlly try this. Now we only need to find the sculptor.. and a few tons of marble, one statue coming up!

[rbray89]

Oh boy, i grabbed it and made 3, quick game load-do some stuff-and exit, in turns. Same game installation, same save, same loading of ships.

A Load. Just my heavy moded installation. Runs around 3 to 3.4 gigs when using max res. in the texture settings, and 2.7 abouts at half res.

B Load. Added your plugin as well, and i got 2.4 to 2.6 with full rez. oO, at half rez i was down to 1.6-1.8 gigs

C Load. I decided i wanted to experiment a bit more, so i removed your plugin, loaded up the A again, and went to match your ~1.6 range in ram.

I could only do it, in lower than quarter rez, where the graphics really start to crack up.

So all and all its good. Only 2 problems i could see in the short play were:

1) If anyone is using a tex reduction pack, B9's for example, you can expect to notice the less detail right away. Just go at Full rez settings you still gonna get more than 500mb of ram free. Or try to experiment, and use that mods full/normal tex pack instead.

2) The other thing of course is the longer initial game load time, since your machine needs to do extra compression? (not sure about that last part, im just saying). But at least its only at the game start, it doesnt affect later loads, like your saves or switching ships/planets etc.

Well that was my quick an dirty run, and the numbers are not measured with a "ruller", but yeah i believe anyone who gots hips of mods and cant breath (and doesnt want to go lower than half rez, i dont), deffinetlly try this. Now we only need to find the sculptor.. and a few tons of marble, one statue coming up!

Haha... Yeah, there is a marginal increase in load time, and a good reduction of texture quality. But it allows for a good bit of breathing room.

[EuSouONumero345]

Man, how can ksp comunity be so awesome?

BEST FORUMS EVER!!!!!

[therealcrow999]

Lol, I was going to ask same thing, I just found that one earlier today.

With this one rbray89, can I use it even though I used a KW and KSP texture reduction mod?

Haha, very similar, but this won't fill your log with errors, and will only load once.

[Helix935]

what happens if i download this while i am already using polecatz's squad reduction

[rbray89]

Lol, I was going to ask same thing, I just found that one earlier today.

With this one rbray89, can I use it even though I used a KW and KSP texture reduction mod?

what happens if i download this while i am already using polecatz's squad reduction

It depends on how those textures are compressed. There is the possibility it could help further, or it might not. Only way to know for sure is to give it a shot.

[localSol]

what happens if i download this while i am already using polecatz's squad reduction

'Awesome happens'. I'm using it. Haven't noticed any side effects or anything.

[therealcrow999]

Feels like Christmas!

[therealcrow999]

Yeah it works ok with KW and KSP reduction pack. I getting 600 free, I am running at 2gb even, which is really good. I am going to try a run without the KSP and KW reduction pack and see where it sets me at.

[NathanKell]

Hey cool!

Suggestion: add support for a CFG file with structure

TextureCompressorBlacklist
{
	texturename1 = true
	texturename2 = true
	texturename3 = true
	// etc
}

And then before compressing a texture, check if the node HasValue(texture url) and if so skip it.

That way any, say, UI textures, that need to be left uncompressed, can be.

[therealcrow999]

With this mod i'm at 2.5gb and without this mod with KSP and KW texture reduction its 2.5gb. With this mod, KSP and KW texture reduction I am at that 2gb mark. This mod trims off another 400mb for me.

Yeah it works ok with KW and KSP reduction pack. I getting 600 free, I am running at 2gb even, which is really good. I am going to try a run without the KSP and KW reduction pack and see where it sets me at.