Jump to content

[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

rbray89

By rbray89
in KSP1 Mod Releases

 Share

Recommended Posts

Shad0wCatcher

Shad0wCatcher

Posted
Posted

Dante, if you haven't yet; open up the texturecompressor.cfg and add in all the folders that contain parts in your install.

I went from 2.213 @ the start screen to 1.9-ish going from 1.1-2.6.

Working on some part loss issues now though; though it may be a PEBKAC issue (never loaded my save after starting with 2.6; just loaded to start then exited).

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
rbray, a question. Right now, 2.6 aggressive destroys my memory (crashing while using about 3.5gb when the loading bar finishes), while 1.1 with the exact same gamedata folder loads ok with about 2.7gb at the start screen. My gamedata folder is about 1.8gb in size, using a lot of mods.

If I understand correctly, 2.6 aggressive has a config file where we can place folders and parts and assign instructions on compression, overrides etc. Everything not specified in the config is compressed with the 1.1 method.

Might my problem be that the 1.1 compression comes at the end of loading instead of in-the-go? Could this be altered in some way?

No, the order is the same as it was. That is odd though. Do you have a log I could see?

Link to comment
Share on other sites
Dante80

Dante80

Posted
Posted (edited)

Sure, here are the three last of them...C:

http://www./download/bj12si6ssfgy1sr/error%20logs.zip

And here is a folder structure for the gamedata (have in mind that B9 and KOSMOS only have a small number of parts included).

01-Jan-1405-29-35_zpsfd015f62.png~original

Tested with 2.6 aggressive with no changes to the config file, and 1.1.

Edited by Dante80
Link to comment
Share on other sites
Shad0wCatcher

Shad0wCatcher

Posted
Posted (edited)

Damn that's funky. My folder structure is identical apart from 2 or 3 folders and my game loads fine (I am however using both the B9 and Squad's texture reduction pack). No parts removed. 1/2 resolution in-game @ 1920x1080 with everything else maxed out.

EDIT: So not a user error; I'm missing the 1.25 meter and 2.5 meter SQUAD probe cores using 2.6. Using the SQUAD texture reduction pack along with the B9 reduction pack (drop-in replacement for 1.1). Nothing missing from B9 that I can see. Made sure to restart the program after it loaded (first to the start menu; then my save-game).

Edited by Shad0wCatcher
Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
Sure, here are the three last of them...C:

http://www./download/bj12si6ssfgy1sr/error%20logs.zip

And here is a folder structure for the gamedata (have in mind that B9 and KOSMOS only have a small number of parts included).

http://i44.photobucket.com/albums/f9/Dante80/bellycancer/01-Jan-1405-29-35_zpsfd015f62.png~original

Tested with 2.6 aggressive with no changes to the config file, and 1.1.

Just took a look at the first log... what is "Squad/Spaces/Common/common003b" from? Looks like the out of mem exception happens right after it can't be read. Try adding this to the LEAVE_READABLE section.

EDIT: added snippet.

PartLoader: Compiling Internal Space 'Squad/Spaces/GenericSpace1/internal/GenericSpace1'

(Filename: C:/BuildAgent/work/ea95e74f6e5f192d/Runtime/ExportGenerated/StandalonePlayer/UnityEngineDebug.cpp Line: 54)

UnityException: Texture 'Squad/Spaces/Common/common003b' is not readable, the texture memory can not be accessed from scripts. You can make the texture readable in the Texture Import Settings.
  at (wrapper managed-to-native) UnityEngine.Texture2D:GetPixel (int,int)

  at GameDatabase.BitmapToUnityNormalMap (UnityEngine.Texture2D tex) [0x00000] in <filename unknown>:0 

  at GameDatabase+TextureInfo.get_normalMap () [0x00000] in <filename unknown>:0 

  at PartLoader.ReplaceTextures (UnityEngine.GameObject model, System.Collections.Generic.List`1 textureNames, System.Collections.Generic.List`1 newTextures) [0x00000] in <filename unknown>:0 

  at PartLoader.CompileModel (.UrlConfig cfg, .ConfigNode partCfg, Single scaleFactor) [0x00000] in <filename unknown>:0 

  at PartLoader.LoadInternalSpace (.UrlConfig urlConf) [0x00000] in <filename unknown>:0 

  at PartLoader+.MoveNext () [0x00000] in <filename unknown>:0 
UnityEngine.MonoBehaviour:StartCoroutine_Auto(IEnumerator)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
:MoveNext()

(Filename:  Line: -1)

DynamicHeapAllocator out of memory - Could not get memory for large allocationCould not allocate memory: System out of memory!
Trying to allocate: 1398128B with 32 alignment. MemoryLabel: Texture
Allocation happend at: Line:388 in 
Memory overview

Link to comment
Share on other sites
gardocki

gardocki

Posted
Posted
Damn that's funky. My folder structure is identical apart from 2 or 3 folders and my game loads fine (I am however using both the B9 and Squad's texture reduction pack). No parts removed. 1/2 resolution in-game @ 1920x1080 with everything else maxed out.

EDIT: So not a user error; I'm missing the 1.25 meter and 2.5 meter SQUAD probe cores using 2.6. Using the SQUAD texture reduction pack along with the B9 reduction pack (drop-in replacement for 1.1). Nothing missing from B9 that I can see. Made sure to restart the program after it loaded (first to the start menu; then my save-game).

Same problem here except I lost the small and large Crew Capsules. I have too many mods to remove the texture pack so it looks like I'm stuck with 2-4 for now.

Link to comment
Share on other sites
Chebalu

Chebalu

Posted
Posted

I had the same problem, some of saved crafd couldn't be loaded, because some parts (all of them were squad's) missed. My solution : recopy the missing parts from the original game zip, and everything will be ok! (it was 4 parts)

Link to comment
Share on other sites
halrick

halrick

Posted
Posted

You have my thanks for this great mod. While there have been hickups (2.4 completly broke my install -> reinstall, but even then, with the previously working 1.1, ksp just wouldn't load), 2.6 is most definitely a step in the right direction, as ksp now requires even less ram than with 1.1 . Best wishes for the new year to you (and this great mod)!

Link to comment
Share on other sites
Frostiken

Frostiken

Posted
Posted (edited)

By the way, I was able to successfully apply severe compression to the following addons:

    folder = BobCatind/
    folder = NearFuture/
    folder = WarpPlugin/
    folder = LLL/
    folder = RLA_ElectricEngines/
    folder = Kethane/
    folder = MagicSmokeIndustries/
    folder = KSPX/

It threw a ****ton of errors, but I didn't notice anything wrong in-game. The only exception I had to make was: 

texture = LLL/Parts/Structural/LLL2x1Hull/HullNorm

It shaved nearly 1Gb off completely.

Note that on the HOME parts, which are very big, the compressed textures look worse than a game from the late 90s. Some of the Interstellar parts are pretty large as well, so again, compressed textures become very obvious and EXTREMELY hideous. It might be prudent to make exceptions for some of these to stop your game from looking like smashed buttholes :P Nobody cares when your little RCS tank is a little blurry... everyone cares when that giant engines looks like a French Impressionist painting.

EDIT: I switched to the Basic and now it freezes while loading with "[Exception]: UnityException: Texture 'Squad/Parts/FuelTank/fuelTank2-2/model001' is not readable, the texture memory can not be accessed from scripts. You can make the texture readable in the Texture Import Settings." Went back to advanced and it still does it. What the ****.

Went back to 1.1 and we're good. Guess I'm sticking with that.

Edited by Frostiken
Link to comment
Share on other sites
Sigma52

Sigma52

Posted
Posted (edited)

whats the trick to adding an exception to the 2-6 Reduction Mod? Using Basic build

Currently I keep getting hung up on command modules during the loading screen of ksp.

at first it was a command module in LLL- lack luster labs so i took it off the folder list.

to be specific it was. LLL/Parts/Command/MKVII/bridge/LLLBridgetest

After removing the folder from the list it now gets further down the loading screen until it gets snagged again

and now im getting stuck at Squad/Parts/Command/Mk1-2Pod/part/Mark1-2Pod

Ksp is not freezing or crashing from the looks of it. It loads up 3/4ths of the way then it just stops and doesn't load up the next part.

These are the folders i have listed in the cfg

FOLDERS

{

folder = Squad/

folder = BoulderCo/

folder = KWRocketry/

folder = B9_Aerospace/

folder = AIES_Aerospace/

folder = ASET/

folder = Bavaria/

folder = BioMass+Science/

folder = chroma works/

folder = heat/

folder = ExtraplanetaryLaunchpads/

folder = KAS/

folder = Kethane/

folder = Klockheed_Martian/

folder = LLL-Extra/

folder = MagicSmokeIndustries/

folder = MechJeb2/

folder = MobiusRocketWorks/

folder = ModsByTal/

folder = NavyFish/

folder = NearFuture/

folder = nothke_DROMOMAN/

folder = NothkeSerCom/

folder = Parts/

folder = RBR/

folder = RealChute/

folder = RemoteTech2/

folder = ScoopOMatic/

folder = The DoPToOT/

folder = ThunderAerospace/

folder = WarpPlugin/

}

Edited by Sigma52
Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted

I realized that there is something going on behind-the-scenes with normal maps and readability. I think KSP converts them. I moved the readability application to the end, and will be packaging a new version soon that won't require readability to be allowed except in very specific circumstances. Should be uploaded soon.

Link to comment
Share on other sites
Frostiken

Frostiken

Posted
Posted

Out of curiosity how complicated was this plugin to make? Sounds like most of the heavy lifting is unlocking access to a default Unity feature. Kind of silly that this isn't a default aspect of the game, considering their approach to loading all textures into memory at game launch.

Link to comment
Share on other sites
Galane

Galane

Posted
Posted

What I'd prefer is a separate program that does automatically what PolecatEZ did manually with the texture packs.

Throw the GameData folder at this program and it would find all the textures of a certain minimum dimension, of the file format that eats up so much memory, then do a bicubic sharper resample when shrinking the dimensions. One and done, no need for an active plugin that can have conflicts with other plugins or parts.

Link to comment
Share on other sites
LameLefty

LameLefty

Posted
Posted
What I'd prefer is a separate program that does automatically what PolecatEZ did manually with the texture packs.

Throw the GameData folder at this program and it would find all the textures of a certain minimum dimension, of the file format that eats up so much memory, then do a bicubic sharper resample when shrinking the dimensions. One and done, no need for an active plugin that can have conflicts with other plugins or parts.

Great, and while you're at it, make it compatible with Windows, OS X and Linux. Let us know when you're done. :)

That's one of the very great joys of KSP - plug-ins and mods are Unity-based, not OS-based.

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
What I'd prefer is a separate program that does automatically what PolecatEZ did manually with the texture packs.

Throw the GameData folder at this program and it would find all the textures of a certain minimum dimension, of the file format that eats up so much memory, then do a bicubic sharper resample when shrinking the dimensions. One and done, no need for an active plugin that can have conflicts with other plugins or parts.

I thought about that, but there are a couple issues:

Due to how KSP handles formats, there is really no image format that is completely suitable for managing textures perfectly. non-normal MBMs and PNGs are always compressed, PNGs don't generate mipmaps, TGAs aren't compressed, and doing it this way would mean that people would have to edit the files on disk. Something that I eventually figured out how to avoid.

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted

UPDATED. Found out what was going on. Ended up re-writing lots of code to handle normal maps, added handling for normalmapped pngs, and should work better than ever, though it *CAN* add quite a bit of time to startup.

I ran at 1.77Gb and it takes ~2.5 Minutes to load to main menu (I have an SSD so YMMV).

Have the following installed for test, but only KW, B9, Squad and BoulderCo are being managed:

B9_Aerospace

BoulderCo

DeadlyReentry

ExsurgentEngineering

Firespitter

HexCans

KineTechAnimation

KWRocketry

ResGen

Squad

TreeLoader

WarpPlugin

Link to comment
Share on other sites
WaRi

WaRi

Posted
Posted

I'm not sure about this. To be clear. This plugin (2.7) only changes how the textures are loaded, It doesn't create, modify or delete any file.

If I delete the plugin folder, KSP ends up like before the instalation of the plugin.

Is that correct?

Tnks

Much appreciated Your work.

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
I'm not sure about this. To be clear. This plugin (2.7) only changes how the textures are loaded, It doesn't create, modify or delete any file.

If I delete the plugin folder, KSP ends up like before the instalation of the plugin.

Is that correct?

Tnks

Much appreciated Your work.

Exactly. There are no file modifications (as much as I want to correct improperly marked files I won't :) )

Link to comment
Share on other sites
Nereid

Nereid

Posted
Posted
Great, and while you're at it, make it compatible with Windows, OS X and Linux. Let us know when you're done. :)

That's one of the very great joys of KSP - plug-ins and mods are Unity-based, not OS-based.

Well... he had just write it in a platform independent language with a minimal amount of discipline. Let's take Java as an example. So the "joy" in KSP is quite some kind of standard these days.

Link to comment
Share on other sites
Nereid

Nereid

Posted
Posted
Exactly. There are no file modifications (as much as I want to correct improperly marked files I won't :) )

I will test 2.7 today. But 2.6 has messed my game up. Many parts weren't loaded, so I'm still using 1.1.

Link to comment
Share on other sites
Frostiken

Frostiken

Posted
Posted (edited)
UPDATED. Found out what was going on. Ended up re-writing lots of code to handle normal maps, added handling for normalmapped pngs, and should work better than ever, though it *CAN* add quite a bit of time to startup.

I ran at 1.77Gb and it takes ~2.5 Minutes to load to main menu (I have an SSD so YMMV).

Have the following installed for test, but only KW, B9, Squad and BoulderCo are being managed:

B9_Aerospace

BoulderCo

DeadlyReentry

ExsurgentEngineering

Firespitter

HexCans

KineTechAnimation

KWRocketry

ResGen

Squad

TreeLoader

WarpPlugin

Giving it a shot.

On the plus side, you can probably look forward to rewriting it all again when Squad once again dramatically changes the texture format in the next update or two :P

EDIT: I'm not actually really seeing any big differences in RAM between 1.1 and 2.7 Basic... going to add some more mods to it and see what happens.

Edited by Frostiken
Link to comment
Share on other sites
Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...