[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

[BrutalRIP]

That's a different mod causing that, I use KSPI and I'm not seeing this behavior. What other mods are you running, that would help a lot in finding the problem.

i think he means Astronomer's Visual Pack - Interstellar as i have the same issue with no clouds

[sarbian]

I had a quick look at the code and had an idea on how to to multi-threading with minimal code change.

[RandulfJ]

i think he means Astronomer's Visual Pack - Interstellar as i have the same issue with no clouds

I'm having the same issue but its with EVE's cloud.cloud.update method as far as I can tell so far. Not sure where to post the logs.. >.<

[Space Cadet]

Thanks for this mod. It did a good job on cutting down my initial memory usage but I'm having a problem with a memory leak though (goes from 2.2 gb used to 3.7 gb before crashing) after simply leaving it on the VAB screen for an hour or so. I tried a new install (downloaded all new files from steam) and tried each mod I use individually and none alone seem to cause the issue. This was very time-consuming due to having to wait after each mod was installed to see if it caused the issue. So now I'm in the situation of trying to determine which combination of two or more mods is causing my memory leak.

Does anyone happen to have suggestions on how I might best go about this as the permutations are too numerous for an ad-hoc approach to be helpful?

[jstnj]

I'm having the same issue but its with EVE's cloud.cloud.update method as far as I can tell so far. Not sure where to post the logs.. >.<

Very frustrating because even if you change the config "Compress flag" to NOT compress files without a config...it still compresses BoulderCo

[rbray89]

I had a quick look at the code and had an idea on how to to multi-threading with minimal code change.

Will it be OK with the use of Unity's functionality? (Mathf, Vector, Color32, etc...)

4-1: should fix the issues seen with EVE and other mods.

https://github.com/rbray89/ActiveTextureManagement/releases/tag/4-1

[Agost]

Will it be OK with the use of Unity's functionality? (Mathf, Vector, Color32, etc...)

4-1: should fix the issues seen with EVE and other mods.

https://github.com/rbray89/ActiveTextureManagement/releases/tag/4-1

Me wants multithread!

However, did you try this release with EVE 7.4 ( the current one ) ?

[rbray89]

Yup! works just fine.

[madlemur]

What if I've already converted my textures? Painfully loading the mods, and adding exceptions to the dds.py script until all the icons worked, and my game loads in less than 15 minutes (it's actually under 2 minutes, but it used to seem to be around 15)... Did I mention I'm running 64-bit Linux?

Would I be better off reinstalling all my mods and letting ATM compress them, or would I run into the same issue of having textures compressed that shouldn't be? Would you like my exception list, so you can merge it with yours (or more appropriately, create new .tcfg files for you to add)? Which reminds me... Kerbal Alarm Clock got mangled because the CKAN entry is installing it in the wrong place... Need to pop over and correct that...

[rbray89]

Will it be OK with the use of Unity's functionality? (Mathf, Vector, Color32, etc...)

4-1: should fix the issues seen with EVE and other mods.

https://github.com/rbray89/ActiveTextureManagement/releases/tag/4-1

Also, this won't require a cache update. So no painful re-loading

[rbray89]

What if I've already converted my textures? Painfully loading the mods, and adding exceptions to the dds.py script until all the icons worked, and my game loads in less than 15 minutes (it's actually under 2 minutes, but it used to seem to be around 15)... Did I mention I'm running 64-bit Linux?

Would I be better off reinstalling all my mods and letting ATM compress them, or would I run into the same issue of having textures compressed that shouldn't be? Would you like my exception list, so you can merge it with yours (or more appropriately, create new .tcfg files for you to add)? Which reminds me... Kerbal Alarm Clock got mangled because the CKAN entry is installing it in the wrong place... Need to pop over and correct that...

The big benefit is that this mod will ensure all your textures get managed automatically without having to micro-manage your mods' textures. Really, it is up to you, but keep in mind that this mod also gives you the ability to set scaling however you want as well.

[madlemur]

The big benefit is that this mod will ensure all your textures get managed automatically without having to micro-manage your mods' textures. Really, it is up to you, but keep in mind that this mod also gives you the ability to set scaling however you want as well.

OK. Then I think I will "downgrade" my installs to the original png/mbm/tga files, and let ATM manage the compression/caching. I'll just have to review the exceptions I set up so I can make sure I don't mangle my toolbars and plugin GUIs...

[Lechuza]

Also, this won't require a cache update. So no painful re-loading

That's great! Should I still re-chache (I am doing that as we speak) the "Clouds" folder for it to work? I tried without doing that and still no clouds so I am hoping the answer is yes

I really appreciate all your hard work! I am sure I am not alone.

[ecat]

Will it be OK with the use of Unity's functionality? (Mathf, Vector, Color32, etc...)

If in doubt just wrap them up in a critical section, or whatever C# calls the exclusive access mechanism. It's best to keep such exclusion sections as short and fast as possible, for obvious reasons, it can also be tricky to debug any problems caused by shared resources that should be in a CS but for some reason are not. Still, iirc from my tests the gains were near linear (2 threads == half the time etc) so well worth investigating?

[rbray89]

That's great! Should I still re-chache (I am doing that as we speak) the "Clouds" folder for it to work? I tried without doing that and still no clouds so I am hoping the answer is yes

I really appreciate all your hard work! I am sure I am not alone.

:/ Hmmm... It shouldn't. I don't understand where these issues are stemming from. I'm using the current release of ATM and EVE that are on Github... You might try deleting the BoulderCo folder inside the texture cache folder.

[rbray89]

If in doubt just wrap them up in a critical section, or whatever C# calls the exclusive access mechanism. It's best to keep such exclusion sections as short and fast as possible, for obvious reasons, it can also be tricky to debug any problems caused by shared resources that should be in a CS but for some reason are not. Still, iirc from my tests the gains were near linear (2 threads == half the time etc) so well worth investigating?

That's the thing though, I use the Unity operations pretty much everywhere. And we all know unity wasn't designed around multithreaded operations.

[ecat]

That's the thing though, I use the Unity operations pretty much everywhere. And we all know unity wasn't designed around multithreaded operations.

From a quick look around it appears the Math functions are thread safe eg

answers.unity3d.com/questions/754873/c-how-to-make-a-job-queue-for-noise-function.html

I'm unsure of other functions but it would appear the main threaded concern comes about when two or more threads try to access the same object. tbh, most functions written in a stack based high level language are thread safe, it's only when accessing single shared resources that problems occur.

Of course ymmv.

Edit:

Color operations could be safe too

http://answers.unity3d.com/questions/486584/how-to-create-a-parallel-for-in-unity.html

Some more info, mentions vectors and some custom helper code

http://forum.unity3d.com/threads/multithreading-solutions.90676/

Edited December 9, 2014 by ecat

[rbray89]

From a quick look around it appears the Math functions are thread safe eg

answers.unity3d.com/questions/754873/c-how-to-make-a-job-queue-for-noise-function.html

I'm unsure of other functions but it would appear the main threaded concern comes about when two or more threads try to access the same object. tbh, most functions written in a stack based high level language are thread safe, it's only when accessing single shared resources that problems occur.

Of course ymmv.

Haha, that is what I worry about Thinking about it, the libsquish implementation would be more likely able to be multi-threaded, but the Nvidia one uses static objects. Probably why the NVidia implementation seems faster as it is(less dynamic allocation & garbage collection). I'll give it a shot, and see where we get.

[Lechuza]

:/ Hmmm... It shouldn't. I don't understand where these issues are stemming from. I'm using the current release of ATM and EVE that are on Github... You might try deleting the BoulderCo folder inside the texture cache folder.

Re caching the Clouds folder did the trick! It works!! Thank you!

[LitaAlto]

You are wrong. DLL used by KSP plugins work on OSX and Linux without any change. They are .Net compiled code and those are platform agnostic.

Very interesting, thanks for the correction.

[ecat]

The quick and dirty method of experimenting with these things:

1) Create a wrapper function for each and every Unity function you wish to use

2) inside each wrapper create a critical section around the the Unity function call

3) Write the threaded code, it should work fine since everything is protected.

4) Start to selectively remove the CS protection.

5) profit Or lose a great deal of hair. Multi-threadding is great fun but can be tricky

I may have some time at the weekend if you hit any problems.

[Agost]

2 hours... still going D:

I should have used the nvidia tool...

[jstnj]

2 hours... still going D:

I should have used the nvidia tool...

Mine took about 8 hours all said and done...but I've got a lot going on in the game data folder.

[sarbian]

Haha, that is what I worry about Thinking about it, the libsquish implementation would be more likely able to be multi-threaded, but the Nvidia one uses static objects. Probably why the NVidia implementation seems faster as it is(less dynamic allocation & garbage collection). I'll give it a shot, and see where we get.

I saw that the Nvidia one is full of static while trying

I ll have a go at libsquish on my side too.

[rodion]

speaking of \BoulderCo\Clouds, caching the latest version of Astronomer's Visual Pack with its 10 or so 8K textures is truly a herculean effort unto itself. i can practically hear the wailing and gnashing of teeth coming from within my GameData folder, but the libsquish algorithm seems to be slowly and painfully getting along with it.

that aside, i'd just like to extend my thanks to you for developing ATM all this time. i feel like you essentially saved the majority of the KSP community twice in a row now; first when the proliferation of good mods resulted in everyone running into severe memory limitations, and then when those people installed so many mods that loading the game was getting to be a ridiculously drawn-out process (especially when troubleshooting; i will not miss the days when i would make a 30-second emendation to GameData, load the game for 6-10 minutes, see that the same problem was still there, and then repeating the same process for a whole evening).