Jump to content

[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!


rbray89

Recommended Posts

Thanks for this mod. It did a good job on cutting down my initial memory usage but I'm having a problem with a memory leak though (goes from 2.2 gb used to 3.7 gb before crashing) after simply leaving it on the VAB screen for an hour or so. I tried a new install (downloaded all new files from steam) and tried each mod I use individually and none alone seem to cause the issue. This was very time-consuming due to having to wait after each mod was installed to see if it caused the issue. So now I'm in the situation of trying to determine which combination of two or more mods is causing my memory leak.

Does anyone happen to have suggestions on how I might best go about this as the permutations are too numerous for an ad-hoc approach to be helpful?

Link to comment
Share on other sites

I'm having the same issue but its with EVE's cloud.cloud.update method as far as I can tell so far. Not sure where to post the logs.. >.<

Very frustrating because even if you change the config "Compress flag" to NOT compress files without a config...it still compresses BoulderCo

Link to comment
Share on other sites

What if I've already converted my textures? Painfully loading the mods, and adding exceptions to the dds.py script until all the icons worked, and my game loads in less than 15 minutes (it's actually under 2 minutes, but it used to seem to be around 15)... Did I mention I'm running 64-bit Linux?

Would I be better off reinstalling all my mods and letting ATM compress them, or would I run into the same issue of having textures compressed that shouldn't be? Would you like my exception list, so you can merge it with yours (or more appropriately, create new .tcfg files for you to add)? Which reminds me... Kerbal Alarm Clock got mangled because the CKAN entry is installing it in the wrong place... Need to pop over and correct that...

Link to comment
Share on other sites

What if I've already converted my textures? Painfully loading the mods, and adding exceptions to the dds.py script until all the icons worked, and my game loads in less than 15 minutes (it's actually under 2 minutes, but it used to seem to be around 15)... Did I mention I'm running 64-bit Linux?

Would I be better off reinstalling all my mods and letting ATM compress them, or would I run into the same issue of having textures compressed that shouldn't be? Would you like my exception list, so you can merge it with yours (or more appropriately, create new .tcfg files for you to add)? Which reminds me... Kerbal Alarm Clock got mangled because the CKAN entry is installing it in the wrong place... Need to pop over and correct that...

The big benefit is that this mod will ensure all your textures get managed automatically without having to micro-manage your mods' textures. Really, it is up to you, but keep in mind that this mod also gives you the ability to set scaling however you want as well.

Link to comment
Share on other sites

The big benefit is that this mod will ensure all your textures get managed automatically without having to micro-manage your mods' textures. Really, it is up to you, but keep in mind that this mod also gives you the ability to set scaling however you want as well.

OK. Then I think I will "downgrade" my installs to the original png/mbm/tga files, and let ATM manage the compression/caching. I'll just have to review the exceptions I set up so I can make sure I don't mangle my toolbars and plugin GUIs...

Link to comment
Share on other sites

Also, this won't require a cache update. So no painful re-loading :)

That's great! Should I still re-chache (I am doing that as we speak) the "Clouds" folder for it to work? I tried without doing that and still no clouds so I am hoping the answer is yes :blush:

I really appreciate all your hard work! I am sure I am not alone.

Link to comment
Share on other sites

Will it be OK with the use of Unity's functionality? (Mathf, Vector, Color32, etc...)

If in doubt just wrap them up in a critical section, or whatever C# calls the exclusive access mechanism. It's best to keep such exclusion sections as short and fast as possible, for obvious reasons, it can also be tricky to debug any problems caused by shared resources that should be in a CS but for some reason are not. Still, iirc from my tests the gains were near linear (2 threads == half the time etc) so well worth investigating?

Link to comment
Share on other sites

That's great! Should I still re-chache (I am doing that as we speak) the "Clouds" folder for it to work? I tried without doing that and still no clouds so I am hoping the answer is yes :blush:

I really appreciate all your hard work! I am sure I am not alone.

:/ Hmmm... It shouldn't. I don't understand where these issues are stemming from. I'm using the current release of ATM and EVE that are on Github... You might try deleting the BoulderCo folder inside the texture cache folder.

Link to comment
Share on other sites

If in doubt just wrap them up in a critical section, or whatever C# calls the exclusive access mechanism. It's best to keep such exclusion sections as short and fast as possible, for obvious reasons, it can also be tricky to debug any problems caused by shared resources that should be in a CS but for some reason are not. Still, iirc from my tests the gains were near linear (2 threads == half the time etc) so well worth investigating?

That's the thing though, I use the Unity operations pretty much everywhere. And we all know unity wasn't designed around multithreaded operations.

Link to comment
Share on other sites

That's the thing though, I use the Unity operations pretty much everywhere. And we all know unity wasn't designed around multithreaded operations.

From a quick look around it appears the Math functions are thread safe eg

answers.unity3d.com/questions/754873/c-how-to-make-a-job-queue-for-noise-function.html

I'm unsure of other functions but it would appear the main threaded concern comes about when two or more threads try to access the same object. tbh, most functions written in a stack based high level language are thread safe, it's only when accessing single shared resources that problems occur.

Of course ymmv.

Edit:

Color operations could be safe too

http://answers.unity3d.com/questions/486584/how-to-create-a-parallel-for-in-unity.html

Some more info, mentions vectors and some custom helper code

http://forum.unity3d.com/threads/multithreading-solutions.90676/

Edited by ecat
Link to comment
Share on other sites

From a quick look around it appears the Math functions are thread safe eg

answers.unity3d.com/questions/754873/c-how-to-make-a-job-queue-for-noise-function.html

I'm unsure of other functions but it would appear the main threaded concern comes about when two or more threads try to access the same object. tbh, most functions written in a stack based high level language are thread safe, it's only when accessing single shared resources that problems occur.

Of course ymmv.

Haha, that is what I worry about :) Thinking about it, the libsquish implementation would be more likely able to be multi-threaded, but the Nvidia one uses static objects. Probably why the NVidia implementation seems faster as it is(less dynamic allocation & garbage collection). I'll give it a shot, and see where we get.

Link to comment
Share on other sites

:/ Hmmm... It shouldn't. I don't understand where these issues are stemming from. I'm using the current release of ATM and EVE that are on Github... You might try deleting the BoulderCo folder inside the texture cache folder.

Re caching the Clouds folder did the trick! It works!! Thank you!

Link to comment
Share on other sites

The quick and dirty method of experimenting with these things:

1) Create a wrapper function for each and every Unity function you wish to use

2) inside each wrapper create a critical section around the the Unity function call

3) Write the threaded code, it should work fine since everything is protected.

4) Start to selectively remove the CS protection.

5) profit :) Or lose a great deal of hair. Multi-threadding is great fun but can be tricky :)

I may have some time at the weekend if you hit any problems.

Link to comment
Share on other sites

Haha, that is what I worry about :) Thinking about it, the libsquish implementation would be more likely able to be multi-threaded, but the Nvidia one uses static objects. Probably why the NVidia implementation seems faster as it is(less dynamic allocation & garbage collection). I'll give it a shot, and see where we get.

I saw that the Nvidia one is full of static while trying :)

I ll have a go at libsquish on my side too.

Link to comment
Share on other sites

speaking of \BoulderCo\Clouds, caching the latest version of Astronomer's Visual Pack with its 10 or so 8K textures is truly a herculean effort unto itself. i can practically hear the wailing and gnashing of teeth coming from within my GameData folder, but the libsquish algorithm seems to be slowly and painfully getting along with it.

that aside, i'd just like to extend my thanks to you for developing ATM all this time. i feel like you essentially saved the majority of the KSP community twice in a row now; first when the proliferation of good mods resulted in everyone running into severe memory limitations, and then when those people installed so many mods that loading the game was getting to be a ridiculously drawn-out process (especially when troubleshooting; i will not miss the days when i would make a 30-second emendation to GameData, load the game for 6-10 minutes, see that the same problem was still there, and then repeating the same process for a whole evening).

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...