[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

[rbray89]

Saves me about 1700mb (saves me from crashing, it runs at about 2.5 Gb and 3.5 without it )

Edit: just updated my texture compresser and it now hangs on the squad parts (Can't actually load ksp now), it takes like 10x more time to load as well, i think i'm gonna go back to the last update, It worked fine, I was using version 1-1 (i think, it may have been later)

Edit2: I use this mod:http://forum.kerbalspaceprogram.com/threads/65754-HotRockets%21-Particle-FX-Replacement could this cause the hangs?

Possibly, Could you please post your KSP.log file?

[Boamere]

Possibly, Could you please post your KSP.log file?

XD let me just see if it loads this last time

Edit: OK i don't need to load my log, it just wouldn't load for like uum 2 times in a row : 3rd time's the charm It loaded now

Edit 3: ok It loaded but some of the parts look very odd, like they're shaded differently ,here's a screenshot of the novapunch parts (sorry if i'm being a pain in the butt)

Is this on purpose?

(I am using a texture reduction pack, and basic compressor, also the orbital orb looks funny as well)

Here's my output log http://www./download/bnmxcdk34mbmvzv/output_log.zip.zip

Edit: sorry for this post being incredibly long winded, i'm a bit confused

Oh lordy there's a lot of things going wrong, this service module is playing the decoupling animation, while in the VAB!

http://imgur.com/gd2ZjtO

Edited January 16, 2014 by Boamere

[ecat]

New update, should resolve most people's issues. See update log in OP for more details.

Excellent stuff, tyvm

While I was waiting for the warp fix I did a little poking around.

My test system was taking 2:30 to load:

A small change to the way you do some of your calculations reduced this to 2:25 - lol.

Splitting the GetPixel calls across two threads reduced the load time to around 2:08 - not bad I guess.

Adding a simple cache mechanism (GameData/BoulderCo/Cache) reduced second and subsequent load times to 1:27 - now we are talking But I don't know if writing to the KSP data directories is generally 'a good idea' or if it will work on linux/Mac or even a standard ProgramFiles/Steam install.

The changes are trivial but let me know if you'd like the code, it may save you some development time.

[Boamere]

Excellent stuff, tyvm

While I was waiting for the warp fix I did a little poking around.

Adding a simple cache mechanism (GameData/BoulderCo/Cache) reduced second and subsequent load times to 1:27 - now we are talking But I don't know if writing to the KSP data directories is generally 'a good idea' or if it will work on linux/Mac or even a standard ProgramFiles/Steam install.

The changes are trivial but let me know if you'd like the code, it may save you some development time.

That sounds interesting interesting enough to send to squad, if it doesn't cause problems

[lincourtl]

New update, should resolve most people's issues. See update log in OP for more details.

I'm sad to report I'm still seeing only about half the memory savings in Aggressive 2-14, as I saw in Aggressive 2-12 -- 714MB vs. 1.5GB. Load time is also still much higher in the latest version compared to 2-12 -- 7m 7s vs. ~5m.

No crashing though, loaded fine, no errors in the log.

[ecat]

That sounds interesting interesting enough to send to squad, if it doesn't cause problems

If all you are doing is loading files, which is mostly what Squad do, then adding a cache just means the files are loaded from a different location - no big saving. This mod does a lot of processing on some of the texture data. How much? If I comment out the data processing 'GetPixel' routine my load time goes from 2:30 all the way down to 1:15! That is a lot of time which can be saved, if it works on other systems.

[team.leit]

Saves me about 1700mb (saves me from crashing, it runs at about 2.5 Gb and 3.5 without it )

Edit: just updated my texture compresser and it now hangs on the squad parts (Can't actually load ksp now), it takes like 10x more time to load as well, i think i'm gonna go back to the last update, It worked fine, I was using version 1-1 (i think, it may have been later)

Edit2: I use this mod:http://forum.kerbalspaceprogram.com/threads/65754-HotRockets%21-Particle-FX-Replacement could this cause the hangs?

No, that wouldn't cause hangs. I am using it and I am loading fine in the latest version of the aggressive

[rbray89]

If all you are doing is loading files, which is mostly what Squad do, then adding a cache just means the files are loaded from a different location - no big saving. This mod does a lot of processing on some of the texture data. How much? If I comment out the data processing 'GetPixel' routine my load time goes from 2:30 all the way down to 1:15! That is a lot of time which can be saved, if it works on other systems.

The problem then becomes saving it where KSP won't load it into memory. That is the tricky part, unless I make some special non-standard-image type cache. Hmmm... that could work.

[rbray89]

I'm sad to report I'm still seeing only about half the memory savings in Aggressive 2-14, as I saw in Aggressive 2-12 -- 714MB vs. 1.5GB. Load time is also still much higher in the latest version compared to 2-12 -- 7m 7s vs. ~5m.

No crashing though, loaded fine, no errors in the log.

Hmmm... did you modify any configs? The format changed, so if there were any configs that you added, they may have to be modified. Aggresive 2-14 should be REALLY aggressive. In my full mod configuration I was saving nearly 3GB of memory. Are those values the saved memory or the config reporting? Could you please post your KSP.log file?

[rbray89]

XD let me just see if it loads this last time

Edit: OK i don't need to load my log, it just wouldn't load for like uum 2 times in a row : 3rd time's the charm It loaded now

Edit 3: ok It loaded but some of the parts look very odd, like they're shaded differently ,here's a screenshot of the novapunch parts (sorry if i'm being a pain in the butt)

http://i.imgur.com/Lrn2Wex.jpg?1

Is this on purpose?

(I am using a texture reduction pack, and basic compressor, also the orbital orb looks funny as well)

Here's my output log http://www./download/bnmxcdk34mbmvzv/output_log.zip.zip

Edit: sorry for this post being incredibly long winded, i'm a bit confused

Oh lordy there's a lot of things going wrong, this service module is playing the decoupling animation, while in the VAB!

http://imgur.com/gd2ZjtO

Do you have an old version of the mod still around? try deleting the plugin and all the old config files. (if you don't have visual enhancements you can just delete BoulderCo.

[Boamere]

Do you have an old version of the mod still around? try deleting the plugin and all the old config files. (if you don't have visual enhancements you can just delete BoulderCo.

Nah i don't have the old boulder co. I deleted it, i think I'm going to go back to my previous version, this version has caused me lots of wierd bugs (like engine effects not showing up and phantom forces) I don't have visual enhancements either i'm using it because i have 37 mods, but thanks anyway

[rbray89]

XD let me just see if it loads this last time

Edit: OK i don't need to load my log, it just wouldn't load for like uum 2 times in a row : 3rd time's the charm It loaded now

Edit 3: ok It loaded but some of the parts look very odd, like they're shaded differently ,here's a screenshot of the novapunch parts (sorry if i'm being a pain in the butt)

http://i.imgur.com/Lrn2Wex.jpg?1

Is this on purpose?

(I am using a texture reduction pack, and basic compressor, also the orbital orb looks funny as well)

Here's my output log http://www./download/bnmxcdk34mbmvzv/output_log.zip.zip

Edit: sorry for this post being incredibly long winded, i'm a bit confused

Oh lordy there's a lot of things going wrong, this service module is playing the decoupling animation, while in the VAB!

http://imgur.com/gd2ZjtO

looks like you have issues with other plugins...

Failed to load assembly C:\Program Files (x86)\Steam\steamapps\common\Kerbal Space Program\GameData\ExtraplanetaryLaunchpads\Launchpad.dll:
System.BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid.
...
Failed to load assembly C:\Program Files (x86)\Steam\steamapps\common\Kerbal Space Program\GameData\Kethane\Plugins\MMI_Kethane.dll:
System.BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid.

Looks like I'll have to double check I got all the normal maps. Looks like some NP2 parts might not have been created correctly.

[rbray89]

XD let me just see if it loads this last time

Edit: OK i don't need to load my log, it just wouldn't load for like uum 2 times in a row : 3rd time's the charm It loaded now

Edit 3: ok It loaded but some of the parts look very odd, like they're shaded differently ,here's a screenshot of the novapunch parts (sorry if i'm being a pain in the butt)

http://i.imgur.com/Lrn2Wex.jpg?1

Is this on purpose?

(I am using a texture reduction pack, and basic compressor, also the orbital orb looks funny as well)

Here's my output log http://www./download/bnmxcdk34mbmvzv/output_log.zip.zip

Edit: sorry for this post being incredibly long winded, i'm a bit confused

Oh lordy there's a lot of things going wrong, this service module is playing the decoupling animation, while in the VAB!

http://imgur.com/gd2ZjtO

Can you post a picture of the orbital orb?

[Boamere]

I'm sad to report I'm still seeing only about half the memory savings in Aggressive 2-14, as I saw in Aggressive 2-12 -- 714MB vs. 1.5GB. Load time is also still much higher in the latest version compared to 2-12 -- 7m 7s vs. ~5m.

No crashing though, loaded fine, no errors in the log.

Same here weird

[Boamere]

Can you post a picture of the orbital orb?

Sure; basically a lot of parts are loading as if they where in shade, like chequerboard shading. Let me get an album.

Edit:Here you go:

Javascript is disabled. View full album

Edited January 16, 2014 by Boamere

[ecat]

The problem then becomes saving it where KSP won't load it into memory. That is the tricky part, unless I make some special non-standard-image type cache. Hmmm... that could work.

Humm. I can think of reasons for loading all files from all directories under GameData, I can also think of reasons for being more selective. If you say all files under GameData are loaded I'll take you at your word but after a few quick tests I can find no trace of my 390MB of cache data when I run KSP.

I'm using GameData/BoulderCo/Cache as my cache directory, no other sub directories, the file names have no extension, eg 'KWRocketry+Parts+Engines+3mTitanI+KW3mengineTitan1F1Emiss8bit'

'Squad+Parts+Aero+advancedCanard+model000'

The data in the files would look like garbage to any self respecting image file loader, they contain a 32bit CRC (used to detected changes in the original unprocessed texture file) and a binary dump of the relevant newPixels[] array. I like to keep things simple, especially when I have no idea what I'm doing

[rbray89]

Humm. I can think of reasons for loading all files from all directories under GameData, I can also think of reasons for being more selective. If you say all files under GameData are loaded I'll take you at your word but after a few quick tests I can find no trace of my 390MB of cache data when I run KSP.

I'm using GameData/BoulderCo/Cache as my cache directory, no other sub directories, the file names have no extension, eg 'KWRocketry+Parts+Engines+3mTitanI+KW3mengineTitan1F1Emiss8bit'

'Squad+Parts+Aero+advancedCanard+model000'

The data in the files would look like garbage to any self respecting image file loader, they contain a 32bit CRC (used to detected changes in the original unprocessed texture file) and a binary dump of the relevant newPixels[] array. I like to keep things simple, especially when I have no idea what I'm doing

Haha, that would work. I was referring to the fact that KSP loads everything with an extension it recognizes, so I wouldn't be able to store it as a "regular image file" (with extension). I really like the idea of a CRC. I could also do the normal map conversion there too.

Sure; basically a lot of parts are loading as if they where in shade, like chequerboard shading. Let me get an album.

Edit:Here you go:

http://imgur.com/a/OA86g

I think I may have stumbled upon how to fix the normalmap issue. KSP/Unity normalmap handling is WEIRD.

[ecat]

Haha, that would work. I was referring to the fact that KSP loads everything with an extension it recognizes, so I wouldn't be able to store it as a "regular image file" (with extension). I really like the idea of a CRC. I could also do the normal map conversion there too.

Cool.

Here's some code you can play with

        static UInt32[] CRCTable = {
            0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419,
            0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4,
            0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07,
            0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
            0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856,
            0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9,
            0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4,
            0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
            0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3,
            0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a,
            0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599,
            0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
            0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190,
            0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f,
            0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e,
            0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
            0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed,
            0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950,
            0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3,
            0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
            0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a,
            0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5,
            0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010,
            0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
            0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17,
            0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6,
            0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615,
            0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
            0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344,
            0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
            0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a,
            0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
            0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1,
            0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c,
            0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef,
            0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
            0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe,
            0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31,
            0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c,
            0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
            0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b,
            0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242,
            0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1,
            0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
            0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278,
            0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7,
            0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66,
            0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
            0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605,
            0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8,
            0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b,
            0x2d02ef8d
        };

        public static void Resize(GameDatabase.TextureInfo texture, int width, int height, TextureFormat format, bool mipmaps)
        {
            Texture2D tex = texture.texture;
            Color32[] pixels = tex.GetPixels32();
            Color32[] newPixels = new Color32[width * height];
            int index = 0;

            bool readFromCache = false;
            UInt32 origDataChkSum = 0;

            //
            // ecat
            // MD5 may be faster but the builtin accepts byte[] so the pixels array would need an additional conversion pass
            //
            foreach (Color32 pixel in pixels)
            {
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.r)) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.g)) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.a)) & 0xff] ^ (origDataChkSum >> 8);
            }

            //
            // ecat
            // for now you MUST MANUALLY CREATE the Cache dir
            //
            String fileName = texture.name.Replace('/', '+');
            String pathStart = (KSPUtil.ApplicationRootPath + "GameData/BoulderCo/Cache/").Replace('\\', '/');

//            UnityEngine.Debug.Log("ecat name " + fileName);
//            UnityEngine.Debug.Log("ecat path " + pathStart);

            //
            // ecat, load the texture from cache if available and has a matching checksum
            //

            if (File.Exists(pathStart + fileName))
            {
                try
                {
                    using (FileStream fs = new FileStream(pathStart + fileName, FileMode.Open, FileAccess.Read))
                    {
                        using (BinaryReader br = new BinaryReader(fs))
                        {
                            index = 0;
                            UInt32 storedChkSum = br.ReadUInt32();
//                            UnityEngine.Debug.Log("ecat Cache O S " + origDataChkSum.ToString() + " " + storedChkSum.ToString());
                            if (storedChkSum == origDataChkSum)
                            {
                                for (int h = 0; h < height; h++)
                                {
                                    for (int w = 0; w < width; w++)
                                    {
                                        Color32 c32;
                                        c32.r = br.ReadByte();
                                        c32.g = br.ReadByte();
                                        c32.b = br.ReadByte();
                                        c32.a = br.ReadByte();
                                        newPixels[index++] = c32;
                                    }
                                }
                                readFromCache = true;
                            }
//                            else
//                                UnityEngine.Debug.Log("ecat Cache Missmatch");  // this may happen when a file is 'OUT OF DATE', whatever that may mean
                         }
                    }
                }
                catch {
                    readFromCache = false;      // in case of error force a rebuild
                }
            }

            if (!readFromCache)
            {

                if (texture.isNormalMap)
                {
                    ConvertToUnityNormalMap(pixels);
                }
                int origWidth = tex.width;
                int origHeight = tex.height;
                for (int h = 0; h < height; h++)
                {
                    for (int w = 0; w < width; w++)
                    {
                        newPixels[index++] = GetPixel(pixels, tex, ((float)w) / width, ((float)h) / height, width, height);
                    }
                }

                //
                // ecat, save the texture to file
                //
                using (FileStream fs = new FileStream(pathStart + fileName, FileMode.Create))   // force create of new file
                {
                    using (BinaryWriter bw = new BinaryWriter(fs))
                    {
                        index = 0;
                        bw.Write(origDataChkSum);               // save checksum of origonal data so we can detect updated textures

                        for (int h = 0; h < height; h++)
                        {
                            for (int w = 0; w < width; w++)
                            {
                                Color32 c32 = newPixels[index++];
                                bw.Write(c32.r);
                                bw.Write(c32.g);
                                bw.Write(c32.;
                                bw.Write(c32.a);
                            }
                        }
                    }
                }

            }
            tex.Resize(width, height, format, mipmaps);
            tex.SetPixels32(newPixels);
            tex.Apply(mipmaps);
        }

[rbray89]

Cool.

Here's some code you can play with

        static UInt32[] CRCTable = {
            0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419,
            0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4,
            0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07,
            0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
            0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856,
            0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9,
            0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4,
            0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
            0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3,
            0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a,
            0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599,
            0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
            0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190,
            0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f,
            0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e,
            0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
            0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed,
            0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950,
            0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3,
            0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
            0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a,
            0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5,
            0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010,
            0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
            0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17,
            0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6,
            0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615,
            0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
            0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344,
            0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
            0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a,
            0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
            0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1,
            0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c,
            0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef,
            0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
            0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe,
            0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31,
            0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c,
            0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
            0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b,
            0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242,
            0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1,
            0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
            0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278,
            0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7,
            0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66,
            0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
            0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605,
            0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8,
            0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b,
            0x2d02ef8d
        };

        public static void Resize(GameDatabase.TextureInfo texture, int width, int height, TextureFormat format, bool mipmaps)
        {
            Texture2D tex = texture.texture;
            Color32[] pixels = tex.GetPixels32();
            Color32[] newPixels = new Color32[width * height];
            int index = 0;

            bool readFromCache = false;
            UInt32 origDataChkSum = 0;

            //
            // ecat
            // MD5 may be faster but the builtin accepts byte[] so the pixels array would need an additional conversion pass
            //
            foreach (Color32 pixel in pixels)
            {
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.r)) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.g)) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.) & 0xff] ^ (origDataChkSum >> 8);
                origDataChkSum = CRCTable[(origDataChkSum ^ (pixel.a)) & 0xff] ^ (origDataChkSum >> 8);
            }

            //
            // ecat
            // for now you MUST MANUALLY CREATE the Cache dir
            //
            String fileName = texture.name.Replace('/', '+');
            String pathStart = (KSPUtil.ApplicationRootPath + "GameData/BoulderCo/Cache/").Replace('\\', '/');

//            UnityEngine.Debug.Log("ecat name " + fileName);
//            UnityEngine.Debug.Log("ecat path " + pathStart);

            //
            // ecat, load the texture from cache if available and has a matching checksum
            //

            if (File.Exists(pathStart + fileName))
            {
                try
                {
                    using (FileStream fs = new FileStream(pathStart + fileName, FileMode.Open, FileAccess.Read))
                    {
                        using (BinaryReader br = new BinaryReader(fs))
                        {
                            index = 0;
                            UInt32 storedChkSum = br.ReadUInt32();
//                            UnityEngine.Debug.Log("ecat Cache O S " + origDataChkSum.ToString() + " " + storedChkSum.ToString());
                            if (storedChkSum == origDataChkSum)
                            {
                                for (int h = 0; h < height; h++)
                                {
                                    for (int w = 0; w < width; w++)
                                    {
                                        Color32 c32;
                                        c32.r = br.ReadByte();
                                        c32.g = br.ReadByte();
                                        c32.b = br.ReadByte();
                                        c32.a = br.ReadByte();
                                        newPixels[index++] = c32;
                                    }
                                }
                                readFromCache = true;
                            }
//                            else
//                                UnityEngine.Debug.Log("ecat Cache Missmatch");  // this may happen when a file is 'OUT OF DATE', whatever that may mean
                         }
                    }
                }
                catch {
                    readFromCache = false;      // in case of error force a rebuild
                }
            }

            if (!readFromCache)
            {

                if (texture.isNormalMap)
                {
                    ConvertToUnityNormalMap(pixels);
                }
                int origWidth = tex.width;
                int origHeight = tex.height;
                for (int h = 0; h < height; h++)
                {
                    for (int w = 0; w < width; w++)
                    {
                        newPixels[index++] = GetPixel(pixels, tex, ((float)w) / width, ((float)h) / height, width, height);
                    }
                }

                //
                // ecat, save the texture to file
                //
                using (FileStream fs = new FileStream(pathStart + fileName, FileMode.Create))   // force create of new file
                {
                    using (BinaryWriter bw = new BinaryWriter(fs))
                    {
                        index = 0;
                        bw.Write(origDataChkSum);               // save checksum of origonal data so we can detect updated textures

                        for (int h = 0; h < height; h++)
                        {
                            for (int w = 0; w < width; w++)
                            {
                                Color32 c32 = newPixels[index++];
                                bw.Write(c32.r);
                                bw.Write(c32.g);
                                bw.Write(c32.;
                                bw.Write(c32.a);
                            }
                        }
                    }
                }

            }
            tex.Resize(width, height, format, mipmaps);
            tex.SetPixels32(newPixels);
            tex.Apply(mipmaps);
        }

Hmmm... I was thinking of just performing it against the texture file itself (as MBMs can't be read anyways) and comparing the width and height separately and add any more non-data fields to the comparison if anything else is updated (eg. format). I will probably end up creating a format similar to MBM. A flag to indicate Normal or Not, a width and height field, MD5 hash, and of course, data. I was also thinking of just cloning the directory structure, so that it is easier to navigate in case you wanted to take a look. The last thing I was starting to think about, was to encode the files as pngs so that they could be viewed by changing the extension and take up less storage, then store the CRC, normal map flag in a paired config.

I have to give you major props for this. Excellent, excellent idea. Much better than my initial attempt by overwriting all the original textures on the file system.

[gath875]

Decided to try Interstellar. I saw a post a few pages back regarding the resource data, and I see where the config was updated, but I'm still having an issue.

This is what the spectrometer shows with only Interstellar:

http://i.imgur.com/PqddVQK.jpg

When I add the BoulderCo folder, I get this:

http://i.imgur.com/8g01bx8.jpg

This is with 2-14 aggressive, but I tried basic as well. I started to try and mess with the config file myself, but I figured I'd post it here first.

[rbray89]

Decided to try Interstellar. I saw a post a few pages back regarding the resource data, and I see where the config was updated, but I'm still having an issue.

This is what the spectrometer shows with only Interstellar:

http://i.imgur.com/PqddVQK.jpg

When I add the BoulderCo folder, I get this:

http://i.imgur.com/8g01bx8.jpg

This is with 2-14 aggressive, but I tried basic as well. I started to try and mess with the config file myself, but I figured I'd post it here first.

What...? Could you please post your ksp.log file? The config should be updated to handle this, but the log file will tell me for sure.

[gath875]

https://www.dropbox.com/s/rhp0r389irm7ix2/KSP.log

Thanks for the quick reply! I've been messing with quite a few new mods lately, so there's no telling what I might have screwed up, hehe.

[ecat]

Hmmm... I was thinking of just performing it against the texture file itself (as MBMs can't be read anyways) and comparing the width and height separately and add any more non-data fields to the comparison if anything else is updated (eg. format). I will probably end up creating a format similar to MBM. A flag to indicate Normal or Not, a width and height field, MD5 hash, and of course, data. I was also thinking of just cloning the directory structure, so that it is easier to navigate in case you wanted to take a look. The last thing I was starting to think about, was to encode the files as pngs so that they could be viewed by changing the extension and take up less storage, then store the CRC, normal map flag in a paired config.

Haha!

See, you know what you are doing. I was just out to see how much time I could knock off the startup, while butchering as little of your code as possible . Either way, if my tests hold true, reducing the overhead by 80% has got to be worth the effort... I'd still like access rights confirmation on Linux/Mac and Win 'program files' directory.

[gath875]

Double post!

Edited January 17, 2014 by gath875

[RatchetTheGamer(RSC)]

Hey Rbray89 what is the diffenters with the two versions ples tell me I what to know. also witch one is the better one to use?