Stock Debloating (WIP)

[PolecatEZ]

This is quick and dirty for a first post, and the release will get a better name, but here goes.

I ran some scripting on the Squad Stock parts and internals and managed de-rezzed some bloated textures. My first pass I hit all the 4mb and 1mb textures, which knocked about 30% off the total load size. I have 2 problems:

1 - My machine is too beefy with dual raid SSD and big fat processor, 30% off just isn't that noticeable for me on a basic install with no other mods.

2 - Because the textures are .tga now, they don't overwrite the default textures, just add to them. I need a quick and dirty way to make a seek-and-destroy script (or upload the entire parts folder, even the parts without changes). If I can figure out how to get a printout of files in the "Cooked" folder, I could definitely batch it that way. It does make install a little messy though (add in files AND run batch...maybe too much for most users to grasp).

I also need the texture resolution settings for 12mb, 16mb (2048x2048 I assume), 3mb, 1.5mb, and 768kb so I can make heading swap files for the remaining textures. As well, nearly ALL textures in the internals folder are duplicated, if we could make reference configs instead, the internals folder could be shrinkalated by about 90%+ (no joke).

Here's the shrunken textures that I have so far, its a tedious process.

If anyone wants to test or help out, feel free.

Also, if anything looks particularly like kerdoodoo, let me know and I'll shrink it less or not at all.

DOWNLOAD 0.1a texture pack

MIT license (means do whatever you feel like, I'm cool unless you aren't).

Edited September 30, 2013 by PolecatEZ

[Greys]

Here's my 1.275 cents.

It might be against the rules to distribute modified Squad assets, and yes this would count. You might want to instead exclusively distribute the script that generates these.

The first thing I did was go and open /Squad/Parts/EngineliquidEngine2/model000.tga in GIMP and noticed two things

- The resolution is 384 square

- There is already an alpha channel on the file

These are both bad things

Textures need to always be powers of 2, Unity will make sure of it happen if you don't. 384 on load gets expanded to the next largest power of 2, which happens to be 512. It does not get scaled, Unity just adds a bunch of empty space to it. The nearest power of 2 happens to be 512. Instantly that texture takes up 22% more memory than intended.

Textures are also never put into memory in the format they're stored in. There's been a lot of misinformation about this lately, some from me, and wholly due to nobody actually looking. But KSP does compress textures. On load all textures are converted into DXT1 or DXT5 DDS format. This is good for us because these are very formulaic as to how much space they take up, DXT1 takes 4 bits per pixel, and DXT5 takes twice as much. What, you ask, is the difference between them? DXT5 has transparency, and KSP uses it for any texture files that have alpha channels in the file.

So now you've got a texture sheet that due to a weird size contains a lot of empty space, and because of unnecessary transparency takes up 2 times more.

(512 * 512) * 8 = 2,097,152 bits ram RAM

Assuming the sheet was originally 1024 square without alpha channel

(1024 * 1024) * 4 = 4,194,304 bits in ram

This is only a 50% reduction in memory usage, but now here's the useful information you're storing

^{if the original sheet was 512 then you're not saving a damn thing and actually taking double the ram}

(384 * 384) * 4 = 589,824 bits in ram

That is to say, 28.125% of the memory you are now using, is useful to KSP.

If you only scaled the original file down to 512, you'd be using the same amount of memory but you'd have better texture resolution for it.

If you get rid of the alpha channel you will use one half the memory you currently are

Or, you could get rid of the alpha channel and round down to 256x256, lose a bit more resolution, and only take up

(256 * 256) * 4 = 262,144 bits in ram

for an overall savings vs the original 1024x1024 of 93.75%

So:

stick to powers of 2 ^{(2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, etc)}

Don't add alpha channels where they don't already exist, and MBMs don't have alpha channels because they're bitmaps.

Edited September 30, 2013 by Greys

[PolecatEZ]

Good thing I saved the original swaps...now to figure out how to strip an alpha channel.

I went with 384 because 256 was too fuzzy but 512 was too big. I can run back through and in my alpha stripping pass change them less.

So far as I know, anything from Squad in the /gamedata folder is fair game to do whatever, its the other stuff they don't want you modifying or mucking with. People wholly rip off the config templates for things and redistribute all the time, why would modified textures be different? I'm not claiming the textures as my work, just the tedious process of converting them. Justice from the admins is generally swift and brutal if there is a violation.

I also solved the installer problem. Seek and destroy batching is easy to construct now.

[Gaius]

So far as I know, anything from Squad in the /gamedata folder is fair game to do whatever, its the other stuff they don't want you modifying or mucking with.

I believe that's correct. From the official rules:

Do not decompile, modify or distribute any of the DLLs or other files KSP comes with beyond stuff in the Parts folder.

They specifically except the "stuff in the Parts folder". Unfortunately, they wrote that before reorganizing how parts and mods are organized on the disk with the new GameData folder, leaving it now somewhat ambiguous (which "Parts" folder? The new one under GameData/Squad? Only the legacy one? Or do they really mean everything under GameData under the new scheme?), but I believe the intent is more or less as you describe.

Edited September 30, 2013 by Gaius

[PolecatEZ]

Anyways, did everything nice and proper for the crewcabin, and here's a comparison in quality.

The top lander can has the old 4mb texture and 1 or 4mb shadow map or whatever that second file is.

The bottom cabin has his 4mb texture + 4mb shadow map shrunk to 512x512 with no alpha for 768kb now, with a 256kb shadow map.

My settings are on the absolute highest levels for everything, people sporting lower resolutions or textures may not notice a difference.

Is this quality difference ok for the average user, or should I stick just to shrinkilating ridiculous things like the 8mb throttle handle?

================

And in an unrelated EUREKA moment, I just figured out how to ensmallen the /spaces folder to almost nothing, without applying any texture modifications. I can probably squeeze that down to 20% original size before texture shrinking.

Edited September 30, 2013 by PolecatEZ

[NoMrBond]

For stuff where the texture is identical, you could also change the part.cfg to use a MODEL{} call with the texture argument to call the other identical texture, and replace the unused redundant texture with a blank 4x4 px image.

[PolecatEZ]

For stuff where the texture is identical, you could also change the part.cfg to use a MODEL{} call with the texture argument to call the other identical texture, and replace the unused redundant texture with a blank 4x4 px image.

Great, perhaps you can share how to do that with the spaces because I got it to work once, redid every space like that, and then it didn't work again at all. Hair-pulling.

Now I can't tell if the model command is being ignored or the texture sub-command is being ignored.

[NoMrBond]

I'll check/test it when I get home and hopefully have some kind progress update later (at work now, still ~6 hours to go)

[WinkAllKerb'']

i m a bit late here but all this are an incredible amount of amazings benchies infos !!!!! (/worship mode on)

i m also wondering one thing:

how could uv map transforms applied could impact the preload and then in game, i suppose that using the simpliest math transfo for applying texture is better for memory than using lot of weird uv transfo ? am i wrong with this or once the part is loaded it's no longer important at all, the way the uvmap been applied is complex or not with the use of many differents transfo ?

cheers :)

[NathanKell]

In the end, each vertex gets UV coords, and these are interpolated by the pixel shader. Doesn't matter how complex your mapping was.

[rbray89]

Textures are also never put into memory in the format they're stored in. There's been a lot of misinformation about this lately, some from me, and wholly due to nobody actually looking. But KSP does compress textures. On load all textures are converted into DXT1 or DXT5 DDS format. This is good for us because these are very formulaic as to how much space they take up, DXT1 takes 4 bits per pixel, and DXT5 takes twice as much. What, you ask, is the difference between them? DXT5 has transparency, and KSP uses it for any texture files that have alpha channels in the file.

Not correct actually. TGA's are not compressed at all currently. There are LOTS of MBMs that are not compressed (a flag in the header determines whether to compress them or not). PNGs are compressed, but no mip-maps are generated.

Hence why my Texture compressor mod works. I had been playing around with the idea of modifying the MBMs before they are loaded, forcing them to be readable textures, then manually reading them in, passing them through a bicubic re-size to lower resolutions. Then compressing them. This would save TONS of memory, and would still be a single dll that wouldn't violate SQUAD's IP.

[rbray89]

So I found out that the reason *some* of the textures are not readable, is that they are MBMs that are set to be a normal/bump map (byte 12, 0-indexed). When I remove the flag, they load as readable textures (in RGB/RGBA format). I am currently working on a way to set the byte to tell KSP it is a normal texture, then load it up, then convert the byte back so we don't screw stuff up (closing during load could still mess stuff up). I should be able to scale the textures this way using a bicubic filter.

[Greys]

Make sure you add some way for users to disable that, compressing normalmaps will save a lot of space, but it will also have a bigger impact on visual fidelity than just about anything else.

[rbray89]

Make sure you add some way for users to disable that, compressing normalmaps will save a lot of space, but it will also have a bigger impact on visual fidelity than just about anything else.

Yeah, it looks like the KSP model loader doesn't like compressed normal maps much either. Instead of compressing, I will probably just re-size those.