[0.23] Squad Texture Reduction Pack - B9 and KW Packs also

[NathanKell]

Alpha channel is usually used for transparency, but really it's just another channel. Bitmaps can usually have at most four channels (though there's no reason you couldn't have an arbitrary number of channels), and the channels are usually 8bits each (though there are 16bit/channel formats). Alpha in games is usually used for transparency, but really it's just used so you can pack more into a single texture. So you can read color + transparency from a texture, or color + height, or color + specular, or color + emissive/self-illumination...

Specular maps tell a shader, basically, how "shiny" to make the object. Emissive maps controls whether the object is self-illuminated, i.e. is bright without light hitting it. (In the best of worlds, it would control the object actually emitting light, but usually it just means "count the object as self-illuminated").

A brief detour re: lighting models and shaders. How these all combine is:

Get the color of the pixel from the color map.

Get the normal at the pixel by averaging the vertex normals and then modifying by the normal map (if there is one).

Calculate the light hitting the pixel by calculating diffuse factor and specular factor (dot product of light incidence and normal, basically, with a power for the specular factor). Multiply diffuse by light, multiply specular color (or diffuse color or white, if no specular color) by specular factor and by that pixel (0=0.0, 255=1.0) in the specular map (if there is one). Add them up, in each (R/G/B) channel, that's your pixel's color.

Get alpha from alpha map. Instead of just writing the pixel, if alpha < 1, blend it with what's under it, by the alpha factor.

[WinkAllKerb'']

You'll ask a lot of modders and get a lot of different opinions. For my 2 cents, norma/shadow maps shouldn't need to be very large, maybe 512 max. All else should be relative to the actual size of the object in game or the level of fine detail you want to preserve. Until Squad makes some kind of dynamic texture quality loading, I would also suggest you release a hi-rez and low-rez pack for your users.

.png's are actually fine, but you need to remember that the file size for .png is does not reflect properly what the RAM usage will be. The size on your hard drive is just an illusion. The game bases things on pixel dimensions when it re-constitutes the graphics files, so 512x512 is about 750k-1mb regardless of the .png file size, for example. In my next update, I'll try running things through the .png converter before I shrink them to see if I get better results.

Another note is consolidation of textures using the MODEL command. This is helpful, but not as much as you want to believe. It does cut down on initial load times and a little on your RAM footprint (and HD space)...until you make a space ship with 20 - 30 copies of the same part. Then, that lower texture size starts to look real nice, as the next lower size knocks off 75% off the total load for that craft. This is the reason why B9 still ran like a pig on some systems even though the creator did a very good job on using MODEL for consolidation.

I can't think of where I did reduce the texture detail of the main "inner" piece for the cockpits, all the detail should still be there. I did reduce some shadow maps, and ripped out an alpha channel, but LOD should still be very much intact. Not all cockpits have a "main piece" though, and are very much just a collection of little bits. You can still see your Kerbal family photos and read the sticky notes just fine.

Testing a few thing around this i should share a part in the next day: (just need to fix a few anims and nodes wich is a bit painfull cause i m new to blender and unity and shortcuts knowledge miss me xD)

I splitted/merged/overlaped the UvMap/objectsFaces with some specific pattern/transform amongst the layers: so some different faces(even if there own geometry are different) that don't require detail use the same small 1x1 monocolor texture mid detail use 128x128 or 256x256 and higher detail use a few 512x512. Dunno how this split within the cpu proc and graphic proc regarding the hard/soft mapping process just need to fix a few more thing to test it properly and see if it help with some of the memory aspect.

(+ the texture used for this part will be used for other part via this uvmap splits/tranforms/'pre planning' that can be shared amongst different models, i intend to reduce the number of texture of a parts sets this way as most of the set's parts will be build with some {model} cfg to take advantage of this "contents" sharing possibility, it all has to do with some uvmaptransform that allow to overlap some differents faces geometry 'within a'/between 3dmodels with the same texture but not sure yet if there is a true gain in perf)

I wonder if doing so help memory a signifiant way too once in flight.

Edited October 4, 2013 by WinkAllKerb''
typo

[Fail-Man 3D]

Notice: This doesn't handle "del /s" properly!

I made a quick-and-dirty bash script to auto-convert debloat.bat

#!/usr/bin/env bash
# License to keep the moderators happy:
# This script is released into the public domain per United States copyright law.
# If your country has no legal equivalent, then the license reverts to your choice of WTFPL or CC-BY
echo '#!/bin/sh' > "$2"
sed -Ee -e '/^[[:blank:]]*$/d' \
        -e 's/[[:blank:]]+$//' \
        's/^del/rm/' \
        -e 's/\\/\//g' < <(
                dos2unix < "$1"
        ) >> "$2"
chmod +x "$2"

Tested on Linux, should probably work on Mac. Cygwin and SfU/Interix may require you to install dos2unix. SfU/Interix may also require you to install bash.

How to use (from within GameData):

chmod +x convert.sh && ./convert.sh <infile> <outfile>

Edited October 5, 2013 by Fail-Man 3D
Added warning about new versions of debloat.bat

[p1t1o]

Alpha channel is usually used for etc...

Couldn't have asked for a more complete answer, thanks!

[PolecatEZ]

Parachute Bug is fixed for the 0.1c Version of the Squad Pack

KickassKyle was cool with a KW version. Another 200mb saved...

See original post in this thread for links.

------------------

Could someone give me a definitive answer and/or drop box link for any Mac/Unix installer files they may have for the Squad Pack. If you are doing the conversion yourself, make sure you get the minor bug fix in the 0.1c version. I honestly can't make heads or tails of the Mac/Unix/Linux conversations in this thread, so if anyone can do this there would be some grateful people.

Edited October 5, 2013 by PolecatEZ

[Commander Gizmo]

This has really been perfect for me. With texture resizing (some of which I've done myself), I am now running B9, EL, IR, KW, NP, JARFR, KAS, Kethane, KOSMOS, Sperhical Tanks, HexCans, all procedurals, Wayland, and 30 others in their entirety (sans non procedural fairings) -- all with memory usage at 2.83GB! That's actually less memory than before I added all of B9 and KOSMOS.

Now if only my rockets would fly before exploding...

[PolecatEZ]

If anyone has any other good candidates, let me know. I'm taking a look at AIES right now (crosses fingers), but there's not a lot of extra fluff. The mod maker on that seems to have left out normal maps entirely, which already effectively cuts the RAM load in half. I'm going to need to get some more practice on .mbm conversion before I go nuts on that pack, if at all.

Some people are very visual so reduced textures aren't for them, but I'm more about the physics so these packs have actually made the game very playable for me as well. I rebuilt a B9 parts behemoth today just to tool around in space, and the lag difference was incredible. I know graphics people will pick these apart, but that's the beauty of these being just supplemental...its your option to trade performance for perfect visuals if you want.

Now time to crash land the KSS Yamato on the Mun and go to bed...rescue mission in the morning.

------------------------

Examining AIES pack, it begs the question...if I already reduce normals to almost nothing in some cases, are they really necessary for low rez people? This shall be examined in the morning.

Edited October 5, 2013 by PolecatEZ

[WinkAllKerb'']

great work polecatz, tetris graphics are basic but it's one of the funniest game ever on the other hand some game with great graphism are craps so ...

[Trann]

Notice: This doesn't handle "del /s" properly!

I made a quick-and-dirty bash script to auto-convert debloat.bat

...

Tested on Linux, should probably work on Mac. Cygwin and SfU/Interix may require you to install dos2unix. SfU/Interix may also require you to install bash.

I was going to try something similar but dos2unix isn't part of the base OSX. Rather than add another install for users, this should work against a vanilla OS.

And I've bloated your script

#!/usr/bin/env bash
# License to keep the moderators happy:
# This script is released into the public domain per United States copyright law.
# If your country has no legal equivalent, then the license reverts to your choice of WTFPL or CC-BY

# parameter test
if [ $# -ne 2 ]; then
  echo 'Syntax: ./convert.sh <batfile> <newfile>'
  exit
fi

echo '#!/bin/sh' > "$2"
# OSX test
if [ `uname -s` = "Darwin" ]; then
  echo '# OSX version' >> $2
  sed -e '/^[[:blank:]]*$/d' \
        -e 's/[[:blank:]]+$//' \
        -e 's/^del/rm/' \
        -e 's/\\/\//g' $1 | col -b >> $2
# Linux test
elif [ `uname -s` = "Linux" ]; then
  echo '# Linux version' >> $2
  sed -Ee -e '/^[[:blank:]]*$/d' \
        -e 's/[[:blank:]]+$//' \
        's/^del/rm/' \
        -e 's/\\/\//g' < <(
                dos2unix < "$1"
        ) >> "$2"
else
  echo 'Untested.'
  exit
fi
chmod +x "$2"

My personal copy has a step to backup the Squad directory first:

  echo '# OSX version' >> $2
  echo 'cp -Rp Squad ../Squad-`date "+%Y%m%d-%H%M"`' >> $2

Edited October 5, 2013 by Trann

[Trann]

For what it's worth, my run generated some noise for files that weren't there (it was a first run so not a remnant of any previous run).

rm: Squad/Parts/Aero/noseConeAdapter/model002.*: No such file or directory
rm: Squad/Parts/Command/probeCoreSphere/model000a.*: No such file or directory
rm: Squad/Spaces/Common/*.mbm: No such file or directory
rm: Squad/Spaces/mk1CockpitInternal/*.mbm: No such file or directory
rm: Squad/Spaces/Common/*.png: No such file or directory
rm: Squad/Spaces/crewCabinInternals/*.png: No such file or directory
rm: Squad/Spaces/cupolaInternal/*.png: No such file or directory
rm: Squad/Spaces/GenericSpace1/*.png: No such file or directory
rm: Squad/Spaces/GenericSpace3/*.png: No such file or directory
rm: Squad/Spaces/landerCabinInternals/*.png: No such file or directory
rm: Squad/Spaces/landerCabinSmallInternal/*.png: No such file or directory
rm: Squad/Spaces/mk1PodCockpit/*.png: No such file or directory
rm: Squad/Spaces/PodCockpit/*.png: No such file or directory
rm: Squad/Spaces/Common/*.tga: No such file or directory
rm: Squad/Spaces/crewCabinInternals/*.tga: No such file or directory
rm: Squad/Spaces/cupolaInternal/*.tga: No such file or directory
rm: Squad/Spaces/GenericSpace1/*.tga: No such file or directory
rm: Squad/Spaces/GenericSpace3/*.tga: No such file or directory
rm: Squad/Spaces/landerCabinInternals/*.tga: No such file or directory
rm: Squad/Spaces/landerCabinSmallInternal/*.tga: No such file or directory
rm: Squad/Spaces/mk1PodCockpit/*.tga: No such file or directory
rm: Squad/Spaces/PodCockpit/*.tga: No such file or directory

My personal preference would be to use 'rm -f' so it would run quietly but there may be some merit to seeing these missed files.

KSP loaded up fine (so far).

[SpacecraftX]

You should be employed at squad to help them with optimization.

[PolecatEZ]

You should be employed at squad to help them with optimization.

I don't think they would hire me, my salary request to return to the games industry would be absurd. I'll tell you first-hand that everything that gamers think about the soul-sucking gaming mega-corporations is true. The only people the upper management at these major corporations hold in more contempt than their player base is their workers. I left a lot of board meetings literally sick to my stomach. A lifetime ago I was a designer then studio manager and producer. I'm quite content now to sit on the sidelines and just help a project I love with a studio that seems not to be corrupted yet.

Besides, these packs aren't exactly skilled labor. If its just .png files, literally anyone could do them if they know a few basic rules.

Edited October 5, 2013 by PolecatEZ

[kyle.mk.howard]

For what it's worth, my run generated some noise for files that weren't there (it was a first run so not a remnant of any previous run).

rm: Squad/Parts/Aero/noseConeAdapter/model002.*: No such file or directory
rm: Squad/Parts/Command/probeCoreSphere/model000a.*: No such file or directory
rm: Squad/Spaces/Common/*.mbm: No such file or directory
rm: Squad/Spaces/mk1CockpitInternal/*.mbm: No such file or directory
rm: Squad/Spaces/Common/*.png: No such file or directory
rm: Squad/Spaces/crewCabinInternals/*.png: No such file or directory
rm: Squad/Spaces/cupolaInternal/*.png: No such file or directory
rm: Squad/Spaces/GenericSpace1/*.png: No such file or directory
rm: Squad/Spaces/GenericSpace3/*.png: No such file or directory
rm: Squad/Spaces/landerCabinInternals/*.png: No such file or directory
rm: Squad/Spaces/landerCabinSmallInternal/*.png: No such file or directory
rm: Squad/Spaces/mk1PodCockpit/*.png: No such file or directory
rm: Squad/Spaces/PodCockpit/*.png: No such file or directory
rm: Squad/Spaces/Common/*.tga: No such file or directory
rm: Squad/Spaces/crewCabinInternals/*.tga: No such file or directory
rm: Squad/Spaces/cupolaInternal/*.tga: No such file or directory
rm: Squad/Spaces/GenericSpace1/*.tga: No such file or directory
rm: Squad/Spaces/GenericSpace3/*.tga: No such file or directory
rm: Squad/Spaces/landerCabinInternals/*.tga: No such file or directory
rm: Squad/Spaces/landerCabinSmallInternal/*.tga: No such file or directory
rm: Squad/Spaces/mk1PodCockpit/*.tga: No such file or directory
rm: Squad/Spaces/PodCockpit/*.tga: No such file or directory

My personal preference would be to use 'rm -f' so it would run quietly but there may be some merit to seeing these missed files.

KSP loaded up fine (so far).

So how do I do this with your script? Can you PM me?

[Umlüx]

staying below 2,5gig now (installed all three packs) where before it was around 3,5-3,7 with the occasional freezes.

THANK YOU! now i can load em up with even more cool and pointless mods!

[Mokmo]

KW just re-realeased an update that further reduces its memory footprint, so i don't know which one of you has the smallest one now...

EDIT: Dat moment you see it linked on KW's page... nice one

Edited October 5, 2013 by Mokmo

[Neutrinovore]

My suggestion for the next mod to turn your attention to would be JARFR/THSS. That can be pretty laggy for me, anyway. And also, even the 'low-res' LazTek pack causes my rig to chug a bit, especially with the landing gear parts.

My 2 cents worth.

[dizzysaurus]

My suggestion for the next mod to turn your attention to would be JARFR/THSS. That can be pretty laggy for me, anyway. And also, even the 'low-res' LazTek pack causes my rig to chug a bit, especially with the landing gear parts.

My 2 cents worth.

Yeah I dropped THSS because it lagged my game so much.

[zzz]

I don't think texture size reduction for THSS will help much.

Edited October 5, 2013 by zzz

[dizzysaurus]

Awesome work, I love this mod

[thunderstar]

thank you so much! now with this i can run the game with a couple of mods more than before and this plus KW redux my gamedata folder weights less than 1GB!

[wasmic]

Nope, resizing THSS won't help - the textures are no more than 8*8 pixels. Except for the Octostrut Garage, which has a 2048*2048 and a 1024*1024 file, but they're in .png format, so you can easily resize them yourself.

EDIT: I believe that the lag that some people are experiencing might be due to an enormous number of small files. I think that with some clever restructuring, it would be possible to make all the parts of THSS require only 4-5 texture files.

Edited October 5, 2013 by wasmic

[PolecatEZ]

Currently looking (just looking!) at AIES, NovaPunch, FireSpitter, and a few other minor ones with friendly licenses. FS is a wierd animal, in that I would actually love to do a project to improve the textures, not just shrink them, but just shrinking them would be useful too. I also need to take another look at his license as it was specially worded to protect his .dll.

LazTek has a locked up tight license, but I certainly see where it could be significantly reduced - probably around 70% or so. If anyone wants to be pro-active and contact the owner for permission, let me know how it goes. I'll be happy to do the work if the answer is positive.

JARFR has no license that I've been able to find, and the original creator hasn't been on in a while. Again, if anyone knows the guy and can ask, I'd be happy to do textures or keep the mod updated.

Edited October 5, 2013 by PolecatEZ

[acealeam]

Well, the first time it loaded I was very pleased, my load time was so quicky, only around 40 seconds. Now it's at 10 or so minutes?

What...?

[PolecatEZ]

Well, the first time it loaded I was very pleased, my load time was so quicky, only around 40 seconds. Now it's at 10 or so minutes?

What...?

I really need more information to go on. What other mods are you using? Did Steam update you for any reason? Did you uninstall and re-install? Which OS are you using? What's your mother's maiden name and how come you haven't called her lately?

[Mokmo]

At first i was a bit skeptical... but i went from 2.78 GB Ram use with KW and other major parts packs to 2.3GB with B9 ADDED! Seriously i never thought i could put B9 back on this computer...

OMG the planets in the loading indicator spin !?!