[0.90] DDSLoader 1.9 (Mar 9) - Loads DDS Texture "Boringly fast" edition

[sarbian]

Thanks fixed. I kinda forgot to add the jenkins callback in GitHub

[Nuke]

There's one DXT5 based format that was made specially for normalmaps, but support is somewhat questionable I guess.

EDIT: Namely DXT5nm

there is also bc5/3dc, but ive never seen a game that uses it. its a 2 channel map which is the same pixel format as a dxt5 alpha channel. dxt5nm just moves red to alpha, leaves the green where it is, and red and blue become ballast (a dirty but effective hack invented by john carmack).

ninjad by a couple pages.

Edited October 14, 2014 by Nuke

[nli2work]

so basically

basic texture => DTX1

texture w/ alpha => DTX5

Normal => DTX5nm

[Ph34rb0t]

Converted my entire GameData folder, including addons to .dds in one go. I ran special conversion parameters (-n4) on every file with a "_NRM" in its name. Started up KSP and my space center was gone, lol.

Threw out ActiveTextureManagement and everything loaded up fine now. Even mod icons seem to work.

RAM usage with OpenGL is at 2.2 GB at the moment.

- E:

If you need to make a backup of all your .png and .tga textures, save this code as a batch file in your KSP root directory and run it.

It will create a backup directory in the parent directory of KSP's root directory.

set local=%cd%
cd..
mkdir KSPbackup
set backup=%cd%\KSPbackup
cd %local%\GameData
pause
robocopy /S "%local%\GameData" "%backup%" *.tga
robocopy /S "%local%\GameData" "%backup%" *.png
pause

Edited October 14, 2014 by Ph34rb0t

[slumpie]

there is also bc5/3dc, but ive never seen a game that uses it. its a 2 channel map which is the same pixel format as a dxt5 alpha channel. dxt5nm just moves red to alpha, leaves the green where it is, and red and blue become ballast (a dirty but effective hack invented by john carmack).

As I said before, BC5 requires at minimum Direct3d 10. Therefore I don't see it as an option for KSP.

so basically

basic texture => DTX1

texture w/ alpha => DTX5

Normal => DTX5nm

Yes pretty much.

It's worth mentioning that DXT1 and DXT5 give the exact same RGB result, but DXT1 only uses 50% of the space DXT5 does (because it doesn't save the alpha channel). Bitrates are fixed by the way.

And yes, 50% means indeed half the space.

Edited October 14, 2014 by slumpie

[sarbian]

1.1 was re-uploaded since it had a stupid error.

[Ph34rb0t]

Transitions between vessels or scenes are really quite a bit faster. Neat!

[Gaalidas]

dxt5 could makes tex bigger if they don't have alpha channel at all.

In such case, better to sue dxt1c format. (bump & spec map for example but as squad tex names are... not meaningful, it have to be made by hand unless you have some tool which detect the use of alpha channel)

Hey, it was a quick and dirty batch file to see if I could do it easily. I look forward to anyone else making something easier and able to distinguish between file types. The alternative is converting the textures one by one since there's no standard naming convention. It sure was a lot easier dealing with textures in, say, the elder scrolls series. Normal maps always ended in _n and spec maps always ended in _s and so forth. You never had stuff with model000, model001, etc. with no indication of what type of file it is without looking at it closely both visually and code-wise. I did find most files got a little bigger than their PNG counterparts, even without mips, but not as large as a fully uncompressed PNG or larger format would produce.

Now I can expand the batch process to check for properly named files (such as normal maps) before converting the rest in a "catch all" loop.

Edited October 14, 2014 by Gaalidas

[Nuke]

As I said before, BC5 requires at minimum Direct3d 10. Therefore I don't see it as an option for KSP.

my bad, i think i skipped a page there. probibly because i had to get up and do something in the middle of a post and didnt get back till several hours later.

[slumpie]

Oh no problem, it wasn't meant to sound harsh.

I also keep missing things all the time, that's normal in forums.

[Ph34rb0t]

Hey, it was a quick and dirty batch file to see if I could do it easily. I look forward to anyone else making something easier and able to distinguish between file types. The alternative is converting the textures one by one since there's no standard naming convention. It sure was a lot easier dealing with textures in, say, the elder scrolls series. Normal maps always ended in _n and spec maps always ended in _s and so forth. You never had stuff with model000, model001, etc. with no indication of what type of file it is without looking at it closely both visually and code-wise. I did find most files got a little bigger than their PNG counterparts, even without mips, but not as large as a fully uncompressed PNG or larger format would produce.

Now I can expand the batch process to check for properly named files (such as normal maps) before converting the rest in a "catch all" loop.

Thanks for your batch template, by the way!

I've expanded it a bit to catch both .png and .tga files in their _NRM and regular state. I don't know if the conversion options are optimal, but so far, I did not see any visual glitches.

(I don't have an eye for details though.)

The converter crashed on me twice or three times during the process, but this didn't have any readily perceivable consequences so far.

Anyway, here's my expanded batch code (meant to be run from KSP's root folder):

:: Initialization of environment states and variables.
SETLOCAL ENABLEDELAYEDEXPANSION
SET localdir=%CD%\GameData
SET DDSTool="J:\DDS Utilities\nvdxt.exe"

FOR /R "%localdir%" %%F IN ("*_NRM.png") DO (
    CALL %DDSTool% -dxt5nm -outsamedir -quality_highest -nomipmap -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)
pause
FOR /R "%localdir%" %%F IN ("*_NRM.tga") DO (
    CALL %DDSTool% -dxt5nm -outsamedir -quality_highest -nomipmap -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)
pause
FOR /R "%localdir%" %%F IN ("*.png") DO (
    CALL %DDSTool% -dxt5 -outsamedir -quality_highest -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)
pause
FOR /R "%localdir%" %%F IN ("*.tga") DO (
    CALL %DDSTool% -dxt5 -outsamedir -quality_highest -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)
pause

- Edit: Removed -n4 switch, which was inappropriate for the conversion.

- Edit²: Implemented -DXT5nm switch for normal/bump maps.

- Edit³: Script updated with -quality_highest for the best conversion results.

Edited October 15, 2014 by Ph34rb0t

[yecode]

i am always wondering how can you guys found out all those *weird* classes, apis in KSP assembly... this is really awesome btw.

[Urgante]

I used the Ph34rb0t code to convert all my gamedata folder again and all textures are working now, including transparent toolbar icons and textures from mods.

I noticed than only images multiple of 4 are being converted, is this intended or i should use another converter for that kind of images? (Around 300 unconverted images due to this, all of them small).

Thanks for the work, this is looking realy promising and evolving fast

[kpstjeev]

@Ph34rb0t, my experience was a little bit different.

This converter will gonna need a list of exceptions, as some plugins are intentionally loading PNGs, without asking database.

With ATM - Toolbar become a purple mess, as well as most icons in the tracking station. 50% of parts now have pure white textures.

Without ATM (and after ditching several mods so it could load) it looked mostly normal, I will wait till DSS support is integrated into ATM (or compatible with) before using it for anything other than test.

[slumpie]

I've expanded it a bit to catch both .png and .tga files in their _NRM and regular state. I don't know if the conversion options are optimal, but so far, I did not see any visual glitches.

• Regarding quality settings:
  

In terms of file size it doesn't matter what quality setting you use. The only difference is in how accurately and how quickly the textures get converted.

Because you only need to convert them once, I recommend -quality_highest.

• Also how to differ between textures with and without alpha:
  

-24 <dxt1c> 24 means 8:8:8 Note: dxt1c is the one you want. (Unless you want 1-bit alpha (then dxt1a), but 24bit textures have no 1bit alpha anyway.)

: compress 24 bit images with this format

-32 <dxt5> 32 means 8:8:8:8

: compress 32 bit images with this format

Note:

-8

-> Some people (I've seen a very few) use 8bit textures for specular maps or similar, so you still have to consider those.

I'd say you better leave them, they're too few to worry about.

-16

-> Not 100% sure about 16bit. You can have either 5.6.5, 1.5.5.5 or 4.4.4.4. I'd say better leave them, they're (just as 8bit textures) too few to worry about and Unity knows better how to differ between such textures than we could with nvdxt commands. Maybe try DXT1a, in my eyes a possible compromise.

Not 100% sure if this is failsafe, but it should work.

Note: Normalmaps still need exception. I recommend using DXT5nm, you call it with -dxt5nm.

Ehm you shouldn't have to use -n4 then if I'm not mistaken.

EDIT: -n4 converts height maps to normal maps, don't use it for existing normal maps!

• Textures with wrong resolutions:
  

-rescale <nearest | hi | lo | next_lo> (select one)

I guess nearest will work well, or maybe hi.

EDIT: -rescale next lo will cut resolution in half, so don't use that one. Read end of post for more information.

I used the Ph34rb0t code to convert all my gamedata folder again and all textures are working now, including transparent toolbar icons and textures from mods.

I noticed than only images multiple of 4 are being converted, is this intended or i should use another converter for that kind of images? (Around 300 unconverted images due to this, all of them small).

That's because it uses 4x4 blocks.

You can easily solve this by using -rescale <nearest | hi | lo | next_lo> (select one)

EDIT: -rescale next lo will cut resolution in half, so don't use that one. Read end of post for more information.

---------------

Rescaling

-RescalePoint

-RescaleBox

-RescaleTriangle

-RescaleQuadratic

-RescaleCubic

-RescaleCatrom

-RescaleMitchell

-RescaleGaussian

-RescaleSinc

-RescaleBessel

-RescaleHanning

-RescaleHamming

-RescaleBlackman

-RescaleKaiser

Not sure which one to choose. I heard Mitchell is the best for upscaling and Sinc the best for downscaling.

Although the question is how much of a difference it would make, since we're only scaling the textures to have a few pixel more or less.

Default is cubic apparently.

EDIT:

-rescale next lo will cut resolution in half, so don't use that one.

-rescale nearest -> nearest power of 2 (does nothing if already power of 2)

-rescale hi -> nearest higher power of 2 (does nothing if already power of 2)

-rescale lo -> nearest lower power of 2 (does nothing if already power of 2)

Edited October 18, 2014 by slumpie

[slumpie]

@Ph34rb0T

Oh I see -n4 converts greyscale height maps to normal maps. You shouldn't use that command for existing normal maps!

Edited October 14, 2014 by slumpie

[Lilleman]

It was a bit tedious to get everything working, and RPM still don't want to hear about dds. But, one mod at a time, I got rid of most PNGs and TGAs.

When I tested it yesterday, lots of textures got lost in the conversion process, so I was afraid results wouldn't be as good this time. Also, I installed KW Rocketry, so my mods folder is now 1.4GB (If I count Squad and NasaMission folder, I get to ~2.2GB).

This time I converted everything (including stock textures), the problem I had with DXT5's quality was due to a poor choice of program (pro tip: don't use ImageMagick for this).

And the new measure, with no missing textures this time : 1min04... While the loading time for my other install grew to more than 4minutes...

It's not promising anymore, it's confirmed awesome!

[shaw]

Some mods modify textures or perform operations on them, that's why they need access to (non-compressed) texture pixels. DDSLoader loads images directly to VRAM, doesn't "shadow" their non-compressed copies in RAM, so such mods can't work. Besides RPMs, at least EVE (^BoulderCo/), community resources (^CommunityResourcePack/), Lazor (^Romfarer/textures/) and Interstellar (^WarpPlugin/) need access to texture data.

[Ruedii]

Sarbian, yeah, I heard that DXT1 and DXT5 variants are the only two that they provide guaranteed support for on desktop. Results may vary for other formats.

There is also a technique ATI used in their research into the their 3Dc and 3Dc+ formats. It involves using reversable color-space transformation on the color-space of normals. Usually this is done with a 45 degree rotation, so you have an LV format texture instead of an RG format texture. I'm not sure how this would work exactly, but I suspect they used color transformation shaders to pull it off before they implemented hardware support.

[Ruedii]

But BC5 requires at minmum Direct3d 10.

ASTC is even worse in this regard.

Yes, but both are widely supported in OpenGL, starting in version 3.x where it became an optional extension.

Edited October 15, 2014 by Ruedii
added quote

[slumpie]

Yes, but both are widely supported in OpenGL, starting in version 3.x where it became an optional extension.

Am I being stupid and the runtime doesn't even matter as long as the hardware matches?

KSP runs by default in Direct3D 9 on windows and in OpenGL 2.1 on OSX if I'm not mistaken.

Edited October 18, 2014 by slumpie

[Nuke]

i am always wondering how can you guys found out all those *weird* classes, apis in KSP assembly... this is really awesome btw.

some of these formats are older than the average user of the ksp forums. therefore there is support for it in practically every piece of hardware and software out there. actually bc5/3dc is pretty old by computer standards, its just that the amount of legacy hardware/software in the wild that cant run it is significant enough to reduce the size of your market. the reason for using a dx<10 is because of the handful of folks still running xp are significant enough to support, and its easier to just go with one api for everyone than to do conditional support, and obviously you go with the one that is most compatible (oldest). its a lot like 64 bit, its been around for a really long time, but because there is a legacy crowd out there, you do 32 bit to keep them happy and the 64bit people really dont care because they can run it too. the only people left out in the cold are the technophiles who want to run software that reflects the capabilities of their hardware.

[blowfish]

. the reason for using a dx<10 is because of the handful of folks still running xp are significant enough to support.

Microsoft doesn't seem to think they're significant enough to support.

[sarbian]

And they are right to stop supporting them. XP is 13 years old, is ill adapted to modern hardware and full of problems. On the steam stats XP is down to 4.4 % (5% if you count the madmen running XP 64).

It has slowed down actual progress in desktop games dev for years (console played a part, but it's an other problem). CPU have been 64bit and multicore for the last 10 years and yet we did not get truly multi-threaded 64 bits games before 2 years ago.

Each time a see someone running a descent rig on XP I die a little. And I get an axe if they do it with an SSD.

[Ph34rb0t]

With ATM - Toolbar become a purple mess, as well as most icons in the tracking station. 50% of parts now have pure white textures.

Without ATM (and after ditching several mods so it could load) it looked mostly normal, I will wait till DSS support is integrated into ATM (or compatible with) before using it for anything other than test.

In my previous post, I've said that I had to ditch ATM after conversion to make everything work.

@Ph34rb0T

Oh I see -n4 converts greyscale height maps to normal maps. You shouldn't use that command for existing normal maps!

Okay. I've removed it from the post containing my script.

I'm not really familiar with NVidia's tools, only with Imagetool that comes with Flight Simulator X' SDK. To generate a valid FSX bump map in DDS format with it, you use its -RedInAlpha switch and simply I assumed that -n4 was something similar. My first attempt at a conversion actually was with Imagetool, but it's awfully picky regarding input format and outright refused to read the second .tga it had to process.

Some mods modify textures or perform operations on them, that's why they need access to (non-compressed) texture pixels. DDSLoader loads images directly to VRAM, doesn't "shadow" their non-compressed copies in RAM, so such mods can't work. Besides RPMs, at least EVE (^BoulderCo/), community resources (^CommunityResourcePack/), Lazor (^Romfarer/textures/) and Interstellar (^WarpPlugin/) need access to texture data.

Umm...

http://i.imgur.com/Boyl3qF.jpg

RPM 18.3, EVE 9-2, KSOS 4.08...all textures in .dds (except maybe two or three).

Even FinalFrontier's ribbons show up fine.

Edited October 15, 2014 by Ph34rb0t