[0.90] DDSLoader 1.9 (Mar 9) - Loads DDS Texture "Boringly fast" edition

[NathanKell]

rbray: assuming one preformats the texture to Unity's internal format, with Z ignored there's 4 bits each for X and Y, paletted at 64bit for 4x4 should quite be decent. I wouldn't like to apply S3TC to an RGB normal map though!

[Gaalidas]

So, I tried this with one of the B9 pods, that oddly shaped drone thing. It seems that the textures, once converted into DDS format, need to be transformed in some way as well to align properly on the model. In this case, it appeared that the texture was loaded in upside down from what it is supposed to be. I wonder if this is related to the flipped state of the textures you extract from the asset files? Those come out flipped vertically and if you want to modify them for use in TextureReplacer you need to flip them vertically again to make them load up properly.

[slumpie]

Were all textures upside down?

[Camacha]

No, DDS files can be edited the same as other formats like tga or png.

.mbm files however are a pain to work with.

Great!

You're one who see the glass half empty aren't you ?

I guess not

[slumpie]

@Camacha

You often need plugins though.

What image editor are you using?

[Camacha]

What image editor are you using?

I currently have access to Photoshop.

Edited October 14, 2014 by Camacha

[slumpie]

https://developer.nvidia.com/nvidia-texture-tools-adobe-photoshop

You must use the installer, you can't just copy it.

If you save a file don't be confused if it only shows DDS as option for the format, another window will popup and give you more options then.

[Ruedii]

There are DDS export tools for The GIMP. Also, AMD has released some texture reprocessing tools that can generate DDS textures from various formats.

On another note, does this work on all platforms, or just Direct3D ones. I know the DDS texture load routines are different on Direct3D (which has driver-level support for DDS textures) and OpenGL (with which you need a loader program, but still has many advantages with DDS textures.)

On one more thing, is there any way you could supplement the texture compression options in Unity3D to add actual modern texture compression formats selected based on what the user's graphics driver supports?

[Nuke]

opengl can do dds just fine. its kind of a misconception that its an m$ format. dxt1 and 5 were actually s3 formats back in the stone age of 3d acceleration, the formats are really old. m$ had its own specifications for them in d3d (likely to avoid paying s3) and the names they used kind of become the defacto terminology over time. they may have created the dds file format, but once the pixel data is loaded the video card doesnt care.

No, DDS files can be edited the same as other formats like tga or png.

yes and no. yes you can open them and edit them, but since the file is lossy. you should load an uncompressed version if you want to make changes. you dont want cumulative data loss over a series of edits.

Edited October 14, 2014 by Nuke

[sarbian]

Lilleman : I'm not keen on using native code. It gets complex fast on multi-platform games.

On another note, does this work on all platforms, or just Direct3D ones. I know the DDS texture load routines are different on Direct3D (which has driver-level support for DDS textures) and OpenGL (with which you need a loader program, but still has many advantages with DDS textures.)

It should work on all, but I did not test it.

On one more thing, is there any way you could supplement the texture compression options in Unity3D to add actual modern texture compression formats selected based on what the user's graphics driver supports?

I'm limited to what Unity supports since before I load the texture I must create it with a format unity know. But Unity know a lot more format than the 3/4 we talked about here, so we'll see.

Shaw (of TextureReplace) sent me some code to load other DDS format (uncompressed RGB/RGBA) and I'll look if Unity can handles others. I'll see if I can do the convention of normal to what unity needs (even if DXT does not seems to be the best idea for those). I could handle the up/down flip too, but I'm not sure it's such a good idea.

If you have texture that fail loading please put it somewhere so I can test it.

[slumpie]

There's one DXT5 based format that was made specially for normalmaps, but support is somewhat questionable I guess.

EDIT: Namely DXT5nm

[Lilleman]

Took a closer look at this today, using KSO as a guinea pig. Problems I had with it yesterday were due to some missing arguments in Crunch while converting it's textures. Everything is loading just fine now. (Used ImageMagick, since it can flip vertically an image during a conversion via a batch).

I thought DXT5 compression would have been less agressive, though: here is a screen to illustrate. The original texture is a 2048*2048 tga file.

Am I missing something or is this what a compressed DXT5 dds should look like? DXT1 is almost the same quality...

Still, I'm sure lots of people won't be bothered by those small artifacts. On smaller objects, it's barely noticeable.

I'll try that again later, with some uncompressed dds, but I'm wondering if the loading time will be as good.

As if you should handle the vertical flip, maybe this whole conversion process is a job for an external tool, not a plugin.

[sarbian]

"DXT5nm is the same file format as DXT5 except before compression the red channel is moved into the alpha channel, the green channel is left as-is, and the red and blue channels are blanked with the same solid color. This re-arranging of the color channels is often called swizzling."

it is how unity does normal map. and how my mod expect them to be

Lilleman : uncompressed is not in yet, but will be soonish.

[slumpie]

But sarbian, if you manually convert a normalmap into DXT5 you won't automatically get DXT5nm.

But if I understand you right, we can use DXT5 for diffuse and specular and DXT5nm for normalmaps.

Edited October 14, 2014 by slumpie

[sarbian]

Yes, you need to move the channels before the convertion, or use a tool that does it like the Nvidia Tools (nvcompress.exe -bc3n). I m quite sure Crunch has an option for that somewhere too. Edit /DXT5_xGxR may be the one for crunch.

Edited October 14, 2014 by sarbian

[Ruedii]

For normal maps you should use DXN/BC5/3Dc or ASTC

Those are the two "modern" standards I was referring to.

Unfortunately Unity only currently supports these on mobile platforms, because most developers find the S3TC algorithms tolerable, even if they aren't the best, and mobile devices don't support S3TC.

Mobile devices instead instead force you to chose between 3Dc, ASTC, and some compression standards that were designed for OpenGL ES 1.0. 3Dc and ASTC are superior to S3TC, but the other standards, well let's just point out that OpenGL ES 1.0 doesn't even support 24bpp/32bpp RGBA mode to give you an idea of what I'm talking about.

Edited October 14, 2014 by Ruedii
fix smily

[slumpie]

But BC5 requires at minmum Direct3d 10.

ASTC is even worse in this regard.

Edited October 14, 2014 by slumpie

[Gaalidas]

Were all textures upside down?

Wow this thread is growing fast. Yes, all the textures that I converted (only the B9 stuff at the time) were upside down as far as I could tell, considering UV mapping makes it hard to tell sometimes. Last night I rearranged my batch file to also invert all the textures that I converted. the nVidia converter also failed on any texture that was not dimensioned on the "power of two" rule, so I made sure that those stuck around as PNG (which has given me the best performance when also using LoadOnDemand mod). Testing so far has shown that everything works as intended. On purpose I am not touching GUI buttons (toolbar, stock-toolbar, custom UI images) as of yet. These are so small anyway that it doesn't matter what format they use. One little error is with the B9 flat panels which appeared to have a quarter of the model (one triangle in the square shape) brighter colored than the rest. When switching the mesh to the open frame I got better results, but the interior of the frame was not consistently textures as they were supposed to be. My thought on this problem is that something funky has happened to the normal mapping, since the effect seems similar to when a normal map contains too many white edges and makes a model appear to be way too shiny at certain close angles in the VAB. My experiments continue...

[slumpie]

Power of two rule? Resolution should be a multiple of 4 in all dimensions, because block size is 4x4.

That's why mipmaps have memory padding if it can't be divided by 4.

EDIT: Damn my english is bad, I misunderstood what power of 2 means.

Edited October 14, 2014 by slumpie

[sirkut]

Wow this thread is growing fast. Yes, all the textures that I converted (only the B9 stuff at the time) were upside down as far as I could tell, considering UV mapping makes it hard to tell sometimes. Last night I rearranged my batch file to also invert all the textures that I converted. the nVidia converter also failed on any texture that was not dimensioned on the "power of two" rule, so I made sure that those stuck around as PNG (which has given me the best performance when also using LoadOnDemand mod). Testing so far has shown that everything works as intended. On purpose I am not touching GUI buttons (toolbar, stock-toolbar, custom UI images) as of yet. These are so small anyway that it doesn't matter what format they use. One little error is with the B9 flat panels which appeared to have a quarter of the model (one triangle in the square shape) brighter colored than the rest. When switching the mesh to the open frame I got better results, but the interior of the frame was not consistently textures as they were supposed to be. My thought on this problem is that something funky has happened to the normal mapping, since the effect seems similar to when a normal map contains too many white edges and makes a model appear to be way too shiny at certain close angles in the VAB. My experiments continue...

Mind sharing that batch file? I started modifying my powershell script utilizing mbm2png but came across some issues with what you said. i couldn't figure out how to know if a file was successful or not during conversion using nvdtx.exe

[Gaalidas]

It's actually pretty simple. I use a "FOR" loop to go through all the files that I want to modify, then use a "CALL" command on the nVidia command-line tool for converting image types, then I use an "IF" command to see if the resulting filename with the new extension exists before deleting the unneeded file. Here's what I have, though the code I'm going to post is not ready to be used outside of my home system as of yet.

ConvertoToDDS.bat

@ECHO off

:: Initialization of environment states and variables.
SETLOCAL ENABLEDELAYEDEXPANSION
SET localdir=%CD%
SET DDSTool="C:\Program Files (x86)\NVIDIA Corporation\DDS Utilities\nvdxt.exe"

FOR /R "%localdir%" %%F IN ("*.png") DO (
    CALL %DDSTool% -dxt5 -outsamedir -quality_normal -nomipmap -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)

I put that batch file in a folder on my computer under "documents" that I call "MyTools" and reference this from a shortcut. I can put that shortcut into any sub-folder I want to start from (for instance, the "B9_Aerospace" folder) so that I do not convert everything in one go (I'm not done testing the effect on GUI elements or TextureReplacer elements at this time) and I simply change the "Start in:" field to match the folder that the shortcut resides in. Otherwise it will look in the "MyTools" folder for the images to convert, and successfully find nothing to do and quit without error. EDIT: Alternatively you could just copy the batch file to whatever folder you want to start recursively checking and it would work just fine out of the box. Also note, the new file will be placed in the same place as the input file and the batch file will recursively select any file in any sub-directory from the starting folder.

As you can see from the code, I first set it to not bore me with echoing commands. I set a local variable that probably isn't needed here but helps if I ever need to expand a variable at a delay from the start of the script. Next I have the path to the tool I want to use, in quotes so that it does not choke on the spaces in the path.

I also define the local "start in" folder to a variable just so that it isn't determining the local path every time I want to reference the default "%CD%" environment variable.

Then I have the loop which places each PNG file in each loop into the variable defined as "%%F" which I can then reference with a variety of interpretations using the "~" operator where "~dF" gets the drive letter, "~pF" gets the path (minus the drive letter), and "~nF" gets the file name (minus the extension). Since I use "CALL" the batch will wait until the nVidia tool is finished before executing the next command. It then checks to see if the new file, at "%%~dpnF.dds" exists (the file drive, path, and name minus extension, with the new extension at the end) and if it does, it deletes the PNG file. Else, it leaves it alone. Note, the nVidia tool continues to output all sorts of crud to the screen despite the echo being turned off, but that can be fixed later if you care.

And that's it. It's not even the most complicated batch file I've written. You should see the one that I use to detect and convert any image downloaded in a mod to the format that I prefer to use. That baby is massive, and makes heavy use of subroutines and the "EXIT /B" command, and even uses several different batch files to process the entire thing. I even implemented a basic progress meter and counter in the title bar. I'm quite pleased with it.

EDIT: Note, I realise I'm using "-nomipmap" in my conversion command. It runs faster, and for testing I decided mips were not required.

Edited October 14, 2014 by Gaalidas

[Justin Kerbice]

ConvertoToDDS.bat

FOR /R "%localdir%" %%F IN ("*.png") DO (
    CALL %DDSTool% [B][U]-dxt5[/U][/B] -outsamedir -quality_normal -nomipmap -flip -file "%%F"
    IF EXIST "%%~dpnF.dds" ( del "%%F" )
)

dxt5 could makes tex bigger if they don't have alpha channel at all.

In such case, better to sue dxt1c format. (bump & spec map for example but as squad tex names are... not meaningful, it have to be made by hand unless you have some tool which detect the use of alpha channel)

[blowfish]

You're my hero sarbian . Why Squad didn't implement this to begin with I will never understand.

[sarbian]

New version : 1.1 Adds support for RGB and RGBA thanks to shaw

I also updated to link to the nvidia tools link in the first post to more recent tools Those have more option and should be able to convert texture normal textures ( those named *NRM*) to DXT5nm in one run.

[Ph34rb0t]

The link to 1.1 doesn't work. Replacing 1.1 in the URL with 1.0 still correctly points to the old version though.

(Might want to make Jenkins work faster.)