[0.90] DDSLoader 1.9 (Mar 9) - Loads DDS Texture "Boringly fast" edition

[slumpie]

If you use as example u1555 or u8888 then the resolution doesn't have to be divisible by 4. That's only a limitation for block compression formats.

There's something else messing with the icons, because they even refuse to load without any changes to the resolution.

Edited October 20, 2014 by slumpie

[Urgante]

I can see the ring, it's very dark though. Not sure if it's supposed to be like this, but actually Jool itself is relatively dark too.

Dark doesn't mean it's grey or black, I can see details and some green, it's just dark.

They look like this in my pc http://i.imgur.com/uExvSRU.jpg but I have a light mod.

I have the version 1.1 of ddsloader yes.

Did you convert the entire KittopiaSpacemod folder to -u8888 or only the jool_ring.png file? (The others textures work for me in DXT5).

Are the other textures in DXT5? Did you used the nvdxt.exe?

Details please.

The icons converted with -u1555 produced this http://i.imgur.com/CkzafEI.jpg

A fast check with -u8888 worked, needs double checking, looks promising.

The files are bigger this way but It's almost irrelevant in my particular case, I had around 300 images non multiple of 4 but they are around 1mb in total.

Edited October 20, 2014 by Urgante

[slumpie]

The jool ring texture was the only texture I converted. I used your save and the mod seemed to work.

I'm currently playing on low settings, because my graphics card is still in RMA...

I used nvdxt with the -u8888 command. Maybe it's relevant to set -nomipmap as well. Oh and no rescaling, you don't have to rescale anything when using -u8888.

Icons with 1555 don't work at all for me, but 8888 works like a charm, without any issues. Maybe Sarbian can tell us why 1555 doesn't work.

Edited October 20, 2014 by slumpie

[Urgante]

Changing only the jool_ring.png file makes the ring darker, changing also the default ring texture "GameData\KittopiaSpace\Textures\Default\rings.png" makes it invisible.

The icons in partcatalog mod become invisible after the -u8888 conversion but maybe there is something wrong in my KSP install, I will check tomorrow from a clean install.

[Lilleman]

I still didn't managed to get normalmaps converting properly (using Cunch, or nvdxt)...

This is only a problem for mbm normal files, normals converted from tga or png don't seems to have any problem.

In-game, though, everything seems fine, but the log spam "Texture 'xxx' requested as normal map but texture is not a normal map and is not readable".

I tried almost every format I could think of, but I'm running out of ideas...

How can I get those files (specifically, they are converted from mbm to tga, then to dds using dxt5nm) recognized by KSP? I saw something earlier in this thread regarding Alpha and Green channels, but no luck with that.

Let's say I have the raw texture (ARGB format) available, and the possibility to save the resulting file to any of those formats, is there anything I can do? I'm starting to think I'll just ignore those files...

Edited October 20, 2014 by Lilleman

[sarbian]

I'll have a more in depth look at how the normal maps are processed. I may need to not mark them as read only from the start. Any stock part where the normal are really visible, so I can see it even with my awful vision ?

[Las-pen]

So basically this makes the game load faster?

[Lilleman]

I'll have a more in depth look at how the normal maps are processed. I may need to not mark them as read only from the start. Any stock part where the normal are really visible, so I can see it even with my awful vision ?

It's quite visible on the Rockomax x200-16.

When I use a format KSP don't recognize, the log just throw something like "can't load texture". So I guess he is loading dds normalmaps after all, I just don't really get why those errors shows.

Edit: Does anyone have any good source for infos on MBM files? I actually made a converter for those, but one of the header's fields is still a complete mystery:

It is called "magicNumber" (or something like this) in the mbm2png source, and it seems to be only 0 or 1. I noticed it's always 1 for normals, and 0 for everything else, but I actually need some confirmation for this one.

It could be a good way to know when a texture is supposed to be a normal, even when it's name don't give any clue.

@Las-pen: Yes, but at the moment it's quite tedious to get everything working. I think this might need some days for someone to come with an "all-in-one" tool for converting all this without errors. For now, it's mostly command-line tools and patience.

Edited October 20, 2014 by Lilleman

[sarbian]

ATM check a flag at pos 12 in the file for mbm and mark the file as a normal map if it is equal to 1.

Edited October 20, 2014 by sarbian

[Lilleman]

Thanks for the confirmation. I wrote a custom converter in .NET assuming this, glad I don't have to re-write that part.

Also, I just got rid of the warning: I saw in the DDSLoader's source that files are forced to be loaded as normal when their filename end with "NRM", so I made a quick test:

-convert the normal to dds using usual process

-rename the texture with something ending with "NRM" (in this case, I changed a "model001.dds" to "modelNRM.dds")

-Hex-edit the corresponding mu file, look for "model001.mbm" and replace it with "modelNRM.dds"

And this seems to work, the warning is gone. Still, I don't know how DDSLoader could be able to recognize a normalmap from a dds file, since their header is not supposed to contain that kind of informations...

[shaw]

That warning is harmless, just make sure you convert normal map to RGBA=YYYX (DXT5nm) format. Some stock models (AV-R8 winglet, 2.5m RCS tank) have normal maps in RGB=XYZ format named model001.mbm or similar (without NRM suffix) and they are kept readable and converted to YYYX only when the model is loaded. If the texture is already a DXT5nm DDS and not readable, everything will work fine.

[rbray89]

I still didn't managed to get normalmaps converting properly (using Cunch, or nvdxt)...

This is only a problem for mbm normal files, normals converted from tga or png don't seems to have any problem.

In-game, though, everything seems fine, but the log spam "Texture 'xxx' requested as normal map but texture is not a normal map and is not readable".

I tried almost every format I could think of, but I'm running out of ideas...

How can I get those files (specifically, they are converted from mbm to tga, then to dds using dxt5nm) recognized by KSP? I saw something earlier in this thread regarding Alpha and Green channels, but no luck with that.

Let's say I have the raw texture (ARGB format) available, and the possibility to save the resulting file to any of those formats, is there anything I can do? I'm starting to think I'll just ignore those files...

the AG channels have to be moved to the RG channels, and the A+B have to be set to full (or half, can't remember wich). to be considered a "regular" normal map.

[rbray89]

I'll have a more in depth look at how the normal maps are processed. I may need to not mark them as read only from the start. Any stock part where the normal are really visible, so I can see it even with my awful vision ?

This was the most annoying thing in ATM. You can check what I do to handle normal maps in the different loaders.

[sarbian]

And this seems to work, the warning is gone. Still, I don't know how DDSLoader could be able to recognize a normalmap from a dds file, since their header is not supposed to contain that kind of informations...

the DDS format has a bunch of unused flag we could hijack for the file who don't have the "NRM" in their names. Or I could just check of the presence of an empty file named <texturename>.dds_nrm, or read text/cfg files with a list of the normals (which end up being what ATM does).

[Lilleman]

@shaw and rbray89: Thanks, I finally figured it out: swizzling from RGBA to RGRA, then compress to dxt5 seems to do the trick. (It was working just fine using nvdxt, but I'm trying to get rid of external programs)

@Sarbian: I was thinking of changing the first 4 bytes of the dds header from "DDS " to something like "DDSn", since your code already has some functions to check this easily. Don't know if this could be a problem for Unity, though (does it check file headers before loading?).

Edited October 20, 2014 by Lilleman

[Apollo13]

Well this is definitely a exciting option. Don't know about anyone else but my heavily modded KSP takes forever to load up...

I hear that! I've got about three dozen mods (few parts mods, mostly gameplay, such as MechJeb). Somewhere, I learned about putting KSP in a window (ALT-ENTER). So, now I start KSP, then slide the window to the side for about five minutes while I do other things. When KSP is ready, I slide it back and ALT-ENTER again to go full-screen.

[sarbian]

Lilleman : if you change those then you might break other program that reads dds. There is 11 unused dual word ( dwReserved1[11] in the spec ). I would use the first of those dual word for our use. So that would be the byte at position 35 if I can still count ?

[Lilleman]

OK then, I'll go with modifying byte #35 for now. Then I'll have to compile a test fork of your dll to check how this works.

Of course if you plan to implement another way to determine if a dds is a normalmap, I'll adapt my code.

I would actually prefer this solution over an external text file, but both have pros and cons.

[shaw]

@shaw and rbray89: Thanks, I finally figured it out: swizzling from RGBA to RGRA, then compress to dxt5 seems to do the trick. (It was working just fine using nvdxt, but I'm trying to get rid of external programs)

You mean RGBA -> BGBR or RGBA -> GGGR? The latter lessen quality loss more. RGBA -> RGRA you mentioned shouldn't work (correctly).

[sarbian]

KSP does RGBA => GGGR, so it should be done in the same.

[Lilleman]

I've done some "reverse-engineering" (okay, maybe it was bruteforce) in order to get my files converted exactly the same way as nvdxt's dxt5nm. For now I got it working with RGBA->RGRA, but I've only tested a few files.

It surprised me too, since I didn't see this format elsewhere yet... But it looks like it works.

[slumpie]

Lilleman are you still taking about .mbm files? Apparently they already have X in alpha.

If you use -dxt5nm on those you will lose the x vector from the alpha.

That what you see after converting a xy_ (sometimes xyz) normal map with the -dxt5nm command is yyyx or gggr. It doesn't matter what's in the red and blue channel though, because they will be ignored. I think it's done to reduce co-compression artefacts, but I'm not sure. Not all tools do this, some leave them blank of fill one or the other.

But again, .mbm are different, at least that's what I got told.

------

I made another annoying discovery:

Out of whatever reason are many regular purple normal maps (xyz or xy_) saved with alpha channel, a full alpha channel to be precise.

Makes no sense, because they absolutely don't use the alpha and it just bloats the file size, but it makes the -32 nvdxt command chatch them.

Therefore my solution a few pages back isn't universal.

Edited October 21, 2014 by slumpie

[Lilleman]

Yep, I was talking about mbm files, but I ended not using the nvdxt5 command. I've made a small program to do the conversion automatically, that use the DirectX API. Since it doesn't have native dxt5nm support, I have to do the swizzling "by hand" before saving the file to dds.

RGBA->RGRA was the result of trial and errors, and even if it does not make any sense (as stated earlier, Unity shouldn't be able to use those), they are exactly the same as nvdxt's files... I don't really get it, but as long as it works, I don't really bother.

MBMs seems to be an exception, though, I think I'll have to use gggr for some other files (the AV-R8 winglet normal, for instance).

It's easy to determine when a mbm is a normal, it has a flag in his header. For TGAs files, however, either I look in the file name for something familiar ("nrm", "normal", "nm"...), or I check if all pixels have their blue channel set to 1 (0xff). Probably not the most efficient solution, and it could be a problem for some textures... Though, I didn't encountered "pure blue" textures in KSP yet, so this will probably do the trick...

[slumpie]

No nvdxt doesn't save them as RGRA. If you open a converted file and compare the channels, then you'll see that the red data landed in the alpha channel and the red and blue channel is just a copy of green.

The red and blue channel will look worse than the green channel and show more artefacts, but that's just a result of the lower bit number.

Unity doesn't care about the red and blue channel:

[LIST]
[*]inline fixed3 UnpackNormalDXT5nm [COLOR=#000000]([/COLOR]fixed4 packednormal[COLOR=#000000])[/COLOR] 
[*][COLOR=#000000]{[/COLOR] 
[*]   fixed3 [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL]; 
[*]   [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]xy[/COLOR] [COLOR=#008000]=[/COLOR] packednormal.[COLOR=#2b91af]wy[/COLOR] [COLOR=#008000]*[/COLOR] [COLOR=#0000c0]2[/COLOR] [COLOR=#008000]-[/COLOR] [COLOR=#0000c0]1[/COLOR]; 
[*][COLOR=#008000]#if defined(SHADER_API_FLASH)[/COLOR] 
[*]   [COLOR=#008000][I]// Flash does not have efficient saturate(), and dot() seems to require an extra register.[/I][/COLOR] 
[*]   [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]z[/COLOR] [COLOR=#008000]=[/COLOR] sqrt[COLOR=#000000]([/COLOR][COLOR=#0000c0]1[/COLOR] [COLOR=#008000]-[/COLOR] [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]x[/COLOR][COLOR=#008000]*[/COLOR][URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]x[/COLOR] [COLOR=#008000]-[/COLOR] [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]y[/COLOR] [COLOR=#008000]*[/COLOR] [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]y[/COLOR][COLOR=#000000])[/COLOR]; 
[*][COLOR=#008000]#else[/COLOR] 
[*]   [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]z[/COLOR] [COLOR=#008000]=[/COLOR] sqrt[COLOR=#000000]([/COLOR][COLOR=#0000c0]1[/COLOR] [COLOR=#008000]-[/COLOR] saturate[COLOR=#000000]([/COLOR]dot[COLOR=#000000]([/COLOR][URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]xy[/COLOR], [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL].[COLOR=#2b91af]xy[/COLOR][COLOR=#000000])[/COLOR][COLOR=#000000])[/COLOR][COLOR=#000000])[/COLOR]; 
[*][COLOR=#008000]#endif[/COLOR] 
[*]   [COLOR=#008de4]return[/COLOR] [URL="http://unity3d.com/support/documentation/ScriptReference/30_search.html?q=normal"][COLOR=#8b6ba6]normal[/COLOR][/URL]; 
[*][COLOR=#000000]}[/COLOR] 
[/LIST]

EDIT: Wow, why is the code box so small...

Oh and btw, I found out why nvdxt mirrors the green channel. It is in fact done to reduce compression artefacts.

Edited October 21, 2014 by slumpie

[Lilleman]

Well, I was focusing on mbm files. And they seems to have their own rules regarding swizzling... I just ran some tests with TGAs, and this time Unity expect xGxR format. So you were all right about that, RGRA isn't a format Unity should use (I really don't get why this is working.)

As a result, my code is a mess (programming in VB don't help), since it now has two separate functions for swizzling, and it has to perform a bunch of tests to get the right conversion parameters...

I'm starting to understand why my first batch with Crunch did not convert half of my textures properly...