[1.9.x] Textures Unlimited - PBR-Shader, Texture Set, and Model Loading API

[Delbrutis]

13 hours ago, Manwith Noname said:

You'll also want to find a suitable replacement for...

shader = SSTU/PBR/StockMetallicBumped

So doing a find replace for PBR/Metallic and TEXTURE worked great on my old configs. Thank's for the suggestion ! Now part of my older configs are working but 85% of my shaders are using  SSTU/PBR/StockMetallicBumped Would you happen to have a suggestion on what would be a suitable equivalent  shader ? Or is their some place with documentation listing what shaders are  available so a person could experiment  with finding a replacement ?

Thanks

 

 

[Electrocutor]

8 hours ago, Delbrutis said:

So doing a find replace for PBR/Metallic and TEXTURE worked great on my old configs. Thank's for the suggestion ! Now part of my older configs are working but 85% of my shaders are using  SSTU/PBR/StockMetallicBumped Would you happen to have a suggestion on what would be a suitable equivalent  shader ? Or is their some place with documentation listing what shaders are  available so a person could experiment  with finding a replacement ?

Thanks

 

 

To mimic old behavior:

MATERIAL {
	shader = TU/Metallic
	mode = update
	keyword = TU_STOCK_SPEC
}

[Shadowmage]

3 hours ago, Electrocutor said:

To mimic old behavior:

MATERIAL {
	shader = TU/Metallic
	mode = update
	keyword = TU_STOCK_SPEC
}

Yep, that is correct (though that 'update' mode will cause errors unless used very carefully, but should acceptably for model-data style patches; more problematic in runtime texture switching).

The trick to enable the stock-specular equivalent decoding is the third line 'keyword = TU_STOCK_SPEC '.

[Munarmunar]

Hello, i'm having some trouble trying to modify the shine/PBR of some parts. I'm trying to modify it via the difficulty menu, i tweaked all of the settings but nothings changes, my question is if there is another way around making this.

Thank you

[Delbrutis]

12 hours ago, Electrocutor said:

To mimic old behavior:

MATERIAL {
	shader = TU/Metallic
	mode = update
	keyword = TU_STOCK_SPEC
}

I'll have to try that and see if I can get my old configs back up and running.

Thanks for the info.

Edited February 24, 2020 by Delbrutis

[Mlgisawsome02]

When will support for making history come out? I have some making history tanks and parts in my builds sometimes and I don't like using the shaders on the supported parts and then having the unsupported parts being stock. Also lower the strength of the shadows because it gets super dark on the side with the shadow

[Shadowmage]

On 2/23/2020 at 11:01 AM, Munarmunar said:

Hello, i'm having some trouble trying to modify the shine/PBR of some parts. I'm trying to modify it via the difficulty menu, i tweaked all of the settings but nothings changes, my question is if there is another way around making this.

Thank you

This mod by itself does not change any part textures or enable PBR for anything.  All that Textures Unlimited does is provide the shaders for others to use, and provide a framework for runtime material adjustment.

To actually change parts, you need a mod that either uses the features of TU for its textures -- you can see some of those in the OP, or check out SSTU or KerbalFoundries from my signature.  The other recommended option is @Manwith Noname's TexturesUnlimitedRecoloringDepot, which does add PBR support for stock parts (and some mods!).

2 hours ago, Mlgisawsome02 said:

When will support for making history come out? I have some making history tanks and parts in my builds sometimes and I don't like using the shaders on the supported parts and then having the unsupported parts being stock. Also lower the strength of the shadows because it gets super dark on the side with the shadow

Please see the answer above.  You'll need to find a mod that has configs for whatever parts you want updated.  TexturesUnlimited is a framework, an API for use by other modders.

2 hours ago, Mlgisawsome02 said:

Also lower the strength of the shadows because it gets super dark on the side with the shadow

TexturesUnlimited has absolutely nothing to do with shadow-strength or lighting settings.  You are barking up the wrong tree on that one...  maybe try the built-in stock settings?

[Shadowmage]

Hopefully I'll be releasing a 1.9 update over the weekend.  Have been spending the past ~two weeks working on tracking down some odd normal-map related bugs in regards to the use of the stock textures, and finally made some headway the past few evenings.  Hint:  The stock textures are wrong (encoded incorrectly).

For this, I must say 'Thank You' to @Manwith Noname for his initial report of the issue, and his patience while helping work through what has been, to date, one of the strangest problems I've investigated.  Its been... fun (well, as fun as bug-hunting can be).

In the next release(s) the shader packs are likely to ~double in size due to the additional variants needed to support the incorrectly encoded stock normal maps while still allowing for use of correctly encoded textures everywhere else and adding support for BC5 encoded normal maps (something new!).

The alternate solution would have been to do run-time correction of the stock textures directly in the GameDatabase, but the startup impact for that solution turned out to be quite high, and could have easily added minutes to the startup time (worse on slower CPUs).  This method will still be available (already wrote the code, no reason to remove it), but it will be up to patch authors if they want to fix the textures (though they shouldn't need to).

The shader based solution should have no runtime/performance impact whatsoever, and should be fully compatible with all existing stock + other correctly BC3/DXT5nm encoded normal maps (as long as there is Y axis data in the G channel and X axis data in the A channel).  Use of BC5 encoded normal maps will require activation of a keyword in each of the MATERIAL blocks ('keyword = TU_BC5_NRM'), and when activated will only decode BC5 (not BC3/DXT5nm).

(now I just need to update all of my tools to support BC5 as an export format...)

[Gaalidas]

5 hours ago, Shadowmage said:

now I just need to update all of my tools to support BC5 as an export format...)

Dunno what tools you're using or speaking of... but I do a lot of texture modification, daily, and I've developed a few tricks to handling DDS.

First off... if you're using GIMP or another tool with outdated plugins for DDS, I suggest converting any texture you intend to edit into a TGA format first.  I found that the GIMP plugin left artifacts on any exported DDS image, and PNG takes a lot longer to export to than TGA in the same image when I timed it.

Then, to handle DX11 texture formats, I had to get creative.  There are multiple command line utilities out there for handling texture conversions and I've discovered one that works in about 95% of the cases I run into.  I'd called "DirectXTex" or, just "texconv.exe" and it's pretty powerful.  Unsure where I got it originally though.  I've spent the last few years developing a set of complicated batch files to handle all of my texture formatting and converting needs.  Even went as far as you write one to separate the filenames from extracted unity textures for easy editing, and then restitch them together.  Helps a lot when you want to update texture IDs for new versions of games without re-editing all the files by hand.  I'm a bit of a mad man with batch files... so titled by my old school buddies.

Anyway... if you can find that converter, it can handle just about everything you'd need including a few obscure cases where there isn't a specified format or compression in the file.  I've only discovered a few different types that I couldn't use this tool on, and in those cases, the batch-processing utility in Irfan-View did the job (if I could live with the alpha channel disappearing).

I've even handled 8k BC7 textures with relative ease, and those usually don't even import into a DDS handling program except for the higher end ones.

[Shadowmage]

4 hours ago, Gaalidas said:

Dunno what tools you're using or speaking of... but I do a lot of texture modification, daily, and I've developed a few tricks to handling DDS.

A collection of custom C#-WPF designed tools to handle texture conversion, texture merging (putting specular values in alpha channels), and general... fun.  https://github.com/shadowmage45/TexturesUnlimited/tree/master/Plugin/TexturesUnlimitedTools

They currently use a collection of NVDXT.exe for texture compression/exporting(DXT1/DXT5), XNA-DXUtils source code for DXT1/5 for texture reading/decompression, and an assortment of custom code for doing the in-memory data manipulation (once it is in a byte array of RGBA pixels...simple enough to deal with).

 

4 hours ago, Gaalidas said:

First off... if you're using GIMP or another tool with outdated plugins for DDS, I suggest converting any texture you intend to edit into a TGA format first.  I found that the GIMP plugin left artifacts on any exported DDS image, and PNG takes a lot longer to export to than TGA in the same image when I timed it.

Yeah, I gave up using the GIMP plugin for DDS conversion long ago.  It works well enough for -reading- the textures to look at them in GIMP, but for exporting... yeah, it was terrible.  Hence the use of the NVDXT.exe for compression (Nvidia DXT Utilities; a legacy application from Nvidia).

I internally store all of my (exported) art assets as PNG -- compressed, lossless, supports full alpha, relatively simple to decode and encode with built-in system libraries (well, built-in to C#/.NET anyway).  The original art assets are of course buried inside of GIMP or Substance Painter projects, but I still use PNG for exporting from those tools and as the base for conversion to DDS.  I'm not concerned about the time-to-export, and would far rather have the wider support that PNG offers compared to TGA (everything I know can handle PNG (even Windows Paint...), very few things do TGA (properly) that I'm familiar with).

 

4 hours ago, Gaalidas said:

I'd called "DirectXTex" or, just "texconv.exe" and it's pretty powerful.

Possibly this one:  ( https://github.com/Microsoft/DirectXTex/wiki/Texconv )?

Indeed, I started down that road at one point, but as I was trying to do it all 'in-application', I opted to not use their command-line utility.  Technically they are supposed to have libraries as well so that I could link to them and use them all from within my own code, but I could never figure out how to get the references sorted out (always missing some random DirectX library, that apparently doesn't exist in the wild anymore).

Edit:  Hmm... this might fit the bill ( https://github.com/deng0/DirectXTexNet )

 

Still looking for code-only solutions if possible -- some sort of a C# library that I can reference to use the functions from within my own code.  Sadly I have not been able to locate anything that meets the requirements of:

a.) 'free' (libre) - such that I can include the library or source in my tools and redistribute it; e.g. open-source compatible licensing
b.) modern enough to support the recent formats (BC3/BC3n/BC5)
c.) modern enough to actually be usable as a library and not require some deprecated windows-only DirectX dependencies that no longer exist, and preferably not windows-only at all.
d.) supports both decompression (reading into memory) and compression (writing to disk).  I've found some that can do one or the other (for a limited number of formats), but none that can do both for all of the required formats.
e.) supports managed C#/.NET.  Quite a few C++ libraries available, but I hate C++ about as much as I hate python (and only slightly more than I despise Java) (all for different reasons )

Yeah, its a pretty tall order (compared to what is available...), which is why I have been slowly writing my own stuff.  But its... slow and tedious work, at best.

 

Anyhow, thanks for the suggestions, and I'll certainly give them a look over before I decide on what direction to go for these features.

 

Edited February 28, 2020 by Shadowmage

[Manwith Noname]

To comment on the texture tools, I find GIMP better than nVidia texture tools, at least as far as exporting colour masks to DXT1. I did a test a while back and it produced the least artefacts. I compared GIMP, nVidia Texture Tools and DXTBMP. The main selling point for GIMP was the perceptual error checking. Without that, all results were much the same but the error check got rid of a fair few 4x4 blocks in certain scenarios. This may still be possible to achieve with the nVidia Texture Tools command line interface but I just found it much simpler to tick a box during export in GIMP where the rest of the project is based. From what I could make out, most tools use the nVidia tools as the basis for their conversion.

More recently, I've been looking at something that can handle BC7 out of curiosity. I haven't tried anything yet but I'll post the things I have come across so far that I'll be looking at checking out...

https://github.com/GameTechDev/Intel-Texture-Works-Plugin

If you use Photoshop, then this ^. Most of what I have read is that the Intel algorithms provide the best result. If someone knows of a standalone tool using this then I'm all ears.

https://vvvv.org/contribution/texconvgui

This is what I'm probably going to test at some point as I have not used Photoshop for years. It is essentially a GUI for Microsoft's texconv.exe previously mentioned from what I can tell.

I didn't check but I imagine both will have support for BC5...at least I'm hoping so.

[Shadowmage]

I also just ran into: https://github.com/GPUOpen-Tools/Compressonator

Which is AMD's open-source asset compression (textures and models) pipeline.  Seems like it has both GUI and command-line utilities available, for BC1-7, and a few more I'm unfamiliar with.  Likely will be playing around with it in the near future, will let you know what I find out

 

13 minutes ago, Manwith Noname said:

I find GIMP better than nVidia texture tools,

There are some settings you have to use for NVDXT.EXE to get good results, at least for DXT5.  I don't use DXT1 as much myself, but I use the same settings for those..  ( https://github.com/shadowmage45/TexturesUnlimited/blob/master/Plugin/TexturesUnlimitedTools/TextureConvertWindow.xaml.cs#L66-L84 ).  IIRC it amounts to:

nvdxt.exe -file "InputFileName.png" -output "OutputFileName.dds" -Triangle -flip -quality_highest -dxt[1/5/5nm]

The important bits are the '-Triangle' filter mode, and '-quality_highest' flags, which seem to do some magic to smooth out many of the artifacts that I would see from the GIMP plugin.  Honestly haven't compared the DXT1 outputs though (or done much comparison at all in recent years), so could be that GIMP does those just as well or better.  Mostly I've been concerned with normal maps and what where diffuse/specular maps (yep, haven't even re-checked since I started using the metallic/gloss setup).

29 minutes ago, Manwith Noname said:

The main selling point for GIMP was the perceptual error checking.

Is this an export setting in GIMP somewhere?  Not familiar with that term in reference to the GIMP/DDS plugin.  Perhaps my plugin is out of date...

[Manwith Noname]

1 hour ago, Shadowmage said:

Is this an export setting in GIMP somewhere?

Yeah, you need to expand the + Advanced options during export and tick the checkbox. Like I said, it might be essentially the same as some of the  arguments you are putting in the nVidia tools command line, I just did a quick comparison some time ago with arguments I found. Anyway, here's an example to show what I mean because a picture is worth a thousand words...

Spoiler

The top part which is ultimately the "background" in this image is the "artwork" in GIMP. The middle is a straight DXT1 export without any fiddling and matches pretty much exactly what I experienced with everything I compared with at the time (tested by exporting the GIMP stuff as a PNG and converting in other applications), including the limited nVidia arguments I happened to find back then. Bottom is a GIMP export with the error checking box ticked.

I suspect those arguments (triangle and quality, which I don't recall having at hand because to be fair, I only searched for arguments so much) you list are essentially the same thing as I'm fairly sure the nVidia tools are the base for pretty much all DDS / DXT conversion tools of this era.

 

Edit: ...and I only did this testing because of changes to how TU was handling the recolour. In ye olde way back with tinting mode, these blocks were barely noticeable. It was only when moving over to the normalisation I found these artefacts to cause me grief (and this was by far the worst scenario I encountered). In the end though, everything turned out for the best 

Edited February 28, 2020 by Manwith Noname

[Arco123]

I have no idea on anything but do you know why panther/whiplash textures are borked? Or am I missing something. Also, I may be wrong but proc wings doesn't seem to work at all. (TURD, the prco wings work fine tho.) Thanks!  

[Manwith Noname]

25 minutes ago, Arco123 said:

...why panther/whiplash textures are borked? Or am I missing something.

This should automagically fix itself in the next TU release. There was an odd issue when using an old release of TU in newer versions of KSP, though I found they didn't fully manifest themselves unless KS3P was installed. If KS3P wasn't present, they worked.

[Shadowmage]

12 hours ago, Manwith Noname said:

The top part which is ultimately the "background" in this image is the "artwork" in GIMP. The middle is a straight DXT1 export without any fiddling and matches pretty much exactly what I experienced with everything I compared with at the time (tested by exporting the GIMP stuff as a PNG and converting in other applications), including the limited nVidia arguments I happened to find back then. Bottom is a GIMP export with the error checking box ticked.

Ahh, yeah, that nasty messy output was what I was seeing out of GIMP (but didn't see out of the other tools I was using at the time), which is why I simply wrote-off the GIMP plugin and never looked back.  Good to know that it has some settings to clean up those issues.

Huh.. looking at it more, it claims to be able to do BC5 exports as well... interesting...

[Arco123]

11 hours ago, Manwith Noname said:

This should automagically fix itself in the next TU release. There was an odd issue when using an old release of TU in newer versions of KSP, though I found they didn't fully manifest themselves unless KS3P was installed. If KS3P wasn't present, they worked.

Oh, that explains it. Do you know why proc wings won't work? I have ks3p btw. I'll redownload both TURD and TU today. Also, it's very dark when not hitting light and I did not find any white shiny presets with out it changing dark.

Edited February 28, 2020 by Arco123

[Shadowmage]

5 hours ago, Arco123 said:

I'll redownload both TURD and TU today.

You need to wait for the next TU update for it to actually fix anything, which likely won't be until sometime over the weekend.

Once that is available, then yes, the updated TU should fix many of the issues with the stock textures (and possibly some modded ones?).

[Gaalidas]

Either way you go... I wouldn't suggest anyone export into DDS from gimp.  There have been a lot of reports, mine as well, of nasty artifacts on some Gump exported DDS images.  That's why I export to TGA first, then go to DDS from there.  I'd do PNG, but it's quicker to export to TGA and, this ay, my batch files can differentiate between png files that I'm allowing to remain like that, or haven't yet vertically flipped, with those that are ready for conversion.  Then I can just iterate thought the entire directory structure.

Anyway, yeah... I'd like to go code-only at some point but it's proving a bit difficult to do at my current knowledge level.  Even some of the other tools I've used before were simply calling up an external command line utility to do the actual work.  I'm gonna look at that github repo you posted though... very interesting.

Also.. I agree, C++ is ugly.  C# is the future... maybe...

Edited February 28, 2020 by Gaalidas

[Manwith Noname]

On 2/28/2020 at 3:11 PM, Arco123 said:

Oh, that explains it. Do you know why proc wings won't work? I have ks3p btw. I'll redownload both TURD and TU today. Also, it's very dark when not hitting light and I did not find any white shiny presets with out it changing dark.

Additionally to what @Shadowmage mentioned, make sure you have reflections enabled in the game options, without that, parts configured to make use of them tend to appear quite dark. Also, if you're sending mirror like objects in to space they tend to look black too. 

As for why pwings doesn't work, I'd need to see the config as I take it you managed to get the recolour pack working.

[Shadowmage]

Took a bit of time this week (beyond updating things) to play around with getting an updated post-process stack functional in KSP, using the updated Post Processing Stack v2 that is supported by the new Unity versions.  This will be released as a stand-alone mod at some point in the future, and is intended as a spiritual successor to KS3P as it is no longer being maintained.

So far I've gotten the code imported to KSP, the shaders compiled in Unity and loading into KSP, and can enable some basic effects and play with settings.  Yet to-do is to investigate each built-in effect to see which work/don't work, what the settings do, and what the default values are/should be.  Further, I will need to build a profile parsing and loading system, in-game config editing UI (wip, above), and some method to save the edited configs.

Likely will be packing up a TU release a bit later this afternoon; was waiting to see if it would require any changes/fixes to support the post-process stuff.  So far everything looks to be working...

[Shadowmage]

Updated release is available:

https://github.com/shadowmage45/TexturesUnlimited/releases/tag/1.5.10.25

See the link for change-log and downloads.

Notably it fixes some issues with stock normal maps (for reuse of those textures by config/patch authors), and adds optional support for the BC5 normal map format. Should be backwards compatible with existing configs.  Should also continue to work fine with KSP 1.8.

[dtoxic]

On 3/1/2020 at 7:55 PM, Shadowmage said:

Updated release is available:

https://github.com/shadowmage45/TexturesUnlimited/releases/tag/1.5.10.25

See the link for change-log and downloads.

Notably it fixes some issues with stock normal maps (for reuse of those textures by config/patch authors), and adds optional support for the BC5 normal map format. Should be backwards compatible with existing configs.  Should also continue to work fine with KSP 1.8.

Hi i am getting a couple of nullrefs with this update, nothing is breaking everything is working fine i just thought you might want to look at this error

KSP 1.9.1

Spoiler

NullReferenceException: Object reference not set to an instance of an object
    KSP.UI.Screens.ApplicationLauncher.RemoveModApplication (KSP.UI.Screens.ApplicationLauncherButton button) (at <55ba45dc3a43403382024deac8dcd0be>:0)
    KSPShaderTools.Addon.TexturesUnlimitedDebug.Awake () (at <d05d146f0f32458d9a30cf17e61f35a8>:0)
    UnityEngine.DebugLogHandler:LogException(Exception, Object)
    ModuleManager.UnityLogHandle.InterceptLogHandler:LogException(Exception, Object)
    UnityEngine.GameObject:AddComponent(Type)
    AddonLoader:StartAddon(LoadedAssembly, Type, KSPAddon, Startup)
    AddonLoader:StartAddons(Startup)
    AddonLoader:OnLevelLoaded(GameScenes)
    AddonLoader:OnSceneLoaded(Scene, LoadSceneMode)

Edit: I think it;s happening on every scene change, or at the very least switching from SC to VAB,SPH, etc..(all the building), i am not sure about in flight to SC or Tracking station switch, hope it helps

 

Edited March 3, 2020 by dtoxic
Additional Info

[Shadowmage]

19 hours ago, dtoxic said:

Hi i am getting a couple of nullrefs with this update, nothing is breaking everything is working fine i just thought you might want to look at this error

KSP 1.9.1

  Reveal hidden contents

NullReferenceException: Object reference not set to an instance of an object
    KSP.UI.Screens.ApplicationLauncher.RemoveModApplication (KSP.UI.Screens.ApplicationLauncherButton button) (at <55ba45dc3a43403382024deac8dcd0be>:0)
    KSPShaderTools.Addon.TexturesUnlimitedDebug.Awake () (at <d05d146f0f32458d9a30cf17e61f35a8>:0)
    UnityEngine.DebugLogHandler:LogException(Exception, Object)
    ModuleManager.UnityLogHandle.InterceptLogHandler:LogException(Exception, Object)
    UnityEngine.GameObject:AddComponent(Type)
    AddonLoader:StartAddon(LoadedAssembly, Type, KSPAddon, Startup)
    AddonLoader:StartAddons(Startup)
    AddonLoader:OnLevelLoaded(GameScenes)
    AddonLoader:OnSceneLoaded(Scene, LoadSceneMode)

Edit: I think it;s happening on every scene change, or at the very least switching from SC to VAB,SPH, etc..(all the building), i am not sure about in flight to SC or Tracking station switch, hope it helps

 

Thanks for the report.  Should be fixed up for the next release.

Looks like the code that should be spawning (or removing) the 'TU Debug' app-launcher button was not properly checking its references.  Shouldn't have had any impact unless you had debug features enabled, and the only impact in that situation was the loss of ability to open the debug GUI.

On 2/28/2020 at 4:43 PM, Gaalidas said:

Anyway, yeah... I'd like to go code-only at some point but it's proving a bit difficult to do at my current knowledge level.  Even some of the other tools I've used before were simply calling up an external command line utility to do the actual work.  I'm gonna look at that github repo you posted though... very interesting.

Yep, I'm in pretty much the same boat.  Really want to go code-only, but I've got a bit to learn yet before I can implement it myself.  Getting there, slowly.  I can reason through the 'decompression' algorithms well enough, but writing the compression end of things is still a bit beyond my understanding.

Thankfully, at this point, I think I'm okay with the compression being done by an external tool.  As long as I can load the textures into memory (uncompressed and compressed sources), and manipulate the data there, I'm less concerned with the final compression process itself (which is usually the last step in the process, to be used for finished textures).  Sure, would be nice to be able to do it all in-memory, but as long as the external tools are reliable, no reason not to use them.

 

On 2/28/2020 at 4:43 PM, Gaalidas said:

Also.. I agree, C++ is ugly.  C# is the future... maybe...

My issues with C++ stem from.. well, yeah, how ugly it is, and how painful it is to set up projects.  References (&), pointers (*), and standard variable types (they each have their uses, but are often abused poorly).  Manual memory management can be a complete PITA.  I probably would be much better with it now than I was last I tried it, but I still think I would prefer the 'quicker to use' scripting languages.

No doubt C++ is better suited for the implementation of specific functions, or for use in specific performance-critical places; but a general purpose scripting language, it is not.

 

 

 

[dtoxic]

Just now, Shadowmage said:

Thanks for the report.  Should be fixed up for the next release.

Looks like the code that should be spawning (or removing) the 'TU Debug' app-launcher button was not properly checking its references.  Shouldn't have had any impact unless you had debug features enabled, and the only impact in that situation was the loss of ability to open the debug GUI.

Great, thx for looking into it! (i am allergic to nullref's) :D