[1.9.x] Textures Unlimited - PBR-Shader, Texture Set, and Model Loading API

[Manwith Noname]

With regard to how the colour masks work, am I reading it right that they need to be "flat" RGB, as in you are not reading individual channel data for the layer as such? I'm probably not explaining it too well. If, for example, the mask has yellow parts, which would be red 255 and green 255 with blue 0, can this mod recognise that the primary layer is "red" and the secondary layer will place the "green" channel on top of the primary?

I have a whole bunch of masks I made for KerbPaint but got slightly demotivated due to various rendering issues like the light flare on landing gear being "painted" or parts with masks not casting shadows. It looks to me like these issues are resolved here.

[Shadowmage]

19 minutes ago, Manwith Noname said:

With regard to how the colour masks work, am I reading it right that they need to be "flat" RGB, as in you are not reading individual channel data for the layer as such? I'm probably not explaining it too well. If, for example, the mask has yellow parts, which would be red 255 and green 255 with blue 0, can this mod recognise that the primary layer is "red" and the secondary layer will place the "green" channel on top of the primary?

I have a whole bunch of masks I made for KerbPaint but got slightly demotivated due to various rendering issues like the light flare on landing gear being "painted" or parts with masks not casting shadows. It looks to me like these issues are resolved here.

Any place in the mask texture that is colored RED will have that portion of the model recolored by the 'Main Color' selection in the recoloring GUI.
Any place in the mask texture that is colored GREEN will have that portion of the model recolored by the 'Second Color' selection in the recoloring GUI.
Any place in the mask texture that is colored BLUE will have that portion of the model recolored by the 'Detail Color' selection in the recoloring GUI.

If you mix colors anywhere in the mask, you MUST normalize them to a range of 0-255(byte) or 0-1(float).  If you do not... you will get 'blown out' colors, as they calculated through color-add processing.  So, if you color something R255-G255-B0 -- you'll simply blow-out the color range, as the output will end up in the 0-510(byte) value range, so everything will be super bright, and the upper-half of the sliders will be useless.

Basically, DONT mix colors in the mask.  Make it red, OR make it green, OR make it blue, OR make it black.  Anything else will result in rendering artifacts.

 

 

If you want to see how the system is supposed to be setup, and how it works in practice -- go download and check out SSTU.  Nearly every part in there is recolorable, using the process I have laid out over my last few posts.  Not only does it work, but it works extremely well, allowing the largest degree of user control over the resultant color scheme and look of the parts that is possible with PBR shaders (it exposes every possible parameter to use control).  But only if you follow the proper setup and create the proper mask and diff/spec/met textures.

 

30 minutes ago, Manwith Noname said:

can this mod recognise that the primary layer is "red" and the secondary layer will place the "green" channel on top of the primary?

No.. why would it do that?  How does that even remotely make sense?  Why wouldn't you just create the mask properly in the first place?

[Manwith Noname]

1 hour ago, Shadowmage said:

No.. why would it do that?  How does that even remotely make sense?  Why wouldn't you just create the mask properly in the first place?

You might do it that way so to repaint the entire texture in a uniform colour in game you only need to adjust one layer, then the other two are used for detail or not at all. There's not really a "proper" way to do it. KerbPaint did it the way I described and you've done it another, neither is right or wrong it just depends on how you approach it.

Spoiler

In the image above, the red channel sets a "basecoat" and the green and blue add detail.

I only wished to clarify how it functioned to see if I could easily configure my current masks to work with this mod so stock becomes available. So thanks for clarifying.

Edited December 28, 2017 by Manwith Noname

[Electrocutor]

@Manwith Noname

From my understanding...

_MainTex

• Where _MaskTex is black, this should be used as per normal with WYSIWYG
• Where _MaskTex is not black , this should be gray in areas that you want recolored. 127,127,127 means 100% recolor.
• Higher and lower values from 127,127,127 will blend the recolor with the diffuse, with intensity depending on how bright the _MaskTex color is (brighter _MaskTex means less diffuse).
• Alpha is ignored

_BumpMap

• Used normally
• Note that you must make your _MaskTex match this because there is no masked normal
• Uses DXT5nm format

_MaskTex

• Should be black in areas where you want to use the diffuse as usual
• Red channel is Main, Green channel is Secondary, Blue channel is Detail
• The total color value should not exceed 255, but blending between channels is allowed. f.e. a gradient of 255,0,0 at one end, 0,255,0 at the other and 127,127,0 in the middle.
• Alpha is ignored

_SpecMap

• Where _MaskTex is black, the Red channel is specular and Alpha channel should be metallic (I cannot confirm since metallic currently is set to use the full RGBA)
• Where _MaskTex is not black, this should be gray in areas that you want recolored. 127,127,127,127 means 100% recolor specular and metallic
• Higher and lower values from 127,127,127,127 will blend the recolor with the _SpecMap, with intensity depending on how bright the _MaskTex color is (brighter _MaskTex means less _SpecMap)

_AOMap

• Standard grayscale occlusion map

_Emissive

• Standard emission map for KSP

_MaskColor1, _MaskColor2, _MaskColor3, _MaskMetallic

• These are the variable parameters that get set when using the recolor GUI
• _MaskColor alpha channel contains the specular
• _MaskMetallic is a vector, so x, y, z are 1, 2, and 3

Meshes

• You can utilize multiple meshes by grouping meshes into their own named sections which will all show up in the recolor GUI.
• This allows you to have more than 3 recolorable regions on parts.
• You can blend the 3 regions in a single section depending on what color you make the _MaskTex, but you cannot blend between sections.
• Because each section has its own properties, you can utilize this to also change normal maps per section.

Overrides

• The KSPTextureSwitch PartModule works on a top-down and cumulative workflow. This means that you could in fact have two instances of KSPTextureSwitch on the same part where one had a single TextureSet that specified your masked shader, mask texture, and recoloring defaults while another allowed you to choose between a of number normal maps and specular maps to change the look of the part; for example, plates and bolts vs foil or pristine vs cracked glass.

Edited December 28, 2017 by Electrocutor

[Shadowmage]

8 minutes ago, Manwith Noname said:

In the image above, the red channel sets a "basecoat" and the green and blue add detail.

Well, if you were to directly render the existing mask textures onto the meshes -- they would look very similar to that.  Here is a render out of blender showing a model with the mask texture used for diffuse coloring:

(windows are a separate mesh, and are black in the mask texture, so they pull all of their color directly from the diffuse texture without any adjustments)

The actual diffuse, specular, and metallic textures used for that model are basically that grayscale noise texture I posted yesterday.  This is because it uses the diffuse/spec/metallic textures as 'detail input' so that you can still have some highlights/etc in the textures, while still allowing for them to be recolorable, and not having to deal with trying to 'tint' things.

The way the shader works in recoloring mode is (simplified for a single recoloring channel)

if mask texture is black the output color = the input color
if mask texture is R the output color = ( ( inputColor * 2 ) - 1 ) + userSpecifiedColorForMainColor
(repeat for G, B channels)

 

11 minutes ago, Manwith Noname said:

to repaint the entire texture in a uniform colour in game you only need to adjust one layer, then the other two are used for detail or not at all.

Interesting concept, and with that explanation I can see what you were referring to.  Makes quite a bit more sense in that context.  But no, that is not how the TU masked/recoloring shader functions.  It expects that any particular spot on the mesh will only have one of the three 'recoloring channels' applied to it (or an appropriately normalized mix across boundary regions).

(might be willing to create a special mode in the shader that functioned that way that could be enabled through a keyword; feel free to post up an issue ticket with that as a feature request if it is something you are interested in; please note that it might take a few weeks to get it done though)

[Manwith Noname]

@Electrocutor Thanks for that information.

@Shadowmage Another thing it makes very easy is blending. Nose cones and fairings for example I typically applied a base coat and then overlaid in another channel a slowly fading to transparent layer from the tip.. I don't have any screenshots handy to demonstrate it but hopefully you can picture what I mean.

Edited December 28, 2017 by Manwith Noname

[Jimbodiah]

30 minutes ago, Shadowmage said:

 

 

 

HAH!   When you posted that some time ago I just thought it was your attempt at the recoloring functionality.  

[Manwith Noname]

On 28/12/2017 at 6:37 PM, Shadowmage said:

(might be willing to create a special mode in the shader that functioned that way that could be enabled through a keyword; feel free to post up an issue ticket with that as a feature request if it is something you are interested in; please note that it might take a few weeks to get it done though)

I kinda missed this part but if that is at all on the cards, it would be fantastic. Primarily because I have most of the stock parts already masked in some way shape or form and the thought of going through them all again to "re-layer" them fills me with dread, plus I have a workflow built around that method. There are some parts I wanted to revisit and some with only a red base coat but due to KerbPaint having some issues due to lack of maintenance I kind of lost the enthusiasm. I'm not a coder and I'm not even a graphic artist but I can "paint by numbers" in 3 colour channels and fudge MM patches!  I have only made the masks I have because I didn't want to see what KerbPaint offered die completely and this seems like the perfect mod to move those on to with all it's bells and whistles on top.

I have an idea of how to move forward short term which I will take to @Electrocutor's thread since it is entirely about the config work he has already done more than functions of this mod.

[Shadowmage]

52 minutes ago, Manwith Noname said:

I kinda missed this part but if that is at all on the cards, it would be fantastic.

If you post up the issue ticket..sure.  No ticket = no work (I'll forget about it before I would ever start working on it).

Still will be a few weeks/months out though.  Far too overloaded with work that I actually need for my mods to be spending too much time on 'extras' that are of no use to me.

--Or post up a PR with a new shader that does what you want right out of the box.  I'll gladly accept PRs as long as they don't cause problems elsewhere in the code/mod/etc, and it would likely be available much sooner than if you are waiting for me to do the work.

[Shadowmage]

1 hour ago, Manwith Noname said:

going through them all again to "re-layer" them fills me with dread,

You know... if you have them all as distinct files (why wouldn't they be..they are textures)...

It would be a relatively simple affair to write a script that looped through them and re-layered them according to some simple rules.

//pseudocode
//for each pixel in the texture
Color c = pixelColor;
Color o;
if(c.b>0){o = 0,0,1,1}
else if(c.g>0){o=0,1,0,1}
else if(c.r>0){o=1,0,0,1}

Using C# (or Java) it wouldn't be too many more lines of code than that.  A few dozen lines of file reading/writing code, and a few more to access the pixel data structure.  Much faster than trying to redo them by hand anyway.

Now, replicating the blended layering would be slightly more difficult, but likely still doable.  Just more math.

 

(just brainstorming some ideas to try and help you get what you are looking for a bit quicker)

[Manwith Noname]

Honestly, you don't know how many times I have wished for an automated conversion process for various things...

To give you more of an idea of what the current masks look like...

Spoiler

Individual channels followed by composite.

https://imgur.com/a/3lFje

Edit: Apparently I don't know how to make an imgur album either...so it's now a link to a page...

 

...which ends up looking like this (for the cones at least)...

Spoiler

Given that 1.4 could potentially bin it all I think I'll hold fire on doing anything beside learning how this works. I've got a tax return to do...

Edited December 30, 2017 by Manwith Noname

[stuChris]

my god this mod looks amazing.

[Electrocutor]

In case people haven't realized, you can actually use these shaders without doing any PBR work at all or anything if you just want specular light and reflections to work.

Just set the shader to SSTU/PBR/StockMetallicBumped with _Metal at 0 and you're done.

Edited January 3, 2018 by Electrocutor

[stuChris]

35 minutes ago, Electrocutor said:

In case people haven't realized, you can actually use these shaders without doing any PBR work at all or anything if you just want specular light and reflections to work.

Just set the shader to SSTU/PBR/StockMetallicBumped with _Metal at 0 and you're done.

i mean, i cant figure out how to get the mod to work in the first place  ive installed it like any other mod, but get no difference in-game (am i just an idiot and am missing something?)

[Eric S]

12 minutes ago, stuChris said:

i mean, i cant figure out how to get the mod to work in the first place  ive installed it like any other mod, but get no difference in-game (am i just an idiot and am missing something?)

 

I think that's because this mod does nothing by itself, it just allows modders access to the ability.  You need a parts pack or MM patch that has parts or alters parts to use it.

 

[Shadowmage]

9 minutes ago, stuChris said:

i mean, i cant figure out how to get the mod to work in the first place  ive installed it like any other mod, but get no difference in-game (am i just an idiot and am missing something?)

Because this mod, by itself, does nothing that a user would see.  This mods purpose is to load the shaders and make them available to other users/developers to make use of, as well as providing a framework for managing and updating reflection probes in a fashion that works in KSP.

You need another mod / set of configs that apply the shaders to parts.  @Electrocutor has created quite a few of these patch sets, covering stock and a few other popular mods, which can be found:

(or, you can check out SSTU -- the only mod that I know of that makes full integrated use of the PBR + recoloring systems;  you'll need both the regular SSTU release + the SSTU-PBR expansion pack in order to make use of it all)

[stuChris]

2 minutes ago, Shadowmage said:

Because this mod, by itself, does nothing that a user would see.  This mods purpose is to load the shaders and make them available to other users/developers to make use of, as well as providing a framework for managing and updating reflection probes in a fashion that works in KSP.

You need another mod / set of configs that apply the shaders to parts.  @Electrocutor has created quite a few of these patch sets, covering stock and a few other popular mods, which can be found:

(or, you can check out SSTU -- the only mod that I know of that makes full integrated use of the PBR + recoloring systems;  you'll need both the regular SSTU release + the SSTU-PBR expansion pack in order to make use of it all)

ooohhh, thank you, i hadnt realized this

[Manwith Noname]

I've been having some fun playing with this, making and breaking things. I noticed that the tinting mode behaves quite differently in a number of ways with regard to how colour, specular and metallic properties are handled and without going too much in to the things I like about each I decided to setup both modes in a similar fashion via texture sets and switching to play about. It seems that once the tinting mode has been applied to a part, it gets stuck on and does not revert to the "normal" mode.

This screenshot shows how the part first appears with the tinting mode not set...

Spoiler

This screenshot shows how it appears when switching to the tinting mode texture set...

Spoiler

This screenshot shows how it appears when switching back to the non tinting mode texture set...

Spoiler

I hope I got them in the right order...

Edited January 4, 2018 by Manwith Noname

[Shadowmage]

11 hours ago, Manwith Noname said:

It seems that once the tinting mode has been applied to a part, it gets stuck on and does not revert to the "normal" mode.

You've hit a limitation of the current implementation of KEYWORDs -- namely, there is no way to disable them once enabled.

(it is also a bit of fallout of the re-use of existing material instances, rather than recreating a blank material for every texture-set change)

The first bit I can solve (very ugly code though to fix it, and it would break every other config/mod using recoloring shaders); the second bit... can't be solved without breaking Electrocutors' patch setups -- you would be limited to a single TextureSwitch effecting any specific mesh; you couldn't have separate texture-switch modules handle metal/specular values.

So, sadly, nothing really can be done about either right now.  Pick a shader/keyword combo, and stick with it for all of the texture sets for any given part.

(some of this -will- be changing during the update to KSP 1.4; new TU releases past that point will not be backwards compatible)

 

Speaking of which --

@Electrocutor --  might you have some time to 'chat' over the weekend a bit?   Something a bit more real-time than forum PMs if possible (IRC, steam chat, whatever).  I'd like to discuss what features you need/would like from both shader and plugin code, as well as go over some of the already planned config-breaking changes.  As you are the main external party making use of TU, and would likely be more impacted by the changes than others, I would like to see what you think of them and give you a chance to help figure out feature-set.

Mostly I would like to figure out a new 'stock recoloring' shader, so I can remove the tint-mode keyword stuff from the SSTU masked shaders.  As I have no personal use for such a shader, I'm going to need help/input on figuring out what it should do and how it should function.  A few other bits to discuss regarding the 'property bleeding' and the multi-texture-switch support; things that will need to be sorted out before solutions can be put in place.

(if anyone else has a vested interest in, and knowledge of, the shader setup, and would like to take part in these discussions, please speak up -- not trying to be exclusive, will gladly expand the conversation to anyone with knowledge and interest)

Edited January 4, 2018 by Shadowmage

[Manwith Noname]

Righto.  I did snoop the github and stumbled upon that discussion shortly after I posted. I should also point out something I forgot to mention, I had each texture set up to use different colours in the primary layer. One was set to absolute black, the other absolute white. These also didn't switch but maybe this is the same problem.

As far as what I would wish for in terms of how things function...

My own preference is to recolour stock parts and existing mods as easily as possible. It turns out, I have actually made my existing masks work well enough on their own without the need to create more textures, though I still need to remap the way the RGB is setup to get the right colours from the values. (I'm really wishing I used paths when making these now)

Spoiler

The reason I tried to setup two texture sets was from my current testing, tinting mode clamps values within a certain range and gives a really rich black. Without tint mode, black is always kinda grey but I can really blow out spec and metallic values to produce a nice chrome type effect which is not possible with my current configs in tint mode unless RGB layers overlap. Like below...

Spoiler

Ideally, I personally would like as much creative freedom from within game via sliders and essentially a kind of mixed mode between what tinting and non tinting settings currently offer. Rich blacks while also being able to over saturate to compensate for things like high metallic levels.

These are simply my wishes and desires so far, not demands. Heh, it might even be possible already I just haven't found the magic in the time I have spent mucking about. There's still some property settings I need to mess with.

[fourfa]

Is there an easy way to make this mod default to Appearance = "Metal" setting, instead of "Default Metal"?

[Manwith Noname]

@fourfa

Assuming you are talking about @Electrocutor's Porkjet config, open it up in a text editor and find...

currentTextureSet = PorkjetStock_DefaultMetal

...replace all with...

currentTextureSet = PorkjetStock_Metal

 

[fourfa]

ah sorry, should be in Electrocutor's thread.  Actually it's for the stock configs, but that'll set me in the right direction

[Shadowmage]

15 hours ago, Manwith Noname said:

My own preference is to recolour stock parts and existing mods as easily as possible.

Contrary to my original beliefs, this actually is the single most requested feature from this mod.  I thought that reflections and proper PBR would have been the bigger feature and driver of adoption... but no... its people who want recoloring. 

In regards to recoloring of existing textures (stock / mods) -- until/unless someone can offer me a better way to do the shader, I'm afraid it isn't going to improve much.  The options are extremely limited when using existing textures and without requiring additional inputs to map things out.  I've given it a ton of thought, and cannot find any method to allow for proper recoloring that does not also require new textures be created.

The options are:  (diffColor = diffuse texture color; userColor = user selected color from GUI; baseColor = color input from 'base color' map)

1. 'Tinting' mode.  This is what you currently have when you enable tinting mode in the shader.  Requires only a user/dev provided 'mask' texture for each part.  The base diffuse/spec textures influence the range of colors/values available through the sliders. 
   • OutputColor = diffColor * userColor
2. 'Grayscale' mode.  This is the SSTU mode, using a grayscale diff/spec/met base texture.  Results in authentic/proper recoloring -- the color of the part is exactly what was specified in the GUI. 
   • OutputColor = ((diffColor*2)-1) + userColor
3. 'Mapped' mode.  Supply a recoloring mask, and a base color mask.  The 'base color' mask allows for extracting the details out of an existing diffuse texture, so they can be remapped on top of the user-selected colors.  Results in authentic/proper recoloring -- the color of the part is exactly what was specified in the GUI.  (not yet implemented, just a concept; nobody was interested in the 'grayscale' mode, so I doubt they would be interested in this one). 
   • OutputColor = (diffColor - baseColor) + userColor

The method of blending of the layers in the mask texture really doesn't matter as long as the 'tinting' problem exists.  This is going to be the single largest problem with adoption of recoloring for existing parts, without requiring new textures be created, and I'm not sure that it can be solved (without adding more textures).  I certainly haven't found any solutions to it, even after approaching the subject multiple times, and spending probably 40-80 hours working on it (you think the grayscale mode that I use was the first thing I tried?  not even close....).

How would you propose solving this tinting problem?  (provide math/proofs please)  (the rest of the discussion is moot unless this problem can be solved)

 

15 hours ago, Manwith Noname said:

Ideally, I personally would like as much creative freedom from within game via sliders and essentially a kind of mixed mode between what tinting and non tinting settings currently offer. Rich blacks while also being able to over saturate to compensate for things like high metallic levels.

As I've said above -- maths please.  How would such a thing be accomplished without requiring additional data inputs?  I really don't think what you are asking for is even possible.  I would be glad for someone to prove me wrong.  Ecstatic even.  But I'm not holding my breath.

 

To detail the specifics of the problem --

• The goal of a 'recoloring' shader is to provide user control over the base coloring of a part.  Whether it is a single color applied to the whole part, or using a mask input for multiple areas, the goal is the same - changing the displayed color of the object.
• Recoloring of a mesh in a 'raw' fashion is simple -- just use the color set in the GUI directly, and render it to the mesh.  The problem with this is that the mesh will then lack texture-based details.  No noise, no panel lines, no gradients, no dirt, etc.
• The root of the problem -- how to extract just the 'details' from an existing arbitrary diffuse texture, and apply those on top of the user-selected colors?
  • You can either author a 'detail only' texture (grayscale mode),
  • Or add a secondary input mask that denotes the 'base color' for every pixel in the original diffuse texture, which allows for 'nulling out' the base color, and extracting of the detail data.

 

The only thing I've been able to think of that even partially alleviates this problem without requiring additional/new textures, would be to add three new color parameters to the shader -- 'baseColor1/2/3'.  These would take the place of the 'base color' texture listed above, and be applied to the main,second,detail recoloring channels respectively.  This could work just as well as the 'base color' mode from above in specific setups.  Notably you would be limited to three 'base colors', -and- they would be forced to line up with the mask texture -- so if the actual part needed more base colors, or more sections, some of them would still recolor improperly  (also would not work well with mask textures that used any sort of blending, unless they were pre-normalized properly - and the base colors also needed blended).

 

It just seems that I've already given all of the options that are needed to have full and proper recolorable parts, but nobody wants to do the work to make it happen (e.g. creating new textures).  Can't eat the cake without baking it first (and can't even make it without the proper ingredients)....

[Manwith Noname]

@Shadowmage

I understand a bit better now, thank you for that explanation. To be clearer, I'm not expecting to make no textures and magically see exactly what I want. My initial concern from reading what the required textures to set this all up need to look like was that you might need to redistribute modified textures of others work. I have found ways of working around this but maybe with your latest post I might not even need that trick. I think I'm getting muddled up in the procedure to use when actually creating a mod and integrating this system and the much simpler, "let's take what a modder has done and add colour masks".

I'm not an artist or a coder so I'm never going to be able to make textures akin to a lot of the part creators here. Likewise, asking me to read or write code would be like asking me to 讀寫中文. At least for that we have google. I really do appreciate what is currently possible, please don't misunderstand what I'm saying. While I don't know the inner workings of shaders I am simply expressing things that may not have been considered from a programmers perspective but as a user looking to enable more creative freedom within the game. For me, this is the epitome of what KSP is, the freedom to design things in a physics sandbox. KerbPaint and now this mod are wonderful extensions to further that.

Saying that, I can make simple "flat" texture masks that overlay the real art and muddle together module manager patches. I will keep playing with configs and masks to see what I can achieve. Like I said previously, some of the things I'm looking to do may well be possible right now, as is and I just have not found it.