[1.9.x] Textures Unlimited - PBR-Shader, Texture Set, and Model Loading API

[Shadowmage]

18 minutes ago, Philaphlous said:

So something I'm not always sure of with mods and version compatability is backwards compatability. I'm using rss/ro 1.3.1 and I've got the version of tu for 1.3.1 but I'm wondering if the later versions will still run in 1.3.1 with the additional enhancements???? If it's screwed up do I just delete the folder and download the supported version with my game?

Newer versions (compiled for KSP 1.4+) will not work with KSP 1.3x due to changes in the KSP-API.

You might be able to use configs/textures/shader packs from the newer versions in your legacy install, but the plugin absolutely will not work.  The configs/textures/shaders also may or may not work depending on exactly what they do and if they touch any features that were changed between the versions (notably, there was a syntax change in the 1.4+ configs that will render them unusable in older versions).

So I would say that the short-answer is -- no, newer versions of the mod will not work properly/reliably on KSP 1.3.x.

 

[BRAAAP_STUTUTU]

i'm not sure if it's even TU that's causing this but I have no knowledge of wheter it already was like this before I installed TU but:

it seems to only be like this for the stock 2.5m station parts (MPL-G, Hitchhiker and Cuppola)

[alexus]

welcome to my topic with TU config`s work

[Gordon Dry]

2 hours ago, alexus said:

welcome to my topic with TU config`s work

How about a link?

->

 

[Manwith Noname]

22 hours ago, BRAAAP_STUTUTU said:

i'm not sure if it's even TU that's causing this but I have no knowledge of wheter it already was like this before I installed TU but:

 

Are you using the recolour pack? If so, I have half addressed this with a patch to remove the texture switch from stockalike station parts that are affected...

@PART[crewCabin,cupola,Large_Crewed_Lab,dockingPort2]:FOR[ZZZZZZ_NEEDSMOARZEDS_ZZZZZZ]:NEEDS[StationPartsExpansionRedux]
{
	!MODULE[KSPTextureSwitch],* {}
	!MODULE[SSTURecolorGUI]
}

At least, that's the parts I am aware of anyway.

If you are not using the recolour pack, the patch may still work.

 

Edit: If you don't know what to do with that....

1) Make a blank text file somewhere in your GameData folder.

2) Paste the stuff in the code box above into that text file and save it.

3) Rename the text file to "anythingyoulike.cfg". Without the quotes. You'll get a warning about changing the file format but it's fine.

Edited October 13, 2018 by Manwith Noname

[alexus]

 

On 10/14/2018 at 3:03 AM, Gordon Dry said:

How about a link?

 

https://forum.kerbalspaceprogram.com/index.php?/topic/178932-textures-unlimited-configs-and-retexturing-of-some-models/

I am sorry

Edited October 15, 2018 by alexus

[Shadowmage]

Heads up -- there will be a TexturesUnlimited update for KSP 1.5 as soon as I get an updated PartTools and can look at the new stock shaders to ensure compatibility.  If this does not seem forthcoming, I will put out a quick 'recompile' update while I await the updated part-tools.

There are no 'breaking' changes planned for this update, but there are new features I would like to get developed before any official releases.  So you may continue to work on/use the existing features and patches without fear of them breaking.

 

[Shadowmage]

Hmmm... so I'm stuck at work for a few hours yet, so can only do abstract research on the subject.  Anyone have any insight on the new stock shader variant, and how precisely it functions?

This blurb from the release notes has me intrigued --

Some parts now take advantage of the new "Bumped Specular (Mapped)"

Which leads me to believe that it is using an environment map of some sort (meaning a cube-map).  Beyond that, I can only speculate -- but I'm hoping that I can leverage some of the assets and functions they are using to try and get a more unified look between TU and stock parts.  Really won't know until I can look at the shader source-code... which will need a PartTools update...

[Electrocutor]

From the stay-put-nik, it appears there is a real reflection being used (at least visible in the VAB). It's tough to tell what all it renders though as it's not shiny enough to be a mirror. I'm a little surprised that they didn't use this new shader on the windows. If it is a true reflection, then you could just tap into that cubemap instead of rendering your own.

[Electrocutor]

It would appear that 'Bumped Specular (Mapped)' is a PBR shader.

@PART[landerCabinSmall] {
	MODULE
	{
		name = ModulePartVariants
		baseVariant = White
		VARIANT
		{
			name = White
			displayName = #autoLOC_8007116
			themeName = White
			primaryColor = #ffffff
			TEXTURE
			{
				shader=KSP/Bumped Specular (Mapped)
				_SpecMap= Squad/Parts/Command/mk1LanderCan/ksp_s_landerCan_psd_illum
				_Shininess= 0.95
			}
		}
	}
}

The metallicy seems to be set via the _SpecMap, and the smoothness is fixed per material using _Shininess. I've not figured out the exacts yet though.

This shader seems to not support lights.

 

It defaults to 100% metal if you do not specify a _SpecMap.

@PART[mk2Cockpit_Standard] {
	MODULE
	{
		name = ModulePartVariants
		baseVariant = White
		VARIANT
		{
			name = White
			displayName = #autoLOC_8007116
			themeName = White
			primaryColor = #ffffff
			TEXTURE
			{
				shader=KSP/Bumped Specular (Mapped)
				_Shininess= 0.95
			}
		}
	}
}

 

Edited October 15, 2018 by Electrocutor

[Shadowmage]

6 minutes ago, Electrocutor said:

It would appear that 'Bumped Specular (Mapped)' is a PBR shader.

(you probably already know this)  There is a lot more to 'PBR' shaders than supporting environment maps/reflections -- it is the actual lighting calculations and math that make it PBR.  What they have would more properly be termed 'legacy with environment-mapped reflection' shader (i.e. it still uses Blinn-Phong algorithm for lighting -- https://en.wikipedia.org/wiki/Blinn–Phong_shading_model)  (all just guesses at this point, but I doubt they implemented the full BDRF required for 'PBR' shaders -- https://en.wikipedia.org/wiki/Bidirectional_reflectance_distribution_function ).

Still really interested in seeing how they've accomplished this.  It looks like the reflection on the window in your screenshot includes some sort of convolution (blurring), which is the missing technical bit that has prevented me from further optimizing TU's rendering setup.  Although from the images posted by forum users, it looks like they have simply done a baked and pre-convolved cubemap for each of the VAB/SPH editors, and then are using a flat color or gradient based reflection-skybox in the flight scene.  I'm hoping that this is not the case, and that they have done proper run-time rendered cubemap reflections, but I'm not seeing any proof of it so far.

I suppose I'll have to d/l and boot up 1.5 this evening and check it out for myself....

17 minutes ago, Electrocutor said:

This shader seems to not support lights.

Are you referring to emissives, or that the reflections do not pick up light sources?

[Electrocutor]

40 minutes ago, Shadowmage said:

(you probably already know this)  There is a lot more to 'PBR' shaders than supporting environment maps/reflections -- it is the actual lighting calculations and math that make it PBR.  What they have would more properly be termed 'legacy with environment-mapped reflection' shader (i.e. it still uses Blinn-Phong algorithm for lighting -- https://en.wikipedia.org/wiki/Blinn–Phong_shading_model)  (all just guesses at this point, but I doubt they implemented the full BDRF required for 'PBR' shaders -- https://en.wikipedia.org/wiki/Bidirectional_reflectance_distribution_function ).

Still really interested in seeing how they've accomplished this.  It looks like the reflection on the window in your screenshot includes some sort of convolution (blurring), which is the missing technical bit that has prevented me from further optimizing TU's rendering setup.  Although from the images posted by forum users, it looks like they have simply done a baked and pre-convolved cubemap for each of the VAB/SPH editors, and then are using a flat color or gradient based reflection-skybox in the flight scene.  I'm hoping that this is not the case, and that they have done proper run-time rendered cubemap reflections, but I'm not seeing any proof of it so far.

I suppose I'll have to d/l and boot up 1.5 this evening and check it out for myself....

There is a bug: the editor/VAB has a reflection probe or cubemap, but during runtime, anything that is 100% metal is solid black, anything that uses the new shader is darker (the % that is supposed to be reflection is black blended).

Edited October 15, 2018 by Electrocutor

[blowfish]

Welp, when the 1.5 PartTools come out we'll be able to figure out exactly what that shader is doing... 

[Manwith Noname]

I'm inclined to say there's "real time reflection" going on. I stuck a sputnik and a QBE in orbit and you can see the faces updating and the "reflection" matches the colour of the planet below.

[Shadowmage]

Well, whatever they are doing for the cube-maps, they haven't solved the DX9 cubemap convolution issues -- check out the seams in this reflection:

Edit:  Added better img showing the seams...

Still doesn't tell me if they are pre-baked environment maps, or dynamically rendered and updated with reflection probes.

Debating a bit about how much digging into the new shader I want to do prior to PartTools being released.  Certainly I could poke around a bit and find out more than I know now, but if/when I can see the .shader file, that will tell me everything.

 

Ohh, and can confirm the lack of reflections in flight scene -- not sure how they could have missed something as large as that...

 

5 minutes ago, Manwith Noname said:

I'm inclined to say there's "real time reflection" going on. I stuck a sputnik and a QBE in orbit and you can see the faces updating and the "reflection" matches the colour of the planet below.

Would love to see some screenshots of this, as I'm unable to see any reflections in the flight scene at all on any of the updated parts.

Wonder if they have some bugs in their cubemap generation setup, or how they are 'handling' the creation of the reflections.

Edited October 15, 2018 by Shadowmage

[Manwith Noname]

[Shadowmage]

@Manwith Noname

Interesting, must just be broken in my game then    (un-modded steam installation that I use for grabbing game updates).

Hmm... I bet there is a setting somewhere that controls it

 

@Electrocutor

You were correct in that they are using a standard shader variant.  Customized StandardSpecular to be precise.  Investigating the source of the reflection maps ATM.  Almost certain there will be conflicts if we are both using reflection probes...

[Electrocutor]

5 minutes ago, Shadowmage said:

@Electrocutor

You were correct in that they are using a standard shader variant.  Customized StandardSpecular to be precise.  Investigating the source of the reflection maps ATM.  Almost certain there will be conflicts if we are both using reflection probes...

Are they actually including the Standard shader includes and stuff? If so, and also selected the checkbox to include all variants, it should support other unused Unity features, like light sources from emission maps and what not.

[Shadowmage]

16 minutes ago, Electrocutor said:

Are they actually including the Standard shader includes and stuff?

No, not really.  Just using the Standard shader BRDF to enable the reflections  ( #pragma surface surf StandardSpecular ).  The .shader file is available in the updated Part-Tools if you wanted to peruse it.

 

TU loads just fine in KSP 1.5.  There is definitely some conflict/overlap in functionality though, as the TU cubemaps (from the reflection probes) override whatever stock is using, and give some ugly mess (far worse than the stock setup), at least under DX9.

My first course of investigation will be to compile a version of TU that simply loads textures/materials, but doesn't touch the reflection system, and see if I can get TU materials to pick up the stock reflection stuff.  Then... I'm going to investigate the source of these reflections (Reflection Probes, pre-rendered skybox textures, ??).

I'm in no huge rush on this.  Obviously have a bit of stuff to sort out beyond the basic updating that I had imagined...

 

 

[Shadowmage]

Whatever the reflection source is, it works fine with the TU shaders, at a technical level.  If I disable all of TU's reflection-probe functionality, the shaders/materials seem to pick up the stock environment map.  I can also confirm that the seams in the cubemap are only present on DX9 -- so they are either doing dynamic run-time map generation, or the problem with seams is inherent in the dx9 texture sampling and not baked into the prebuilt maps.

However, the stock system is not capturing the environment as it should for use as a proper environment map -- notably it lacks lighting effects, so the reflection source seems extremely dark compared to what it should be (see the TU shot below for comparison).

 

Still can't get it working in the flight scene though...

I'm hoping someone is filing bugs about the above, because it doesn't seem to work regardless of what graphics API I use or what settings I try.

 

TU reflection setup still works in editor:

and flight scene

 

However -- when the stock parts with the new stock shader use TU's reflection probes, they are not nearly as reflective (note it is still showing blue from the sky outside, but it is very dull)--

There are major differences between how Unity's reflection probe is doing the convolution/blurring, and whatever stock is using.  I'm still trying to figure out exactly where the stock reflections are coming from and how they are generated, but I have a feeling that there are some 'quick hacks' at play somewhere to get the stock stuff working for DX9.

Doesn't look too bad though, even if it is not as shiny as it should be.

Might be something that I could patch to fix, even if it is just converting the stock parts to use a TU shader

 

Still investigating.  At this point it appears that there will be some 'incompatibilities' between TU and the new stock shader, but there might be some parts of the system that I can use to clean up some of the DX9 nastiness... if I can figure out where they are and how to make use of them.  I likely won't be using the stock environment mapping as-is though, as it is not suitable for standard PBR texturing due to the difference in blurring/convolution  (likely they are doing simple blurring/mip-map tricks rather than convolution).

 

[Shadowmage]

More investigations, more questions..... and a few answers.

Stock is not using reflection probes, but rather they are using the ability to specify a 'default reflection' environment map in the Unity render settings, and it is this environment map that the stock shader is picking up (and is what TU will pick up if reflection probes are not enabled).

As near as I can tell this is only used in the VAB and SPH, and there are no reflections in the flight-scene (no cubemap is assigned to RenderSettings.customReflection, no reflection probes, and no skybox).

@Manwith Noname Are you sure you didn't have TU installed in your screenshot above?

 

As soon as I can get at these cubemaps and dump them to disk, I'll take a look at the MIP levels and see how the blurring/convolution is done.

 

On an interesting note, I was able to find out where the clouds in the VAB/SPH are rendered at/from....  (just a cube-map used as a background...)

[Shadowmage]

Pretty sure the 'fix' for the stock-shader compatibility will be to patch the _Smoothness values to something more reasonable given the improved env. maps used by TU.  This is with a value of 0.75  (default appears to be ~0.5 on this part) --

Stock shader, just using TU to patch the material properly (and handle reflection probes...).

 

Another option might be to create a TU/Specular shader that uses the specular workflow.  The stock textures appear to be authored with smoothness data in the specular-map alpha channel, so could leverage the existing information there to set the smoothness appropriately. 

Never did manage to get a dump of the environment map data used in the stock reflections.  I'm convinced at this point though that they are only active in the editors, and not using any sort of convolution (just the 'blurring' caused by mip-map sampling).  Will need to await confirmation from the devs that the lack of flight scene reflections are either a bug, or intentional.

[Manwith Noname]

9 hours ago, Shadowmage said:

@Manwith Noname Are you sure you didn't have TU installed in your screenshot above?

Yeah, I sort of figured this out after posting as I went to bed. "Why am I getting reflections when noone else is?" kind of thought, then guessed that even though it's not using TU shaders, it's picking up the reflection mapping somehow. I haven't put the whole project in to 1.5 yet but I had started planning for new parts...well...one part...

Spoiler

 

 

Edit: Oh, be wary of this...

7 hours ago, Shadowmage said:

The stock textures appear to be authored with smoothness data in the specular-map alpha channel, so could leverage the existing information there to set the smoothness appropriately. 

...go look at the HECS textures I think it was. They don't have information in the alpha...well they do but it's "1" everywhere.

Another edit: It's the rovemate textures I was thinking of. HECS doesn't currently have separate textures.

Edited October 16, 2018 by Manwith Noname

[Electrocutor]

6 hours ago, Shadowmage said:

The stock textures appear to be authored with smoothness data in the specular-map alpha channel, so could leverage the existing information there to set the smoothness appropriately. 

I don't want to reinstall unity right now to look at the shader code. What is the math for smoothness? _SpecMap.A * _Glossiness or something?

Summary so Far:

• Stock reflection cubemap missing in Flight mode
• Stock PBR shader does not support emission (no pod/cabin lights unless on separate mesh)
• Reflection cubemap still renders top surface inverted in dx9
• Reflection cubemap seems to use a quick-draw method that is less accurate than TU

Did I miss anything?

 

One side effect here is that people can now make models using this shader, then TU could just automatically replace that shader and pull from the same channels to add more functionality.

Edited October 16, 2018 by Electrocutor

[Shadowmage]

2 hours ago, Electrocutor said:

One side effect here is that people can now make models using this shader,

Kind of...

Due to the terrible stock reflection setup, if they author textures to look good with the stock shader/reflections... they'll end up with results like the stayputnik -- far less reflection when using proper reflection probes and convolved reflection maps.

2 hours ago, Electrocutor said:

What is the math for smoothness? _SpecMap.A * _Glossiness or something?

Just using the _Shininess property directly for smoothness -- no texture sampling or any other inputs.  One of the most powerful features of PBR... and they leave it to a material-wide property....

2 hours ago, Electrocutor said:

Did I miss anything?

That about sums it up...

Its like whoever was writing these shaders didn't know what he was doing.  Or was given an absurd set of 'requirements' from external sources.  Either way, someone was out of touch with the functions/features of these shaders...

4 hours ago, Manwith Noname said:

Edit: Oh, be wary of this...

Good point.. I really only looked at the StayPutnik.  In that case there was an alpha channel in the specular texture, with different values than were in the RGB channels.  Ohh... and the new shader did not use the value for whatever reason -- they sample the specular.rgb and use that for specular color, but they ignore the specular map alpha channel entirely.