[WIP][1.9.x-1.12.x] Scatterer-atmospheric scattering (0.0838 - 14/08/2022) Scattering improvements, in-game atmo generation and multi-sun support

[blackrack]

Giant post ahead

So I've been working on a few things. The blending used in scatterer was rather primitive and relied on using the blend operations built-in to blend the results into the background without having to read back the background, this limits what we can do in term of blending to additive and soft-additive and alpha blending. However since the transition to depth buffer mode, and switching away from MSAA we can now use any blending mode we want. Here I manage to get HDR-like blending without applying HDR tonemapping on the background colors (the game's shaders aren't really built for HDR)

This improved the colors a bit.

I then revisited the config tool, I just wanted to modify it to not rely on compute shaders in the beginning. The config tool contains all the Proland code for generating the atmosphere and baking it into look-up textures, and I haven't touched it pretty much since the first version of Scatterer. As I got into it, I ended up doing a deep dive into the atmosphere code to try and fix some of the issues/artifacts it had.

The original code had some imprecision issues at boundaries, often resulting in these concentric circles that get reported often as bugs:

This was easy to fix, but a relatively more complex issue is this discontinuity and artifacts that occur around the horizon line:

This is a well-known limitation of the precomputed scattering technique that Scatterer uses, to my surprise I found that the new version of Proland doesn't fix this

Thankfully, there are smart people out there who propose a better parametrization of the precomputed textures to concentrate detail on the horizon. After some time implementing it and fixing various issues, it works quite well. Notice the clean horizon line in these screenshots:

After that I still found several imprecision issues that result in these bands over the horizon whenever the atmosphere starts getting thick, making foggy atmospheres mostly unusable:

I was able to track these down as coming from the precomputed extinction having some issues at boundaries. This is also fixes the small band that appears over the horizon and for which I had the "viewdirOffset" workaround. After fixing it I had some fun testing some foggy atmospheres (better click the gfycat link as the preview butchers the quality):

https://gfycat.com/lastlavishasiantrumpetfish

I also went ahead and simplified the atmosphere generation process, removing variables like Rt and RL, calculating the atmosphere height directly from the parameters you enter, and generating the atmo directly in-game. I'm sure you have seen this video:

Now you no longer need to include .half files, they are simply generated on the fly when needed and cached on disk for reuse.

Then I decided to try a different tonemap operator, the built-in proland/bruneton tonemapper being a bit washed out. I settled on using the well-known uncharted 2 tonemapping for now, here's how it looks (left: old, right: new)

Note that this tonemapping doesn't affect the colors of the background/game, only the colors of the scattering and how it is blended to the background.

After all this I still found that Kerbin in map view looked a bit bland, although it improved.

Comparing it to RSS it seemed to me that earth textures had a lot more pop and more contrast:

After comparing the textures, I decided that Kerbin needed a bit more contrast, and darker oceans, so I added a small configurable shader that modifies the texture in-game, here's how kerbin looks now:

I was happy to get a more clear separation between land and sea, and a bit less haze when looking straight down.

This should only be needed for stock planets, if you're creating your own planets you're free to create your scaled textures however you want so shouldn't need this, plus it duplicates the texture wasting VRAM.

Also, TAA appears to be fixed in 1.12 so will now be enabled by default, looks better in movement than SMAA.

Some screenshots:

My favourite atmosphere views are probably on Laythe

Comparing the same atmo to 5x stronger mie scattering:

Different mie asymmetry settings to look more fog-like or more dust-like feel:

Thanks everyone for reading and thanks for the support.

[Delay]

1 hour ago, blackrack said:

So I've been working on a few things.

A few, huh?

Scatterer just keeps getting prettier! I like the new tonemap and Kerbin colors. Looks much more realistic, and I haven't even begun to talk about the atmospheres! This is the most beautiful the game has ever been. Hopefully my computer can run this upcoming version!

[Al2Me6]

1 hour ago, blackrack said:

Giant post ahead

So I've been working on a few things. The blending used in scatterer was rather primitive and relied on using the blend operations built-in to blend the results into the background without having to read back the background, this limits what we can do in term of blending to additive and soft-additive and alpha blending. However since the transition to depth buffer mode, and switching away from MSAA we can now use any blending mode we want. Here I manage to get HDR-like blending without applying HDR tonemapping on the background colors (the game's shaders aren't really built for HDR)

This improved the colors a bit.

I then revisited the config tool, I just wanted to modify it to not rely on compute shaders in the beginning. The config tool contains all the Proland code for generating the atmosphere and baking it into look-up textures, and I haven't touched it pretty much since the first version of Scatterer. As I got into it, I ended up doing a deep dive into the atmosphere code to try and fix some of the issues/artifacts it had.

The original code had some imprecision issues at boundaries, often resulting in these concentric circles that get reported often as bugs:

This was easy to fix, but a relatively more complex issue is this discontinuity and artifacts that occur around the horizon line:

This is a well-known limitation of the precomputed scattering technique that Scatterer uses, to my surprise I found that the new version of Proland doesn't fix this

Thankfully, there are smart people out there who propose a better parametrization of the precomputed textures to concentrate detail on the horizon. After some time implementing it and fixing various issues, it works quite well. Notice the clean horizon line in these screenshots:

After that I still found several imprecision issues that result in these bands over the horizon whenever the atmosphere starts getting thick, making foggy atmospheres mostly unusable:

I was able to track these down as coming from the precomputed extinction having some issues at boundaries. This is also fixes the small band that appears over the horizon and for which I had the "viewdirOffset" workaround. After fixing it I had some fun testing some foggy atmospheres (better click the gfycat link as the preview butchers the quality):

https://gfycat.com/lastlavishasiantrumpetfish

I also went ahead and simplified the atmosphere generation process, removing variables like Rt and RL, calculating the atmosphere height directly from the parameters you enter, and generating the atmo directly in-game. I'm sure you have seen this video:

Now you no longer need to include .half files, they are simply generated on the fly when needed and cached on disk for reuse.

Then I decided to try a different tonemap operator, the built-in proland/bruneton tonemapper being a bit washed out. I settled on using the well-known uncharted 2 tonemapping for now, here's how it looks (left: old, right: new)

Note that this tonemapping doesn't affect the colors of the background/game, only the colors of the scattering and how it is blended to the background.

After all this I still found that Kerbin in map view looked a bit bland, although it improved.

Comparing it to RSS it seemed to me that earth textures had a lot more pop and more contrast:

After comparing the textures, I decided that Kerbin needed a bit more contrast, and darker oceans, so I added a small configurable shader that modifies the texture in-game, here's how kerbin looks now:

I was happy to get a more clear separation between land and sea, and a bit less haze when looking straight down.

This should only be needed for stock planets, if you're creating your own planets you're free to create your scaled textures however you want so shouldn't need this, plus it duplicates the texture wasting VRAM.

Also, TAA appears to be fixed in 1.12 so will now be enabled by default, looks better in movement than SMAA.

Some screenshots:

My favourite atmosphere views are probably on Laythe

Comparing the same atmo to 5x stronger mie scattering:

Different mie asymmetry settings to look more fog-like or more dust-like feel:

Thanks everyone for reading and thanks for the support.

This looks absolutely amazing. Kudos.

[blackrack]

10 hours ago, Delay said:

Hopefully my computer can run this upcoming version!

There won't be a difference in performance, only perhaps if enabling TAA but you can always use SMAA like now.

[Delay]

1 hour ago, blackrack said:

There won't be a difference in performance,

That is downright insane. What kind of black magic is this?

[MashAndBangers]

20 hours ago, blackrack said:

So I've been working on a few things.

So the question now is for custom planet packs (KSRSS, JNSQ, etc.), will the Scatterer Re-Scattered still work with the old configs, or will the mod authors have to make new configs?

[RyanRising]

41 minutes ago, MashAndBangers said:

Scatterer Re-Scattered

LGG isn't adopting this too, is he? It's far too soon!

 

Edited November 4, 2021 by RyanRising

[Kerminator K-100]

My man over here making KSP2

Why hasn't Intercept hired you yet?

This seriously looks amazing, I can't wait for my new GPU to get here so I can try this out

[blackrack]

2 hours ago, MashAndBangers said:

So the question now is for custom planet packs (KSRSS, JNSQ, etc.), will the Scatterer Re-Scattered still work with the old configs, or will the mod authors have to make new configs?

They will work, but old versions of the config tool left it up to the user to pick the atmosphere height, instead of matching it to the density scale heights entered. So depending on how the old configs were made, they may work fine or the scale may be completely different.

On average I think configs will need to be readjusted. The good news is, readjusting is now much easier.

 

[wisdomsavingthrow]

All this is genuinely making me really look forward to retooling my custom Scatterer settings. Super excited for when it releases!

[y2kcobrar]

Can anyone tell me how to increase the distance at which Scatterer begins to take effect? I'm playing with GPP at 10x scale and about half way to the nearest moon from Gael (Kerbin)  the game will hang for a moment and Scatterer stops showing. The inverse happens when returning from Gaels moons. The exact distance escapes me right now but I did try and find a correlation in the configs and didn't see any. TIA~!

[blackrack]

2 hours ago, y2kcobrar said:

Can anyone tell me how to increase the distance at which Scatterer begins to take effect? I'm playing with GPP at 10x scale and about half way to the nearest moon from Gael (Kerbin)  the game will hang for a moment and Scatterer stops showing. The inverse happens when returning from Gaels moons. The exact distance escapes me right now but I did try and find a correlation in the configs and didn't see any. TIA~!

LoadDistance and unloadDistance in planetsList file/patch: https://github.com/LGhassen/Scatterer/wiki/PlanetsConfig#planets-list

Just multiply it by 10 I guess

Edited November 5, 2021 by blackrack

[y2kcobrar]

3 hours ago, blackrack said:

LoadDistance and unloadDistance in planetsList file/patch: https://github.com/LGhassen/Scatterer/wiki/PlanetsConfig#planets-list

Just multiply it by 10 I guess

I tried adding a zero to each and no change.

[space_powder]

Is there a way to enable/disable the long distance shadow stuff while in-game? I sometimes turn them off during launches with a hefty amount of parts, but I do like the crisp shadows once I'm in space and things have settled down in terms of performance.

[Poodmund]

Blackrack, this is an insane improvement. 

[Maple Kerman]

Very pretty!

[blackrack]

New version is up

[Poodmund]

The multiple cloud interaction fixes are so very welcome.  

[Delay]

Oh my GOD IS THIS GORGEOUS!

I just have a quick question, though: Is the fog supposed to be this bright in the background?

On the mountains, it doesn't look that bad, but grass is a bit bright.

Other than that, this mod makes KSP look better than ever. Here are some more shots I took on that test flight:

Very nice colors on the atmosphere!

And this... this is just an indescribable feast for the eyes!

Fantastic work!

[blackrack]

4 minutes ago, Delay said:

Oh my GOD IS THIS GORGEOUS!

I just have a quick question, though: Is the fog supposed to be this bright in the background?

On the mountains, it doesn't look that bad, but grass is a bit bright.

Other than that, this mod makes KSP look better than ever. Here are some more shots I took on that test flight:

Very nice colors on the atmosphere!

And this... this is just an indescribable feast for the eyes!

Fantastic work!

Thanks. It's indeed not supposed to be this bright. Are you using different scatterer configs than the included ones? Or maybe a TUFX profile tweaked for the old atmo?

Edited November 10, 2021 by blackrack

[Delay]

8 minutes ago, blackrack said:

Are you using different scatterer configs than the included ones? Or maybe a TUFX profile tweaked for the old atmo?

I have an old EVE configs at least, dating back to the original releases (the BoulderCo thing). But that's not doing anything to the atmosphere as far as I can tell.

If it helps, here's the atmosphere.cfg for Kerbin. It should be the unaltered one Scatterer comes with:
 

Scatterer_atmosphere
{
	Atmo
	{
		name = Kerbin
		atmosphereStartRadiusScale = 1
		HR = 3.74399996
		HM = 0.561600029
		m_betaR = 0.0120000001,0.0279309992,0.0684830025
		BETA_MSca = 0.00854699966,0.00854699966,0.00854699966
		m_mieG = 0.850000024
		averageGroundReflectance = 0.100000001
		multipleScattering = True
		godrayStrength = 0.699999988
		flattenScaledSpaceMesh = 0.600000024
		rimBlend = 20
		rimpower = 600
		specR = 80
		specG = 72
		specB = 65
		shininess = 75
		cloudColorMultiplier = 3
		cloudScatteringMultiplier = 0.200000003
		cloudSkyIrradianceMultiplier = 0.0500000007
		volumetricsColorMultiplier = 1
		EVEIntegration_preserveCloudColors = False
		adjustScaledTexture = True
		scaledLandBrightnessAdjust = 1.32000005
		scaledLandContrastAdjust = 1.79999995
		scaledLandSaturationAdjust = 1.29999995
		scaledOceanBrightnessAdjust = 0.839999974
		scaledOceanContrastAdjust = 1
		scaledOceanSaturationAdjust = 1
		configPoints
		{
			Item
			{
				altitude = 0
				skyExposure = 0.200000003
				skyAlpha = 1
				skyExtinctionTint = 1
				scatteringExposure = 0.200000003
				extinctionThickness = 1
				postProcessAlpha = 1
				postProcessDepth = 0
				extinctionTint = 1
			}
		}
	}
}

Also, to be fair, the images were taken at high noon.

Edited November 10, 2021 by Delay

[blackrack]

11 minutes ago, Delay said:

I have an old EVE configs at least, dating back to the original releases (the BoulderCo thing). But that's not doing anything to the atmosphere as far as I can tell.

If it helps, here's the atmosphere.cfg for Kerbin. It should be the unaltered one Scatterer comes with:
 

Scatterer_atmosphere
{
	Atmo
	{
		name = Kerbin
		atmosphereStartRadiusScale = 1
		HR = 3.74399996
		HM = 0.561600029
		m_betaR = 0.0120000001,0.0279309992,0.0684830025
		BETA_MSca = 0.00854699966,0.00854699966,0.00854699966
		m_mieG = 0.850000024
		averageGroundReflectance = 0.100000001
		multipleScattering = True
		godrayStrength = 0.699999988
		flattenScaledSpaceMesh = 0.600000024
		rimBlend = 20
		rimpower = 600
		specR = 80
		specG = 72
		specB = 65
		shininess = 75
		cloudColorMultiplier = 3
		cloudScatteringMultiplier = 0.200000003
		cloudSkyIrradianceMultiplier = 0.0500000007
		volumetricsColorMultiplier = 1
		EVEIntegration_preserveCloudColors = False
		adjustScaledTexture = True
		scaledLandBrightnessAdjust = 1.32000005
		scaledLandContrastAdjust = 1.79999995
		scaledLandSaturationAdjust = 1.29999995
		scaledOceanBrightnessAdjust = 0.839999974
		scaledOceanContrastAdjust = 1
		scaledOceanSaturationAdjust = 1
		configPoints
		{
			Item
			{
				altitude = 0
				skyExposure = 0.200000003
				skyAlpha = 1
				skyExtinctionTint = 1
				scatteringExposure = 0.200000003
				extinctionThickness = 1
				postProcessAlpha = 1
				postProcessDepth = 0
				extinctionTint = 1
			}
		}
	}
}

Also, to be fair, the images were taken at high noon.

I did notice that at noon the shading on the terrain was bright but I attributed it to the stock terrain shaders (lol always blaming the game). I don't remember it being this bright but I will check again when I can. I admit I do most my testing at the more interesting times of day.

[Gordon Dry]

First quick test with latest JNSQ, Tracking Station tabbing through all celestials:
Everything fine but the gas giants + Eve, they render black with a thin halo.

btw if I understood correctly I just could remove all .half files inside JNSQ and be good?

Edited November 10, 2021 by Gordon Dry
Eve also does

[SiCaRiO31]

The water looks a bit...flourecent?

 

[blackrack]

1 hour ago, Delay said:

So I just looked at this again from my PC and it definitely doesn't look right. The area around KSC in particular just looks weird. I fired up my game again and to my surprise I found the same thing, which doesn't happen on my previous unreleased version. Investigating it now