[WIP][1.9.x-1.12.x] Scatterer-atmospheric scattering (0.0838 - 14/08/2022) Scattering improvements, in-game atmo generation and multi-sun support

[Pointblank66]

1 hour ago, Nightside said:

Why would you release it before it was finished? 

There are mod devs that release their mods bit by bit or in parts for redundency reasons, Just so the users can fall back on something.

5 hours ago, blackrack said:

Soon (TM)

Aight, I'll sit tight for the release

[EdwardB3020]

My frame-rate decreases when I'm on EVA or when a goo canister opens  

[Nightside]

Does anyone know if it is possible to get a lens flare effect on rocket engines?

[soulsource]

On 11/17/2018 at 10:30 AM, blackrack said:

You probably have a nullref spam. Press alt+F12 and see what is going on, if it's the case make a proper bug report, check in the OP for a bug report example.

I'm seeing the same issue. No nullrefs, but a huge FPS impact of scatterer's post processing. Disabling post processing in the scatterer menu immediately yields good FPS again... The issue does not exist with 0.0331b.

Edit: It's certainly caused by the shaders, as running 0.0331b with the shaders file from 0.0336 also yields significantly lowered fps on Linux, while running 0.0336 with the shaders file from 0.0331b does not yield the fps drop. (Might also be hardware specific - I have an AMD card with radeonsi drivers).

Edit 2: I'll try to get Unity set up on my machine. If I manage to get it working (last time I tried it was a bit a PITA on Linux), I'll try to do a git bisect this weekend, to find the offending commit.

Edited December 14, 2018 by soulsource

[soulsource]

Hmmm, it seems performance on Linux went down the drain with

Quote

commit ddbaecc1a751db0393701eab1699e06c8f988556
Author: [moderator snip]
Date:   Mon May 21 18:24:54 2018 +0200

    add dithering to post-processing + ocean surface

The culprit seems to be the added dithering in atmospherePostProcessing32bitViewSpaceDepth.shader. By undoing

--- a/scatterer/Shaders/scattererShaders/Assets/shaders/atmospherePostProcessing32bitViewSpaceDepth.shader
+++ b/scatterer/Shaders/scattererShaders/Assets/shaders/atmospherePostProcessing32bitViewSpaceDepth.shader
@@ -306,7 +306,7 @@ Pass {
 //                             inscatter*=eclipseShadow;
 //#endif
 
-                return float4(hdr(inscatter)*_global_alpha*returnPixel, 1);                                            
+                return float4(dither(hdr(inscatter)*_global_alpha, screenPos)*returnPixel, 1);

I got performance back up to acceptable levels, also with shaders from 0.0336.

I'll investigate this further, because the dither function itself looks reasonably benign, so there has to be something crazy going on in either Unity's HLSL->GLSL converter, or in the AMD shader compiler...
(I originally thought that we get register spilling, but meanwhile I'm rather certain that we can rule that out...)

Edited December 15, 2018 by Vanamonde

[soulsource]

I might get crazy: By just removing the dither function completely (meaning: just making it return the input value) performance drastically increased on Linux. Like, from 35 to 45 fps on my machine...

Edit: It has something to do with the size of the bayerPattern.

Edit 2: That gives me an idea. The matrix can be replaced with a formula (https://en.wikipedia.org/wiki/Ordered_dithering). I'll try that tomorrow, and if it indeed improves performance, there will soon be a pull request.

Edited December 14, 2018 by soulsource

[blackrack]

12 hours ago, soulsource said:

I might get crazy: By just removing the dither function completely (meaning: just making it return the input value) performance drastically increased on Linux. Like, from 35 to 45 fps on my machine...

Edit: It has something to do with the size of the bayerPattern.

Edit 2: That gives me an idea. The matrix can be replaced with a formula (https://en.wikipedia.org/wiki/Ordered_dithering). I'll try that tomorrow, and if it indeed improves performance, there will soon be a pull request.

Let me know how it goes, in my experience the shader compiler is full of bugs when it comes to compiling to opengl. One thing you can do is try to add the #pragma glsl directive to the shader, although I'm not sure if it still does something since unity 5

[soulsource]

What I've found up to now is that the Bayer pattern array seems to be placed in private memory, instead of being hardcoded or stored in registers...

Quote

*** SHADER STATS ***
SGPRS: 72
VGPRS: 48
Spilled SGPRs: 0
Spilled VGPRs: 0
Private memory VGPRs: 64
Code Size: 4424 bytes
LDS: 0 blocks
Scratch: 17408 bytes per wave
Max Waves: 5
********************

I think the following snippet compares the same shader, once with the Bayer pattern as an array (old file), and once with it computed with the formula from Wikipedia (new file): https://pastebin.com/GAuSLWPx (think, because I have no experience when it comes to reading assembly).

Here's a pull request for your consideration: Edit 3: Please disregard, it's not building for Windows... https://github.com/LGhassen/Scatterer/pull/52

As said, this change improves performance on Linux with AMD hardware a lot, but as it does more computations (instead of using precomputed values), it might decrease performance on other platforms and should therefore be tested before merging (I would actually even consider conditional compilation, in the worst case)...

Edit: The merge request requires shader model 4. I don't know if scatterer supports d3d9, but if it does, please tell me, then I'll rewrite it so it does not use any bitwise operators.

Edit 2: I just saw that when building for Windows shaders for SM 3 are being built. I'll fix that.

Edit 4: See post below.

Edited December 15, 2018 by soulsource

[soulsource]

So, sorry for the confusion. The pull request is open again, and should now work on shader model 3 as well. https://github.com/LGhassen/Scatterer/pull/52

[blackrack]

13 hours ago, soulsource said:

So, sorry for the confusion. The pull request is open again, and should now work on shader model 3 as well. https://github.com/LGhassen/Scatterer/pull/52

It's integrated, thanks!

[blackrack]

On 12/8/2018 at 11:53 PM, EdwardB3020 said:

My frame-rate decreases when I'm on EVA or when a goo canister opens  

Please make a proper bug report (see section in the OP)

On 12/14/2018 at 7:39 PM, Nightside said:

Does anyone know if it is possible to get a lens flare effect on rocket engines?

Not possible now

[soulsource]

As a small sidenote: I could not stop playing with this Linux issue, and it seems that the HLSL->GLSL converter of Unity is to blame. Maybe we should file a bug report there...

I've looked at the generated GLSL code using Mesa's shader replacement feature, and the generated code is everything but compiler friendly. Here's an example (one of the shaders that uses dither):  https://pastebin.com/KUpVnT2R
Unity converts the local compile time constant array into an uninitialized global array, that then is copy-initialized in the shader's main function . Obviously the llvm AMD shader compiler is seeing this write access happening, and decides to make it a real, in-memory array instead of compiling it in as constants. I've made some trivial edits to the generated GLSL code, and that also brought performance up to speed (tested with Mesa's shader replacement feature again - I don't know how to make Unity do the same thing ): https://pastebin.com/yCbXqHa1. The only difference being that the array is const again, and initialized right at declaration instead of copying the single values into it in the main function...

So at least we now know what was going on...

[blackrack]

15 minutes ago, soulsource said:

As a small sidenote: I could not stop playing with this Linux issue, and it seems that the HLSL->GLSL converter of Unity is to blame. Maybe we should file a bug report there...

I've looked at the generated GLSL code using Mesa's shader replacement feature, and the generated code is everything but compiler friendly. Here's an example (one of the shaders that uses dither):  https://pastebin.com/KUpVnT2R
Unity converts the local compile time constant array into an uninitialized global array, that then is copy-initialized in the shader's main function . Obviously the llvm AMD shader compiler is seeing this write access happening, and decides to make it a real, in-memory array instead of compiling it in as constants. I've made some trivial edits to the generated GLSL code, and that also brought performance up to speed (tested with Mesa's shader replacement feature again - I don't know how to make Unity do the same thing ): https://pastebin.com/yCbXqHa1. The only difference being that the array is const again, and initialized right at declaration instead of copying the single values into it in the main function... 

So at least we now know what was going on...

Yeah, the HLSL -> GLSL converter is wonky in many ways, I often had to work around unexplained bugs.

If you would like to proceed with the bug report to unity, make sure to test it against the most recent unity version as the one used in KSP is a bit dated.

[soulsource]

On 12/16/2018 at 2:32 PM, blackrack said:

Yeah, the HLSL -> GLSL converter is wonky in many ways, I often had to work around unexplained bugs.

If you would like to proceed with the bug report to unity, make sure to test it against the most recent unity version as the one used in KSP is a bit dated.

Okay, so I'm going to test if the issue persists with the latest Unity version, and if yes, file a bug report.

Edit: I can confirm that the same issue persists with the latest Unity build for Linux (2018.2.something). Bug report is in the process of being filed.

Edit 2: Bug report has been sent using Unity's Bug Reporting tool, including a minimal project with a really minimal shader for easy reproduction. Let's see if there's an answer.

Edit 3: No news from Unity yet. I should have filed the bug report at work, where we have Pro licenses, instead of home using a free license..

Edited December 23, 2018 by soulsource

[Cataclism]

----(please ignore this post)__________  

Edited December 18, 2018 by Cataclism

[EdwardB3020]

On 12/16/2018 at 5:52 AM, blackrack said:

Please make a proper bug report (see section in the OP)

Not possible now

I think it was a memory glitch with the in game version 1.5.1. also could you update the mod to 1.6 please so I can use this mod with out having glitches from 1.5.1 in 1.6. thank you. If you watch @ShadowZone's video on 1.6 then you'd probably know what I'm talking about.

Edited December 20, 2018 by EdwardB3020

[Snark]

6 hours ago, EdwardB3020 said:

also could you update the mod to 1.6 please so I can use this mod with out having glitches from 1.5.1 in 1.6.

Dude.  KSP 1.6 was released literally not even eight hours ago.  And it's the holiday season, and modders do in fact have real lives with real things to do with their time other than the long, unpaid volunteer hours that they put into their mods, in order to give us all these shiny toys for free.

So please, be patient.  Rest assured that modders know that 1.6 has released, and if you haven't seen an update yet, it's because it hasn't been convenient (or even possible) for them to do so. The modder will release an update if and when they're inclined to do so, if and when they have time.  Their convenience is relevant; yours isn't.  It'll be ready whenever it's ready.

Personally, I hope they're taking a few weeks off to relax over the holiday season-- goodness knows they've earned it! 

[EdwardB3020]

13 hours ago, Snark said:

Dude.  KSP 1.6 was released literally not even eight hours ago.  And it's the holiday season, and modders do in fact have real lives with real things to do with their time other than the long, unpaid volunteer hours that they put into their mods, in order to give us all these shiny toys for free.

So please, be patient.  Rest assured that modders know that 1.6 has released, and if you haven't seen an update yet, it's because it hasn't been convenient (or even possible) for them to do so. The modder will release an update if and when they're inclined to do so, if and when they have time.  Their convenience is relevant; yours isn't.  It'll be ready whenever it's ready.

Personally, I hope they're taking a few weeks off to relax over the holiday season-- goodness knows they've earned it! 

nvr mind sry. ygacuhbadidhidhiuadhdahchuahuidaiaciuchaidbhcsbadssdacdadvdghvajcdghad crash. 

[blackrack]

Preview of next release features (incoming image wall):

Extinction is now applied to the main light, this means that objects now receive a nicely tinted light at sunset/sunrise, helping with the perception of early morning/dusk

whereas before everything remained bright white at any time:

Comparison of KSC at sunrise

 

Glinted reflection and light matching the light on the clouds (here using textures unlimited for the part shaders, but stock shader will get the light tint as well):

On Eve this looks interesting as well, although the purple ground gets extra dark with green light

You can even see this from the Mun surface if you get the planetary alignment just right:

At the moment eclipses aren't handled at PQS level however so the mun surface stays red instead of going completely black when Kerbin covers the sun, this will be handled in the future.

The same approach to changing the light is now also used underwater, to make the actual underwater surface dimmer:

You can see thsi doesn't affect secondary lights and makes them more important to have underwater.

Moving on, the ocean now also responds to ambient light, meaning it's no longer pitch black at night

 

And I have added shadows on the ocean surface

This should help a lot with judging how close you are to the water surface when landing or flying really close.

EVE cloud shadows aren't handled yet but I plan to support them as well.

Fixed the small break in the postprocessing between the two game cameras:

 

Fixed small aliasing at the edge of scaled space planet surface (both are with AA on, left shows aliasing anyway)

 

Improved atmosphere blending:

Improvements to shadow precision and less blobbiness:

 

The release will as well contain a few more changes and bug fixes that I will detail later.

One of the things I want to work on before releasing as well is to fix the aliasing on the horizon.

Edited December 28, 2018 by blackrack

[Dafni]

Great news @blackrack! thanks for all the work you put into this! I cant wait to try it out.

Edit: this might finally be a real reason to leave 1.3.1 behind

Edited December 28, 2018 by Dafni

[Poodmund]

Thank you for your continued efforts, Blackrack, they are truly appreciated!  

5 hours ago, blackrack said:

One of the things I want to work on before releasing as well is to fix the aliasing on the horizon.

I think Scrawk's https://github.com/Scrawk/Brunetons-Improved-Atmospheric-Scattering specifically fixes this issue: 

Quote

Improved texture coordinate mapping which removes the horizon artifact in the previous version.

 

[Pkmniako]

The hype is real

[Damon]

5 hours ago, blackrack said:

Preview of next release features (incoming image wall):

(many many many awesome previes were listed here)

Fixed the small break in the postprocessing between the two game cameras:

I cannot say how thankful I am for this. I've been promoting this bug since the 1.4 update and now the very own scatterer mod has fixed a stock bug. Amazing! Thank you!

[CobaltWolf]

Awesome!!!

[The Cuttlefish Empire]

Holy crap, this is beautiful. And I thought Scatterer could only be so pretty...  

5 hours ago, Dafni said:

this might finally be a real reason to leave 1.3.1 behind

Same...and my install is just about as unstable as possible, so that's saying something for me