Jump to content

[WIP][1.9.x-1.10.x-1.11.x] Scatterer-atmospheric scattering (v0.0723 - 31/12/2020)


Recommended Posts

1 hour ago, Nightside said:

Why would you release it before it was finished? 

There are mod devs that release their mods bit by bit or in parts for redundency reasons, Just so the users can fall back on something.

5 hours ago, blackrack said:

Soon (TM)

Aight, I'll sit tight for the release

Link to post
Share on other sites
On 11/17/2018 at 10:30 AM, blackrack said:

You probably have a nullref spam. Press alt+F12 and see what is going on, if it's the case make a proper bug report, check in the OP for a bug report example.

I'm seeing the same issue. No nullrefs, but a huge FPS impact of scatterer's post processing. Disabling post processing in the scatterer menu immediately yields good FPS again... The issue does not exist with 0.0331b.

Edit: It's certainly caused by the shaders, as running 0.0331b with the shaders file from 0.0336 also yields significantly lowered fps on Linux, while running 0.0336 with the shaders file from 0.0331b does not yield the fps drop. (Might also be hardware specific - I have an AMD card with radeonsi drivers).

Edit 2: I'll try to get Unity set up on my machine. If I manage to get it working (last time I tried it was a bit a PITA on Linux), I'll try to do a git bisect this weekend, to find the offending commit.

Edited by soulsource
Link to post
Share on other sites

Hmmm, it seems performance on Linux went down the drain with

Quote

commit ddbaecc1a751db0393701eab1699e06c8f988556
Author: [moderator snip]
Date:   Mon May 21 18:24:54 2018 +0200

    add dithering to post-processing + ocean surface

The culprit seems to be the added dithering in atmospherePostProcessing32bitViewSpaceDepth.shader. By undoing

--- a/scatterer/Shaders/scattererShaders/Assets/shaders/atmospherePostProcessing32bitViewSpaceDepth.shader
+++ b/scatterer/Shaders/scattererShaders/Assets/shaders/atmospherePostProcessing32bitViewSpaceDepth.shader
@@ -306,7 +306,7 @@ Pass {
 //                             inscatter*=eclipseShadow;
 //#endif
 
-                return float4(hdr(inscatter)*_global_alpha*returnPixel, 1);                                            
+                return float4(dither(hdr(inscatter)*_global_alpha, screenPos)*returnPixel, 1);

I got performance back up to acceptable levels, also with shaders from 0.0336.

I'll investigate this further, because the dither function itself looks reasonably benign, so there has to be something crazy going on in either Unity's HLSL->GLSL converter, or in the AMD shader compiler...
(I originally thought that we get register spilling, but meanwhile I'm rather certain that we can rule that out...)

Edited by Vanamonde
Link to post
Share on other sites

I might get crazy: By just removing the dither function completely (meaning: just making it return the input value) performance drastically increased on Linux. Like, from 35 to 45 fps on my machine...

Edit: It has something to do with the size of the bayerPattern.

Edit 2: That gives me an idea. The matrix can be replaced with a formula (https://en.wikipedia.org/wiki/Ordered_dithering). I'll try that tomorrow, and if it indeed improves performance, there will soon be a pull request.

Edited by soulsource
Link to post
Share on other sites
12 hours ago, soulsource said:

I might get crazy: By just removing the dither function completely (meaning: just making it return the input value) performance drastically increased on Linux. Like, from 35 to 45 fps on my machine...

Edit: It has something to do with the size of the bayerPattern.

Edit 2: That gives me an idea. The matrix can be replaced with a formula (https://en.wikipedia.org/wiki/Ordered_dithering). I'll try that tomorrow, and if it indeed improves performance, there will soon be a pull request.

Let me know how it goes, in my experience the shader compiler is full of bugs when it comes to compiling to opengl. One thing you can do is try to add the #pragma glsl directive to the shader, although I'm not sure if it still does something since unity 5

Link to post
Share on other sites

What I've found up to now is that the Bayer pattern array seems to be placed in private memory, instead of being hardcoded or stored in registers...

Quote

*** SHADER STATS ***
SGPRS: 72
VGPRS: 48
Spilled SGPRs: 0
Spilled VGPRs: 0
Private memory VGPRs: 64
Code Size: 4424 bytes
LDS: 0 blocks
Scratch: 17408 bytes per wave
Max Waves: 5
********************

I think the following snippet compares the same shader, once with the Bayer pattern as an array (old file), and once with it computed with the formula from Wikipedia (new file): https://pastebin.com/GAuSLWPx (think, because I have no experience when it comes to reading assembly).

Here's a pull request for your consideration: Edit 3: Please disregard, it's not building for Windows... https://github.com/LGhassen/Scatterer/pull/52

As said, this change improves performance on Linux with AMD hardware a lot, but as it does more computations (instead of using precomputed values), it might decrease performance on other platforms and should therefore be tested before merging (I would actually even consider conditional compilation, in the worst case)...

Edit: The merge request requires shader model 4. I don't know if scatterer supports d3d9, but if it does, please tell me, then I'll rewrite it so it does not use any bitwise operators.

Edit 2: I just saw that when building for Windows shaders for SM 3 are being built. I'll fix that.

Edit 4: See post below.

Edited by soulsource
Link to post
Share on other sites
On 12/8/2018 at 11:53 PM, EdwardB3020 said:

My frame-rate decreases when I'm on EVA or when a goo canister opens :( 

Please make a proper bug report (see section in the OP)

On 12/14/2018 at 7:39 PM, Nightside said:

Does anyone know if it is possible to get a lens flare effect on rocket engines?

Not possible now

Link to post
Share on other sites

As a small sidenote: I could not stop playing with this Linux issue, and it seems that the HLSL->GLSL converter of Unity is to blame. Maybe we should file a bug report there...

I've looked at the generated GLSL code using Mesa's shader replacement feature, and the generated code is everything but compiler friendly. Here's an example (one of the shaders that uses dither):  https://pastebin.com/KUpVnT2R
Unity converts the local compile time constant array into an uninitialized global array, that then is copy-initialized in the shader's main function :0.0:. Obviously the llvm AMD shader compiler is seeing this write access happening, and decides to make it a real, in-memory array instead of compiling it in as constants. I've made some trivial edits to the generated GLSL code, and that also brought performance up to speed (tested with Mesa's shader replacement feature again - I don't know how to make Unity do the same thing :(): https://pastebin.com/yCbXqHa1. The only difference being that the array is const again, and initialized right at declaration instead of copying the single values into it in the main function...

So at least we now know what was going on...

Link to post
Share on other sites
15 minutes ago, soulsource said:

As a small sidenote: I could not stop playing with this Linux issue, and it seems that the HLSL->GLSL converter of Unity is to blame. Maybe we should file a bug report there...

I've looked at the generated GLSL code using Mesa's shader replacement feature, and the generated code is everything but compiler friendly. Here's an example (one of the shaders that uses dither):  https://pastebin.com/KUpVnT2R
Unity converts the local compile time constant array into an uninitialized global array, that then is copy-initialized in the shader's main function :0.0:. Obviously the llvm AMD shader compiler is seeing this write access happening, and decides to make it a real, in-memory array instead of compiling it in as constants. I've made some trivial edits to the generated GLSL code, and that also brought performance up to speed (tested with Mesa's shader replacement feature again - I don't know how to make Unity do the same thing :(): https://pastebin.com/yCbXqHa1. The only difference being that the array is const again, and initialized right at declaration instead of copying the single values into it in the main function... 

So at least we now know what was going on...

Yeah, the HLSL -> GLSL converter is wonky in many ways, I often had to work around unexplained bugs.

If you would like to proceed with the bug report to unity, make sure to test it against the most recent unity version as the one used in KSP is a bit dated.

Link to post
Share on other sites
On 12/16/2018 at 2:32 PM, blackrack said:

Yeah, the HLSL -> GLSL converter is wonky in many ways, I often had to work around unexplained bugs.

If you would like to proceed with the bug report to unity, make sure to test it against the most recent unity version as the one used in KSP is a bit dated.

Okay, so I'm going to test if the issue persists with the latest Unity version, and if yes, file a bug report.

Edit: I can confirm that the same issue persists with the latest Unity build for Linux (2018.2.something). Bug report is in the process of being filed.

Edit 2: Bug report has been sent using Unity's Bug Reporting tool, including a minimal project with a really minimal shader for easy reproduction. Let's see if there's an answer.

Edit 3: No news from Unity yet. I should have filed the bug report at work, where we have Pro licenses, instead of home using a free license..

Edited by soulsource
Link to post
Share on other sites
On 12/16/2018 at 5:52 AM, blackrack said:

Please make a proper bug report (see section in the OP)

Not possible now

I think it was a memory glitch with the in game version 1.5.1. also could you update the mod to 1.6 please so I can use this mod with out having glitches from 1.5.1 in 1.6. thank you. If you watch @ShadowZone's video on 1.6 then you'd probably know what I'm talking about.

Edited by EdwardB3020
Link to post
Share on other sites
6 hours ago, EdwardB3020 said:

also could you update the mod to 1.6 please so I can use this mod with out having glitches from 1.5.1 in 1.6.

Dude.  KSP 1.6 was released literally not even eight hours ago.  And it's the holiday season, and modders do in fact have real lives with real things to do with their time other than the long, unpaid volunteer hours that they put into their mods, in order to give us all these shiny toys for free.

So please, be patient.  Rest assured that modders know that 1.6 has released, and if you haven't seen an update yet, it's because it hasn't been convenient (or even possible) for them to do so. The modder will release an update if and when they're inclined to do so, if and when they have time.  Their convenience is relevant; yours isn't.  It'll be ready whenever it's ready.

Personally, I hope they're taking a few weeks off to relax over the holiday season-- goodness knows they've earned it!  ;)

Link to post
Share on other sites
13 hours ago, Snark said:

Dude.  KSP 1.6 was released literally not even eight hours ago.  And it's the holiday season, and modders do in fact have real lives with real things to do with their time other than the long, unpaid volunteer hours that they put into their mods, in order to give us all these shiny toys for free.

So please, be patient.  Rest assured that modders know that 1.6 has released, and if you haven't seen an update yet, it's because it hasn't been convenient (or even possible) for them to do so. The modder will release an update if and when they're inclined to do so, if and when they have time.  Their convenience is relevant; yours isn't.  It'll be ready whenever it's ready.

Personally, I hope they're taking a few weeks off to relax over the holiday season-- goodness knows they've earned it!  ;)

nvr mind sry. ygacuhbadidhidhiuadhdahchuahuidaiaciuchaidbhcsbadssdacdadvdghvajcdghad crash. 

Link to post
Share on other sites

Preview of next release features (incoming image wall):

Extinction is now applied to the main light, this means that objects now receive a nicely tinted light at sunset/sunrise, helping with the perception of early morning/dusk

sEru0fb.jpg

whereas before everything remained bright white at any time:

8BcxDvO.jpg

Comparison of KSC at sunrise

2Bg4MSm.jpg

 

Glinted reflection and light matching the light on the clouds (here using textures unlimited for the part shaders, but stock shader will get the light tint as well):

1tAJyGh.jpg

PVs3lNV.jpg

2IL60gR.jpg

mGUb1LB.jpg

On Eve this looks interesting as well, although the purple ground gets extra dark with green light

mONVszz.jpg

You can even see this from the Mun surface if you get the planetary alignment just right:

usSGmF7.jpg

7oS3Byx.jpg

At the moment eclipses aren't handled at PQS level however so the mun surface stays red instead of going completely black when Kerbin covers the sun, this will be handled in the future.

The same approach to changing the light is now also used underwater, to make the actual underwater surface dimmer:

CnLPrAa.jpg

You can see thsi doesn't affect secondary lights and makes them more important to have underwater.

Moving on, the ocean now also responds to ambient light, meaning it's no longer pitch black at night

nQrltDo.jpg

 

And I have added shadows on the ocean surface

jCr1Uo8.jpg

This should help a lot with judging how close you are to the water surface when landing or flying really close.

3TcbkkV.jpg

EuUjzkP.jpg

7Vd8558.jpg

Pxv1j9v.jpg

SlI3euf.jpg

EVE cloud shadows aren't handled yet but I plan to support them as well.

Fixed the small break in the postprocessing between the two game cameras:

8CphS2v.jpg

 

Fixed small aliasing at the edge of scaled space planet surface (both are with AA on, left shows aliasing anyway)

ssOpHbD.jpg

 

Improved atmosphere blending:

uVPfERG.jpg

Improvements to shadow precision and less blobbiness:

xKjnLgR.jpg

 

The release will as well contain a few more changes and bug fixes that I will detail later.

One of the things I want to work on before releasing as well is to fix the aliasing on the horizon.

Edited by blackrack
Link to post
Share on other sites

Thank you for your continued efforts, Blackrack, they are truly appreciated! :D 

5 hours ago, blackrack said:

One of the things I want to work on before releasing as well is to fix the aliasing on the horizon.

I think Scrawk's https://github.com/Scrawk/Brunetons-Improved-Atmospheric-Scattering specifically fixes this issue: 

Quote

Improved texture coordinate mapping which removes the horizon artifact in the previous version.

 

Link to post
Share on other sites
5 hours ago, blackrack said:

Preview of next release features (incoming image wall):

(many many many awesome previes were listed here)

Fixed the small break in the postprocessing between the two game cameras:

8CphS2v.jpg

I cannot say how thankful I am for this. I've been promoting this bug since the 1.4 update and now the very own scatterer mod has fixed a stock bug. Amazing! Thank you!

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...