• Enter the Shadows.....


    JPLRepo

    There have been a steadily growing list of shadow issues since KSP moved to Unity 5. This list now included (but was not limited to) shadow issues with Trees, KSC buildings, Green lines over water and Sun shadows flickering in flight.

    Looking at them one by one, each of these issues appeared to have different root causes.

    So let’s look at them one by one.

     

    Trees

    As soon as I started looking at the tree issue it was quickly apparent this was a shader issue given it only affected one particular tree model and its vertex shapes.

    Where do we start?

    vGQyt5t5z4ubNIL_Lfs4zCTndoBdu3maS3f9rCjTx3MLrG46bIoQQRw2Z46csMP6TScUOA-h4H84r86J

     

    Digging into the shader being used by our tree that is causing us issues I quickly found that the alpha cutoffs and fallback shader are just not quite right because of changes Unity made in Unity 5.

    A few adjustments later to the shader and we have our result.

     

    oDrCjyNNxNySNsx5XDGTzfynDAoggxXNBzx5zSND

     

     

    KSP's Camera System

    KSP uses a two Main camera system for the main game view. Of course there are a number of other cameras for displaying UI, Graphic FX, etc.

    But these two are the ones that show you the game world. Let’s look at a picture so I can describe it.

    W4aeXLPijpzog0JDgQlhzXZbVBDlv5cmgmOKMRHJ

     

    There are two cameras located where the camera icon is in the top left of the picture.
    This is their location in the game world.
    They are both looking in the same direction and are linked to the same camera transform.
    The first camera, or Camera00, is known as the Near Camera. It is set to show only the first few hundred meters of the game world and it’s view of the world ends at it’s Near Camera Far clipping pane, as seen in the picture.
    The second camera, or Camera01, is known as the Far Camera. It is set to show the game world where the Near camera ends out to a very very long distance away.
     

    There have been long standing issues with this system including shader issues where the two cameras overlap, or where the Near Camera’s Far Clipping Pane ends and the Far Camera’s Near Clipping Pane starts. This has been particularly visible as a green line over water.

    The primary reason for why these two cameras overlap is to provide a seamless view of the game world. So changing the camera settings we can reduce the overlap to the very minimum and we see that the green line that used to be visible over water has now vanished.

     

    YTzwBdnxeZYaQoMQB1PBM37gkPHXtSBoPuRxC6iQ

    2p2XEVk1xhzZe-_KyX8sxpZidl2wi2t2ZThdmWH2


     

    KSC Buildings

    The KSC buildings had also been suffering from this same two camera system since Unity had introduced changes to the way it draws shadows.

    In Unity's built in shadow shaders is an automatic fade out of shadows as you approach the far clipping plane (the furtherest the camera can see).

    This is why we had things like this

    MX6l3pycmAErHQsQkR4RvbTu6H8VrNJ5d0UG0d4o

    where the shadow would fade away, get cut, or not appear around the area of where the closest game camera far clipping plane ended.
    This problem has been known about for some time.

    Here is a link to an article written by Harvester asking for help about the problem. Here also, is a picture by Harvester describing the problem (from the article).


    UJWlWzkWvZXpYAkz6vmr77dznekaVeIoRky-5yfX

     

    So how do we fix this? Well luckily as a developer I am able to get access to the Unity built in shaders and create a modified version that does not have this fade out feature. So I take the Unity built in shader used for the building shadows and change its code to remove the fade out processing.

    Once I have the shader fixed to not fade the shadows I also need to write a script (code) that will replace the built-in Unity shadow shader with our modified one.

    Finally I add this script to the KSP cameras and we get a much better result.


    M_1Q11YahA-SXkgPwetN0pzfiVnKTfFaTXYhspHn

     

    But we weren't quite done with the building shadows. Another known issue in Unity 5 is the way sharp edges in object geometry cause gaps in shadows to appear.

    Luckily this is much easier to resolve by changing the settings of the geometry on our building objects to use two-sided shading.


    apVsIWwMXA5RCS2b69DVph3HkP09qfyIKqGGvbuz
     

     

    The Sun and Flight Scene.

    Everyone has probably noticed the flickering/sliding shadow effects that have plagued the game in flight mode since the move to Unity 5.

    Without a doubt the trickiest problem of the bunch and hardest to resolve. There is a combination of factors at play here. First is a known issue with the way Unity draws shadows with directional lights and the second issue, that only tends to exacerbate the problem, is floating point precision issues calculating very large numbers which represent the very large distances (even at the scale the Kerbin system is in).

    The SunLight we see in the game is generated by a Unity Directional Light. This light is rotated over time based on the position of the celestial bodies.

    The position and rotation is calculated every tick of the clock and the light itself is rotated based on the body positions.

    Due to floating point imprecision this means the calculations between the bodies can vary even the smallest amount between clock ticks and this is exacerbated further by a known

    bug in Unity with regard to shadows drawn by directional lights.

    By reducing the precision of the calculations between the celestial bodies we can reduce these tiny variations so we have a solid number that gradually changes over time.

    The downside of reducing the precision is that the rotation of the sun is less smooth, so we end up with a sun that tends to move in small steps rather than one fluid motion across the sky.

    But the plus side is we reduce the difference between clock ticks and we don't get light shadows flickering all over the place.

    This work-around solution, whilst not ideal, is the best compromise to alleviate the issue working within the known limitations and Unity issues that we have.
    These known issues with Unity 5.4 directional lights and shadows have been fixed and much improved in Unity 5.6. Along with improved shaders, effects and other enhancements that come with Unity 5.6.

    Edited by JPLRepo

    69 people like this




    User Feedback




    Awesome! Are these included in 1.3 or are they coming in a future update?

    Share this comment


    Link to comment
    Share on other sites

    ...does that last sentence allude to us getting version of KSP on Unity 5.6.x in the future? /fingerscrossed

    Share this comment


    Link to comment
    Share on other sites
    11 minutes ago, legoclone09 said:

    Awesome! Are these included in 1.3 or are they coming in a future update?

    Read the changelog in the release announcement:

    Quote
    
    * Fix Tree shaders - shadows.
    * Fix green line appearing in water.
    * Fix shadows of buildings in KSC scene.

    There's a few other related items.

    3 people like this

    Share this comment


    Link to comment
    Share on other sites

    +1 for Unity 5.6. It has greatly improved for me developing my game as well. 

    Share this comment


    Link to comment
    Share on other sites

    Oh, these look amazing!!! 

    I take a ton of screenshots, and I really love seeing these fixes, especially for the tree shadows!!!

    Thank you!!!  :D

    Share this comment


    Link to comment
    Share on other sites

    Nice.  i really like these in-depth explanations of bug hunting.  

    2 people like this

    Share this comment


    Link to comment
    Share on other sites

    Awesome, these articles are great, keep up the good work @JPLRepo!

    So while we're on the topic of clipping planes, let's talk about the near camera's near clipping plane...

    2mip3P9.png

    9 people like this

    Share this comment


    Link to comment
    Share on other sites
    52 minutes ago, The Moose In Your House said:

    Is this for 1.3?

    Yes. all the things discussed above were implemented in 1.3.

    1 person likes this

    Share this comment


    Link to comment
    Share on other sites

    Wow... I never actually noticed how bad KSP shadows were until you pointed them all out and fixed them! Incredible work and really insightful (I have extremely limited knowledge of game development - had a go at making mods for Skyrim, I could create worlds, new items, spells etc, but lighting/shadows/cameras were never something I really got my head around at all).

    Share this comment


    Link to comment
    Share on other sites

    Why use two cameras instead of one?

    1 person likes this

    Share this comment


    Link to comment
    Share on other sites
    Quote

    By reducing the precision of the calculations between the celestial bodies we can reduce these tiny variations so we have a solid number that gradually changes over time.

    The downside of reducing the precision is that the rotation of the sun is less smooth, so we end up with a sun that tends to move in small steps rather than one fluid motion across the sky.

    But the plus side is we reduce the difference between clock ticks and we don't get light shadows flickering all over the place.

    This work-around solution, whilst not ideal, is the best compromise to alleviate the issue working within the known limitations and Unity issues that we have.

    Can't help but disagree with this assessment, Changing the way the sun moves (an important orbital transition) to fix a shadow issue (a relatively unimportant issue) seems like an odd compromise in this game about orbital mechanics, especially since the benefit is only really noticeable on Kerbin and less so in space, having the shadows move incrementally seems worse to me than lower quality smooth transitions since they tend to draw attention as they move, I can't really say I've ever noticed the shadows to be all that bad previously.

    Personally I would prefer smooth sunsets and sunrises, watching the Mun eclipse over kerbin, etc to better shadow fidelity, after all I tend to be looking up in this game more than looking down.

    That said, I appreciate the work you have put into these issues and this post of course, so thanks :) 

    2 people like this

    Share this comment


    Link to comment
    Share on other sites
    3 hours ago, rob593 said:

    Can't help but disagree with this assessment, Changing the way the sun moves (an important orbital transition) to fix a shadow issue (a relatively unimportant issue) seems like an odd compromise in this game about orbital mechanics, especially since the benefit is only really noticeable on Kerbin and less so in space, having the shadows move incrementally seems worse to me than lower quality smooth transitions since they tend to draw attention as they move, I can't really say I've ever noticed the shadows to be all that bad previously.

    Personally I would prefer smooth sunsets and sunrises, watching the Mun eclipse over kerbin, etc to better shadow fidelity, after all I tend to be looking up in this game more than looking down.

    That said, I appreciate the work you have put into these issues and this post of course, so thanks :) 

    And others will say that the shadows annoyed them to no end and that is more important to them.. There was and is no easy answer. But there is light at the end of the tunnel.

    8 people like this

    Share this comment


    Link to comment
    Share on other sites
    1 hour ago, JPLRepo said:

    And others will say that the shadows annoyed them to no end and that is more important to them.. There was and is no easy answer. But there is light at the end of the tunnel.

    I understand this, it is a situation where you can not please all people, which is why i felt it was important to post that I personally value the smooth sun transition over shadows as I consider it a more important contributor to the game. But I am aware that is MY opinion and it's not shared by all.

    You did however dedicate resources to making this compromise, which to me implies that you consider shadows over smooth sun transition to be more beneficial to the game, as a result I felt my opinion on this issue (being seemingly opposite to yours) is valuable feedback.

    Thanks for taking the time to respond, and again for the original post.

    1 person likes this

    Share this comment


    Link to comment
    Share on other sites
    Speadge

    Posted (edited)

    @JPLRepo

    Thank you very much for this dev article and your time for explaining what happens "behind the scenes".
    I really like when the devs speak about what is actually happening and tend to do more public documentation than just logging "its fixed now".

     

    I really look forward playing 1.3 and the expansion now AFTER i read this article - not because i expect fixes to be in the expansion - but i appreciate the "fanservice for techs" u do from time to time

    Edited by Speadge
    4 people like this

    Share this comment


    Link to comment
    Share on other sites
    Errol

    Posted (edited)

    ..U5.6!?

    Edited by Errol
    1 person likes this

    Share this comment


    Link to comment
    Share on other sites
    On 6/9/2017 at 1:37 AM, Damjan said:

    Why use two cameras instead of one?

    Because of the way the Z coordinate is rendered in what is called the Z-buffer.
    So if you look at the picture imagine there are slices like a sheet of paper between where the camera is and it's far clipping pane.
    The amount of "slices" you can have between the near and far clipping pane are relatively fixed by Unity. So the greater the distance between the two then the greater the space between these slices of paper becomes.
    This creates other problems such as Z-fighting.
    So the two camera system allows a small distance between these Z planes at closer distance to the player. Which gives greater Z-buffer depth and less Z-fighting where it matters the most (closest to the player).

    2 people like this

    Share this comment


    Link to comment
    Share on other sites

    Great write-up @JPLRepo - I finally understand why the game uses two cameras for rendering scenes.

    Any chance the Shader rework that comes with Unity 5.6 will fix the IVA shadow casting issue?

    Share this comment


    Link to comment
    Share on other sites
    9 hours ago, sumghai said:

    Great write-up @JPLRepo - I finally understand why the game uses two cameras for rendering scenes.

    Any chance the Shader rework that comes with Unity 5.6 will fix the IVA shadow casting issue?

    We will have to see what the future holds. :cool:

    Share this comment


    Link to comment
    Share on other sites

    @JPLRepo So glad that the tree shadows were finally sorted out and to hear about future lighting improvements. Could you talk a bit about camera clipping and why it became rather severe a few updates ago? And indeed if there is a possible solution.

    Share this comment


    Link to comment
    Share on other sites
    13 hours ago, JPLRepo said:

    We will have to see what the future holds. :cool:

    :wink:

    Share this comment


    Link to comment
    Share on other sites
    11 hours ago, HatBat said:

    @JPLRepo So glad that the tree shadows were finally sorted out and to hear about future lighting improvements. Could you talk a bit about camera clipping and why it became rather severe a few updates ago? And indeed if there is a possible solution.

    Which clipping are you referring to?
    The one referred to here:

     

    Share this comment


    Link to comment
    Share on other sites

    Great write up @JPLRepo! I always love to see the care and attention put into the game, as well as the process of identifying issues and figuring out which solutions works best. They make a great resource for the Unity community too.

    On 11/6/2017 at 2:05 AM, JPLRepo said:

    Which clipping are you referring to?
    The one referred to here:

    Yes, either the near camera's near clip plane or the scroll wheel FoV control changed noticeably between KSP 1.0 and KSP 1.1, reducing how close you could get to the craft without cutting it.

    Here's a screenshot of KSP 1.1.2.1260, zoomed in to a capsule on the launchpad, up the the point of first clipping:

    MFNlrRR.png

    Here's a similar scene in KSP 1.0.5.1024, with no clipping at this zoom level:

    7KEHvXM.png

    The clipping only appears when the capsule's window is zoomed in close enough to fill the whole screen. One more level of zoom beyond this causes the first sign of clipping:

    Fd2DFyh.png

     

    Edited by pizzaoverhead
    2 people like this

    Share this comment


    Link to comment
    Share on other sites
    18 hours ago, pizzaoverhead said:

    @JPLRepo

    Yes, either the near camera's near clip plane or the scroll wheel FoV control changed noticeably between KSP 1.0 and KSP 1.1, reducing how close you could get to the craft without cutting it.

     

    Gives me an itch every time.

    Share this comment


    Link to comment
    Share on other sites



    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!


    Register a new account

    Sign in

    Already have an account? Sign in here.


    Sign In Now