Why I'm Excited - An overview of HDRP and CBT Studies

[RayneCloud]

9 hours ago, The Aziz said:

It's all cool and pretty but these are all demo scenes. The rendered environment it's all there is. Now how does that work when there's a full game around it? Are there games that use HDRP and fit somewhat inside KSP2s hardware requirements? Sure the devs said they're not set in stone but me, and many other I bet would hate to see a "hey guys, we're switching to a much more demanding rendering tech so the minimum requirements are going up, kthxbye"

So, if you want to see that for yourself, here's a space game using HDRP - https://store.steampowered.com/app/1161580/Hardspace_Shipbreaker/
Nebulous Fleet Command also uses HDRP and has an incredibly demanding simulation on the backend - https://store.steampowered.com/app/887570/NEBULOUS_Fleet_Command/

6 hours ago, JoeSchmuckatelli said:

 

We don't know.  We are working off two pieces of information - a Shadowzone (maybe others) pre-launch interview with Nate where he refers to a massive overhaul of the PQS and the post by Modoc (Dev 18).  No timelines were suggested. 

Rayne...  

Hmm?

[JoeSchmuckatelli]

52 minutes ago, RayneCloud said:

Hmm?

Contextual. 

I'm certainly being presumptuous - but out of respect.  And... you know what.

[RayneCloud]

49 minutes ago, JoeSchmuckatelli said:

Contextual. 

I'm certainly being presumptuous - but out of respect.  And... you know what.

Oh, I just, didn't understand why you mentioned me is all Joe  Just confused. 

[Moons]

Im not that excited - i think the main issues will be:

- how easy are they to implement

- is it even worth it when already mid development

- how do those things scale - KSP doesnt just render a bigger area - it prett much renders a planet and a solar system

 

Would be interresting to know if those things are easy to implement and will be used for KSP2.

 

Anyways anything that improves engines for space/flight sims is always welcome by me - the space-flight and flight genere needs way more games - simulators need to have a comeback

The clouds by the way look pretty similar to the ingame clouds?

And i also hope that the DEVs are looking into DLSS - while that tech somewhat cheats a bit and reduces image quality - the results when used correctly still are great since it enables better looking games on cheaper hardware. Hopefully it wont be used to compensate for bad optimization in the future.

 

And regarding tech in general - while one can argue that Star Citizen is aweird Kickstarter - i hope the tech developed for Star Citizen will improve many new games. The detail of planets, ships and even seamless landing on planets is mindblowing but i doubt it would work with that much focus on physics like in KSP due to hardware limitations:

 

Edited March 12, 2023 by Moons

[Starhawk]

Some content has been removed.

Please keep your discussion on the topic at hand.

Thank you,
Forum Moderation Team

[Moons]

Just found that on reddit:

Reddit

Looks even better than the cloud videos you posted - or is that just a mod that uses new features of the unity engine?

 

Honestly what the ...: - this seriously looks amazing - would it be easy to implement that in KSP2?

(effects like that could also make other planets way more interresting - this seriously has a wow effect to it that could really help KSP2)

 

Edited March 12, 2023 by Moons

[The Aziz]

2 hours ago, RayneCloud said:

So, if you want to see that for yourself, here's a space game using HDRP - https://store.steampowered.com/app/1161580/Hardspace_Shipbreaker/
Nebulous Fleet Command also uses HDRP and has an incredibly demanding simulation on the backend - https://store.steampowered.com/app/887570/NEBULOUS_Fleet_Command/

It's all fair, I see only one difference - none of these render vast, detailed open environments.

[RocketRockington]

3 hours ago, The Aziz said:

It's all fair, I see only one difference - none of these render vast, detailed open environments.

Yeah, in the context of HDRP, the things you want it for are kind of orthogonal to the things you care about when render big environments, other than ray tracing.  Ray tracing could be fantastic for many things in KSP, but tbh if they hadn't already built clouds, penumbras and shadows with ray tracing, I don't see them doing it now.  And it certainly wouldn't improve the key performance problems they're having.

Thr more I dig into it, the more HDRP seems like a boondoggle to me.  But they may want just one specific feature out of the pipeline. *Shrug*.

[SimonTheSkink]

Well this is very beautiful looking. But how hard would it be to transition from the current setup to this new one?

[Streetwind]

No one in the community can answer that. Even Mortoc probably can only estimate, not know precisely.

He mentioned that it's a "medium term" project. If we define "short term" as "in the next couple of patches", and a patch cycle is three to four weeks, then medium term would be a minimum of three months and most likely quite a few more. My own blind guesstimate, based on nothing more than this, is Q3/2023 for previews, Q4/2023 for an initial release, and the first half of 2024 for bugfixing and polishing.

But since I have no idea about the state of the project, it could come faster than that. or take longer. Who knows? Not me!

[Moons]

2 hours ago, Streetwind said:

No one in the community can answer that. Even Mortoc probably can only estimate, not know precisely.

He mentioned that it's a "medium term" project. If we define "short term" as "in the next couple of patches", and a patch cycle is three to four weeks, then medium term would be a minimum of three months and most likely quite a few more. My own blind guesstimate, based on nothing more than this, is Q3/2023 for previews, Q4/2023 for an initial release, and the first half of 2024 for bugfixing and polishing.

But since I have no idea about the state of the project, it could come faster than that. or take longer. Who knows? Not me!

If modders can add something like this to KSP1 - shouldnt it be a reasonable possible thing to add something similar to KSP2?

 

 

Edited March 13, 2023 by Moons

[Streetwind]

21 minutes ago, Moons said:

If modders can add something like this to KSP1 - shouldnt it be a reasonable possible thing to add something similar to KSP2?

Sure. But what does that have to do with my post that you quoted?

[Moons]

14 hours ago, SimonTheSkink said:

Well this is very beautiful looking. But how hard would it be to transition from the current setup to this new one?

Werent you answering to this post - if not then sorry.

[Majorjim!]

Thems some damn shiny keys! 

[foonix]

After a lot of poking my nose under the hood, I have to say I'm pretty excited about HDRP.  Below are just my random assessments and observations.

KSP2 uses unity's legacy built-in render pipeline, but "retrofitted" with a lot of HDRP-like features.  For instance, they're already using PBR-style shaders that use a cubemap for environmental lighting.  HDRP uses PBR by default, and its "Environmental Lighting" system also uses a cubemap.

KSP2 uses third-party deferred light/shadow shaders that support soft shadowing, where HDRP supports PCSS and screen space shadows out of the box.

Most opaque materials in KSP2 use deferred rendering, but transparent materials use forward.  HDRP supports mixed deferred and "Forward+" rendering, which may perform better in certain lighting situations.

KSP2 uses some of the post-processing effects HDRP supports.  However, HDRP combines a bunch of them into a single "uber shader", where as the old render pipeline has to apply them one-at-a-time while moving the result between buffers.  There are various other postprocessing quality improvements available such as SSAO.

KSP2 uses camera stacking for certain things like UI layering.  Camera stacking is generally a bad idea in HDRP, but HDRP has features that would eliminate the need for most of the cameras.  I've done a single camera that can render out to 100km, and the PQS/CBT might want to be engineered not as a separate camera (see HDRP "custom pass").  The UI camera can probably be eliminated just by moving the UI shaders to the right draw phase.  So basically, it might be possible to shove the whole camera stack into one camera.  Although HDRP cameras are a lot slower than old cameras, eliminating extra cameras has the benefit potentially reducing the number of culling operations, which might partially make up for it.

KSP2 uses a setup that pumps planet cloud data into the volumetric cloud system.  I've not used HDRP clouds so I can't comment on how "easy" this would be to use,  But, HDRP's various volumetric effects are integrated with each other.  For example "god ray" effects would be out of the question if volumetric processing is already turned on.

IMHO a big potential win, from a performance standpoint, is might be SRP batching.  In certain situations (eg, landed at KSC), draw call batching takes a significant portion of the frame loop.  KSP2 uses instancing for a lot of the terrain clutter, which is quite fast in theory, but in practice gets split up into tiny batches due to a combination of material differences and distance sorting.  There is potentially some chance to leverage the SRP batcher for this stuff, depending on how the materials are set up.

Finally, a big difference is that all of the above (except PQS/CBT) in KSP2 are kind of mix and matched to work in spite of the old render pipeline, more so than with it.  Doing them with HDRP means working with a product that was designed from the ground up to do these things, and shares various computational steps between them.

The challenges I see:

• Every material needs to be retouched.  At minimum, converted to HDRP/lit, HDRP/unlit, or some shadergraph shader.  This process for some assets could be as simple as a tedious repack of the textures/ material.  For some assets it may need to be re-baked for HDRP's specific type of PBR and AO mapping.
• Lighting in general works differently and needs to be retouched.
• Camera stacking has to be reduced.
• The way PQS and/or CBT interface with HDRP is different and will require reworking.  However this might actually be _simpler_ as an HDRP custom pass.
• Environmental lighting works differently and may require some custom lighting, depending on how married to the current aesthetic they are.

[runner78]

At the weekend I tried to create an experimental orbit scene in HDRP with realistic stars (HDRI skybox) but had quite a problem with the exposure, as soon as an object such as a planet was even slightly in sight, the whole scene was completely overexposed .

However, I think that the move to HDRP is a bit late and maybe it should have been used from the start, or at least upgraded earlier in development since it's quite a hassle now.

[sumeee]

5 hours ago, runner78 said:

However, I think that the move to HDRP is a bit late and maybe it should have been used from the start, or at least upgraded earlier in development since it's quite a hassle now.

True, it's quite a hassle now, but it's not too late. Allot of assets like interstellar engines, new solar system, radiators and colony parts aren't released yet. And while they work on the new rendering pipeline they could focus more on science, progression, heating, etc.

[RocketRockington]

42 minutes ago, sumeee said:

True, it's quite a hassle now, but it's not too late. Allot of assets like interstellar engines, new solar system, radiators and colony parts aren't released yet. And while they work on the new rendering pipeline they could focus more on science, progression, heating, etc.

Yes but a development process that was already too slow and already far behind where it should have been doesn't need a detour that's going to require significant rework.  If they don't improve performance on their flight gameplay for 6 months, for instance, a lot of people will have tuned out by then.

[sumeee]

3 hours ago, RocketRockington said:

Yes but a development process that was already too slow and already far behind where it should have been doesn't need a detour that's going to require significant rework.  If they don't improve performance on their flight gameplay for 6 months, for instance, a lot of people will have tuned out by then.

Well, the purpose of HDRP and CBT is to improve the game performance and contrary to popular opinion you have to redo or create new systems to increase the game performance. Small optimizations, have usually, small impact on game performance.

Lets take Satisfactory as an example, the performance of that game improved allot over the years, but they had to rewrite systems from the ground up to get this level of performance. KSP2 team will have to do the same thing, and it will take allot of time.

[RocketRockington]

14 minutes ago, sumeee said:

Lets take Satisfactory as an example, the performance of that game improved allot over the years, but they had to rewrite systems from the ground up to get this level of performance. KSP2 team will have to do the same thing, and it will take allot of time.

 I'm not disputing that drastic measures seem necessary to get KSP2 performance up to snuff - I'm disputing whether its 'too late' or not. 

And in that context - I don't think KSP2, in its current nose dive, has the sort of time that Satisfactory did.  Satisfactory, after all, was fun and well reviewed right from the start.  I can't get a historgram chart for Satisfactory from steam from its first days, since it was exclusively on Epic for about a year, but you compare the review from RPS of Satisfactory  vs KSP2.  Performance wasn't even mentioned as a concern for Satisfactory - the only negatives brought up was the fact that multiplayer could be glitchy, and that the game felt a little grindy.  As I recall, Satisfactory ran just fine on middle-range hardware right from the start, and now runs on pretty well on integrated graphics.

Compare that to KSP2 that doesn't even have its promised multiplayer, and there's literally nothing to grind as the game only has its sandbox, and runs badly on anything except the absolutely latest gen, top end machine.

Edited March 15, 2023 by RocketRockington

[sumeee]

KSP2 is in early access, has bad reviews and high system requirements, so I think that the majority of people who want the game didn't buy and are willing to wait.

If the game gets better and allot of positive reviews than will get allot of new players.

[mcwaffles2003]

1 hour ago, RocketRockington said:

 I'm not disputing that drastic measures seem necessary to get KSP2 performance up to snuff - I'm disputing whether its 'too late' or not. 

Too late for what? 

It's obviously a worse choice to just ignore current rendering issues and continue on with outdated software reducing its potential ceiling, just work on it over time while developing and fleshing out other systems that aren't dependent on it

[RayneCloud]

21 hours ago, foonix said:

After a lot of poking my nose under the hood, I have to say I'm pretty excited about HDRP.  Below are just my random assessments and observations.

KSP2 uses unity's legacy built-in render pipeline, but "retrofitted" with a lot of HDRP-like features.  For instance, they're already using PBR-style shaders that use a cubemap for environmental lighting.  HDRP uses PBR by default, and its "Environmental Lighting" system also uses a cubemap.

KSP2 uses third-party deferred light/shadow shaders that support soft shadowing, where HDRP supports PCSS and screen space shadows out of the box.

Most opaque materials in KSP2 use deferred rendering, but transparent materials use forward.  HDRP supports mixed deferred and "Forward+" rendering, which may perform better in certain lighting situations.

KSP2 uses some of the post-processing effects HDRP supports.  However, HDRP combines a bunch of them into a single "uber shader", where as the old render pipeline has to apply them one-at-a-time while moving the result between buffers.  There are various other postprocessing quality improvements available such as SSAO.

KSP2 uses camera stacking for certain things like UI layering.  Camera stacking is generally a bad idea in HDRP, but HDRP has features that would eliminate the need for most of the cameras.  I've done a single camera that can render out to 100km, and the PQS/CBT might want to be engineered not as a separate camera (see HDRP "custom pass").  The UI camera can probably be eliminated just by moving the UI shaders to the right draw phase.  So basically, it might be possible to shove the whole camera stack into one camera.  Although HDRP cameras are a lot slower than old cameras, eliminating extra cameras has the benefit potentially reducing the number of culling operations, which might partially make up for it.

KSP2 uses a setup that pumps planet cloud data into the volumetric cloud system.  I've not used HDRP clouds so I can't comment on how "easy" this would be to use,  But, HDRP's various volumetric effects are integrated with each other.  For example "god ray" effects would be out of the question if volumetric processing is already turned on.

IMHO a big potential win, from a performance standpoint, is might be SRP batching.  In certain situations (eg, landed at KSC), draw call batching takes a significant portion of the frame loop.  KSP2 uses instancing for a lot of the terrain clutter, which is quite fast in theory, but in practice gets split up into tiny batches due to a combination of material differences and distance sorting.  There is potentially some chance to leverage the SRP batcher for this stuff, depending on how the materials are set up.

Finally, a big difference is that all of the above (except PQS/CBT) in KSP2 are kind of mix and matched to work in spite of the old render pipeline, more so than with it.  Doing them with HDRP means working with a product that was designed from the ground up to do these things, and shares various computational steps between them.

The challenges I see:

• Every material needs to be retouched.  At minimum, converted to HDRP/lit, HDRP/unlit, or some shadergraph shader.  This process for some assets could be as simple as a tedious repack of the textures/ material.  For some assets it may need to be re-baked for HDRP's specific type of PBR and AO mapping.
• Lighting in general works differently and needs to be retouched.
• Camera stacking has to be reduced.
• The way PQS and/or CBT interface with HDRP is different and will require reworking.  However this might actually be _simpler_ as an HDRP custom pass.
• Environmental lighting works differently and may require some custom lighting, depending on how married to the current aesthetic they are.

All of this is why I brought it up. 

[JoeSchmuckatelli]

On 3/15/2023 at 4:12 AM, foonix said:

After a lot of poking my nose under the hood, I have to say I'm pretty excited about HDRP.  Below are just my random assessments and observations.

KSP2 uses unity's legacy built-in render pipeline, but "retrofitted" with a lot of HDRP-like features.  For instance, they're already using PBR-style shaders that use a cubemap for environmental lighting.  HDRP uses PBR by default, and its "Environmental Lighting" system also uses a cubemap.

KSP2 uses third-party deferred light/shadow shaders that support soft shadowing, where HDRP supports PCSS and screen space shadows out of the box.

Most opaque materials in KSP2 use deferred rendering, but transparent materials use forward.  HDRP supports mixed deferred and "Forward+" rendering, which may perform better in certain lighting situations.

KSP2 uses some of the post-processing effects HDRP supports.  However, HDRP combines a bunch of them into a single "uber shader", where as the old render pipeline has to apply them one-at-a-time while moving the result between buffers.  There are various other postprocessing quality improvements available such as SSAO.

KSP2 uses camera stacking for certain things like UI layering.  Camera stacking is generally a bad idea in HDRP, but HDRP has features that would eliminate the need for most of the cameras.  I've done a single camera that can render out to 100km, and the PQS/CBT might want to be engineered not as a separate camera (see HDRP "custom pass").  The UI camera can probably be eliminated just by moving the UI shaders to the right draw phase.  So basically, it might be possible to shove the whole camera stack into one camera.  Although HDRP cameras are a lot slower than old cameras, eliminating extra cameras has the benefit potentially reducing the number of culling operations, which might partially make up for it.

KSP2 uses a setup that pumps planet cloud data into the volumetric cloud system.  I've not used HDRP clouds so I can't comment on how "easy" this would be to use,  But, HDRP's various volumetric effects are integrated with each other.  For example "god ray" effects would be out of the question if volumetric processing is already turned on.

IMHO a big potential win, from a performance standpoint, is might be SRP batching.  In certain situations (eg, landed at KSC), draw call batching takes a significant portion of the frame loop.  KSP2 uses instancing for a lot of the terrain clutter, which is quite fast in theory, but in practice gets split up into tiny batches due to a combination of material differences and distance sorting.  There is potentially some chance to leverage the SRP batcher for this stuff, depending on how the materials are set up.

Finally, a big difference is that all of the above (except PQS/CBT) in KSP2 are kind of mix and matched to work in spite of the old render pipeline, more so than with it.  Doing them with HDRP means working with a product that was designed from the ground up to do these things, and shares various computational steps between them.

The challenges I see:

• Every material needs to be retouched.  At minimum, converted to HDRP/lit, HDRP/unlit, or some shadergraph shader.  This process for some assets could be as simple as a tedious repack of the textures/ material.  For some assets it may need to be re-baked for HDRP's specific type of PBR and AO mapping.
• Lighting in general works differently and needs to be retouched.
• Camera stacking has to be reduced.
• The way PQS and/or CBT interface with HDRP is different and will require reworking.  However this might actually be _simpler_ as an HDRP custom pass.
• Environmental lighting works differently and may require some custom lighting, depending on how married to the current aesthetic they are.

Have you seen the KSP2 / GDC talk?  There is a brief portion where they discuss HDRP & CBT. 

I'm curious as to what you think now that we've had several weeks. 

Can they do HDRP independent of the CBT work - or do they need to do both together? 

@RayneCloudany thoughts? 

Edited April 29, 2023 by JoeSchmuckatelli

[RayneCloud]

I have no idea honestly, If I had to wager a guess, it's not something we're going to see within the next 3 to 6 months. Again, that's a GUESS, I have NO CLUE and I don't work on this project, and I'm not an expert on HDRP and CBT at all. I'm just assuming several working months ahead here.