Physics and Part Count Developer Insight Request :)

[Anth]

Hello.

I saw the last Developer Insight and saw that there is an obvious improvement to the terrain looking more consistent zooming in and out which is excellent. Good work!

However my last 2 years of KSPing have been more about large part count and KSP1 hates large part count. I don't use any visual mods except for waterfall to maximize performance.

I was wondering how part count and physics calculations are going to be changed for improved performance in the following areas:

1. How the CPU will calculate physics per part
2. How the CPU will calculate physics per vessel (per core etc)
3. How the CPU will be affected for vessels within render range vs physics range because 1000+ parts within just render range still causes a significant drop in performance on my i9 9900K + Geforce 1080 GTX
4. How KSP2 will counter the game freezing for a moment when entering render range and physics range exclusively
5. How KSP2 will counter the game freezing when a complicated craft with a large part count explodes

 

[The Aziz]

Physics LOD, devs mentioned it long ago.

Also 

 People should stop thinking in terms of "ksp1 bad, how ksp2 good". New game has no legacy code that would slow it down, no "technical debt", works on an engine which, if properly utilized, is one of the best out there...

Ksp2 doesn't have to "counter" freezing if there's no freezing in the first place because it was coded to not have any freezing.

[Guest]

1 hour ago, The Aziz said:

Physics LOD, devs mentioned it long ago.

Also 

 People should stop thinking in terms of "ksp1 bad, how ksp2 good". New game has no legacy code that would slow it down, no "technical debt", works on an engine which, if properly utilized, is one of the best out there...

Ksp2 doesn't have to "counter" freezing if there's no freezing in the first place because it was coded to not have any freezing.

Also, this time around we'll have proper big ship and big station parts, no more need to use 2000+ 16 seat plane fuselages to build anything bigger than the ISS.

[Anth]

4 hours ago, The Aziz said:

Ksp2 doesn't have to "counter" freezing if there's no freezing in the first place because it was coded to not have any freezing.

If that's true, then the developers haven't explained how it works yet. 

4 hours ago, The Aziz said:

This above doesn't talk about how 1000 parts interact in an explosion.

The above doesn't talk about how render range/physics range works in KSP2

The above doesn't talk about how the freezing was happening in KSP1 and how it will be countered in KSP2

4 hours ago, The Aziz said:

Physics LOD, devs mentioned it long ago.

What was Physics LOD again?

Also part of the reason these Developer Insights are happening is to show off how its improved over KSP1 no?

[MechBFP]

KSP1 != KSP2

[Anth]

Fine. How its different to KSP1...  

[HebaruSan]

I agree this would be a good topic. I don't expect them to cover it though, probably because not much improvement is possible. 1000 parts interacting with each other is on the order of 1,000,000 interactions to calculate every frame, or 16 nanoseconds per interaction at 60 FPS. A modern CPU will take that much time to access the L3 cache a time or two, and you can forget about pulling inputs in from RAM and pushing outputs back out to RAM that fast. That's not technical debt or bad coding, that's just the reality of the task presented.

Large part count vessels were always allowed by the engine but not the main thing it was made to support. Expect that to continue. The best bet, for devs and players alike, will be to find ways to avoid it, such as the big parts mentioned above.

EDIT: Oops, the original version of the above text accidentally assumed "one frame" and "one second" were interchangeable, contrary to most players' expectations that the game should run at more than 1 FPS. Dividing/multiplying by 60 makes the problem much worse, and I've attempted to correct the error.

Edited December 11, 2021 by HebaruSan

[mcwaffles2003]

8 hours ago, Anth12 said:

What was Physics LOD again?

Also part of the reason these Developer Insights are happening is to show off how its improved over KSP1 no?

 

6 hours ago, HebaruSan said:

I agree this would be a good topic. I don't expect them to cover it though, probably because not much improvement is possible. 1000 parts interacting with each other is on the order of 1,000,000 interactions to calculate every frame, or 16 nanoseconds per interaction at 60 FPS. A modern CPU will take that much time to access the L3 cache a time or two, and you can forget about pulling inputs in from RAM and pushing outputs back out to RAM that fast. That's not technical debt or bad coding, that's just the reality of the task presented.

Large part count vessels were always allowed by the engine but not the main thing it was made to support. Expect that to continue. The best bet, for devs and players alike, will be to find ways to avoid it, such as the big parts mentioned above.

EDIT: Oops, the original version of the above text accidentally assumed "one frame" and "one second" were interchangeable, contrary to most players' expectations that the game should run at more than 1 FPS. Dividing/multiplying by 60 makes the problem much worse, and I've attempted to correct the error.

As I understand it, it's being speculated that regions of connected parts may be hard welded together and treated all as single parts to reduce the amount of joints as needed to reduce the simulated part count. It may sacrifice some "accuracy", but is having a kraken attack blow a ship apart for no reason accurate anyway?

 

Really curious as to how much of the will be automated and how much will allow user intervention.

Edited December 11, 2021 by mcwaffles2003

[Gotmachine]

"physics" is a rather vague term.

If you're talking about rigidbody/joint/collision physics, this is handled by the PhysX engine and there is little they can do to optimize that. Unless they decide to roll out a 100% custom rigidbody physics engine, which would be quite justified as PhysX (or any "standard" physics library) is quite inadequate for KSP use case. But I doubt this will be happening, this would be a huge investment for not that much immediate gains.

As mentioned, the most likely optimization they will do is just to disable RB physics altogether in some cases. RB physics are essentially useless when there are no external forces (collisions, gravity, atmospheric drag...). The most likely case will be bases and stations, and other vessels beside the active one. Or even anything that is in space, since they intend to have "unloaded" vessels being able to produce thrust, this mean they are planning to handle thrust/attitude control forces without relying on the RB simulation (but how coherent the simulation will be between active and unloaded vessels remain to be seen).

Also, contrary to popular belief, the rigibody/joints physics is NOT the main KSP performance issue. The KSP other "physics" subsystem are the main bottleneck : the aerodynamic and thermal simulations. Each of these subsystems consume about as much CPU time as the PhysX RB simulation. Using modern Unity features (jobs/burst), those could be massively optimized without that much effort. There are many other potential optimizations in various specific subsystems like the orbit solver, the resource flow/request system, the engine thrust solver, various partmodules...

This being said, I don't expect a paradigm shift in what is possible in term of part count. The current KSP "playable" threshold is somewhere between 300-400 parts. If they do a decent job at optimizing the various subsystems compared to KSP 1, we could maybe get a 3x improvement on that figure, but don't expect being able to have thousands of parts without lag.

Edited December 12, 2021 by Gotmachine

[HebaruSan]

On 12/11/2021 at 3:55 AM, Anth12 said:

1. How KSP2 will counter the game freezing when a complicated craft with a large part count explodes

On 12/11/2021 at 1:07 PM, Anth12 said:

This above doesn't talk about how 1000 parts interact in an explosion.

This may partly be caused by creating lots of new Vessel objects per frame. Remember that when a craft breaks apart, each cluster of parts that are still attached to each other becomes a new vessel in the tracking station (usually with a vessel type of "debris"). So before the explosion, you have 1 vessel with 1000 parts, and afterwards you have many vessels with a few parts each. During the explosion, the game has to create all of those Vessels, each of which has its own calculated center of mass,  resources, heating, Orbit objects, etc. And while parts in the same vessel can be set to not interact, that's not the case for parts from different vessels.

So maybe another optimization to pursue, if KSP 2 uses a similar structure, would be to make the Vessel object (or equivalent) more lightweight in anticipation of needing to create lots of them at once.

[Spaceman.Spiff]

2 hours ago, Gotmachine said:

Using modern Unity features (jobs/burst), those could be massively optimized without that much effort. There are many other potential optimizations in various specific subsystems like the orbit solver, the resource flow/request system, the engine thrust solver, various partmodules...

Could KSPCommunityFixes implement something along these lines to improve performance?

[K^2]

14 hours ago, Gotmachine said:

If you're talking about rigidbody/joint/collision physics, this is handled by the PhysX engine and there is little they can do to optimize that.

As mentioned previously, the optimization that can be done is combining multiple parts into a single rigid body. If you don't care about flex between the parts, combining multiple moment of inertia tensor into one is a rather simple procedure. You also don't have to sacrifice ability to model joint strength limits, as for a rigid body, constraint matrix never changes, so you only ever have to invert it once. Combined with ability to build a reduced LoD collision model for the complex, a collection of welded parts can be as cheap to simulate in PhysX as a single part precisely because from PhysX perpsective it's a single part. It sounds like something like this is, or at least was, in the works for KSP2.

And yeah, the non-PhysX portions of the simulation will mostly benefit from better ability to spread the work between the cores. Between these two optimizations, it should be possible to have a drastically higher part count and/or create breathing room for other systems.

[Gotmachine]

18 hours ago, Spaceman.Spiff said:

Could KSPCommunityFixes implement something along these lines to improve performance?

Not really, for the same reason Squad couldn't do it either : this would imply major breaking changes in core legacy code, breaking ton of other code down the line, including mods. Squad already did what they could in terms of "afterthought" optimization, there is actually not that much left on the table given KSP technical debt and desire to ensure backward compatibility with the existing modding ecosystem.

6 hours ago, K^2 said:

You also don't have to sacrifice ability to model joint strength limits, as for a rigid body, constraint matrix never changes, so you only ever have to invert it once.

I don't see how you can keep joint strength limits if you merge everything into a single PhysX RB, unless you roll out a custom implementation for that.
At this point, if you have that custom implementation, you've already reproduced most of what PhysX is used for in KSP.

Side note : KSP relying on PhysX joints at all is quite questionable, since :
- it has to use a floating origin (which cause a significant overhead on the PhysX side)
- it doesn't use jointed RBs collisions (no "intra-part" collision)
- it has to spam hundreds of extra PhysX joints to be vaguely successful at implementing a "lightweight parts should result in as much rigidity as heavy ones" behavior
- it has to compute a significant proportion of the RB physics results independently
- it has more generally to do tons of dirty hacks to avoid various issues

On the other hand, it gives KSP its distinctive "noodle rocket / wonky physics" touch. Depending on your POV, this is either a bug or a feature.
My personal stance is that the KSP lego parts paradigm fundamentally can't result in a good "structural strength design" gameplay element. This isn't a bridge construction game, there is fundamental conflict in trying to mix the "parts are functional elements" and "parts are structural elements" paradigms. The whole "joints structural gameplay" is vaguely fun for the first two hours you discover the game, then it just become a constant annoyance.

This being said, rolling out a completely custom joint/RB physics implementation is no trivial task, and it imply loosing a lot of "standard" features from the Unity ecosystem, for example the wheels/legs physics plugin KSP relies on (VPP).
I don't expect them to take that path. Even though it would likely be the better option in the long run, it imply a massive upfront development cost.
So yup, maybe they will partially disable the per part RB PhysX simulation we are used to in some cases.
If they do that, I wouldn't qualify that as "optimization", but more as another hacky workaround, in the same realm as spamming PhysX joints to make large vessels vaguely rigid.

There are several technical options there, merging multiple parts in a single RB is one.
For vessels, which are frequently subject to part reconfiguration events (staging, docking, collisions, etc), it would induce a lot of complexity and overhead. But I can see that happening for the "semi-static" large bases/stations, which will likely be different technical entities than regular vessels.

Alternatively, there is the possibility of switching the vessel parts RB to kinematic mode and handling vessel forces/torques manually to update part positions  (KSP already does this when vessels are in the packed state, minus the forces/torques handling). This is a relatively straightforward way to opt-out from joint physics (and the bulk of the performance cost) while keeping some useful RB stats (like per-part CoM), and as I said, they will need to implement all that for being able to do out-of-physics/in-timewarp engine thrust and vessel attitude control, unless they decide to use a dumbed down "just aggregate all engines as a single CoM aligned thrust vector" model for that. The only remaining piece is having a collision prediction thing, so they can switch out of kinematic mode before a vessel-vessel or vessel-terrain collision happen and have joints behave as expected in that case. And side note,  that "physics LOD" rumor comes from a 2019 "leak" from somebody that used to work on KSP 2, and what he said was just that : they will try to keep vessels in the packed (ie kinematic RBs) state more often.

But in any case, as long as they keep the flexible joints feature, those always will be temporary optimization opportunities. There will always be a point where full per-part RB physics have to be enabled. So this would result in an user experience with massive FPS variations for the same vessel depending if said optimizations are applicable or not. Not to mention it would introduce (more) discrepancies in physics behavior. So the whole thing is a quite questionable decision, and I have some doubts that "physics LOD" rumor will actually materialize into something very significant.

As for the stuttering on docking/undocking/staging/collisions/vessel becoming loaded, there is little they can do here. Those events fundamentally imply initializing and re-initializing a lot of stuff, which is always a costly operation. If your CPU already has no breathing room when those happen, the stuttering is unavoidable. They can use some strategies and good practices to limit the effect, but it always will be here. There is also a tradeoff between stutter during those events and performance the rest of the time. Outside of those events, there are usually a lot of opportunities to cache some costly computation results instead of redoing them continuously. But this mean that when those results become invalid because the vessel state has changed, you need to perform all those calculations at once, inducing temporary stutter.

Anyway, I agree with the OP, I would love to hear some information about what the KSP 2 developments and plans are in regard to this stuff.
"Vessel physics" (be it RB physics, aerodynamics, thermal simulation, resource simulation...) are the main pillar of KSP, what make it distinct from any other game.
It's also the most technically difficult aspect. As a seasoned KSP player, I don't really care about the new planets or scifi engines, all I want is a KSP 2.0 that provide good solutions for those core features.

Edited December 13, 2021 by Gotmachine

[K^2]

22 minutes ago, Gotmachine said:

I don't see how you can keep joint strength limits if you merge everything into a single PhysX RB, unless you roll out a custom implementation for that.
At this point, if you have that custom implementation, you've already reproduced most of what PhysX is used for in KSP.

Not at all. You still have to do collision detection, extract contact points, and solve the constraints for remaining joints and contact points. All of that is still handled by PhysX. The only thing the custom implementation for RB does is take the constraint forces produced by PhysX and multiplies them by a pre-computed matrix to get the forces on the internal joints of the RB which you then test against strength limits. It's a tiny fraction of the full solver.

29 minutes ago, Gotmachine said:

This being said, rolling out a completely custom joint/RB physics implementation is no trivial task

This is a bit of an understatement. Less than a year ago, at a former job, I have been involved in evaluating Unreal's new Chaos solver against our in-house engine that I was a lead engineer on. We have found problems. Now, UE5 was not released yet, and this was advertised to us as WIP, and the bugs and issues were being rapidly fixed, and I'm sure it's in way better shape now. So I'm not trying to talk crap about UE5 here. But this was a physics engine years in the making with Epic's resources behind it and it was still in need of work. A good physics solver is a huge project.

Now, Chaos does a lot of things for a lot of different kinds of games. You don't need all of that for a game like KSP, obviously. You can cut a lot of corners and simplify a lot of things. Basic BVH-accelerated collision detection and a constraints solver are reasonably sized problems.  And if you were clever about how Jacobians for your constraints are generated, adding special constraints to handle wheels and landing gear isn't a huge amount of work. But it's still a good amount of work, and if you consider that Intercept has only about a dozen engineers to make the entire game, only one of whom is a physics engineer, making a custom solver does not seem feasible.

43 minutes ago, Gotmachine said:

My personal stance is that the KSP lego parts paradigm fundamentally can't result in a good "structural strength design" gameplay element.

I agree with that part. I think going with wobbly rockets was a mistake from the start, and they should have treated the entire structure as a rigid body. Early versions probably didn't even need stress limits simulation, especially not with the way old aerodynamics worked. It could have been added in later versions, causing rockets to simply fall apart, rather than start bending. The end effect would have been exactly the same. You could still have rockets undergo RUD due to aerodynamic forces if you lost control at high speeds or simply due to impact with terrain, but you wouldn't have to fight Kraken just to get your large rocket to orbit. This is the kind of thing that's challenging, but isn't fun.

But it seems that we're stuck with it. All of the footage we've seen of rockets under any kind of stress still show deformation consistent with PhysX weld joints. So it seems that this is still the design choice for KSP2.

[flart]

21 hours ago, Gotmachine said:

The KSP other "physics" subsystem are the main bottleneck : the aerodynamic and thermal simulations.

does the systems distinct enough for being able to disable them in the cheat menu in the ksp1?

[MechBFP]

11 minutes ago, K^2 said:

But it seems that we're stuck with it. All of the footage we've seen of rockets under any kind of stress still show deformation consistent with PhysX weld joints. So it seems that this is still the design choice for KSP2.

Have we actually seen anything along those lines some time after Intercept took over though? I can't recall.

[K^2]

36 minutes ago, MechBFP said:

Have we actually seen anything along those lines some time after Intercept took over though? I can't recall.

Yeah, there was a shot of a rocket firing engines relatively recently. Nowhere near as much bouncing as in early shots, but definitely some flex. Let me see if I can find the footage I'm thinking of.

Edit: Ok, "recently" might have been a stretch. I'm definitely thinking of the outro sequence of Next Gen Astronauts. You can see the boosters react to change in acceleration.

Spoiler

Time-stamped at 9:53

There is also a much easier to miss example in another part of this video, at 9:17 you can see two ships collide, with boosters on smaller one shaking as a result. Of course, collisions might be treated differently.

Also, notably, all the flexing we've seen here is at separators. So it is possible that separators are still treated as PhysX joints, while the solid welds between components aren't. That would be an interesting choice for optimization and one I'd be in support of. But we also see parts breaking apart in the collision video, meaning that either there is stress limits evaluation as described above in this thread, or the welds are treated as joints at least during the collisions. Both of these are plausible, but so is the possibility that everything is joints, just like in KSP, and they are just too stiff for us to see any flexing in these examples.

Edited December 13, 2021 by K^2

[Gotmachine]

1 hour ago, K^2 said:

Not at all. You still have to do collision detection, extract contact points, and solve the constraints for remaining joints and contact points. All of that is still handled by PhysX.

Yeah, I should have been more specific. I meant not using PhysX RB joints, but rolling out a custom "rigid but breakable" part joint implementation. There is zero justification for not using PhysX colliders and not reusing RBs CoM computations. In other words, KSP could be handling forces and part positions by itself, instead of relying on the wholly inadequate for the purpose PhysX joints. But yep, this would be a complex hybrid implementation, and as you said, it's unlikely they have the resources for a low return on investment project like that. KSP1 joint physics are from ideal, but they still kinda work from a gameplay perspective, are (unfortunately) a huge part of KSP perceived "identity" and again, there are other much lower hanging fruits in terms of performance optimization.

1 hour ago, flart said:

does the systems distinct enough for being able to disable them in the cheat menu in the ksp1?

I'm' not aware of any "kill switch". Those two systems are quite deeply interlinked, you can't really have one without the other. They are also quite strongly interlinked with the orbit solver, and many various subcomponents (not to mention mods). I doubt it's possible to disable meaningful pieces of code there without cascading side effects everywhere.

[MechBFP]

@K^2 pretty sure that outro at9:53 is KSP1. The earlier one at 9:17 is KSP2 for sure.  

It is definitely interesting that this hasn’t been shown at all since then though.

 

[K^2]

7 hours ago, Gotmachine said:

KSP1 joint physics are from ideal, but they still kinda work from a gameplay perspective

So the sad thing is that even despite coming from physics background and working on multiple titles, I used to think that PhysX and (now old) Havok is as good as these systems get, and you just have to eat the instability problems if you want dynamic joints. By "dynamic joints" I mean, something that's entirely defined by the constraints equations that can change - be they contact points, ragdoll joints, wheels, etc. But then I've worked on an engine that actually handles it properly, and now I'm just bemused that PhysX still hasn't fixed their stuff. Interestingly, I can trace pretty much all of the major changes in how people do physics engine to Erin Catto's work on Box2D and at Crystal Dynamics. Following that, I can point out at least Havok, Roblox, and Epic's Chaos as engines that have implemented similar approach and now have good joint simulation that doesn't suffer from nearly the level of stability problems that PhysX does. I'm guessing Blizzard's Overwatch engine is also benefiting from that, since Erin worked on it, but I don't know how much use they're getting out of constraints. Erin has done a lot of work to improve their dynamic BVH, though. In general, if you're interested in game physics, I'd definitely look up his talks at various game dev conferences.

Tangent aside, if PhysX got their stuff together and implemented proper constraint solver, even with using basic Unity welds, the KSP simulation would be way, way more stable. Alternatively, if KSP2 was to take a different route and use Havok physics for their components, but my understanding is that they initially wanted to recycle some of the KSP work in addition to Unity ECS, which is a requirement, being  still kind of raw when KSP2 development started. So I think we missed that particular train.