Possible improvements to the physics system?

[FiendChain]

 

Jan Hloušek on Space Engineer's developer team has a rather interesting thread on various third-party physics engines. In particular if you scroll down, Havok physics 2022 seems much better than the rest of the alternatives. If KSP 2 is still using Unity's PhysX underneath, would it be beneficial to swap it out during early access to see if there are performance and stability improvements? 

What are people's thoughts on his thread?

[K^2]

This is all fairly well known. Well, at least if you work with physics engines. I won't go as far as saying that KSH wasted their time testing it, because it's always good to fact-check yourself, but the results are nothing new. Unreal's Chaos might be a new entry for some people, but I was actually doing similar testing for a different studio a couple of years ago, and we had reached similar conclusions. So on paper, yes, the new Havok, which is available in Unity, is a far better choice than basically any other off-the-shelf solution on the market, and all else being the same it would be a fantastic replacement for KSP2.

Unfortunately, switching KSP2 to Havok would likely require a significant overhaul of the game at this point. In particular, enabling Havok requires switching from GameObject model of Unity over to the ECS model. At the time that KSP2 would have been kicked off at Intercept, this was only supported as a preview, IIRC. ECS became a production-supported feature some time last year. If the game was only starting production now, I think it'd be an easy choice. Retrofitting KSP2 with Havok now, however, is a much trickier proposition.

In principle, converting the components themselves shouldn't be too much work. Looking at the saves and modules setup, there might actually be one main component script that makes calls to relevant modules based on what's hooked up for any given part. (So, like, an engine part will have an engine module hooked up in the script. And it's the engine module that will do all of the actual updates to make the engine work.) Alternatively, there could be a unique script for each type of module, in which case, there would be more scripts to convert, but even then, it should be fairly boilerplate on the surface. Lots of copy-and-pasting. The problem is that this will upset all the thread timings, as the GameObject scripts and ECS scripts are executed fundamentally differently by Unity. For GameObject components, the game objects are updated one at a time, each of its component scripts running in sequence. For ECS components, every type of component is updated together independently of the entity (part) that these components belong to. If built properly, that should actually improve the game's performance, as the workload is easier to split between threads under ECS, but if these threads start modifying shared data, the quick fix is mutexes... Long story short, you might end up with a significant performance degradation that you would then have to fix. Likewise, a lot of new crashes and memory leaks are likely to result from it, and these would also have to be fixed. The game's quality is likely to get worse, and might potentially require a huge amount of work to bring it back to even the state that the early access is in right now.

tl; dr: It's a huge risk. Even in the optimistic outcome, I'm not sure there is time for this work right now. In the worst case, if Intercept starts this, we'll get our next update some time next year. I don't have all the info to say for sure that this is impossible, but based on what I've read about this tech over the past year or so, it doesn't seem to be viable at the moment.

Once the game is shipped, and we get a few patches, assuming that the game does well enough for the work on it to continue, and if there is a quieter moment before Intercept has to ship the DLCs, there might be time to attempt switching from GameObject to ECS. If that's done, actually toggling on Havok is trivial. But that's a lot of conditionals, and best possible ETA of late 2024.

With all of that in mind, Intercept has to figure out how to make the game play sufficiently well with PhysX. I don't think there's a magic bullet approach that lets them bypass that.

[Gotmachine]

4 hours ago, K^2 said:

switching from GameObject to ECS.

That's not gonna happen. The GameObject entity/component is a fully OOP driven framework, whereas DOTS/Entities is a fully data-oriented paradigm with no support whatsoever for classes or reference types, and completely different workflows.
There is absolutely no conversion path possible without scrapping 100% of the existing game logic codebase, and doing an insane amount of rewiring for the rest.

Beside, they don't have to use the built-in PhysX or Havoc integration, it's perfectly possible to use any physics engine from Unity, they "just" have to build the interop layer for it.
It's relatively common to have Unity games plugging into C++ libraries for specific stuff (there is even an example in the KSP modding scene, the n-body integrator of Principia is written as a C++ library).
This would be vastly more doable than rewritting the whole game with the DOTS stack (and I would argue that DOTS Physics might actually be a better fit than Havoc in that case), especially since KSP 2 (at least currently without robotics) is using a relatively narrow subset of features (edit : actually, a major pain point would wheels and landing legs, those would require a lot of work instead of just being a thin wrapper over VPP).

Still, that would be a major undertaking, and the potential benefits are low, so I don't see how they can justify diverting dev resources to such an endeavor while they are still struggling to deliver a playable and commercially viable game.
And on a side note, while *some* issues of KSP 1/2 physics can be traced down to limitations of PhysX that would be alleviated in Havoc, a lot of them are purely on the KSP side, and some of them are *conceptual* issues rather than technical issues.

Edited April 30, 2023 by Gotmachine

[K^2]

11 hours ago, Gotmachine said:

The GameObject entity/component is a fully OOP driven framework, whereas DOTS/Entities is a fully data-oriented paradigm with no support whatsoever for classes or reference types, and completely different workflows.
There is absolutely no conversion path possible without scrapping 100% of the existing game logic codebase, and doing an insane amount of rewiring for the rest.

I haven't experimented with it myself yet, but the documentation and tutorials suggest that you can go mixed bag. You only need simulated parts to be in ECS. So all of the game logic, FX systems, UI, etc can be kept in GameObject format. And the way the game relies on modules, apparently for moddability, the GameObject scripts for the parts are merely invoking these, it would seem. The cross section of code that has to be replaced might actually not be that great.

11 hours ago, Gotmachine said:

Beside, they don't have to use the built-in PhysX or Havoc integration, it's perfectly possible to use any physics engine from Unity, they "just" have to build the interop layer for it.

That might actually be more work. If you go this way, you have to manage the collision scene pretty much manually. Anything with collisions will have to introduce or remove itself from the scene. Dynamic objects will have to update their possitions on every game tick. Terrain's LOD and streaming will have to be tightly integrated into this setup. Any FX that might need collision data will have to be completely redone, and any game logic that uses collisions re-written, which includes wheels, lander legs, kerbals, and colony placement and editor. Some of it is still going to have to be touched even with ECS Havok, but with custom Havok it's basically everything.

It's an option, but I'm not convinced it's an easier option. And same constraints on time would certainly apply.

[Gotmachine]

12 minutes ago, K^2 said:

the documentation and tutorials suggest that you can go mixed bag

You can go mixed bag, but it's a quite delicate setup even if you plan your design around it.

14 minutes ago, K^2 said:

all of the game logic [...] can be kept in GameObject format

No. KSP game logic is heavily tied to parts, and involves physics engine queries all the time. That would be a nightmare to put put together and maintain.

16 minutes ago, K^2 said:

And the way the game relies on modules, apparently for moddability, the GameObject scripts for the parts are merely invoking these, it would seem.

Modules are actually part components, nothing more. They tried to decouple their core logic from the Unity GameObjects / Monobehaviors with some sort of MVC pattern, but in practice there is very high level of coupling between everything.

20 minutes ago, K^2 said:

The cross section of code that has to be replaced might actually not be that great.

Well, all I can say is that I peeped long enough into said code to be quite doubtful of that.

52 minutes ago, K^2 said:

colony placement and editor.

hahahahahahahahahahahaha deep breath hahahahahahahahaha

23 minutes ago, K^2 said:

you have to manage the collision scene pretty much manually [...]
Dynamic objects will have to update their positions on every game tick

They more or less already do that anyway, for various reasons, but mostly because of the floating origin. While PhysX actually has support for a floating origin, the Unity wrapper doesn't expose it, so they already are repositioning everything manually at regular intervals (and tracking the position of everything independently as well).

As I said, there is actually not that much relying on the physics engine in KSP (be it 1 or 2). It's used for a bunch of spatial queries, applying their independently computed thrust/atmospheric forces to the jointed rigidbodies and getting the frame velocity out of it, terrain and inter-vessel collisions, and that's it for the most part. The only thing that I can think of that would require them to roll out a significant new subsystem is the wheels/suspensions because currently they are relying on a third party asset dependent on the Unity PhysX wrapper.

This is still is a major undertaking, but I would estimate on the same order of magnitude as swapping everything from using the built-in render pipeline to the HDRP pipeline, which they have apparently decided to do.
The difference is that swapping the rendering pipeline is part of a necessary refactor to address the huge issues they have with rendering performance, while swapping the physics engine wouldn't bring much benefit as long as they stick with the idea that the un-physical KSP 1 behavior of part connections is a design requirement.
And arguably, if they somehow have the good idea to throw it away and actually think about something that make sense from a game design perspective, there are other approaches in terms of physics that are better fitted to create elements of structural integrity gameplay, in the realm of beam / soft-body solvers.

[K^2]

3 hours ago, Gotmachine said:

Well, all I can say is that I peeped long enough into said code to be quite doubtful of that.

If you've taken a deep dive into how it's currently set up, I'm prepared to take your word for it.