You can't effectively use a GPU and not be parallel - so yes, the GPU portion of PhysX runs in parallel. The complication is that you need to be parallel in a specific way. Although every CPU-bound task in modern games can be parallelized, they can't all be parallelized effectively in the way required by GPUs. Some tasks still require CPU-style code execution.
For tasks that run nicely on GPUs the performance benefit of using a GPU is significant, but quite as drastic as it seems. An RTX 2080 Ti gives you a max theoretical performance of ~14 teraflops @ 250W. An AMD Epyc 7742 gets you ~3 teraflops @ 225W. That's a factor of five.
GPUs are really good at vector operations, when a bunch of tasks are doing exactly the same thing on adjacent data. They're bad at indirection, for example in object oriented programming, or anything that involves non-adjacent data. They're also bad at small tasks - if you don't have a 100,000 items to process, it's not worth bothering the GPU with it. The algorithms I describe above would probably work fine on a GPU depending on a bunch of details, but your ships in KSP probably don't have enough parts to be worth bothering the GPU with - with a parallel algorithm a decent CPU will be faster on even thousands of discrete parts.
With a CPU-parallel physics engine, KSP2 could definitely get us 4x the performance on 8 core CPUs over KSP1, or 20x the performance for people gaming on top of the line server CPUs. With a GPU physics engine, they might be able to get us 50x the simulation performance of KSP 1 for users who have a high end dedicated physics GPU.