[WIP] KSP Threading System: Multithreading Utility, Now with Pre-release for Modders!

[ferram4]

This is a little project I started up after becoming frustrated at seeing KSP slow down even with CPU usage only at ~27%. The intent is to create a simple dependency that plugin authors can bundle (much like ModuleManager) that can be used to handle multithreading within KSP while maintaining sync with Unity's various functions (Update, LateUpdate and FixedUpdate). With this utility managing the worker threads, plugin authors don't need to worry about synchronizing-based issues and will only need to create thread-safe code in order to make good use of multithreading.

Further, due to the way it is implemented, it will allow plugin authors to start a task at the beginning of one of the Unity loops and end it at the very end of that loop; i.e. you can start a physics-related function at the beginning of FixedUpdate, let it run in another thread, and then it will sync up with the Unity thread at the end of FixedUpdate. Which means you don't have to stick to the main thread to keep things running in real-time as opposed to lagging by a frame.

Demonstration and Experimentation Pre-Release for Modders Only

Source

License

This is currently a work-in-progress and is certainly not ready for public release, given that it has not been heavily tested. However, the above version is available for any interested modders to try playing around with, and I certainly welcome feedback on this.

Functions to register with KSPTS:

object PreFunction()
{
//Do stuff that isn't necessarily thread-safe
//example: getting state for a PartModule that requires lots of Unity functions
return someObjectFromPre;
}

//This is the stuff we want somewhere other than the main Unity thread
object ThreadedTask(object someObjectFromPre)
{
//Do stuff that is thread-safe in parallel with main KSP thread
//example: doing aerodynamic calculations on data passed through using someObjectFromPre
return threadedTaskOutputObject;
}

//Runs after the threaded task is finished in main Unity thread; optional function used to do something with data
void PostFunction(object threadedTaskOutputObject)
{
//Do something that isn't necessarily thread-safe with the output of ThreadedTask
//example: taking results of aerodynamic calculations and applying forces to the relevant Part
}

//Runs first in main Unity thread; optional function used to set up data for threaded task

A note: the pre- and post-functions are not strictly necessary, since data can be accessed across threads, but it makes things cleaner and provides a place to run not-thread-safe code that is necessary for interfacing with Unity / KSP.

Example of how to use these sort of things

With the current state of this in mind, are there any plugin authors who would be interested in using something like this / sees related features that they'd like / thinks they have a better way of doing X, Y, or Z / thinks my naming scheme is bad and should change to something else?

Edited August 20, 2014 by ferram4

[mrBlaQ]

Oh man, I'd love to see MechJeb using this. The DV read-out just chugs. Thanks for your contribution, Ferram4!

[KAO]

I hope this project will gain the momentum and attention it should receive! I've been hoping for multi-threading since 0.15.2

[MK3424]

WOW! this really catches my eyes!

I wonder if the devs ever heard of LOOM?

(not the rubber bands ofcourse)

http://u3d.as/content/michiel-frankfort/loom-multi-threading-framework-v1-7/4fx

video example:

[ferram4]

Oh man, I'd love to see MechJeb using this. The DV read-out just chugs. Thanks for your contribution, Ferram4!

Ah, another use for this! I've noticed the dV readout being slow, and I suspect that it can be multi-threaded, but I think that getting the data to the thread might be a little bit complicated in that situation. Especially if it requires a large amount of operations that can't be multithreaded.

I hope this project will gain the momentum and attention it should receive! I've been hoping for multi-threading since 0.15.2

It won't be able to multithread the stock game, unfortunately, nor will it function without modders implementing code to make use of it. Multithreading is not magical. That said, hopefully it ends up as useful as I think it will be.

WOW! this really catches my eyes!

I wonder if the devs ever heard of LOOM?

(not the rubber bands ofcourse)

http://u3d.as/content/michiel-frankfort/loom-multi-threading-framework-v1-7/4fx

video example:

LOOM seems surprisingly similar to what I'm working on here, but it's quite a bit more general than this project. In any case, the same caveats that apply to LOOM apply here.

So an update on progress:

I switched from having dedicated thread groups for each loop (Update, FixedUpdate, LateUpdate) to instead having a single thread pool with a group of prioritized queues of tasks leading into it. This should allow it to take less memory overhead for threads while allowing greater throughput, while also allowing me to change priority as necessary to keep thigns running smoothly, such as giving all unfinished Update() tasks get highest priority into the thread pool as Update() finishes up.

[Tinweasele]

Scansat load and ORS resource detectors come to mind.

[Stevie_D]

That's a stunning idea, Ferram. The only drawback to DLing all the mods a person wants at the moment is the insane amount of slowdown lagging up the game. To change that would be to enable the modding community to really have a sense of "Have at it!" and not worry about concepts for our mods adding to other people's mods slowing everything down. Kudos! I REALLY hope you succeed with this, as a modder and more importantly as a KSP player.

[ferram4]

So, while it's certainly not battle-tested enough to allow full redistribution rights, it is likely stable enough for modders to start poking around and seeing what they can do with it. To that end, I've made a pre-release version available. I'd be quite happy to hear what modders think of this where it currently stands.

As for users, don't bother downloading the plugin, it is not useful to you alone; it requires exploitation by a mod before it can provide any benefit.

[R0cketC0der]

Nice project, I hope it gets implemented in many CPU-intensive mods. By the way I think you should clarify in the OP that this won't make any part of the stock game multithreaded and that it is purely meant as a resource for modders to enhance the performance of their mods, because I think some people might missunderstand it otherwise.

[ferram4]

It mentions that it's for plugin authors, and the entire OP focuses on how it allows plugin authors to do X, Y, and Z; if it was capable of rewriting code like that, I logically wouldn't need to mention to plugin authors anything.

I'm not concerned about that kind of misunderstanding, since it basically falls under, "didn't read the OP at all."

[Padishar]

Oh man, I'd love to see MechJeb using this. The DV read-out just chugs. Thanks for your contribution, Ferram4!

Ah, another use for this! I've noticed the dV readout being slow, and I suspect that it can be multi-threaded, but I think that getting the data to the thread might be a little bit complicated in that situation. Especially if it requires a large amount of operations that can't be multithreaded.

KER runs the deltaV simulation in a background thread (using ThreadPool.QueueUserWorkItem). Getting the data basically involves creating a copy of the part tree with all the data that may be needed by the simulation. This is done in the main thread during an Update call (if a simulation run has been requested in the UI handling function) and then the work item is queued. Once the work item completes it signals that the output data is available and the main thread reads the data (again in the Update call). I have a very rough library version of this "background operation" mechanism separated out but I've not touched it for months and it's quite a way from being publishable.

I switched from having dedicated thread groups for each loop (Update, FixedUpdate, LateUpdate) to instead having a single thread pool with a group of prioritized queues of tasks leading into it. This should allow it to take less memory overhead for threads while allowing greater throughput, while also allowing me to change priority as necessary to keep thigns running smoothly, such as giving all unfinished Update() tasks get highest priority into the thread pool as Update() finishes up.

I can certainly see the attraction of this sort of system for the sort of use you (presumably) intend it for. Firing off an aero calculation for each part during Update and then waiting for them all to have finished during LateUpdate (or possibly at the end of the Update loop as you describe) sounds ideal for FAR though is probably less useful for KER (or MJ) as the calculations can take significantly longer than a frame (sometimes many frames) and it isn't obvious how to parallelise the implementation of the simulation code. Have you integrated this into FAR yet?

So, while it's certainly not battle-tested enough to allow full redistribution rights, it is likely stable enough for modders to start poking around and seeing what they can do with it. To that end, I've made a pre-release version available. I'd be quite happy to hear what modders think of this where it currently stands.

Well, it sounds interesting and I'll definitely take a look due to a general interest in multithreading. The most awkward thing when doing multithreaded stuff in KSP/Unity is that you have to limit your interaction with KSP/Unity objects to the main thread or you risk all kinds of horrible bugs/crashes when things change underneath you. The PrepareSimulation code in KER is what does the building of the private data structures and it needs to access lots of core game objects; Parts, AttachNodes, PartModules (engines) etc. though it may be possible to get some of this off the main thread (and maybe split it across multiple cores) by careful splitting of it into smaller tasks that can be run in the background. This could potentially move quite a bit of the PrepareSimulation processing off the main thread though the extra overhead of the task queuing and the extra encoding of object reference information (e.g. part->part links) may defeat the point.

In terms of the simulation itself, there are a few things that could be parallelised but it will take considerable effort for an unknown benefit. E.g. when working out which "active" engines still have access to fuel, the recursive scan of the (private version of the) part tree could be done by a separate task for each engine, but, in general, most of the bits that could be parallelised in this way aren't very complex and the extra threading overhead would probably drown out any improvement.

[ferram4]

I can certainly see the attraction of this sort of system for the sort of use you (presumably) intend it for. Firing off an aero calculation for each part during Update and then waiting for them all to have finished during LateUpdate (or possibly at the end of the Update loop as you describe) sounds ideal for FAR though is probably less useful for KER (or MJ) as the calculations can take significantly longer than a frame (sometimes many frames) and it isn't obvious how to parallelise the implementation of the simulation code. Have you integrated this into FAR yet?

I haven't started refactoring FAR to be thread-safe to take advantage of this, and it would have to run its code in FixedUpdate, not Update (whether the results are returned at the end of FixedUpdate or if they're allowed to wait until the next FixedUpdate to take advantage of the unused processor time while the physics engine chugs is something I'll leave up to the user). I'd actually considered implementing this straight into it, but I figured that if anyone else wanted to make use of the system and duplicated the code we'd end up with the overhead of all the controllers and threads eating up any performance benefits.

Currently, I can see it having uses for the aerodynamics code in FAR, for some planned heat distribution code in DRE, for an overhaul to the way AJE calculates jet properties, probably some of the work that kOS does, and likely for some moderate GUI applications, where the calculations are heavy enough to warrant being sent to a thread pool, but are still needed in a timely fashion.

Well, it sounds interesting and I'll definitely take a look due to a general interest in multithreading. The most awkward thing when doing multithreaded stuff in KSP/Unity is that you have to limit your interaction with KSP/Unity objects to the main thread or you risk all kinds of horrible bugs/crashes when things change underneath you.

Which is really the frustrating thing with this. I'll have to do some experiments to see if there are any Unity classes / structs that can be used outside the main thread and in what forms, so that hopefully we don't have to scrap everything.

In terms of the simulation itself, there are a few things that could be parallelised but it will take considerable effort for an unknown benefit. E.g. when working out which "active" engines still have access to fuel, the recursive scan of the (private version of the) part tree could be done by a separate task for each engine, but, in general, most of the bits that could be parallelised in this way aren't very complex and the extra threading overhead would probably drown out any improvement.

Yeah, I can see that. If there's really no reason for the task to be tightly-synced with the main thread, you're probably better of implementing your own threading solution.

[rhoark]

I will use this so hard.

[Sean Mirrsen]

Ferram, for "calculations heavy enough to use another thread", have you considered taking a second look at raycasting or some other methods for detecting the geometry of the craft in FAR versus the airflow? I recall you trying it at one point, and it didn't work very well - could it work better with this?

[blizzy78]

One thing I don't really like about the implementation is that you don't seem to have to implement an interface. It just seems to work by using magic method names.

[ferram4]

Ferram, for "calculations heavy enough to use another thread", have you considered taking a second look at raycasting or some other methods for detecting the geometry of the craft in FAR versus the airflow? I recall you trying it at one point, and it didn't work very well - could it work better with this?

Depends. Lots of those things depend on if I can separate out the Unity classes and their methods, since those can't be used outside of the main thread.

One thing I don't really like about the implementation is that you don't seem to have to implement an interface. It just seems to work by using magic method names.

I'll admit that I'm somewhat confused by your complaint; this certainly doesn't work by using magic method names, you need to register each of the methods before they will be called, because I think Unity's magic method name stuff is silly. Would you mind pointing to where you'd like to see a change?

[Valerian]

I will definitely use that!

[S1gmoid]

So this means that before forking off to a thread, you have to create a deep copy in the main thread of any game data you want to use?

[Padishar]

So this means that before forking off to a thread, you have to create a deep copy in the main thread of any game data you want to use?

Well, yes and no. Basically, you need to be certain that any changes the core game might make to its data structures can't cause problems for your code. The easiest way to do this is to avoid accessing any Unity/KSP objects from other threads but, theoretically, this isn't the only way. E.g. if you know for certain that the main thread is currently running your code in, for example, FixedUpdate, then you can be sure that KSP and/or Unity aren't going to be changing any data and, as long as you stop accessing the data before the main thread leaves your FixedUpdate routine and goes back into KSP/Unity code, then everything should remain happy. However, this places very significant constraints on what you can do with multi-threading, basically reducing it to using multiple threads to speed up small bits of code that would otherwise be done by a single thread in a single plugin function. If you have considerable knowledge of the internal workings of Unity then you may be able to extend this usage depending on what KSP/Unity data you need to access but it is almost certainly much easier (and less prone to changes in KSP/Unity completely breaking your code in very hard to diagnose ways) to copy the data you require for your other threads to access.

[S1gmoid]

However, this places very significant constraints on what you can do with multi-threading, basically reducing it to using multiple threads to speed up small bits of code that would otherwise be done by a single thread in a single plugin function. If you have considerable knowledge of the internal workings of Unity then you may be able to extend this usage depending on what KSP/Unity data you need to access but it is almost certainly much easier (and less prone to changes in KSP/Unity completely breaking your code in very hard to diagnose ways) to copy the data you require for your other threads to access.

Hm... makes sense.

I am completely new to Unity... anyway, from my previous experience I'd say that since we're talking about a game, normally I'd want to stay synchronized to the main loop anyway so I can update in time for each Vsync... for truly "parallel running" logic (though I'd be hard pressed to name any right now), I guess it might be best to utilize these synchronization points as uplinks and downlinks, ie. update UIs and gather data for the long-running thread...

[NAMROG]

Sooo, this is a fairly old thing, but I recently found it searching through the forums. I'm kind of curious. If I wanted KSP to perform its physics on multiple threads, would there be a way to use this?

 

Or, better yet, could I modify FAR to use it?

Edited December 24, 2015 by NAMROG

[DMagic]

Perform what physics? The part modelling physics? No. 

This is a system for syncing thread-safe operations (most commonly math heavy operations) to the Unity update cycles.

[linuxgurugamer]

I can't believe I'm necro'ing such an old thread :-)

Anyway, I'm wondering if any mods use this, or if there is even a released version anywhere.  There isn't a released version on Github.

I need to add threading to at least one mod, (coroutines aren't cutting it for this mod)  and rather than reinventing the wheel, I'd like to use this.

@ferram4 The license currently prevents me from distributing this (it's an ARR).  Can we discuss this via PM if you like?

[smotheredrun]

On 3/26/2019 at 11:28 AM, linuxgurugamer said:

I can't believe I'm necro'ing such an old thread :-)

Anyway, I'm wondering if any mods use this, or if there is even a released version anywhere.  There isn't a released version on Github.

I need to add threading to at least one mod, (coroutines aren't cutting it for this mod)  and rather than reinventing the wheel, I'd like to use this.

@ferram4 The license currently prevents me from distributing this (it's an ARR).  Can we discuss this via PM if you like?

Please!!