[Brainstorming] Multi-threaded plugins

[ialdabaoth]

So, I'm about to delve into my first foray with multi-threaded Unity development. Here's my roadmap so far:

Mission Plan

Create a MonoBehaviour which will spawn a second thread, and synchronize between KSP and that thread.

The MonoBehavior will contain a MemoryStream for passing information between another Part, PartModule, or MonoBehaviour and the spawned thread during OnStart, and will contain a ConcurrentDictionary<PartUID, struct T> for synchronizing information during FixedUpdate.

Use Case

Ferram Aerospace Research would like to do a lot of really complex calculations during FixedUpdate without impacting performance. So, ferram4 creates an app using my ThreadedKSP API.

He defines a new ThreadedMonoBehaviour called "AerodynamicCalculations", a new PartModule called PartAeroFX, and a new struct called AeroForces.

AeroForces looks like this:

{
 int vesselID;
 float timestamp_out;
 float timestamp_in;
 Vector3D atmVelocity;
 float atmDensity;
 float atmTemperature;
 bool atmCompressible;
 Vector3D dragForce;
 vector3D partCoD;
}

AerodynamicsCalculations has the following methods and fields exposed:

int AerodynamicsCalculations.StreamToThread(MeshDataStruct meshData);
ConcurrentDictionary<PartUID, AeroForces> AerodynamicsCalculations.UpdateData;

And then PartAeroFX has the following behavior:

- PartAeroFX.OnStart(StartState.FLIGHT) grabs the Part's entire mesh, builds it into a struct that he decides to name meshData, and shoves it through the MemoryStream by calling AerodynamicCalculations.StreamToThread(meshData).

- PartAeroFX.Update then does:

 if(this.lastUpdate < Aerodynamics.UpdateData[this.part.uid].timestamp_in)
 {
   this.lastUpdate = Aerodynamics.UpdateData[this.part.uid].timestamp_in;
   part.rigidBody.AddForceAtPosition(Aerodynamics.UpdateData[this.part.uid].dragForce, Aerodynamics.UpdateData[this.part.uid].partCoD);
 }

- PartAeroFX.LateUpdate then does:

 Aerodynamics.UpdateData[this.part.uid].vesselID = vessel.rootpart.uid;
 Aerodynamics.UpdateData[this.part.uid].atmVelocity = part.rigidBody.velocity + Krakensbane.GetFrameVelocity();
 Aerodynamics.UpdateData[this.part.uid].atmDensity = vessel.atmDensity;
 Aerodynamics.UpdateData[this.part.uid].atmTemperature = vessel.flightIntegrator.getExternalTemperature();
 Aerodynamics.UpdateData[this.part.uid].atmCompressible = CheckUnderwater(part);
 Aerodynamics.UpdateData[this.part.uid].timestamp_out = Planetarium.fetch.time;

Meanwhile, the thread is spinning merrily away, using all that data to calculate the next dragForce and partCoD for each part, based on the last available UpdateData[part_uid]. Whenever it finishes calculating those values for a part, it just updates UpdateData[part_uid].timestamp_in, and that part will asynchronously apply the force on the next Update!

This technique should be useable for just about anything that needs to do a lot of heavy calculation; since the thread doesn't recognize any Unity objects, it should ignore Unity's lack of thread-safeness.

Edited May 11, 2014 by ialdabaoth

[Faark]

Afaik Unity doesn't come any concurrent collection, so you have to provide them based on MS's reference implementation (not a real problem). I not yet sure what exactly you try to accomplish. Its probably not worth the effort to re-write a lot of unity's engine just for multi threading. To parallelize stuff, organize the code into 3 parts... a first that collects all data from Unity&KSP, a second that does the calculation but doesn't touch KSP at all and a last that consumes the resulting data. Sure, a helper class for stuff like synchronization might be useful, but most of the work would have be done manually anyway by one of the three code parts.

Since we are talking about FAR... what does consumes the most processing time of it in the first place? I always thought those were raycasts and that they are impossible to parallelize (unless Unity already does / allows it / you replace unity). Stuff like nbody physics is ofc an entirely different story.

[Padishar]

I'm not sure what purpose the memory stream really serves in your description other than to slow it down.

KER performs the bulk of the fuel flow simulation to calculate deltaV in a different thread by simply calling System.ThreadPool.QueueUserWorkItem(WaitCallback callBack, object state). This allows you to call a static function and pass an object instance to it. KER creates a Simulation object in the main thread that copies all of the data it requires from the core data structures into its own private structures and then passes this object to the function run in the background. The Simulation generates an array of Stage objects containing all the results which the main thread reads once it is safe to do so.

The code is in SimManager.cs and BuildEngineer.cs.

I have thought about writing a library that encapsulates (and improves on) this mechanism for others to use but have not yet got around to it.

Edited May 11, 2014 by Padishar
Changed links to point at Cybutek's repo rather than mine as he originally wrote that bit

[Padishar]

If your background thread happens to be reading the values from Aerodynamics.UpdateData[this.part.uid] when PartAeroFX.LateUpdate is halfway through updating them then the calculations could be messed up. This updating of the data the background thread is reading must be made atomic and synchronised.

In the case of FAR's aerodynamics calculations, the mechanism you describe will be calculating the drag for different parts of the vessel at slightly different points in time and hence with slightly different input parameters. I suspect this will introduce some odd errors into the calculations that will result in undesirable forces on the vessel (e.g. one wing producing more lift/drag than the other). Also, what will happen when a part's mesh changes (e.g. a solar panel)? You would somehow have to detect and pass the change to the thread.

It is certainly an interesting idea but I think you will definitely have issues if the calculations take longer than a frame. If the thread simply sat in a loop and processed items from a queue and stuck them in an output dictionary then you could make each part submit its data in Update and continue. The background thread would wake up when it sees some data in the queue and start processing. This processing would go on in parallel with the core calling the Update of each part. Then in LateUpdate (I presume, I'm not that hot on the KSP/Unity specifics) you would wait for the data for the part to be ready before reading it and continuing. It would also require a cache of your meshData structures that are passed into the queue every time so that they can be updated easily when the part meshes change. It would be possible to share the input queue and output dictionary between multiple background threads if they were appropriately synchronised.

It won't really help in this case but my now work-in-progress job manager allows a plugin to create a job object which includes the input and output parameters and pass this object to the job manager for running in the background. There are mechanisms to determine if a job has completed and for the job to provide progress information which the plugin can safely read (for very long jobs). It is intended that multiple plugins can share the job manager so it will require a hard dependency on the job manager plugin to use it so there will only be one copy of the code running. As soon as I'm fairly sure the interface is reasonably stable I'll put it up on GitHub...