Background Processing 0.4.4.0 [KSP 1.1.3]

[jamespicone]

EDIT: Now has a SpaceDock page: http://spacedock.info/mod/288/BackgroundProcessing. The SpaceDock page is the best source of information on what the mod currently does and how to use it.

I'm working on a little mod/library to make it easier to write PartModules that do something when a vessel is not being actively controlled by the player - say, because they've left it landed or orbiting and gone back to the space center, or undocked it, whatever. For an example of mods that currently do something like this, SCANSat's scanners can continue mapping a planet even when the vessel with the scanners on them is in the background. I'm also intending to fake resource handling for certain resources while vessels are backgrounded - ElectricCharge in particular. Essentially I'm replicating some of the functionality of ASMI's RealTimeModules with a more convenient license (MIT).

The source is here, but keep in mind that this is early days and the mod doesn't do much of use for players yet, and is only just usable by mod authors for backgrounding. I'm posting here because it has, at least, gotten to the point where as a mod author you could theoretically use it and it would be theoretically useful.

It works like this: It creates a MonoBehaviour at startup, BackgroundProcessing.Addon, with a RegisterHandler method, and provides an interface, IBackgroundHandler. PartModules that want background processing should, in their OnAwake method, create a new, static object implementing IBackgroundHandler (that is, there is one IBackgroundHandler per PartModule /type/), and should then call BackgroundProcessing.Addon.Instance.RegisterHandler(). Once again, that's all in OnAwake - don't register handlers after that.

Then, during play, if the background processing library picks up a backgrounded vessel, it calls IBackgroundHandler.FixedBackgroundUpdate() on the appropriate handler for every partmodule in the vessel, passing in a reference to the vessel (which may or may not be all there - be very careful with it. It won't be null though), and a part flightID which will allow the handler to determine exactly which partmodule it was that needs handling. It's extremely simple.

The next step is to implement some handling for electric charge in the background.

EDIT: The method mod authors should use to interface with this library/mod has changed. Right now, on startup, the mod checks all PartModules in the system to see whether they have certain methods. If they do, those methods are called at the appropriate times. If you want your PartModule to perform any kind of processing when the Vessel it is on is out of load range, define:

 

class MyPartModule : PartModule {
  public static void FixedBackgroundUpdate(Vessel vessel, uint partFlightID);
}

it'll get called at fixed intervals with the relevant vessel and the partFlightID of the part that owns the module that triggered the background call (so you'll get multiple calls per vessel if there are multiple PartModules with backgrounding on the same vessel). Think of it as FixedUpdate for when you don't exist.

If your PartModule should produce or consume a resource in the background, or if your mod wants a particular resource handled in the background, you can do that too. Override these methods:

 

class MyPartModule : PartModule {
  public static List<string> GetInterestingResources();

  public static int GetBackgroundResourceCount();
  public static void GetBackgroundResource(int i, out string resourceName, out float rate);
}

GetInterestingResources() should return a List<string> containing the resource name of every resource your PartModule thinks should be handled in the background. ElectricCharge is an interesting resource by default, no others are. For example, a hydrogen ramscoop PartModule that sucks up interstellar hydrogen from a future-tech mod that has a Hydrogen resource might return {"Hydrogen"}.

GetBackgroundResourceCount() and GetBackgroundResource() form a pair used to indicate that your PartModule produces or consumes a particular resource in the background consistently - either your PartModule is a constant drain or some kind of generator. GetBackgroundResourceCount() should return the number of different resources your PartModule produces in the background, and GetBackgroundResource() should return the resource name and the rate (in units per second) that that resource is produced for the i-th resource your PartModule produces. For example, the hydrogen ramscoop PartModule I just mentioned would look like this:

 

public static int GetBackgroundResourceCount() {return 1;}
public static int GetBackgroundResource(int i, out string resourceName, out float rate) {resourceName = "Hydrogen"; rate = "0.001f";}

I'm currently working on providing some methods backgrounded PartModules can use as an analog to RequestResource, so that the ramscoop above could be speed-dependent, for example. Also bugfixes, improving some code, making solar panels behave properly, cleaner timewarp handling, etc. etc.

Edited August 3, 2016 by jamespicone

[KAO]

do you think this system will be able to implement orbital decay?

[duckofthemoon]

do you think this system will be able to implement orbital decay?

KAO asking the real the real questions.

[Drew Kerman]

The next step is to implement some handling for electric charge in the background.

Yes! Currently I just test that my probes can successfully operate under any darkness conditions in a particular orbit and then just assume they always contain enough electrical charge. This works fine for stuff like comms sats that I never really switch to but for an active probe, it's not very realistic for it to have full battery charge if I switch to it halfway through a night transition and want to run experiments. Feels a bit cheaty and I generally back out of the game, edit the SFS and come back to it.

[jamespicone]

do you think this system will be able to implement orbital decay?

This probably isn't the right way to handle it. Orbital decay isn't a partmodule thing, and it's something you want every vessel to experience, not just those in those in the background. It could certainly be worked into it, but I don't plan to do it.

I'm not actually sure how possible it is - don't know how KSP handles modifying orbital parameters for backgrounded vessels.

[Unit327]

Looks like it could be quite handy. I see you are going through every part on every vessel on every physics frame (FixedUpdate), do you have an idea of the performance impact that will have?

[jamespicone]

Looks like it could be quite handy. I see you are going through every part on every vessel on every physics frame (FixedUpdate), do you have an idea of the performance impact that will have?

I'm only going through parts that have registered a background callback every FixedUpdate, not every part (but yes for every vessel). I'm not sure what the performance implications will be in actual use. Mostly that's an implementation detail - this is early-days yet, and I'd prefer to get the resource stuff going before I look at setting up a more delta-t-based update system.

[Ruedii]

Is there a way you could make this do things on timers or triggers configured by the config file. For instance causing something to occur automatically every 5 minutes of gameplay.

Another idea is one that can attach a behavior of a part to any routine in the game, for instance a light that changes color when the craft changes state to touched down.

To be able to do this from config files would be an incredibly powerful tool.

[jamespicone]

Some resource-fakery is implemented and functioning. Not in a way that would be useful to players, as of yet - the modules to fake resources for and how much to fake are hardcoded and wrong, just for testing - but the code is there and as far as I can tell works.

Some thought needs to go into how to represent fakery well. ElectricCharge is really the only resource in stock KSP that's sensible to handle like this, which at least reduces the space, but I still don't really want to hardcode handling for ModuleDeployableSolarPanel, ModuleGenerator, etc. to extract the rate and whether or not its actually relevant in the case ModuleGenerator. I'm going to need to consider a bit but I think there's going to need to be a config file that specifies what modules to fake resource handling for, what resource they generate, and any ConfigNode values to check to get rate information, relevancy information, power-curve information. Also going to need to be some flags to specify whether relevancy/power curve is appropriate. If done properly this should be extensible to arbitrary resources defined by mods.

There aren't that many things that drain ElectricCharge over time, but they should have moderately simple configs AFAICT - module name, resource name, config node for rate.

Is there a way you could make this do things on timers or triggers configured by the config file. For instance causing something to occur automatically every 5 minutes of gameplay.

Another idea is one that can attach a behavior of a part to any routine in the game, for instance a light that changes color when the craft changes state to touched down.

To be able to do this from config files would be an incredibly powerful tool.

That's not really the goal here, although yes that would be powerful. I'm just aiming at getting vessels that aren't currently controlled by the player to execute some mod-relevant code, and to fake some resource handling for them. Having individual background modules specify the timing they want to get called on could be a thing, but I'm not sure it offers any real benefit over calling into them as fast as performance allows and letting them delta-time it out.

[jamespicone]

Doubleposting to note that as of right now I have functioning backgrounded resource handling for anything using ModuleGenerator or ModuleDeployableSolarPanel, although I don't currently do anything with power curves or shadows for solar panels (they're just assumed to output at constant output all the time). I also don't have any resource consumption backgrounded yet - really the only relevant thing in stock KSP are the probe bodies. Keep in mind I'm only background-handling ElectricCharge, but the current system is resource-agnostic - you can ask it to do anything.

In fact it's got all the appropriate hooks to allow arbitrary PartModules to request arbitrary resources (including mod-defined ones) be background-handled, as long as a partmodule can tell me what the resource is and what rate it consumes or produces it at.

I've changed the way background methods are found and hooked onto so that mods using this library don't actually need it to compile or run. You can think of it as an optional extra - mods can support background processing for their partmodules, and then if the BackgroundProcessing mod is present they will get processed in the background.

[jamespicone]

Tripleposting to note that BackgroundProcessing, as of now, is considered usable. The initial post in this thread has the link to KerbalStuff if you want the mod or the mod source, and instructions for mod authors who want background processing in their mod.

As of right now solar panel powercurves and kerbol-visibility is handled, although orientation isn't (all panels are considered optimally-oriented). Also PartModules can now request resources during a background update.

[FreeThinker]

So the current version should update my satelites powers level. Perhaps finally those batteries get some real use!!!

Does it take into account the satelite facing to the sun?

Now it would be nice if I could somehow monitor the power level of all my satelites ...

Edited October 29, 2014 by FreeThinker

[jamespicone]

So the current version should update my satelites powers level. Perhaps finally those batteries get some real use!!!

Does it take into account the satelite facing to the sun?

Now it would be nice if I could somehow monitor the power level of all my satelites ...

Yes. It also takes into account obscured solar panels, panel tracking, and the power curve where distant solar panels are less effective. One caveat - solar panels that are blocked from the sun by the vessel itself or another backgrounded vessel probably won't be considered obscured, as far as I can tell the colliders for the vessel's parts aren't present. That's nontrivial to fix and hopefully shouldn't matter too much.

[Kerbas_ad_astra]

KSP itself doesn't notice if e.g. one set of solar panels is shaded by another set when the vessel is active, so a vehicle's behavior won't change if you don't implement that feature into BackgroundProcessing. I've got a probe flying "sideways" to the Sun with solar panels deployed in an X-configuration -- even though the two panels closer to the Sun are clearly casting shadows on the panels further away, I'm still getting power from all of them.

[dtoxic]

just encountered this warning, nothing crashed or got broke thou but still maybe you want to take a look at it

using ksp 0.25 x32 with Lionhead Probes pack mod

(Filename: C:/BuildAgent/work/d63dfc6385190b60/artifacts/StandalonePlayerGenerated/UnityEngineDebug.cpp Line: 49)

BackgroundProcessing: PartModule/ProtoPartModuleSnapshot sync error processing part LH.Hermes. Something is very wrong.

and the logs

https://www.dropbox.com/s/3f1bh2psmkqt54e/KSP%200.25-BP.rar?dl=0

hope it helps

[jamespicone]

That error pops up when the code can't match modules between the protovessel for a vessel and the actual instantiated vessel. It could be trouble if there are background-processing-using modules present on the vessel doing that. Fortunately, I'm not aware of any modules using it yet. I'll have to look into the Lionhead Probes pack mod to see what they're doing, but it looks like at the very least the mechanism I use to try and match the modules doesn't quite work and I'll have to be cleverer.

Thanks for the report and for providing the logs.

[dtoxic]

That error pops up when the code can't match modules between the protovessel for a vessel and the actual instantiated vessel. It could be trouble if there are background-processing-using modules present on the vessel doing that. Fortunately, I'm not aware of any modules using it yet. I'll have to look into the Lionhead Probes pack mod to see what they're doing, but it looks like at the very least the mechanism I use to try and match the modules doesn't quite work and I'll have to be cleverer.

Thanks for the report and for providing the logs.

no problem if you need anything else just drop me a line

[jamespicone]

Alright, finally got around to having a look at this.

I suspect there's some weird interaction with ModuleManager going on. Earlier on your log has these lines, from, as far as I can tell, KSP's internal loading code:

[WRN 16:48:06.152] [Part]: PartModule indexing mismatch at LH.Hermes, index 4.

Node 'ModuleScienceExperiment' found in loaded data, but 'ModuleDataTransmitter' is defined in prefab.

Looking for ModuleScienceExperiment in other indices...

[WRN 16:48:06.152] ...ModuleScienceExperiment module found at index 5.

That's the same problem that I'm running into. Is it possible that this vessel was loaded with a different set of ModuleManager configs from when it was saved, or something like that? Does the same thing happen if you just make a new vessel of the same kind on the pad?

I've just updated BackgroundProcessing to have a newer method for trying to sync up partmodules and protopartmodulesnapshots. Could you try version 0.2.5, from KerbalStuff, and see what log messages you get?

Thanks.

[dtoxic]

Alright, finally got around to having a look at this.

I suspect there's some weird interaction with ModuleManager going on. Earlier on your log has these lines, from, as far as I can tell, KSP's internal loading code:

That's the same problem that I'm running into. Is it possible that this vessel was loaded with a different set of ModuleManager configs from when it was saved, or something like that? Does the same thing happen if you just make a new vessel of the same kind on the pad?

I've just updated BackgroundProcessing to have a newer method for trying to sync up partmodules and protopartmodulesnapshots. Could you try version 0.2.5, from KerbalStuff, and see what log messages you get?

Thanks.

hey cool will let you know what i found

on the other hand i do use MM patch to modify that part LH.Hermes...(RemoteTech) does it and is the only one that i know of, that modifies the part

will give the new version a go see what i find

======================================

just tried it and now the log get's spammed with nullref's

NullReferenceException: Object reference not set to an instance of an object

at BackgroundProcessing.Addon.GetVesselData (.Vessel v) [0x00000] in <filename unknown>:0

at BackgroundProcessing.Addon.FixedUpdate () [0x00000] in <filename unknown>:0

here are the logs

https://www.dropbox.com/s/ral559h6ohzegw6/BP-Null.rar?dl=0

note: i did not use the same vessel just a random one build, but i don't think that makes any difference

if you need anything more let me know

Edited December 5, 2014 by dtoxic

[toadicus]

I'm considering using this for a future update and have a few questions.

As of right now solar panel powercurves and kerbol-visibility is handled, although orientation isn't (all panels are considered optimally-oriented). Also PartModules can now request resources during a background update.

Yes. It also takes into account obscured solar panels, panel tracking, and the power curve where distant solar panels are less effective. One caveat - solar panels that are blocked from the sun by the vessel itself or another backgrounded vessel probably won't be considered obscured, as far as I can tell the colliders for the vessel's parts aren't present. That's nontrivial to fix and hopefully shouldn't matter too much.

These two answers appear to conflict. Did something change? If a vessel's sun-facing is considered as suggested by the second answer, is some form of stabilization implemented and/or possible, to maintain an optimal orientation with relation to the sun?

I see you have the dll packaged without any path, ala ModuleManager. The current state of the code seems to contradict the OP which suggests that IBackgroundHandler is provided and should be implemented, so I assuming a need to link against your DLL is out. Is the intent for distribution for mods to package your DLL and link to your thread / KerbalStuff entry ala ModuleManager, or do you want to distribute it yourself?

[jamespicone]

I'm considering using this for a future update and have a few questions.

These two answers appear to conflict. Did something change? If a vessel's sun-facing is considered as suggested by the second answer, is some form of stabilization implemented and/or possible, to maintain an optimal orientation with relation to the sun?

I see you have the dll packaged without any path, ala ModuleManager. The current state of the code seems to contradict the OP which suggests that IBackgroundHandler is provided and should be implemented, so I assuming a need to link against your DLL is out. Is the intent for distribution for mods to package your DLL and link to your thread / KerbalStuff entry ala ModuleManager, or do you want to distribute it yourself?

I haven't kept the OP here up to date. The KerbalStuff page is the current description of how to use/what it does. At present, solar panel orientation is handled, both for static solar panels and ones that can swivel. Swivellers swivel to the optimal position given craft orientation, static panels just get as much as they get given the craft orientation, I don't attempt to change the orientation.

I haven't looked into changing vessel orientation with respect Kerbol while off-rails to maintain solar panel output, but it should be possible.

No need to link into the DLL any more. The intent is that the mod should act as an 'extra' - mods can implement that static methods mentioned in KerbalStuff/the edit at the bottom of the OP (Read from "The method mod authors should use to interface with this library/mod has changed."). Then, if BackgroundProcessing is present, they get on-rails processing.

I haven't written any code to detect and prevent multiple instances running, so if distributing with your mod, it's probably a good idea to get people to put it in the root directory of GameData and have them overwrite if it's present.

[jamespicone]

hey cool will let you know what i found

on the other hand i do use MM patch to modify that part LH.Hermes...(RemoteTech) does it and is the only one that i know of, that modifies the part

will give the new version a go see what i find

======================================

just tried it and now the log get's spammed with nullref's

NullReferenceException: Object reference not set to an instance of an object

at BackgroundProcessing.Addon.GetVesselData (.Vessel v) [0x00000] in <filename unknown>:0

at BackgroundProcessing.Addon.FixedUpdate () [0x00000] in <filename unknown>:0

here are the logs

https://www.dropbox.com/s/ral559h6ohzegw6/BP-Null.rar?dl=0

note: i did not use the same vessel just a random one build, but i don't think that makes any difference

if you need anything more let me know

Okay, I'm kind of confused. The only way that should happen is if there's a part on a vessel that doesn't have a prefab... I've added the missing check that was causing the NRE, there's another version on Kerbalstuff, 0.2.6. It might not help.

You have kind of a significant amount of mods installed, which makes it tricky for me to test this stuff here. I've tried just a ModuleManager/RemoteTech install together and they don't seem to be causing any issues over here.

[dtoxic]

Okay, I'm kind of confused. The only way that should happen is if there's a part on a vessel that doesn't have a prefab... I've added the missing check that was causing the NRE, there's another version on Kerbalstuff, 0.2.6. It might not help.

You have kind of a significant amount of mods installed, which makes it tricky for me to test this stuff here. I've tried just a ModuleManager/RemoteTech install together and they don't seem to be causing any issues over here.

ohh ok didn't know it was depending on how many mod's one has installed (but then again i am no coder so )

ill give it a run in moded and clean install see what happens and then report here

=============================

still no luck, thou now there is far less spaming in the log

Edited December 6, 2014 by dtoxic

[philotical]

this is a cool project..

will be heplfull for many things..

I've read about the performance question with the loops on every frame..

in my experiance, you can easely reduce it to once every 0.2 seconds..

you will not "feel" muh difference but gain a lot performance..

especially if there are many vessels in the loop..

just my 2 cents..

[jamespicone]

ohh ok didn't know it was depending on how many mod's one has installed (but then again i am no coder so )

ill give it a run in moded and clean install see what happens and then report here

=============================

still no luck, thou now there is far less spaming in the log

Each mod you run is another thing my mod is interacting with - if I do something they don't expect, or vice-versa, it can cause problems.

Far less spamming in the log is good, but what's currently being printed?