[1.1.3] BackgroundProcessing

[FlowerChild]

I'm an idiot, and the test case I was looking at wasn't the complex FixedBackgroundUpdate, which I indeed broke (it was expecting a delegate type thing, not a Func<Vessel, float, string float>). Should be fixed in 0.4.0.1, up on kerbalstuff right now.

Woot! Not a problem man. Glad we managed to track it down

Seriously, thanks for all your ongoing support with this. Will download and try it out right away.

EDIT: And works like a charm. Double "Woot!"

Edited February 7, 2015 by FlowerChild

[FlowerChild]

Have a question about the following function:

public static List<string> GetInterestingResources()

Under which circumstances should this be implemented? Does it have any impact if we are implementing our own FixedBackgroundUpdate() handler, or should it only be used with the GetBackgroundResourceCount() and GetBackgroundResource() way of doing things? What is it actually intended to do?

Looking over the BP code it seems to only impact the stock module implementation (solar panels, generators, etc.), so I'm wondering under what circumstances it should be applied to a custom one.

And a small comment on these functions:

public static int GetBackgroundResourceCount()

public static void GetBackgroundResource(int index, out string resourceName, out float resourceRate)

I just tried using these for a simpler custom module implementation and quickly realized the utility of them is extremely limited due to not having access to any config node information at all. With the current parameters the only thing I could see these being used for is if you have a module that produces or consumes resources at a fixed rate regardless of circumstance (which is what I do have) that also has its production rate hard coded to a specific value within the module itself (which I don't), with no option to have a constant rate parameter even specified in the config file for the part.

Not sure what the solution would be there as I think it would rapidly turn into the equivalent of doing things through FixedBackgroundUpdate() if stored data access was added or what have you, but I thought it worth point out that the actual use cases for these functions may currently be limited beyond the point of them being helpful to all but a very narrow set of circumstances.

Maybe if the partModule from the part prefab were passed in as a parameter it would become more useful so that the function could access stuff like the default consumption or production rates set for the module? Assuming of course that GetBackgroundResource() is called form a portion of the BP code that has access to it.

Edited February 8, 2015 by FlowerChild

[jamespicone]

Have a question about the following function:

public static List<string> GetInterestingResources()

Under which circumstances should this be implemented? Does it have any impact if we are implementing our own FixedBackgroundUpdate() handler, or should it only be used with the GetBackgroundResourceCount() and GetBackgroundResource() way of doing things? What is it actually intended to do?

Looking over the BP code it seems to only impact the stock module implementation (solar panels, generators, etc.), so I'm wondering under what circumstances it should be applied to a custom one.

It should be implemented iff any of your partmodules believes that a resource should be processed in the background by BP from ModuleGenerators/ModuleCommand/ModuleDeployableSolarPanel parts. Say you're writing a life support mod, and part of that mod is that you have some Parts with PartModules that generate "oxygen", you might return List("Oxygen") from GetInterestingResources() and then any ModuleGenerator that outputs oxygen with no inputs will output oxygen in the background. (and any ModuleCommand that consumes oxygen will consume it, but that probably wouldn't happen).

It doesn't interact with FixedBackgroundUpdate at all.

And a small comment on these functions:

public static int GetBackgroundResourceCount()

public static void GetBackgroundResource(int index, out string resourceName, out float resourceRate)

I just tried using these for a simpler custom module implementation and quickly realized the utility of them is extremely limited due to not having access to any config node information at all. With the current parameters the only thing I could see these being used for is if you have a module that produces or consumes resources at a fixed rate regardless of circumstance (which is what I do have) that also has its production rate hard coded to a specific value within the module itself (which I don't), with no option to have a constant rate parameter even specified in the config file for the part.

Not sure what the solution would be there as I think it would rapidly turn into the equivalent of doing things through FixedBackgroundUpdate() if stored data access was added or what have you, but I thought it worth point out that the actual use cases for these functions may currently be limited beyond the point of them being helpful to all but a very narrow set of circumstances.

Maybe if the partModule from the part prefab were passed in as a parameter it would become more useful so that the function could access stuff like the default consumption or production rates set for the module? Assuming of course that GetBackgroundResource() is called form a portion of the BP code that has access to it.

Passing in the part to those is a good idea. At the moment this background resource handling for non-electric-charge modules is a bit simplistic. Ideally you'd like to be able to return the consumption properties when something goes into the background, the same way I retrieve them for ModuleCommand/ModuleGenerator/ModuleDeployableSolarPanel. I should probably change how those functions work.

The intended usage is if you have a PartModule that consumes or produces a fixed quantity of some resource every tick, you can just have it tie into the same system I use to handle electric charge in the background. It's intended to be the extremely-simple background update, I guess.

[FlowerChild]

It should be implemented iff any of your partmodules believes that a resource should be processed in the background by BP from ModuleGenerators/ModuleCommand/ModuleDeployableSolarPanel parts.

Ah, gotcha. Thanks for the explanation there. The explanation in the OP and on KerbalStuff doesn't really give an impression of when it should be used and what it actually does

The intended usage is if you have a PartModule that consumes or produces a fixed quantity of some resource every tick, you can just have it tie into the same system I use to handle electric charge in the background. It's intended to be the extremely-simple background update, I guess.

Cool stuff. Yeah, I think having access to the part prefab or the specific module within would be a minimum to make it useful in that context, and should still keep things relatively simple given the user wouldn't have to deal with parsing proto snapshots or worrying about the limited and sometimes dodgy initialization state of stuff within a background vessel.

The context I was going to use it in for example was that I have replacement custom ModuleGenerators within my mod, once again to get around stock rounding errors in energy management. So, this essentially means that I have to handle my own background processing for them, but given they're just producing energy at a constant rate (defined in their .cfg file mind you), using GetBackgroundResource() seemed to be a better fit.

Granted, it was super simple for me to just use FixedBackgroundUpdate() instead, especially given I had already written a bunch of helper functions for parsing background vessel data while working on the probe core stuff.

Speaking of which, thought you might like to see this:

Progress

That one is mildly interesting in that I'm actually modifying the damage state of the probe core within the part snapshot during a FixedBackgroundUpdate() and after running multiple tests, it appears that state is automatically successfully saved and loaded once the vessel becomes active again.

Relevant code in case it's a useful reference to anyone:

        public static void FixedBackgroundUpdate( Vessel vessel, uint uiPartFlightID, Func<Vessel, float, string, float> ResourceRequestFunc, ref System.Object customData )        
        {
            // Background Processing callback

            if ( customData != null )
            {
                BTSMModuleProbePowerBackgroundUpdateData probePowerCustomData = (BTSMModuleProbePowerBackgroundUpdateData)customData;

                if ( probePowerCustomData.m_bIsActive )
                {
                    float fResourceDesired = probePowerCustomData.m_fEnergyConsumedRate * TimeWarp.fixedDeltaTime;

                    float fResourceConsumed = ResourceRequestFunc( vessel, fResourceDesired, "ElectricCharge" );

                    // Check energy consumed and damage probe core on lack of power if it's out for too long

                    if ( fResourceConsumed + 0.01F * fResourceDesired < fResourceDesired )
                    {
                        probePowerCustomData.m_iOutOfResourceUpdateCount++;

                        if ( probePowerCustomData.m_iOutOfResourceUpdateCount >= m_iNumOutOfEnergyUpdatesForDamage )
                        {
                            FixedBackgroundUpdateOnProbeCoreOutOfPower( vessel, uiPartFlightID, probePowerCustomData );
                        }
                    }
                    else
                    {
                        probePowerCustomData.m_iOutOfResourceUpdateCount = 0;
                    }

                }
            }
        }

        private static void FixedBackgroundUpdateOnProbeCoreOutOfPower( Vessel vessel, uint uiPartFlightID, BTSMModuleProbePowerBackgroundUpdateData probePowerCustomData )
        {
            // the core has been out of power for awhile.  Disable further background updates for it and damage it within its config node as well

            probePowerCustomData.m_bIsActive = false;

            ProtoPartSnapshot partSnapshot = BTSMUtils.GetProtoPartFromVessel( vessel, uiPartFlightID );

            if ( partSnapshot != null )
            {
                ProtoPartModuleSnapshot moduleSnapshot = BTSMUtils.GetProtoModuleSnapshotFromPart( partSnapshot, "BTSMModuleProbePower" );

                if ( moduleSnapshot != null )
                {
                    Part partPrefab = PartLoader.getPartInfoByName( partSnapshot.partName ).partPrefab;

                    if ( partPrefab != null && partPrefab.Modules != null && partPrefab.Modules.Contains( "BTSMModuleProbePower" ) )
                    {
                        BTSMModuleProbePower modulePrefab = (BTSMModuleProbePower)partPrefab.Modules["BTSMModuleProbePower"];

                        if ( modulePrefab != null )
                        {
                            if ( modulePrefab.canBeDisabled )
                            {
                                // the probe core can be disabled, so just shut it down

                                moduleSnapshot.moduleValues.SetValue( "coreDisabled", "True" );

                                ScreenMessages.PostScreenMessage( vessel.vesselName + ": " + partPrefab.partInfo.title + 
                                    " shut itself down due to lack of power", 
                                    10F, ScreenMessageStyle.UPPER_LEFT );
                            }
                            else
                            {
                                // damage the probe core

                                moduleSnapshot.moduleValues.SetValue( "m_bProbeDamaged", "True" );

                                ScreenMessages.PostScreenMessage( vessel.vesselName + ": " + partPrefab.partInfo.title + 
                                    " was damaged due to lack of power", 
                                    10F, ScreenMessageStyle.UPPER_LEFT );
                            }
                        }
                    }
                }
            }
        }

Next up I'll be tackling converting my life support system to Background Processing.

Edited February 8, 2015 by FlowerChild

[FlowerChild]

Just put out the first public release of BTSM that includes support for Background Processing:

BTSM thread link

At present the support is partial in that energy and life support are handled through BP, while the late-game resources processors are not. Those are rather more involved and I wanted to get this out into the wild so people could start testing it beyond my capacity to do so alone. If anyone is looking for a reference on how to implement support for Background Processing feel free to consult the source for the mod.

Much thanks for all your help with this james. Handling resources for inactive vessels has been a problem with the mod for a very long time now, and I really appreciate what you've done here

[Renegrade]

BackgroundProcessing is working pretty well. I run a fairly intensive BTSM campaign with the new experimental BP-based build, with tons of rotating solar panels, and the performance hit seems very low (mostly just longer scene changes):

These vessels are floating around out there in continual service with no problems (well, I forgot to redeploy the panels on one of the life support tankers and it shut down, but that's expected ), plus about half a dozen more large vessels and a handful of smaller ones.

(Note that these vessels are typical for a Tier 8 BTSM campaign and that I'm not just spamming solar panels for the fun of it - the Duna vessel in particular needs every extended panel on the main body to keep life support running at that range, and the ones on the attachment on the nose are necessary for recharges after transmission and after being on the dark side of Duna)

Great job on the mod so far and thanks

[codepoet]

This is interesting. So I could use it for a potential sanity mod to make it much easier for kerbals to slowly loose their minds while the player is busy trying to launch another satellite to join the comnet?

[FlowerChild]

This is interesting. So I could use it for a potential sanity mod to make it much easier for kerbals to slowly loose their minds while the player is busy trying to launch another satellite to join the comnet?

In theory, yes, but it might get a little tricky as the background update functionality for the mod revolves around part modules, rather than Kerbals

I suspect if you wrote a ModuleManager script to add a part module of your own creation to any part that can contain Kerbals, then you could hook it into Background Processing fairly easily.

[FlowerChild]

@jamespicone: Just a heads up man that there's a lot of stress testing going on by players in the BTSM thread right now that you might find interesting. So far BP is holding up remarkably well to everything that is being thrown at it within the confines of what one might expect in a typical game.

As a result I'm going to be moving forward with making BP a full dependency for BTSM and implementing the remaining modules that could benefit from it (namely the resource production parts I have in the late game)

[Subcidal]

Does the orientation of the vessel get taken into account? Like i have some comm sat's that have more than enough panels in total, but only at certain angles. Does BP base its calculations off the angle you last left the vessel, Or does it actually account for rotation?

[jamespicone]

@jamespicone: Just a heads up man that there's a lot of stress testing going on by players in the BTSM thread right now that you might find interesting. So far BP is holding up remarkably well to everything that is being thrown at it within the confines of what one might expect in a typical game.

As a result I'm going to be moving forward with making BP a full dependency for BTSM and implementing the remaining modules that could benefit from it (namely the resource production parts I have in the late game)

Sweet. Had a look through the thread, and it's good to hear that it takes significant amounts of stuff before I'm a major performance drag.

If performance does become a problem in future, I'll probably look into obviating the linear-time search for the appropriate ProtoPartSnapshot more directly than allowing people to store arbitrary data, and *maybe* some kind of multithreading, although I'm not sure I want to impose threading-related restrictions on people's background updates.

Does the orientation of the vessel get taken into account? Like i have some comm sat's that have more than enough panels in total, but only at certain angles. Does BP base its calculations off the angle you last left the vessel, Or does it actually account for rotation?

Vessel orientation should be taken into account, as should panel location on the vessel, panel orientation, and the panel's ability to track if it's a tracking panel. If one of those isn't being taken into account, it's a bug. Vessels don't rotate in the background, unfortunately - it'll just be whatever orientation the vessel was in when you left it. I can probably fix that - vessels have an 'angularVelocity' vector.

[Subcidal]

Sweet. Had a look through the thread, and it's good to hear that it takes significant amounts of stuff before I'm a major performance drag.

If performance does become a problem in future, I'll probably look into obviating the linear-time search for the appropriate ProtoPartSnapshot more directly than allowing people to store arbitrary data, and *maybe* some kind of multithreading, although I'm not sure I want to impose threading-related restrictions on people's background updates.

Vessel orientation should be taken into account, as should panel location on the vessel, panel orientation, and the panel's ability to track if it's a tracking panel. If one of those isn't being taken into account, it's a bug. Vessels don't rotate in the background, unfortunately - it'll just be whatever orientation the vessel was in when you left it. I can probably fix that - vessels have an 'angularVelocity' vector.

Ok cool, Fix it when i build a satellite that i don't have to move 4 times a year =P

No but seriously, it would be pretty cool if it took rotation into account, That was my biggest concern. Even cooler if somebody got a persistant rotation mod working correctly.

[FlowerChild]

Sweet. Had a look through the thread, and it's good to hear that it takes significant amounts of stuff before I'm a major performance drag.

If performance does become a problem in future, I'll probably look into obviating the linear-time search for the appropriate ProtoPartSnapshot more directly than allowing people to store arbitrary data, and *maybe* some kind of multithreading, although I'm not sure I want to impose threading-related restrictions on people's background updates.

If it comes to that, I think there may be some wiggle room for some fairly straight forward optimization in for example cutting down on the number of conditionals required to process modules. Like from previously looking through the code, I think there's an opportunity to move the stock modules you are handling (solar panels, generators, etc.) into update functions that are consistent with those used for custom module handling so that they wouldn't have to be constantly tested for with each update. It's also possible that branch prediction may make any gains from that negligible anyways.

But really, from the results that are being reported, I don't think even going that far is really necessary at present. I think I mentioned before but one of the things that attracted me to BP was the way it was obviously structured for performance in the first place, and those kind of high level optimizations are usually where you get the most gains anyways. Very happy to see it standing up to this kind of stress testing, even if I'm not entirely surprised by it

I can probably fix that - vessels have an 'angularVelocity' vector.

I think that might be more confusing than anything, especially if SAS can't be enabled or what have you. Given tracking solar panels are taken into account as well, I'm not sure if it would serve much purpose since you can just orient your vessel to make sure the sun remains along the axis of rotation of your panels.

Edited February 13, 2015 by FlowerChild

[Subcidal]

Oh wait, I'm an idiot. My satellites will still fall out of service due to orbiting the sun lol. Would actually be more useful to have rotation taken into account.

[FlowerChild]

Oh wait, I'm an idiot. My satellites will still fall out of service due to orbiting the sun lol. Would actually be more useful to have rotation taken into account.

Why not just use tracking panels to make sure they can always "see" the sun? That's taken into account by BP, and will work fine as long as you make sure to orient your ship appropriately.

If you take a look at the screenshots Rengrade posted on the previous page, that's what the vessels are designed to do, and it looks like the one that he has in orbit around Duna in the middle is oriented for maximum sun exposure as the ship orbits Duna, and Duna in turn orbits the sun. Those are also all far more complicated cases than you are likely to encounter in stock.

Edited February 13, 2015 by FlowerChild

[Subcidal]

It's just the design, I have like 14 non tracking panels on either side with about 5 tracking panels on the other 2 sides, It needs both the tracking and non tracking panels to retain a proper charge. Using a customized version of BTSM actually.

But now i'm confused, just did a little test and it seems that they didn't loose their orientation after a quarter of a kerbin year, when they should of been "sideways". Does anyone know if the game save crafts orientation relative to the body their orbiting or the sun?

[FlowerChild]

It's just the design, I have like 14 non tracking panels on either side with about 5 tracking panels on the other 2 sides, It needs both the tracking and non tracking panels to retain a proper charge.

You'd have to constantly reorient that vessel manually even if it was active then, and say if you were applying time accel. It sounds more like a problem with the vessel design that's just been exposed by it having resource processing applied to it in the background, rather than a problem with how the resource processing is being applied.

I'd say it would probably make sense to complete that mission without BP installed, and then start designing vessels suited to it with it installed afterwards.

But now i'm confused, just did a little test and it seems that they didn't loose their orientation after a quarter of a kerbin year, when they should of been "sideways". Does anyone know if the game save crafts orientation relative to the body their orbiting or the sun?

Not certain on that one, but I didn't think so.

Edited February 13, 2015 by FlowerChild

[Subcidal]

It is indeed a problem with vessel design, I didn't design it with this in mind, TBH I thought it wasn't being used still. Either way i'm going to roll with it for now, I either found a game bug that's going to cause a bit of a problem or something i have is causing vessels to not rotate as you would imagine as they orbit the sun. Tried twice now, If you leave a vessel pointed towards the sun, it's going to be pointed towards the sun when you re-load it.

[FlowerChild]

I either found a game bug that's going to cause a bit of a problem

Well, even if it's the case, it may not be a bug, but rather just the way KSP works. I also doubt it would be much of a problem during play considering it would mean you can just orient a craft towards the sun even with fixed panels and not worry about it.

I'll have to check that myself the next time I have a chance as I'm interested in knowing which way it behaves.

[Nitrous Oxide]

Hmm, I thought it was strange when building my RT networks... I gave them absurdly huge batteries they apparently didn't need for stock's electric charge processing... Basically as long as the satellite had at least 1 unit of EC, it was an infinite com. relay when not in focus... even satellites without solar panels...

Huge kudos for this, now I can build a real satellite network.

[Renegrade]

If you take a look at the screenshots Rengrade posted on the previous page, that's what the vessels are designed to do, and it looks like the one that he has in orbit around Duna in the middle is oriented for maximum sun exposure as the ship orbits Duna, and Duna in turn orbits the sun. Those are also all far more complicated cases than you are likely to encounter in stock.

The middle one is indeed designed for that very operation. Once I have rotating panels, I always aim for that sort of dealie (I make the axis of the rotation parallel to the axis of the system ecliptic)... except on that craft on the left. I threw that together just for appearances, it's not a very practical design without something like IR (which I've never used, even though I frequently reference it heh).

This talk of ship orientation being sun-relative rather than fixed-stars-relative is alarming me though. I think I'll do some tests...

Huge kudos for this, now I can build a real satellite network.

Yep, it addresses a long-standing KSP issue, and will be perfect for RT. It doesn't seem very computationally expensive, with surprised me (pleasantly, which is nice, so few surprises are of the pleasant variety these days..)

UPDATE: A bit of testing in a sandbox stock game has revealed some weirdness. Sometimes, after jumping back to the test probe from KSC after warping a quarter orbit, the test probe is facing weird directions. Normally during high warp while in direct control of a vessel, the sun should appear to move from the right to the left on the screen. If facing the sun and warping for a quarter orbit, it should then be on the left (using default orbital view). However, when performing that test, I found it was often behind the probe, or to it's right, or even still in front of it. NB: this is a stock game, without BP. So whatever's happening, it's not BP's fault.

I wonder if that's related to the effect where a vessel spontaneously disassembles itself when you switch to it?

Edited February 13, 2015 by Renegrade

[FlowerChild]

This talk of ship orientation being sun-relative rather than fixed-stars-relative is alarming me though. I think I'll do some tests...

Just a note on this which might be helpful in testing it (I'll try to look at it myself tomorrow), is that I suspect if it is maintaining orientation relative to anything, it would be to the current reference body, rather than just the Sun. In other words, if you're in orbit around Kerbin or the Mun, I suspect it would maintain orientation relative to it rather than the Sun.

Not certain of that of course, but it's a hunch I had when the previous poster mentioned this that I was planning to test out myself, and which might be helpful in your own tests. The only reason I could see for it doing this is if it was keeping track of its position and orientation in some kind of local coordinate system.

Edited February 13, 2015 by FlowerChild

[Subcidal]

It seems to me like it saves its orientation on a XYZ plane, And lets say X is the direction of the sun (center). When you load up the flight again regardless of what planet your in orbit around and where that planet is at in its orbit KSP is loading up the plane and again aligning X with the sun, (center).

I figure if it is saving orientation relative to the current reference body it would rotate as you would expect it to, simply because the relativity of the body hasn't changed, just the suns position in respect to it. If you get what I'm trying to say....

So long story short i think its centered around the sun, But i don't think (and haven't tested) that a vessel would rotate as you would expect it to if it were in a solar orbit either, due to my theory above. But hell I might be wrong.

[Renegrade]

Just a note on this which might be helpful in testing it (I'll try to look at it myself tomorrow), is that I suspect if it is maintaining orientation relative to anything, it would be to the current reference body, rather than just the Sun. In other words, if you're in orbit around Kerbin or the Mun, I suspect it would maintain orientation relative to it rather than the Sun.

It seems to me like it saves its orientation on a XYZ plane, And lets say X is the direction of the sun (center). When you load up the flight again regardless of what planet your in orbit around and where that planet is at in its orbit KSP is loading up the plane and again aligning X with the sun, (center).

Well, my testing seems to imply that it's rather.. random-y. My tests I've done so far involve a craft in orbit of the Sun in a pure stock game. I would point the craft directly at the sun, go back to the tracking center, and run in high warp until the craft had gone in an approximate quarter circle.

When I came back to the craft though, it was facing in a random direction. It was pretty much at right angles to the Sun still (within a reasonable limit, since I was eyeballing the angle during warp), but it could be left, or right, or towards, or away. It's main axis was still in the ecliptic plane.

Not certain of that of course, but it's a hunch I had when the previous poster mentioned this that I was planning to test out myself, and which might be helpful in your own tests. The only reason I could see for it doing this is if it was keeping track of its position and orientation in some kind of local coordinate system.

That could very well be the key. KSP uses a mobile / local coordinate system AFAIK, that may very well have some weird local rotational frame of reference.

(apparently some of the physics / inter-part calculations happen in single precision, you can imagine what would happen at gigameter ranges )

It seems to me like it saves its orientation on a XYZ plane, And lets say X is the direction of the sun (center). When you load up the flight again regardless of what planet your in orbit around and where that planet is at in its orbit KSP is loading up the plane and again aligning X with the sun, (center).

That's the thing, it's NOT aligning with the Sun, except by chance. It's very weird!

I'll run some planetary tests during my next testing battery (plus moar solar tests) but man, this is some weird stuff.

[Drew Kerman]

KSP uses a mobile / local coordinate system AFAIK, that may very well have some weird local rotational frame of reference.

bring the kOS devs in on this - they'll be able to fill you in on just how whacky the reference frame is. You're right, is is pretty messed up from what I understand. There's a ton of helper functions in kOS so you can just point your ship the way you want without wanting to pull your hair out