[SOLVED] Properly remove a PartModule from a part at runtime - Serialization questions

[Shadowmage]

Hi all,

I'm in the process of writing an advanced mesh-switch module that allows for swapping in/out meshes and their corresponding functionality(PartModules). This would allow, for example, a part to -optionally- have landing legs, RCS thrusters, solar panels, or whatever, dynamically added/removed/swapped as part variants.

So far I have it working well with swapping out -custom- PartModules that track their own internal enabled/disabled state and make the corresponding changes (e.g. disable GUI actions and update methods when the module is disabled). It works, but the disabled modules remain in the Part.Modules list, still receive update ticks, and are still serialized into the parts persistent data (an empty update method is not much overhead, but it -is- un-needed overhead..., I don't really care about the serialization data overhead, disk space is cheap). However this method is not easily extensible as it requires writing a custom PartModule for any module that is intended to be swapped via the mesh-switching functionality, and even then it requires special coding/care to make sure everything is properly disabled.

What I would like to do is dynamically add/remove the modules from the part completely at runtime. This should completely remove any functionality from the module (gui actions, action groups); the mesh/colliders are already taken care of though the basic mesh-switch code. In theory I would grab (and save) a reference to all of the mesh-switch controlled partModules; remove all of the disabled ones from the part.Modules list, and add/swap them as needed for the part variants. And then -before- the part was serialized (saved) I would have to re-add (in original config order!) all of the part-modules so it did not mess up part loading/de-serialization. At least that is how I understand the current part/module serialization process (it serializes all partModules, in listed config order, and having them not present/extra modules present will royally mess-up any reloading of that part, as it will feed the wrong data to any out-of-place modules).

Is anyone aware of how to properly add/remove modules at runtime while handling the serialization problems issues? Is there a callback/event for a part during the loading process -after- the part has had the modules instantiated, but -before- they have been fed the loading data? Is there a similar callback -before- a vessel/part has begun its saving process?

At work for the moment, or I would probably just dive in and skip the asking of questions; however I'm hoping someone has some experience with these issues they are willing to share.

Thanks in advance,

Shadowmage

Edit: Further questions would be - How does the part.ModuleList handle null values? Can I remove (null) a module by index after it is initialized without messing up the OnSave()/serialization code? Alternatively, could I insert a dummy module into the original (disabled) module index, feeding it the original modules config data (by calling OnSave() on the original/disabled module manually and storing the config node in the dummy module), and allow the dummy-module to write out the original modules data in its OnSave method (thus preventing any deserialization issues due to mismatching module indices and/or data).

Hmm..going to have to try that out later, as that is probably the most likely-to-work solution I've come across/thought through.

Edited July 29, 2015 by Shadowmage

[Diazo]

The big limitation you are going to run into is that you can't change the "default part".

By this I am talking about the ProtoPart database where a single, default copy of each part is stored. Then when a part is loaded, KSP copies this default part from the ProtoPart database and if saved data is present, loads the values from said saved data.

This all happens before any of your code can run, so only partModules that are present in the part.cfg file or added via ModuleManager will correctly load data.

Note that when saving, only partModules present on the part are saved.

To add/remove partModules dynamically on an instance of a part is simply the Part.AddModule and Part.RemoveModule methods, but this will not be sufficient for your needs.

Scenario:

Place a part in the editor, so KSP makes a copy of the part from the ProtoPart database.

Add PartModuleA via the Part.AddModule code.

Save and exit the editor. KSP will save the data for the PartModuleA data correctly.

Return to the editor and load the same vessel.

KSP copies the part from the ProtoPart database and while is sees the data for PartModuleA in the saved file, it can't find PartModuleA on the module and so skips loading it.

You code runs adding PartModuleA back onto the part but it has default values as the Load sequence has already run before your code.

I'm not sure how to work around that as I've never thought about it, but if you want data to save/load correctly you need to have your part module in the ProtoPart database somehow. (You could also manually save/load data I guess, but keeping things in sync would get complicated very quickly.)

D.

[Shadowmage]

Thanks for the info.

I intend to have the modules on the parts in the config files / in the game database / on the prefabs. So they will exist on the ProtoPart/AvailablePart. They will only be removed from the part (replaced) after everything has loaded/initialized; this is to prevent some of the problems you point out (the initial loading issues). The biggest problem that I'm facing is how to handle the save-out of an already existing part that has had module changes. As far as I'm aware simply removing the module would run afoul of the problems you have pointed out during reloading of the vessel; as the indices would get out of synch and it would not restore the proper data to the proper module on reload.

I'm really leaning towards inserting a 'dummy' module to replace the actual module instead of fully removing the module at that index. The dummy module would then act as a delegate for the config-data/ConfigNode from the original (removed) module, and re-save out that data to the config node passed in to the OnSave() method. This -should- allow the part to be reloaded from the prefab with all of the original modules (and associated config data), which would then be re-replaced by the MeshSwitch module dummies once loading had finished.

I was hoping that there would be a more robust/supported way of accomplishing this; but I'm (grudgingly) willing to venture into the land of 'dirty hacks' when I have to...

Anyhow... off work here in a bit, so I'll do some testing of a few things and let you know what I find out.

[Diazo]

Seeing the edit to your first post, I do not believe a "dummy" module would work. To the best of my knowledge, KSP uses the string name of the part module to save load data, not the index count of the module. As part module names have to be unique, you can't have a "dummy" module of the same name.

(I have not directly tested this, but I can't think of any other way the "PartModule xxxx not found at index X, looking in other indexes" error message you sometimes see in the log.)

Rather, if you are going to go the switch out route, make a master module and remove all other modules. These removed modules would exist as sub-nodes of the master module.

Then when your master module loaded, it would call Part.AddModule() and populate all the KSPFields correctly. To save data, it would simply be a foreach over the Part.Modules list, following by a foreach over the PartModules.Fields list to save the data.

You can take advantage of the fact that only data that is tagged with a KSPField saves persistence data and that KSP always populates the PartModule.Fields list with all KSPFields on that part module.

D.

[Shadowmage]

After a bit of setup and testing, and it is -so far- looking promising using the dummy module route. The dummies are just there to re-save/persist the data from the original module. The trick is when the OnSave code is called on the dummy module, it needs to alter the config node that is passed in to alter the name it is saved under. As you are correct, KSP references the module by a string during save/load, specifically the Type/Class name for that module. But KSP also feeds you the entire config node for the module in the OnSave() method; so you can alter the auto-written dummy module class name and replace it with the original class name and add back in any persistent data from the original module.

So far, it is working about as I had intended. I just need to find the best time in the part lifecycle to do the initial swapping, and some way to re-enable the removed module/swap it back in as needed (instatiate a -new- module, feed it back the config data I had stored in the dummy, and swap it back in while removing the dummy? seems like it might work...)

It should work out like this in the end:

Editor - new part:

Part is cloned from prefab/ProtoPart/AvailablePart (editor)

At the earliest life-cycle event where -all- modules have been initialized, the ModuleSwap partmodule does its swapping, replacing original modules with dummies while grabbing the current config node data from the original.

When OnSave() is called on the dummy module, it hacks in the ConfigNode data from the original module that it replaced (e.g. replacing class name, adding persistent data that was saved from original module).

Part is then saved out when editor exited or vessel is launched. The persistence file for the craft will -look- exactly as it would had those original modules never been removed/swapped.

Vessel is reloaded:

Part is cloned from prefab/ProtoPart/AvailablePart (editor re-load, vessel launch, vessel re-load)

The -original- module was cloned from the prefab, and is fed the data that was saved out from the dummy module. Since it saved out the original modules' data, there is no problem here. The original module loads properly, and it is unaware anything has happened (in theory).

At the earliest lifecyle event where all modules have been initialized, the ModuleSwap partmodule does it swapping again, same as above (saving original module data for the dummy module to save out).

Gameplay proceeds...tick, tock, etc.

Game is saved; the craft persistence file again looks exactly as it would had those original modules been active and present the entire time.

...

Will update later after a bit more testing and verification.

Will also give some thought to your proposed method as it might work out to be cleaner in the end (master module, with sub-modules).

I'm thinking it would require defining the to-be-removed modules as a bunch of config nodes -inside- the ModuleSwap module config (so they are not loaded/added to the part.ModuleList initially). But as KSP uses the simple config-node system to instatiate the modules in the first place, there shouldn't be too much trouble manually instantiating them (only if needed/enabled from the ModuleSwap). Still trying to fathom how that would work on the save-out though; if these sub-modules were added to the part.ModuleList proper (for updates ticks/etc), they would end up saving to the persistence file for the part/craft. I would also have to duplicate any persistence data for that sub-module inside the master module so that the master could have access to the data to recreate the module layout on craft/part reload.

Hmm...worth a bit more though, and perhaps some poking and prodding (testing..)

Edit: After more thought, I think I am going to try fleshing out your idea a bit more.

I realized that for most purposes I probably don't need the sub-modules to be added to the part.ModuleList. I need them for the script behavior that they contain. I could route all calls to KSP-derived methods (OnSave, OnLoad, GetInfo), from the master module to the sub-modules on an as-needed / sanitized basis (e.g. only calling update/save on the enabled/loaded modules). And as they would be MonoBehaviors attached to the part (rather the parts Unity gameObject) just as any other module/script, they would automagically receive the Unity-derived method calls and lifecycle events (FixedUpdate, Update, etc).

I anticipate that there might be some compatibility issues with a few modules, but I'm going to give it a try and see where things lead.

Thanks for the input and ideas

Edited July 29, 2015 by Shadowmage

[Shadowmage]

Update:

The sub-module method does work. I had to add the modules to the part.ModuleList otherwise GUI actions would not be available (I'm actually just feeding the config node data in through part.AddModule()). That actually simplifies things a bit because now I don't need to worry about other lifecycle events, as long as I call the initial OnLoad / OnStart that would be expected. This results in the sub modules writing their data out to the persistence file as regular modules, and then they are stored a second time as data within the master-modules persistence data (so I have easy access during the next OnLoad of the master module/manually manage their data).

The part does complain about having extra module info in the persistence file upon part or vessel reload...

[WRN 23:28:34.925] [Part]: PartModule ModuleReactionWheel at SSTU.LanderCore.LCTest, index 1: index exceeds module count as defined in cfg.
Looking for ModuleReactionWheel in other indices...
[ERR 23:28:34.925] ...no ModuleReactionWheel module found on part definition. Skipping...

...but as long as there are no duplicate modules in the base un-switched set of modules, I don't -think- it should cause any problems. I'll be doing some thorough investigation on it though before I'll be willing to call this solution 'stable'. Ideally I will find a way to trim that extra config data out of the part/vessel ConfigNode. Will check on what events I can hook into; there might be 'vessel saved' or 'about to write to disk' even that I can hook into and clean out the extra data.

Thanks again for your input Diazo, put me on a totally different path than I would have originally gone down, and it does seem like it will work out to be cleaner in the end.

[Diazo]

Glad to help.

The warning message you are worried about is a non-issue. All that is telling you is that the saved data contains a partModule that KSP can not find on the ProtoPart as currently loaded in the part database. All KSP does in this case is skip loading it which isthe intended behavior for what you are doing.

Having said that, spamming that message every time a vessel using this loads does not look good to the player, even if it is a non-issue, but I'm not sure how to go about suppressing it.

D.

[Shadowmage]

Crazy idea, regarding the solving of the 'extra data'.

At one point you could subclass 'Part', the same way that you currently do PartModules. I believe it is still used on a couple of stock parts, at work so I cannot verify through their configs. Is this still possible?

I have no problem with parts that use the ModuleSwitch functionality requiring a special Part sublcass to be specified in their part.config fie; it is a fairly heavy-duty game-altering mechanic and a few caveats are likely unavoidable. If it is still possible, it seems like I should be able to hook into (read: override) the Part.OnSave() method to scrub out the extra PartModule data from the extra inserted modules.

The only other alternative that I could think of would be to subscribe to the OnPartPacked gameEvent and remove the extra PartModules from the ModuleList before they were serialized; this would likely have some issues though.

Alternatively, if part sublcassing is allowed/functional, I might be able to find a better way to manage the entire system.

-- I suppose this will be my experiment for tonight.

[Diazo]

While Part sub-classing is still available, it should only be used by a part's creator.

The reason for this is that a part can only belong to one part class, if multiple people start trying to set the part class on the same part, only one mod will be able to and the other mods will break.

Therefore the "one person" who gets to set a part's class is the part creator, otherwise we will start seeing mods that are incompatible with one another start appearing and that leads to nothing good.

No issue if you are making new parts to release this code on, but I would strongly suggest you do not do this to mod onto other peoples parts.

I don't even know if making a custom part class like this would allow you to override the OnSave either, at least not how you need to.

I would actually start with the OnSave and OnLoad GameEvents and see if they fire at the right point in the life cycle that you could strip out the extra data there.

D.

[Shadowmage]

Aye, I intend the module to only be used on parts that I create, as I realize there would be issues where other mods' parts (or even stock) are concerned especially where they might use a custom part subclass.

I'll begin by investigating the OnSave/etc game events; would much prefer to not subclass Part as it is a bit of a hacky solution, and stock itself has been moving away from it in favor of PartModules.

Thanks again for the info; will let you know how things turn out.