Parts incorrectly marked shielded - overwrites manual fixes

[bdleitner]

Disclosure: I have done some manual editing of craft and save game files, so I may have gotten myself into trouble, though I've tried to be careful and haven't run into anything obviously caused by bad editing.

TL:DR; Parts that are not inside a cargo bay or a fairing show signs of being "stowed": e.g. Magnetometer boom cannot extend because "cannot deploy while stowed". Looking at the save file I see lots of parts have "shielded = True" even though they are not inside a fairing, cargo bay, or anything similar.  Manually editing the save file to set this to false sees those values overwritten when the craft is reloaded

Detail:

I built up my largest ever craft for a manned mission to Sarnus (Outer Planets Mod, complete mod list included below).  On my ship I have a large lab from stockalike station parts expansion redux" and some science parts (and an antenna) attached to the outside of it.

Once I was ready to do my Kerbin departure burn, I double checked my antenna was extended (it wasn't), but I was blocked from doing so in game by "cannot deploy while stowed"  The antenna is on the surface of the lab (sspx-lab-5-1) lab along with both types of Magnetometer (Magnetometer and DTMagnetometer).  Rather than fight with it I just edited the save file to change the state of the antenna to EXTENDED and all was good.  Then I decided to grab some science on the way out to get some data into my lab.  That's when I discovered that I couldn't extend the Magnetometer either as it gave a similar "cannot deploy while stowed" error.

Things I've tried:

1. Doing an EVA and using construction mode to move the magnetometer to other locations, including other parts.

2. Editing the file to make it EXTENDED - when I reloaded it was extended but I still couldn't run the experiment (similar "cannot run while stowed" error). When I retracted it in game I was again unable to extend it.

3. Deploying all remaining fairings on the vessel. I do have a total of 4 fairings on the ship. One had already been deployed, but the other 3 hadn't. I tried deploying them to see if there was some weird interaction going on. It did not help (though I didn't try that in combination with other things). Also, see below for more on the fairings.

4. Finally, I "launched" a new iteration of the lander bit that I have docked with the main craft, but with a magnetometer added there.  After confirming that on this ship I could extend the magnetometer and run the experiment,  I wrote some code to find the nodes in the save game data for the two magnetometers and compare them. I found the "shielded" property as the only thing that was different and seemingly relevant (of course they had different ids, positions, etc.)  So then I wrote more code that found all parts on the ship that had "shielded = True" and for each of those, look through its ancestry (via parent parts) to see if it was a descendant of a fairing that didn't have "fsm = st_flight_deployed" set. If so, I figured that would mean it should be shielded and I left it alone. But parts that were "shielded = True" but were not descended from a non-deployed fairing, then I set it to "shielded = False" There were over 150 parts that needed this edit, including things that made no sense.. parts that were never in a fairing or similar.  But I hoped that would be the end of it. But when I loaded the new save (which parsed and loaded correctly, yay!) and went to that ship I ran into the same issue. A new save from inside the game revealed that the parts had had their shielded properties reset to True. In fact running the same code on the new save showed a larger number of "shielded = True" parts outside of fairings (though I didn't do digging to see what was different).

A word on the fairings: In order to get symmetry I tried to alt-click parts in the VAB to get a copy, but the copied fairing didn't have its enclosure copied. To get the symmetry I edited the craft file by copying the XSECTION nodes in MODULE(ModuleProceduralFairing) from the built fairing to the other ones. As far as I  could tell the data there was all relative to the fairing itself and didn't need to be adjusted to account for fairings in different locations. The 3 matching fairings (one original and 2 copies, the 4th fairing was also manually built as isn't symmetric with the others) all appeared fine in the VAB and once the craft was launched. But maybe I introduces some weird issue where some part of the system thinks that everything is inside one of the fairings.

Anyway, that's where things are. I have built up a pretty good code library for editing save files, so I have no fear of trying to poke around, but with the game overwriting my "shielded = False" edits, I'm not sure where to go from here.

Link to Player.log:
Player.log: https://www.dropbox.com/scl/fi/1nz7igwrmkd76945ot6ui/20240817-KSP-Player.log?rlkey=ssi8xikejapiqz9gi2bq3civm&st=nffo6rkc&dl=0
Player-prev.log: https://www.dropbox.com/scl/fi/vmmyirbcou4pdyqv7pp61/20240817-KSP-Player-prev.log?rlkey=qe12u7jlrjb0ky9ds8e7dnneu&st=8vajq33n&dl=0

MOD LIST (CKAN export):

[x] Science! Continued (xScienceContinued 6.0.2)
Action Groups Extended (AGExt 1:2.4.1.3)
Antenna Helper (AntennaHelper 2:1.0.7.7)
Astronomer's Visual Pack (AstronomersVisualPack 3:v4.13)
Astronomer's Visual Pack-2k Textures (AVP-2kTextures v1.13)
AtmosphereAutopilot (Fly-By-Wire) (AtmosphereAutopilot v1.6.1)
B9 Aerospace Procedural Wings - Fork (B9-PWings-Fork 3:0.46.0)
B9 Part Switch (B9PartSwitch v2.20.0)
Background Resources (BackgroundResources 1:v0.18.0.0)
BetterCrewAssignment (BetterCrewAssignment 1.4.1)
BetterTimeWarpContinued (BetterTimeWarpCont 2.3.13)
Biome Corrections (BiomeCorrections 1.1)
Breaking Ground (BreakingGround-DLC 1.7.1)
ClickThrough Blocker (ClickThroughBlocker 1:2.1.10.21)
CommNet Antennas Consumptor (CommNetAntennasConsumptor 3.5.7)
CommNet Antennas Extension (CommNetAntennasExtension 2.1.7)
CommNet Antennas Info (CommNetAntennasInfo 3.1.2)
CommNet Relays (ContractConfigurator-CommNetRelays v2.1.0)
Community Category Kit (CommunityCategoryKit v112.0.1)
Community Parts Titles (CommunityPartsTitles 0.10.3)
Community Resource Pack (CommunityResourcePack v112.0.1)
Community Tech Tree (CommunityTechTree 1:3.4.5)
Community Terrain Texture Pack (CommunityTerrainTexturePack 1:1.0.5)
Contract Configurator (ContractConfigurator v2.11.0.0)
Custom Barn Kit (CustomBarnKit 1.1.22.0)
Distant Object Enhancement /L (DistantObject v2.2.0.2)
Distant Object Enhancement /L default config (DistantObject-default v2.2.0.2)
Docking Functions (DockingFunctions v1.0.4)
Docking Port Alignment Indicator (DockingPortAlignmentIndicator 6.10.0.0)
Easy Vessel Switch (EVS) (EasyVesselSwitch 2.3)
Environmental Visual Enhancements Redux (EnvironmentalVisualEnhancements 3:1.11.7.2)
Ferram Aerospace Research Continued (FerramAerospaceResearchContinued 3:0.16.1.2)
Harmony 2 (Harmony2 2.2.1.0)
Hide Empty Tech Tree Nodes (HideEmptyTechNodes 1.3.2)
HyperEdit (HyperEdit 1.5.8.0)
Infernal Robotics - Next (InfernalRoboticsNext v3.1.17)
Infernal Robotics Next - ConnectionSystem (IR-ConnectionSystem v1.0.5)
Interstellar Fuel Switch (InterstellarFuelSwitch 3.30.0)
Interstellar Fuel Switch Core (InterstellarFuelSwitch-Core 3.30.0)
Interstellar Redistributable  (InterstellarRedistributable 1.4)
Kerbal Alarm Clock (KerbalAlarmClock v3.14.0.0)
Kerbal Engineer Redux (KerbalEngineerRedux 1.1.9.5)
Kerbal Joint Reinforcement - Next (KerbalJointReinforcementNext v4.2.27)
Kopernicus Planetary System Modifier (Kopernicus 2:release-1.12.1-212)
kOS KerbalEngineer (kOS-KerbalEngineer v0.1.1)
kOS: Scriptable Autopilot System (kOS 1:1.4.0.0)
KSP Community Fixes (KSPCommunityFixes 1.35.2)
KSP Interstellar Extended (KSPInterstellarExtended 1.29.6)
KSP Recall (KSP-Recall v0.5.0.2)
KSPBurst (KSPBurst v1.5.5.1)
Making History (MakingHistory-DLC 1.12.1)
MechJeb 2 (MechJeb2 2.14.3.0)
Minimum Ambient Lighting Updated (MinAmbLightUpd 1.2.6.2)
ModularFlightIntegrator (ModularFlightIntegrator 1.2.10.0)
Module Manager (ModuleManager 4.2.3)
Near Future IVA Props (NearFutureProps 1:0.7.2)
Outer Planets Mod (OuterPlanetsMod 2:2.2.10)
Parallax (Parallax 2.0.8)
Parallax - Stock Planet Textures (Parallax-StockTextures 2.0.8)
Parallax - Stock Scatter Textures (Parallax-StockScatterTextures 2.0.8)
Parking Brake (ParkingBrake 0.4.4)
Patch Manager (PatchManager 0.0.17.6)
Photon Sailor (PhotonSailor 1.7.4)
PlanetShine (PlanetShine 0.2.6.6)
PlanetShine - Default configuration (PlanetShine-Config-Default 0.2.6.6)
Precise Editor Continued (PreciseEditor 1:1.5.0.1)
Precise Maneuver (PM) by Morse (PreciseManeuver 2:2.4.99.0-adoption)
Procedural Parts (ProceduralParts v2.5.9.0)
RasterPropMonitor (RasterPropMonitor 1:v0.31.13.4)
RasterPropMonitor Core (RasterPropMonitor-Core 1:v0.31.13.4)
RemoteTech Redev Antennas (RemoteTechRedevAntennas 0.1.1)
ReStock (ReStock 1.4.5)
SCANsat (SCANsat v20.4)
Scatterer (Scatterer 3:v0.0878)
Scatterer Default Config (Scatterer-config 3:v0.0878)
Scatterer Sunflare (Scatterer-sunflare 3:v0.0878)
Science Lab Info (and 6 Crew Lab) (ScienceLabInfo 2.0.2)
SpaceTux Library (SpaceTuxLibrary 0.0.8.6)
SpacetuxSA (SpacetuxSA 0.3.13.1)
Stockalike Station Parts Expansion Redux (StationPartsExpansionRedux 2.0.11)
Stockalike Station Parts Expansion Redux - Internal Spaces (StationPartsExpansionRedux-IVAs 2.0.11)
TAC Fuel Balancer (TacFuelBalancer v2.21.5.3)
TAC Life Support (TACLS) (TACLS 1:v0.18.0.0)
Textures Unlimited (TexturesUnlimited 1.6.1.27)
Toolbar (Toolbar 1:1.8.1.1)
Toolbar Controller (ToolbarController 1:0.1.9.11)
Tracking Station Evolved (TrackingStationEvolved 7.0)
Trajectories (Trajectories v2.4.5.3)
Transfer Window Planner - Fork (TransferWindowPlannerFork v1.9.1.0)
TriggerAu Flags (TriggerAu-Flags v2.11.0.0)
Triple-Z RadioTelescope (ZZZRadioTelescope 1.0.5.1)
TweakScale - Rescale Everything! (TweakScale v2.4.8.3)
TweakScale Redistributable (TweakScale-Redist v2.4.8.3)
Waterfall - Restock (WaterfallRestock 0.2.3)
Waterfall Core (Waterfall 0.9.0)
 

Edited August 18 by bdleitner

[bdleitner]

Well, it's a new day, so I decided to have another crack at it, so I tried some new things:

1. Move all the fairings to the bottom stage  and stage them to make sure all the fairings were deployed. That didn't unshield the parts, so I tried running my code to manually deshield them. This didn't help as they reshielded when the new save was loaded.

2. I'd read somewhere (I don't remember where) that there can be weird interactions with docks, so I tried undocking the lander and redocking it. This is where I saw that something was off as even after unselecting/reselecting the target port, it didn't want to dock at first. I tried saving the game with the ships undocked (and all fairings deployed) and then deshielding and that seemed to work!

3. To try and further isolate the problem's cause, I next tried just doing the undocking without deploying the fairings. I unshielded the parts on the main ship and also tried unshielding any parts on the lander that were mistakenly shielded, but there weren't any. Then I got my next surprise. When I saved that game, My ship had just recently left Kerbin's SOI and both the main ship and the lander were right next to each other (less than 10m apart). My save game edit did nothing except change some "shielded = True" properties on the main ship to "shielded = False".  But when I reloaded the game, The lander had decided it was still (barely) in Kerbin's SOI and was ~80 km from the main ship. I used HyperEdit to get the lander back over and redocked, this time without incident, and again it seemed to work. But after an EVA to manually put the part back where it had been, it and others were showing problems again.

4. Next test: I think I read somewhere that fairings can be finicky if they're deployed via the context/action menu vs. staging, so maybe the fairing that used to enclose the docked lander was being weird, since it was deployed not via staging. So, next experiment:
    * reload the original "What's going on here" save point.
    * Move just the already-deployed fairing to the bottom stage and stage it.
    * Undock the lander.
    * Save that game (call it Save B for reference).
    * Try the Magnetometer without any other edits (it still says stowed).
    * Try redocking (didn't work, the docking ports clearly touched and I was using MechJeb's Docking Assistant, but it did not want to go). Switching back and forth to the main ship and then unset/resetting the docking port as the target allowed it to dock.
    * Try the magnetometer again, no luck
    * Ok, go to the save file I made while the ships were undocked. Run the deshielding code on them and save to new file
    * Load that file. Again my lander is now 85km from the ship, but this time it's the main ship that's still in Kerbin's SOI and the lander is not... ok, hyperedit to rendezvous, this is a different problem.
    * Redock (no problems) (should have tried the magnetometer first, but forgot)
    * Try the magnetometer, it deploys ok!
    * Try a quick EVA to get the focus off that ship (maybe not effective since still within 2.3km and it won't reload), then try the magnetometer again, it works again.
    * and finally, save the game and immediately reload it. Try the magnetometer again.  "[]: Cannot deploy while stowed"
    * Headdesk

5. Ok... let's do that again, but this time move ALL the fairings to the bottom stage and stage them (so similar to #2 above but this time more rigorously documented).
    * After undocking the lander, this time I moved the focus back to the main ship before saving. After the save/edit/reload cycle, the main ship was back in Kerbin SOI and the lander was outside, again about 85km away.  So I've had: lander focused: lander moves back to Kerbin SOI, lander focused main ship moves back to Kerbin SOI and main ship focused: main ship moves back to Kerbin SOI.  Ok, I think I know what's up there... the shielding-removal code ends up passing some of the numbers back and forth through float64s, so there may be little rounding errors and when you're close to the SOI border, that might be enough to cause some issues.
    * Ok... since this time around all the fairings are gone and nothing should be shielded, make the code less smart... no converting nodes to make looking-for-parent-nodes easier. Just deshield everything on that vessel and don't go through float64 conversions. Upon loading that updated save, the lander and main ship are still in proximity. Great, that's one mystery solved.
    * Now that we've reloaded, try the magnetometer before redocking (it works)
    * Redock the lander (no issues)
    * Try the magnetometer again (it works)
    * Save the game and immediate load the new save.
    * Try the magnetometer one more time ("[]: Cannot deploy while stowed." AAAAAAAAAAAAAAAAAAAAAAAAAAARGH)

Ok, I have an idea for what I want to try next, but it's time for a break.

[bdleitner]

Ok, after a break for lunch I am back at it.

6. Let's isolate the docking aspect:
    * Reload the save from #5.
    * Test the magnetometer again (works)
    * Do a save/load cycle *without* redocking the lander.
    * Test the magnetometer again (works, AHA!)
    * Now redock the lander.
    * Do another save/load cycle and as expected the magnetometer is once again shielded.

It seems pretty clear that the docking aspect is key to the problem. While I've had many docked ships with extensible parts and not run into this before, I think this is the first time I've had the docking port orginally inside a fairing. So, for my next experiment:

7. Reload the original problem file so the other fairings are again undeployed. This will allow for checking the dock-fairing interaction on that one fairing without changing the others.
    * Save to a new experiment file.
    * modify the experiment file to remove the fairing part (this is where all my previous coding comes in handy) and also deshield everything that's not inside one of the other fairings, plus a tweak to preserve the orbit data so that issue doesn't come back.
    * Load the game, try the magnetometer (doesn't work)
    * Try to undock-redock the lander - same issue as before where it doesn't want to dock.

8. Ok, let's modify that one a bit. Undock first, then try the same edits (remove the fairing, deshield the parts).
    * After loading the modified save, check the magentometer (success)
    * Do a save/reload cycle and then test the magnetometer again (success)
    * Dock the lander and then test the magnetometer again (success)
    * Do another save/reload cycle while the lander is docked and test the magnetometer again (Failed ARGH, I really thought this one might do it)

Fine, let's go back to the space center and modify the initial craft file to remove the fairing that was enclosing the docking port. Launch that and look at the corresponding docking ports on the two ships (with the lander undocked on both).  I didn't see any differences except for a property "dstg"  that was 0 on the newly launched ship and 2 on the "real" one. I don't know if that matters. Ok, it definitely seems related to docking since the save/reload cycle returned things to a broken state only when the lander was docked.  Maybe there are interactions elsewhere.

Let's try going scorched earth on the fairings:

9. Ok, we'll undock first, then run the code to make the edits. The edits will:
    * Remove all the fairings, changing so the fairings children are directly attached to the fairing's parent. This took some extra work with a "new launch" to make sure I could copy orientations and get them right.
    * Unshield everything
    Ok, did that, let's see what we've got
    * Reload looks good, everything is where I expect it to be. magnetometer test successful
    * Save/Reload cycle while undocked, magnetometer check successful
    * Redock (no issues) and rerun magnetometer test (success)
    * Ok, the part that hasn't worked yet: a save/reload cycle while the lander is docked, and the magnetometer test...fails.   Seriously?

Ok, so it doesn't seem to have anything to do with fairings, since it recurs even after I've removed all of them. It definitely has something to do with docking, since it only recurs on a save/reload cycle if the lander is docked. So maybe there's something wrong with the lander's state? But I'll have to tackle that another time, that's enough for one day.

Oh, and just in case I was being really dumb... I do note that the lab (on which the magnetometer sits) does have "doors" to open to allow for running the "Observe Local Space" experiment. Just in case that was somehow causing problems if the doors were closed, I tried reopening them, but it didn't fix the magnetometer issue... so it looks like it has something to do with docking the lander.

More experimentation to come.

I'll also add links to a couple of save games files:
The baseline starting point: https://www.dropbox.com/scl/fi/l1gqscsi4pewlgtg96ebq/what-is-up-with-stowed-parts.sfs?rlkey=pgoh78b8yloxgxot03sl3fhvu&st=e32ppj9w&dl=0
The save file from the last experiment with the edits but before redocking anddoing save/reload cycles: https://www.dropbox.com/scl/fi/ac3iswyqne662nvft18hn/Experiment-F-Done.sfs?rlkey=bt0wlcxu6s7zg72mrl0rcs8g9&st=q64k3wnd&dl=0

[Lisias]

You may be had bitten by a reroot bug - and when you dock and undock a craft, the reroot code is used as part of the process.

What may be happening is that the reroot code messed up who is connected to who in a way that a whole subtree ended up being connected on an inner node of a cargo bay and so the game concluded that everything is inside it.

Ok,ok, manual editing can also lead to this same result but it's usually pretty hard to accomplish that , the cases I witnessed involved merely docking and undocking, ergo, the reroot.

I think you can workaround the problem by opening the cargobay involved on the mess. Open the cargo bays one by one and see if the part gets "unestowed". If you manage to accomplish it, you had found in which cargo bay the subassembly ended up being reattached to an inner node by accident.

Edited August 20 by Lisias
Kraken damned auto-correctors!

[bdleitner]

I don't have any cargo bays on the ship, though. I have some inventory "small cargo containers" but they don't open. And if it's one of the fairings, the problem persists even after performing surgery to remove all fairings. Though if something is wonky on the lander that might apply somehow.

If this is a problem, is there a pattern in the save file I should look for? I obviously have no problem manually editing files (or writing programs to modify them). Though based on the parent-child relationships of PART nodes in he VESSEL, the expected root is still the root.  I'm planning to try some surgery to "replace" the docking ports with fresh parts (i.e. copy the configs on those parts from new launches on the launchpad) but I just haven't gotten to it yet; probably not until the weekend.

[Lisias]

5 minutes ago, bdleitner said:

If this is a problem, is there a pattern in the save file I should look for?

Check in which nodes the parts are connecting each other. If I'm right, somehow a sub-tree those "sub-root" was attached to a external node on its parent ends up being attached to an inner node by mistake, tricking the engine into believe everything is inside that part those inner node ends up being used instead of a outer one.

 

3 minutes ago, bdleitner said:

I obviously have no problem manually editing files (or writing programs to modify them).

Welcome to the dark side of the Force! We offer the best Health Insurance available! (Angry issues not covered.)

 

[bdleitner]

On 8/21/2024 at 6:09 PM, Lisias said:

Check in which nodes the parts are connecting each other. If I'm right, somehow a sub-tree those "sub-root" was attached to a external node on its parent ends up being attached to an inner node by mistake, tricking the engine into believe everything is inside that part those inner node ends up being used instead of a outer one.

I'm not sure what's meant by internal/external or inner/outer here, and since there are no cargo bays (or fairings at this point) I'm not sure what to look for in the files.

I did try a couple new things today. First I tried having (in separate saves) copies of the main ship and the lander on the launchpad. I then went into the 3 separate saves to find the corresponding docking ports and copy all the properties (except mid and launchID) from the "new" ones to the existing ones. Same issue that once I redocked and reloaded it was back to being stuck.

But in the process I realized that (because it had gone through a subassembly), the lander had its docking port as its root.  Wondering if that had contributed to some kind of re-root bug I went back into the VAB for the lander and re-rooted to a different part. I then saved the game with that vehicle on the launchpad and went back and replaced all the part nodes on the "real" lander with the ones from the new one (not even bothering to preserve mid or launchID). That meant the docking port was both fresh and no longer the root of its craft, and this was essentially a brand new lander.  Sadly, this did not help either. Once again it worked at first but not after both redocking and reloading.

So, finally, I went nuclear. I recreated a modified version of the main ship with no fairings (and it never had cargo bays) and put it on the launchpad, then in my code I replaced all the parts of BOTH ships with the parts from the launchpad versions.  So now back out in space I have two ships that look essentially thesame as they used to be, but in terms of parts they are effectively both fresh off the launchpad. This time, I didn't even need the save/reload cycle. As soon as I docked the magentometer (and antenna since it was no longer extended fresh off the pad) gave the same stowed error.

I don't know what's going on, but it doesn't seem tied to anything I did with the parts post-launch.

I'll try moving the magnetometers to the lander (even though they're useless when landed) just so I can try using them evne if I need do it while undocked. Clearly this is a bug somewhere and not something I got myself into.

But as for solving the issue? I give up.

[Lisias]

7 hours ago, bdleitner said:

I'm not sure what's meant by internal/external or inner/outer here, and since there are no cargo bays (or fairings at this point) I'm not sure what to look for in the files.

<...>

But as for solving the issue? I give up.

Can you share a simple savegame and craft file where the problem happens to you? (as well the KSP.log!).

This is mostly odd, I never caught a similar problem where a part with inner nodes (nodes usually used to attach cargo inside it) being wrongly used before...

[bdleitner]

Ok, I reran the simplest experiment (just mark the parts deshielded) and the most aggressive experiment (full part list replacement) to confirm the results and make sure I had fresh saves.

I compiled the results, the list of parts that get magically reshielded, the relevant saves, and the logs and put them all at:

https://github.com/bdleitner/KSP-Shielding-Issue

Hopefully that'll provide enough information, but I think I'll just try to use EVA construction mode to move the affected parts over to the lander. It'll be a bit weird having to undock the lander to do space-borne experiments but at least they aren't biome specific so I only need to do that once per body.