Can't load with MechJeb installed

[Lisias]

6 hours ago, Problemless Mods Wanter said:

=) I'm listening?

Just silliness. You will be the first to know when it's ready!

 

6 hours ago, Problemless Mods Wanter said:

By the way 10+ mods updated since we last spoke, i tried again, but still no luck.

The bad news about this problem is that it's not simple as someone borking on the code. It's a System Integrations conflict - a specific combination of things, that works perfectly fine by themselves or with each other when you get one of them out of the equation.

Most of the time we misdiagnose the problem and pinpoint just one of the variables of that failing equation thinking we found the problem, but in reality, we had just identified one  chain of the chain of events that culminate to the problem.

On a side note, we are having a glimpse of what they got through while building Saturn V and the Apollo Missions - no joking. At the time most of the problems were on integrating hardware components, not software - but the theory is similar to what we get here: identify the whole chain of events that lead to the problem in order to get the root cause of the glitch and see the most economic/safe/both way to prevent it to happen.

There's no time enough on a lifetime in order to learn all we need to really appreciate what was done!

https://www.zdnet.com/article/to-the-moon-ibm-and-univac-appollo-11s-integrators/

[Lisias]

I got a further insight about this problem. It's way hairy than its appears, and just pinpoint fingers to each other is not going to make things better.

Quote

TL;DR: "Outdated" Add'Ons are not the problem. They never were. They were used as scapegoats to justify problems deeper on the stack, at the same time deflecting the responsibility for fixing the problems - what ends up in perpetuating them.

We are in a mine field. Deactivating only the visible mines does not solve the problem.

First and for once, this is not a problem on MechJeb. It's not a problem on (old) Firespitter neither. It's something deep inside the stack.

To the moment, what I can confirm is that an Exception happening on a very special moment inside the runtime renders the Reflection unusable. In the past, I had misdiagnosed the problem reducing it to a failed DLL loading, but now Firespitter gave me evidence that there are more triggers inducing this misbehaviour, as there're no DLL failing to load on this case (it was shaders failing to be loaded!).

In a way or another, FIrespitter borking lead to MechJeb borking. And MechJeb borking leads to the Expansion borking on loading the DLCs:

[LOG 00:33:07.191] Expansion makinghistory detected in path /Users/lisias/Workspaces/KSP/runtime/1.8.1/GameData/SquadExpansion/MakingHistory
[LOG 00:33:07.191] Expansion serenity detected in path /Users/lisias/Workspaces/KSP/runtime/1.8.1/GameData/SquadExpansion/Serenity
[EXC 00:33:07.209] ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
        System.Reflection.Assembly.GetTypes () (at <ad04dee02e7e4a85a1299c7ee81c79f6>:0)
        AssemblyLoader.GetTypesOfClassesImplementingInterface[T] () (at <394a98b9c7624adc895c04290da62640>:0)
        Expansions.Missions.MissionsUtils.InitialiseAdjusterTypes () (at <394a98b9c7624adc895c04290da62640>:0)
        Expansions.ExpansionsLoader+<LoadExpansions>d__21.MoveNext () (at <394a98b9c7624adc895c04290da62640>:0)
        UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at <7d9ec060e791409ab3eb85c61e312ed6>:0

And its this last failing that is preventing the Main Menu to be loaded.

By deleting Firespitter, you can use MechJeb and the DLCs. By Deleting MechJeb and the DLCs, you can't use Firespitter. [By deleting Module Manager, you can use Firespitter]. Right, we have a pattern.

Firespitter is involved on this mess for sure. Let's keep diagnosing it, however, instead of pinpoint the first symptom detected as being the disease (we do not treat Malaria with antipyretics, you know…).

By deleting the DLCs, you can't use Firespitter and MechJeb, because MechJeb borks on every KSP.IO call it does (that triggers a Reflection Exception, and then something else prevents the Main Menu to be loaded) due the Firespitter's bork.

So I made an experiment, and compiled a MechJeb DLL that doesn't uses the KSP.IO (using an alternate, non reflection dependent, mechanism). It worked a bit more, even with the faulty Firespitter, but then it borked on the 

[MechJeb2] ERROR: module threw an exception in OnLoad: MechJebModuleMenu System.Reflection.ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
  at (wrapper managed-to-native) System.Reflection.Assembly.GetTypes(System.Reflection.Assembly,bool)
  at System.Reflection.Assembly.GetExportedTypes () [0x00000] in <ad04dee02e7e4a85a1299c7ee81c79f6>:0 
  at MuMech.ToolbarTypes.getType (System.String name) [0x0002b] in <715215068edb467e85d0d3c1bcf9e83c>:0 
  at MuMech.ToolbarManager.get_Instance () [0x0002d] in <715215068edb467e85d0d3c1bcf9e83c>:0 
  at MuMech.ToolbarManager.get_ToolbarAvailable () [0x00013] in <715215068edb467e85d0d3c1bcf9e83c>:0 
  at MuMech.MechJebModuleMenu.ClearButtons () [0x0003d] in <715215068edb467e85d0d3c1bcf9e83c>:0 
  at MuMech.MechJebModuleMenu.OnLoad (ConfigNode local, ConfigNode type, ConfigNode global) [0x00001] in <715215068edb467e85d0d3c1bcf9e83c>:0 
  at MuMech.MechJebCore.OnLoad (ConfigNode sfsNode) [0x00347] in <715215068edb467e85d0d3c1bcf9e83c>:0 

And, again, the problem is the runtime's Reflection subsystem. And, I want to say it again, it's not a problem on MechJeb neither Firespitter. They are just the messengers, they are the ones with the unhappy luck of stepping on a mine while walking on the minefield.

This is so true that removing Firespitter without removing KAX, that depends from it, I got this:

[LOG 12:44:57.519] FSCoolant not found in resource database. Propellant Setup has failed.
[ERR 12:44:57.525] Module ModuleEnginesFX threw during OnLoad: System.NullReferenceException: Object reference not set to an instance of an object
  at ModuleEngines.SetupPropellant () [0x000a4] in <394a98b9c7624adc895c04290da62640>:0
  at ModuleEngines.OnLoad (ConfigNode node) [0x000d7] in <394a98b9c7624adc895c04290da62640>:0
  at PartModule.Load (ConfigNode node) [0x001ab] in <394a98b9c7624adc895c04290da62640>:0

And the game loading stopped on the spot. Everything is stalled, except by the Loading Screens and the fancy loading messages - what makes me believe that the loading thread was killed.

Well, we have enough evidences about the nature of the problem already. But one more test is in need to satisfy my death wishes: I need to artificially make MechJeb2 bork the game loading. So I decided to induce it to bork in the very same way Firespitter was borking. I'm working on it at the moment, I'll be back about this.

I'm trying to diagnose this thing since last year, yet on the KSP 1.4 era (didn't tried it on KSP 1.3 and 1.2… Something to be tried this weekend…). This is not something new, this is not about "outdated" Add'Ons. This is a bug on the runtime Reflection subsystem and this is not going away.

Even Squad's own modules are borking under the same circumstances.

In order to get this thing safely handled, do not limit yourselves on making tests on "outdated" Add'Ons. Every single Add'On that uses Reflection are prone to this, and every single Add'On that borks something inside the runtime is prone to be the trigger of the problem (not matter its an "outdated" or "updated" Add'On!!)

My evidences above confirms that MechJeb can be the victim, but also the trigger for this thing.

This is a hint that Firespitter is not the only one borking here. What does not means that Firespitter did not needed to be fixed. Just do not feel safe thinking you solved the problem, because you didn't.

The mines are still on the minefield, you just located and deactivated one of them.

— — POST EDIT — — 

By deleting Module Manager, Firespitter kinda works. Most FS modules will not work, all right. But the parts works and KSP do not crashes on the loading.

Also published on my site.

Edited November 16, 2019 by Lisias
Kraken damned Autocompletes **<n> (who's counting this anyway?) — Post edit.

[Stone Blue]

I just wanted to point out, based on the timeline of this thread, that it appears @Problemless Mods Wanter has/had some of Alista's mods installed, which were *not* updated for KSP 1.8.+ at the time... Allista only started releasing some initial 1.8.1 updates for his mods a day or two ago...

So, i guess, i would say, might want to scrub thru installed mods again, checking for *verified*, official 1.8.1 compatability.

(this part below is NOT just for you, Problemless Mods Wanter)
Remember, CKAN and KSP-AVC do *NOT* version check every mod out there... even tho many that do not get checked may stil have a .version file packaged. Many devs do not want their mods to be handled by CKAN, and theres no guarantee those devs have taken steps to have KSP-AVC do so instead... vOv
Anything that falls in those cracks would *have* to be manually looked up for 1.8.+ compat.

Edited November 15, 2019 by Stone Blue

[Lisias]

45 minutes ago, Stone Blue said:

So, i guess, i would say, might want to scrub thru installed mods again, checking for *verified*, official 1.8.1 compatability.

 

You guessed wrong.

it's a weird interaction when an Add'On borks, leading to ANOTHER Add'On to Bork in a visible way.

Read again my post above. I induced a Stock module to bork the Main Menu loading by using a MM patch.

[PyjackMeat]

https://drive.google.com/open?id=1mu2xjGVXoj_XPu32LQafb8LFqPYMiFM4 -ksp.log
https://drive.google.com/open?id=1Kf-WbJ2IQjDfcFc81HEykkOCsVXyim0A -player.log

Logs, because they were asked for.

Been fiddling with my modlist and found Kerbcam, Probe Control Room, TacFuelBalancer, TacSelfDestruct FlightPlan and EVA Enhancements Continued to be exception sources with MJ.

Edited November 15, 2019 by PyjackMeat

[sarbian]

Install the new MM and MJ I just released and you should get more info about the broken mods on your loading screen when the extension load freeze.

Edited November 16, 2019 by sarbian

[Stone Blue]

5 hours ago, Lisias said:

You guessed wrong.

Actually, no, i did not guess wrong. Allista only updated his AT_Utilities mod (used in all his mods), just yesterday. Look thru the last page or so of most of his mod threads. There have been issues, and no official 1.8.+ compatability stated on most, if not all of them, until just yesterday.

If Allista's mods slipped thru while checking the OP's install to make sure everything was indeed 1.8.+ compatable, who's to say others didnt slip thru. vOv

My post was *just* to say, that before digging into this deeper, everything should be checked again.
Totally seperate issue from having incompatable/interacting mods installed together.

Edited November 15, 2019 by Stone Blue

[Lisias]

On 11/15/2019 at 8:39 PM, Stone Blue said:

My post was *just* to say, that before digging into this deeper, everything should be checked again.
Totally seperate issue from having incompatable/interacting mods installed together.

I think we are talking about different triggers for the same problem.

I'm not concerned about triggers anymore - if we manage to define a solid and effective way to detect the problem happening, the triggers will show by themselves on the log.

The Main Menu not being showing has a reason. A mere patch borking a Module OnLoad event is a different trigger, happening in a different time, but the ending result is the same: the thread dies abruptly, perhaps inducing its father to die in a unhandled exception.

My current postulate about the Reflection thingy is that if a thread dies abruptly, some needed code is not run leaving a frame polluting the stack - perhaps the C++ stack (or something like that, I'm guessing). This is not a trivial problem, this is not a mishap made by amateurs - you have to be a God Damned Professional in order to make a marvelous bork this big  

This bug is a fierce fighter. Merely patching triggers as we are doing since last year (that I'm aware, I arrived here in the early days of 1.4.0) is not helping anymore, as we ending up just throwing the ticking bomb to the next guy.

Using Firespitter as an example: Firestpitter was not incompatible with MechJeb. Firespitter was borking trying to load an inexistent Particle Effect (or Shader, whatever was being tried first). That bork should kill only Firespitter, but by some reason, this triggered a problem on the runtime's Reflection - a thing heavily used by MechJeb and the DLC loading subsystem.

Since MechJeb is loaded before the DLCs are checked, MechJeb was taking the heat. Once you delete MechJeb, the next victim was the DLC loader.

When Firespitter was patched to prevent loading Particle Effects and Shaders, you didn't made Firespitter "compatible" to MechJeb or DLCs, you just prevented the trigger to be pulled by Firespitter. But the trigger is still there, waiting by some other unluck code to trigger it again.

My metaphor is accurate. We are all walking on a mine field, and when one of us steps on a mine, it blows hurting the stepper but also who is next. Bashing the steppers is useless, we need to find the mines!

Edited October 17, 2023 by Lisias
Eliminating attrition with the moderation

[Lisias]

I just tested the new Module Manager. It does what is expected to do, it pinpoints any DLL borking on the critical sections of the KSP life cycle.

However, I want to dispute the error message.

Again, the problem is not (and never were) about outdated DLL "incompatible" to KSP, MM or MechJeb. The problem is faulty DLLs (being it updated or outdated) borking on some critical sections of the KSP life-cycle, inducing KSP to a crash.

By removing a needed resource, you can induce Stock, Squad made ModuleEngines to crash the game loading the same way:

[LOG 12:44:57.519] FSCoolant not found in resource database. Propellant Setup has failed.
[ERR 12:44:57.525] Module ModuleEnginesFX threw during OnLoad: System.NullReferenceException: Object reference not set to an instance of an object
  at ModuleEngines.SetupPropellant () [0x000a4] in <394a98b9c7624adc895c04290da62640>:0
  at ModuleEngines.OnLoad (ConfigNode node) [0x000d7] in <394a98b9c7624adc895c04290da62640>:0
  at PartModule.Load (ConfigNode node) [0x001ab] in <394a98b9c7624adc895c04290da62640>:0

And I have a hard time trying to classify ModuleEngines as an "outdated Mod". To make my point, I induced this problem using a MM patch (the full details are on the KAX Issue #6).

Long story made short, I made a part.cfg and a patch.cfg (replicating a configuration I know borks KSP) to be used before any of Squad's owns. Module Manager applies the patch normally, and after it finishes, KSP kills the loading thread while loaded the patched part. Then I restarted KSP, when MM just reloaded the ConfigCache and called it a day, and then the KSP loading thread continued its job and borked the same way. Of course, deleting ModuleManager and recooking that part to incorporate the patch (as Stock parts do), the KSP crashes the same way.

So we have a confirmation of an anomaly on KSP itself. Ok, it's an Elon Musk kind of Anomaly. But yet, just an anomaly  

Then I made another stunt: installed "my" fork of Firespitter DLLs, that needs KSPe (a lib of mine). But I didn't installed KSPe on purpose, to make the FS DLL to bork on loading and not while its initialization. Guess what? Things kinda worked this time! Of course, FS modules weren't loaded, so they didn't worked - but KSP loaded fine, and the Modules that loaded worked! (and yeah, this was unexpected).

Spoiler

Logs available no KAX Issue #6.

So, and again, I have confirmation that it's not every DLL bork that triggers that Anomaly. The code must throw an unhandled exception on a specific phase of the father's thread life cycle in order to crash KSP.

What again pinpoints Module Manager as only the main involved on the problem due it's popularity. Had I not made a stock module to kill the game loading as above, and then confirmed my hypothesis using my personal forks of the Add'Ons, I would pinpoint MM as the cause of the problem. Now that I know better, I realize that MM was only the preferred screaming victim of a problem on the stack, because before 4.1.2 it was not handling the ReflectionTypeLoadException , allowing it to be raised to it's father thread, that so crashed KSP as I did above.

Again, we don't have a problem caused by "outdated DLLs'. They are just the preferred Screaming Victims, as they are more prone to fall due this problem - the same way MM was until recently. The problem is an anomaly on KSP's thread's life-cycle that kills itself when ir receives an unhandled exception sometimes (other times, it does not - appears to be some missing try catches).

Such anomaly is a problem that, besides looking terribly bad, it's not - it's just an annoyance. What made it terribly bad is people deflecting responsibilities by electing scapegoats to save face.

Also published on my site.

Edited November 16, 2019 by Lisias
Links

[sarbian]

3 hours ago, Lisias said:

What again pinpoints Module Manager as only the main involved on the problem due it's popularity.

I am amazed at how you manage to reach a conclusion so far from the mark. MM did not trigger the problem before and it does not now. You did try to understand how the code added intercepts Exception from other mods and the stock game and went straight to "It was MM all along!".

4 hours ago, Lisias said:

Again, the problem is not (and never were) about outdated DLL "incompatible"

I doubt players would read a 40 lines explanation. However I see that I am missing a verb in the first sentence (?!). The purpose of the message is to help diagnose a problem, not instruct everyone on the fine point of assembly loading. 

As to generating NRE when you have a broken conf it is nothing new.

[Lisias]

25 minutes ago, sarbian said:

I doubt players would read a 40 lines explanation. However I see that I am missing a verb in the first sentence (?!). The purpose of the message is to help diagnose a problem, not instruct everyone on the fine point of assembly loading. 

But Add'On authors will. And the wording can help easy the heat when the user reach the author. The heat is unavoidable (believe me, I know  ), but it can be easied. 

 

26 minutes ago, sarbian said:

As to generating NRE when you have a broken conf it is nothing new.

It's not the NRE the problem. Is when it happens. Sometimes the NRE is "Mostly Harmless" (good book), sometimes it's devastating - it depends on the calling thread being protected by a try catch (or something like that) to avoid being killed - or not.

Since it's nuts to shove a Try Catch everywhere, as it creates and destroys frames on the stack and this costs processing (I would not do that on Fixed Update!), we now have a good insight about where to look for trouble.

[Problemless Mods Wanter]

First of all, I apologize that since I was feeling down about the subject, I tried to stay away from the forums and KSP for the last week. Turns out I missed most of the recent posts, especially on my own thread.

I'm trying to catch up, but as usual, most of the arguments are way beyond my comprehension and unfortunately as "users" we can only point out problems, provide logs and do some tests as suggested. Most of the time I wish I could also help with the coding too...

But I feel thankful that the issues are still being discussed and possible solutions are provided from everybody. The outcomes of these kinds of discussions are always for the good of the community, so I feel glad that it's happening, as long as it is maintained both friendly and in a constructive manner.

As for the regular players who are or will be, stumbling upon this thread, I like to give some updates, starting with;

On 11/15/2019 at 8:41 PM, Stone Blue said:

I just wanted to point out, based on the timeline of this thread, that it appears
@Problemless Mods Wanter has/had some of Alista's mods installed, which were *not* updated for KSP 1.8.+ at the time... Allista only started releasing some initial 1.8.1 updates for his mods a day or two ago...

I thank you for taking the timeline into account and giving a heads up on the Alista's mod updates. Those mods and Alista's topics were already in my radar and was wondering if they were the reason or not and hoping for an update.

I noticed the updates today (along with the newer versions of; B9 Aerospace Procedural Wings - Fork, Community Tech Tree, Kerbal Alarm Clock, Mark IV Spaceplane System, The Janitor's Closet and of course; MechJeb 2 & Module Manager) and immediately gave launching KSP a try. Unfortunately (or fortunately) even with those updates, the game still didn't load.

On the good side, the new addition to MJ code was able to detect (at least one of the) culprit(s) as I reported here;

If it's any help, I also would like to leave a note that, @Belivern reported the same symptoms here;

and the only mutual mods that I have with him/her are the "ModuleManager", "Firespitter" and "DockingPortAlignmentIndicator".

I also love to use "SCANsat" but it was removed along with 40 or so mods while I was generating these reports.

Please remember that I already renunciated from most of my favorite mods in order to narrow it down...

So even with the Kerbal Foundries (mentioned above) update, I don't believe the hunt will be over soon.

But eventually It will I guess...

 

What I actually wonder is; can't MJ just ignore these "particle" or "texture" related dll errors and just let KSP load?

Please do not try to answer this, it is a rhetorical question and I won't understand the answer anyways.

I assume it's some sort of a failsafe to prevent unnecessary reports, so I can understand if it just won't...

 

Anyways, I thank you all again, for all your efforts to make the mods of this game, even better  and keeping the community great...

 

As for anyone interested to see my latest log file, it may be downloaded from this link;

https://gofile.io/?c=JuKNLL

    Peace & out!  

(for now)

[Lisias]

On 11/16/2019 at 7:59 PM, sarbian said:

I am amazed at how you manage to reach a conclusion so far from the mark. MM did not trigger the problem before and it does not now. You did try to understand how the code added intercepts Exception from other mods and the stock game and went straight to "It was MM all along!".

I'm amazed how easy is to startle you. I'm one of the ones telling everybody that MM (and MJ2) are Screaming Victims of a wider problem - and I really fail to understand where you read something of mine talking otherwise. From my post above, from where you didn't quoted enough (emphasis are new):

On 11/16/2019 at 3:49 PM, Lisias said:

What again pinpoints Module Manager as only the main involved on the problem due it's popularity. Had I not made a stock module to kill the game loading as above, and then confirmed my hypothesis using my personal forks of the Add'Ons, I would pinpoint MM as the cause of the problem. Now that I know better, I realize that MM was only the preferred screaming victim of a problem on the stack, because before 4.1.2 it was not handling the ReflectionTypeLoadException , allowing it to be raised to it's father thread, that so crashed KSP as I did above.

Jumping fast into conclusions before reaching the end of the sentence can be a symptom of Anxiety. I'll try to phrase my sentences to avoid startling you again.

But at least you seem to grasp something from what I said, as it appears. 

5 hours ago, sarbian said:

As I explained in an other thread it creates a problem for reflexion call that puts Unity mono in an instable state which then breaks the extension loading that does a lot of reflexion. MJ uses some stock code (KSP.IO) that do reflexion and trigger this but other mods could do it. And it could also break stuff in game silently and mess up saves.

source.

On 11/15/2019 at 2:19 PM, Lisias said:

By deleting the DLCs, you can't use Firespitter and MechJeb, because MechJeb borks on every KSP.IO call it does (that triggers a Reflection Exception, and then something else prevents the Main Menu to be loaded) due the Firespitter's bork.

So I made an experiment, and compiled a MechJeb DLL that doesn't uses the KSP.IO (using an alternate, non reflection dependent, mechanism). It worked a bit more, even with the faulty Firespitter, but then it borked on the 

source.

[sarbian]

@Lisias, as I said in the other thread I am tired of this. You do not read my post and just push your conclusion. The problem is now properly diagnosed and the code I added to MM make it easy for anyone to know the DLLs that trigger it. 

[Lisias]

2 hours ago, sarbian said:

@Lisias, as I said in the other thread I am tired of this. You do not read my post and just push your conclusion. The problem is now properly diagnosed and the code I added to MM make it easy for anyone to know the DLLs that trigger it. 

You diagnosed only the problem under you nose.  Better than nothing, but at the same time perpetuates some misconceptions.

Oh well, I'm pretty sure you did your best. Go get a rest, you surely need it.