[1.4.3 <= KSP <= 1.12.5] KSP Recall - 0.5.0.2- 2024-0521

[TurtleMountain]

2 hours ago, Lisias said:

Send me the KSP.log using dropbox and I will check it.

— — POST EDIT — — 

I just fired up my Windows Test Bed, and other than PD-Launcher downloading an update (still analysing what wold it be), I had no problems on running Stock 1.12.4 KSP and KSP-Recall on a full Steam managed installation.

Please send me your KSP.log - there's something different on your installation for sure.

Heres the KSP.log: https://www.dropbox.com/s/n0mzylwe8nev754/KSP.log?dl=0
Not sure if it matters, but I am running 95 mods and launching with CKAN.

[Lisias]

18 minutes ago, TurtleMountain said:

Heres the KSP.log: https://www.dropbox.com/s/n0mzylwe8nev754/KSP.log?dl=0
Not sure if it matters, but I am running 95 mods and launching with CKAN.

Working on it. I will update this post as soon as I have something.\

— — POST EDIT — — 

@TurtleMountain

I just updated KSP-Recall (0.3.0.6) to add some diagnosing messages when this happens. Can you please install it and send me a new KSP.log? Thanks!

Edited November 25, 2022 by Lisias
post edit

[HebaruSan]

General recommendation: If you're going to print an error message because two values don't agree, make sure it displays both values. Saves a lot of time guessing what might be happening.

Edited November 25, 2022 by HebaruSan

[TurtleMountain]

18 minutes ago, Lisias said:

@TurtleMountain

I just updated KSP-Recall (0.3.0.6) to add some diagnosing messages when this happens. Can you please install it and send me a new KSP.log? Thanks!

Here it is https://www.dropbox.com/s/b7rkj03y1nvzszx/KSP 2.log?dl=0

[Lisias]

23 minutes ago, HebaruSan said:

General recommendation: If you're going to print an error message because two values don't agree, make sure it displays both values. Saves a lot of time guessing what might be happening.

Point taken. I should had printed it on the KSP.log at first place - at very least. 0.3.0.6 is doing it now.

As soon as Black Friday is over, I will rework the whole Dialog.

[Lisias]

55 minutes ago, TurtleMountain said:

Here it is https://www.dropbox.com/s/b7rkj03y1nvzszx/KSP 2.log?dl=0

Bingo.

[LOG 15:25:17.633] [KSP-Recall] Version 0.3.0.6 /L running on KSP 1.12.4
[LOG 15:25:19.511] [KSP-Recall] ERROR: pwd != Application Root! -- pwd=c:\program files (x86)\steam\SteamApps\common\Kerbal Space Program\ ; AppRot=C:\Program Files (x86)\Steam\steamapps\common\
Kerbal Space Program\ at error:0
[LOG 15:25:19.514] [KSP-Recall] "Houston, we have a problem!" about "Your 'pwd' doesn't match KSP's 'Application Root'!" was displayed

By some reason beyound me, your "Program Files" directory is named "program files" in your hard disk.

This leaves me with two problems to solve: why this happened to you (my Windows 10 test bed have it as "Program Files"!), and why KSP didn't reflected it on the Application Root, because I made a test here renaming "C:\Users\lisias\Desktop\Kerbal Space Program" to "C:\Users\lisias\Desktop\Kerbal Space program" and things worked for me - the KSPUtil.ApplicationRoot correctly game me "C:/Users/lisias/Desktop/Kerbal Space program/KSP_x64_Data/../".

Spoiler

(and, as a side note, I deeply despise Unity's bad habit of using "/" on Windows! Damnit, they should be using UNC - none of the many problems we have and had with pathnames would be happening if they did that).

For now, please ignore the Dialog - you can click Cancel or OK, it will just keep loading KSP - and since both PWD and AppRoot are really the same in the end, nothing bad will happen.

I'm working right now on the matter, trying to reproduce the problem on my Windows test bed in order to understand why you are being bitten in the SAS  by this Windows' crappyness.

— — POST EDIT — — 

There's something wrong, and I don't think it's on your file system!

I just realised this: in your log, we have: "c:\program files (x86)\steam\" and this doesn't looks at all like mine, "C:\Program Files (x86)\Steam\" .  And it doesn't looks as you had renamed something because even the "c:" part of your pathname is in lower case! On every case on my rig, the "C:\" is always on upper case!

 

Edited November 25, 2022 by Lisias
POST EDIT.

[HebaruSan]

2 hours ago, HebaruSan said:

upper/lower case disagreement? I see this is implemented just with String.Equals, which is a rather poor approximation of path equality.

16 minutes ago, Lisias said:

Quote

[LOG 15:25:19.511] [KSP-Recall] ERROR: pwd != Application Root! -- pwd=c:\program files (x86)\steam\SteamApps\common\Kerbal Space Program\ ; AppRot=C:\Program Files (x86)\Steam\steamapps\common\Kerbal Space Program\ at error:0

Yup, that's not too surprising. Path equality is non-trivial to check (for reasons reflected above plus several others that you'll probably encounter eventually if you keep this check), kind of like with floating point numbers; and like floating point numbers, it's probably best to avoid checking paths for equality when possible.

Edited November 25, 2022 by HebaruSan

[TurtleMountain]

48 minutes ago, Lisias said:

By some reason beyound me, your "Program Files" directory is named "program files" in your hard disk.

Its lowercase in the logfile but the actual folder is uppercase in Windows Explorer so I don't really know what's going on there.

48 minutes ago, Lisias said:

This leaves me with two problems to solve: why this happened to you (my Windows 10 test bed have it as "Program Files"!), and why KSP didn't reflected it on the Application Root, because I made a test here renaming "C:\Users\lisias\Desktop\Kerbal Space Program" to "C:\Users\lisias\Desktop\Kerbal Space program" and things worked for me - the KSPUtil.ApplicationRoot correctly game me "C:/Users/lisias/Desktop/Kerbal Space program/KSP_x64_Data/../".

I'm no programmer, but I noticed your KSP folder is on your desktop instead of Program Files (x86). Could there be some kind of weird permission issue?

Anyway, regardless of whether we find out what is causing this i'm just glad to know that the scary warning message was a false positive and that my save files aren't going to go poof. Thanks for the help and information @Lisias @HebaruSan.

Edited November 25, 2022 by TurtleMountain
thought i had to leave but i have more time

[Lisias]

Just now, HebaruSan said:

Yup, that's not too surprising. Path equality is non-trivial to check (for reasons reflected above plus several others that you'll probably encounter eventually if you keep this check), kind of like with floating point numbers;  and like floating point numbers, it's probably best to avoid checking for equality when possible.

That's the whole problem! KSPe handles all of that! I wrote tons of code to guarantee that whatever KSPe gives me, it's always normalised (".." and "." resolved, duplicated paths separators eliminated, all paths separators converted to the host operating system, you name it) to reflect exatly what the host operating system expects from a pathname. It even checks if the pathname exists and, if it is a directory, appends a PathSeparator to the pathname automatically.

That code is tested on the field and it's proven to work on Windows, Linux, MacOS and even Steam Deck with or without Proton (running under Proton is a pretty strange environment, as we have a "Windows" runtime environment that have access to a UNIX filesystem and utils - this had bitten me in the SAS horribly in the past).

Whatever the problem is, it's not the Equals. Something different is happening on the user's machine - even that he had renamed "C:\Program Files (x86)\Steam" to "C:\program files (x86)\steam", he would not be able to rename the "C:" drive to "c:".  As far as I now, you can't rename a drive in Windows, they are always in upper case!

[HebaruSan]

6 minutes ago, Lisias said:

KSPe handles all of that! I wrote tons of code to guarantee that whatever KSPe gives me, it's always normalised (".." and "." resolved, duplicated paths separators eliminated, all paths separators converted to the host operating system, you name it) to reflect exatly what the host operating system expects from a pathname. It even checks if the pathname exists and, if it is a directory, appends a PathSeparator to the pathname automatically.

But you're not using any of that to get the current working directory; you're relying on the .NET runtime, which apparently doesn't do any such normalization:

https://github.com/net-lisias-ksp/KSP-Recall/blob/1b44d90466a92226552533b84649892494ce3763/Source/KSP-Recall/SanityCheck.cs#L84

string pwd = System.IO.Directory.GetCurrentDirectory();

To compare two paths using a normalization scheme, both paths would need to be normalized. Which means the code needs to carefully track somehow which paths are and are not normalized, and anytime it gets mixed up, a bug is born. Hence it being a good idea to avoid comparing paths like that.

Edited November 25, 2022 by HebaruSan

[Lisias]

1 hour ago, HebaruSan said:

But you're not using any of that to get the current working directory; you're relying on the .NET runtime, which apparently doesn't do any such normalization:

https://github.com/net-lisias-ksp/KSP-Recall/blob/1b44d90466a92226552533b84649892494ce3763/Source/KSP-Recall/SanityCheck.cs#L84

string pwd = System.IO.Directory.GetCurrentDirectory();

To compare two paths using a normalization scheme, both paths would need to be normalized. Which means the code needs to carefully track somehow which paths are and are not normalized, and anytime it gets mixed up, a bug is born. Hence it being a good idea to avoid comparing paths like that.

Looks like that, but so, why this is working fine on my machine? You see, the user's pathnames are correct:

[LOG 15:25:05.973] AssemblyLoader: Loading assembly at C:\Program Files (x86)\Steam\steamapps\common\Kerbal Space Program\GameData\999_Scale_Redist.dll

Drive letter in upcase, "Program Files" and "Steam" - all the paths are fine.

I agree that I should not be relying on the .NET runtime for that, that was a huge mistake of mine, and I'm going to fix it tonight. But I also need to understand how in hell the user's GetCurrentDirectory got screwed this way.

hummm...

@TurtleMountain, how do you launches KSP? Steam Launcher? Command Line? Shortcut? If by shortcut, how did you created it?

— — POST EDIT — — 

@HebaruSan, you are going to like this one.

Open CMD.exe and type exactly:

cd "c:\program files (x86)"

you will get:

c:\Program Files (x86)>_

Yeah, drive in lowercase, directory name as it is on the file system

Now open the PowerShell and do it again. You will get

PS C:\program files (x86)> _

Yeah, drive letter in uppercase, directory name as you typed it, and not as it is on the filesystem.

.NET framework is innocent on this one. We are dealing with Windows - there's no standard on how the PWD is set on Windows. Anyone trusting Windows is  going to get screwed, being it .NET, Unity or KSPe.

Learnt my lesson. I now what to do - now I just need to figure out how to do it in a way I never had to cope with this mess again.

-- -- POST POST EDIT -- --

Almost;

Both PowerShell and CMD didn't triggered the problem, even with the PWD being set with a different case.

Well, I think I have a new suspect. I need to reproduce the problem on this machine, otherwise I would not be sure if I really had fixed the problem with the code I'm writting on KSPe.

-- -- POST POST POST EDIT -- --

Oukey, now I'm showing off.

I had tried Cygwin, and the damned thing worked the same!

I still have one more hope to reproduce the thing on my machine. Working on it.

Edited November 25, 2022 by Lisias
POST EDIT

[TurtleMountain]

10 minutes ago, Lisias said:

@TurtleMountain, how do you launches KSP? Steam Launcher? Command Line? Shortcut? If by shortcut, how did you created it?

Not sure, I just run the mod manager CKAN then click Launch Game. If I had to guess CKAN executes KSP through command line but I cannot say for sure.

Also not sure if you noticed or if its relevant but there was another case difference in the logfile, pwd was SteamApps and application root was steamapps. So something is selectively lowercasing Program Files and uppercasing SteamApps.

Edited November 25, 2022 by TurtleMountain
typo

[HebaruSan]

For what it's worth, Windows is far from the only platform that is subject to thorny problems with comparing paths. You've used Unix and Unix-like operating systems, so you may be familiar with the concept of symbolic links.

On my Ubuntu system, I have done this for convenience:

ln -s '~/.local/share/Steam/steamapps/common/Kerbal Space Program' ~/.ksp

This allows me to easily get into the game folder without typing long paths:

cd ~/.ksp
./KSP.x86_64

Thanks to the magic of symbolic links, the game is really located at '~/.local/share/Steam/steamapps/common/Kerbal Space Program', but the ~/.ksp path points to the exact same folder. If my app root is '~/.local/share/Steam/steamapps/common/Kerbal Space Program' but my current working directory is ~/.ksp, that's perfectly fine and should not be alerted as incorrect by a mod.

[Lisias]

15 minutes ago, HebaruSan said:

For what it's worth, Windows is far from the only platform that is subject to thorny problems with comparing paths. You've used Unix and Unix-like operating systems, so you may be familiar with the concept of symbolic links.

On my Ubuntu system, I have done this for convenience:

ln -s '~/.local/share/Steam/steamapps/common/Kerbal Space Program' ~/.ksp

This allows me to easily get into the game folder without typing long paths:

cd ~/.ksp
./KSP.x86_64

Thanks to the magic of symbolic links, the game is really located at '~/.local/share/Steam/steamapps/common/Kerbal Space Program', but the ~/.ksp path points to the exact same folder. If my app root is '~/.local/share/Steam/steamapps/common/Kerbal Space Program' but my current working directory is ~/.ksp, that's perfectly fine and should not be alerted as incorrect by a mod.

Ah, but this one I have sorted on KSPe!!! I also have a symlink system on my MacOS. You may be interested on this repo:

https://github.com/net-lisias-ksp/KSP-Hacks

by the way. I have all my add'ons unziped on a directory called _pool, and then I symlink everything on my many KSP installments (all but the one I use on Acceptance Tests - a novelty, I confess, I should had been doing this way earlier). One unzip, many, many installs at the same time.  

So, when I need to "install" TweakScale on a new test bed, it's only "kspln <foldername> TweakScale" and whatever I had unzipped on the _pool folder is symlinked there.

Anyway, back to the problem you described: what you are describing is, in reality, Microsoft's stupidity being exported to all the other O.S.es. DotNet's runtime, by reasons beyound me and reason, are "reparsing" all pathnames on startup (reparsing is the Microsoft lingo for solving symlinks). This completely screws up anyone relying on the symlinks, because internally they are being resolved to an absolute pathname. And so you have some code working over the symlink (and the PWD is set using the symlink!), and some other working over the absolute pathname - with everybody just hoping they all pinpoint to the same place.

Faith Programming. The damned thing works on faith.

I "solved" the problem on KSPe by brute force. I "unreparsed" all pathnames and build an internal conversion table - if KSPe receives a reparsed pathname, it gives back a "unreparsed" one so KSPe clients are 100% shielded from the problem - they just ask for a file using a relative pathname from KSP's "Origin" (or Application Root now), and things will just work no matter the DotNet's runtime is screwing your symlinks or not - and, yeah, KSPe's clients are not allowed to jailbreak the KSP's filesystem. KSPe will throw up an exception if anyone tries that (so if you are willing to access any file on the user's machine, KSPe is not something you want to use).

Anyway, back to the trenches. I'm still trying to reproduce the @TurtleMountainscenario on my Windows Test Bed. I din't managed to reproduce the misbehaviour using CMD, PowerShell neither CygWin (yeah, I'm getting desperate).

I'm trying a last shot.

 

 

[HebaruSan]

Or as a simpler alternative to all of that, you could avoid comparing paths.

[Lisias]

5 minutes ago, HebaruSan said:

Or as a simpler alternative to all of that, you could avoid comparing paths.

But by doing that, I can't diagnose problems related to paths.

[HebaruSan]

Just now, Lisias said:

But by doing that, I can't diagnose problems related to paths.

Do you need to? I would think not. If your mods properly use KSPUtil.ApplicationRoot instead of assuming the current working directory, then they'll work correctly, and other mods are other people's problem.

[TurtleMountain]

Well while all my messages were stuck in limbo for like almost 3 hours I figured out that its partly CKAN's fault.

I guess CKAN automatically assigns the path to "c:\program files (x86)\steam\SteamApps\common\Kerbal Space Program" in "File -> Manage game instances". I had no idea because i've never had a reason to mess with it before but yeah.

When I manually made a new game instance and selected it the path is now normal and I don't get the warning message anymore, so that's a thing @Lisias

[Lisias]

7 hours ago, TurtleMountain said:

Well while all my messages were stuck in limbo for like almost 3 hours I figured out that its partly CKAN's fault.

No, it's not CKAN's fault at all.

It's a Windows idiosyncrasy, and I took the responsibility to handle it on KSP-Recall, so it's kinda my fault. 

Thank you very much for the information. Now I know where to look and how to reproduce the problem on my Test Bed. So I can check if the fix I coded will work!

 

8 hours ago, HebaruSan said:

Do you need to? I would think not. If your mods properly use KSPUtil.ApplicationRoot instead of assuming the current working directory, then they'll work correctly, and other mods are other people's problem.

Yes, I need. ALL my mods are now using KSPUtil.ApplicationRoot *** (because KSPe is using it now),  but it is also prone to filesystem problems - no to mention KSP itself that is misbehaving on the PWD stunt, and besides not having time to purse the problem, I'm getting reports of "game corruption" in Steam Forum.

I can't confirm what really means "game corruption" on that context, and I don't have time to investigate the matter on the short term. So the safer approach is, indeed, check for the problem.

My apologies to any trouble this is causing to you, now I understand why you got involved.

I'm working on the fix, it will be published before sunrise (GMT-3)

— — POST EDIT — — 

*** Well, I let 3 use cases pass trough unchecked… I had fixed it on the newest KSP-Recall, with the newest KSPe.Light

Oh, well… O Mundo gira e a Lusitânia roda… Vida que segue...

Edited November 26, 2022 by Lisias
brute force post merging

[Lisias]

ANNOUNCE

KSP Recall 0.3.0.7 on the Wild, featuring:

• Fixing a naïve approach on checking the pwd.
  • Thanks to Fellow Kerbonaut TurtleMountain for the report and help on the diagnose!
  • Also thanks to HebaruSan for the brainstorming while investigating the matter
  • And my apologies to the CKAN guys for the trouble.

Windows is a mess, it always had. And this week I learnt how to cope with some more of that mess.

— — — — — 

---

This Release will be published using the following Schedule:

• GitHub, reaching manual installers and users of KSP-AVC first. Right Now.
• CurseForge. Right Now.
• SpaceDock (and CKAN users). Right Now.

Since the change is merely some diagnosing messages, I don't see the need of updating the Distribution Channels progressively.

[Gordon Dry]

Wow. I still use KSP 1.12.3 in another folder than Steam and so never had any issues with that obscure launcher - but after updating KSP Recall I get this popup message with a warning like the air raid is coming soon.

And when I click "Ok" then KSP closes.

Edit:
This is the logged part:

[KSP-Recall] Version 0.3.0.7 /L running on KSP 1.12.3 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[KSP-Recall] ERROR: pwd != Application Root! -- pwd=E:\Spiele\Kerbal Space Program\ ; AppRot=e:\Spiele\Kerbal Space Program\ at error:0 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

Unable to use a named GUIStyle without a current skin. Most likely you need to move your GUIStyle initialization code to OnGUI 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

Unable to use a named GUIStyle without a current skin. Most likely you need to move your GUIStyle initialization code to OnGUI 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[KSP-Recall] "Houston, we have a problem!" about "Your 'pwd' doesn't match KSP's 'Application Root'!" was displayed 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

So, the only difference is the capital drive letter E: versus e:

 

Edited November 26, 2022 by Gordon Dry

[Lisias]

1 hour ago, Gordon Dry said:

Wow. I still use KSP 1.12.3 in another folder than Steam and so never had any issues with that obscure launcher - but after updating KSP Recall I get this popup message with a warning like the air raid is coming soon.

And when I click "Ok" then KSP closes.

Edit:
This is the logged part:

So, the only difference is the capital drive letter E: versus e:

 

Oww KRAP.

Click "Cancel" to keep going for while or revert to 0.3.0.3). I'm working on it, I normalised the PWD but completely forgot that the AppRoot could be unnormalised too.

On a side note, can you please upload your KSP.log on dropbox or similar service and post the link here? How are you launching KSP?

[Gordon Dry]

1 hour ago, Lisias said:

KSP.log

Game booted up to main menu as I was eating, here's the log:
https://www.dropbox.com/s/22p1o6jtoxemr4p/2022112601.7z?dl=1

Mainly I use two start parameters; -force-d3d11 -single-instance

but I also dare to launch the game this way, using a command batch file inside the game root folder which contains

@echo off
cd /d "%~dp0"
cls
start "KSP x64 with MemoryAllocationFix" /NORMAL "KSP MemoryAllocationFix.exe"
KSP_x64.exe -force-d3d11 -single-instance
taskkill /IM "KSP MemoryAllocationFix.exe"

using

 

Edited November 26, 2022 by Gordon Dry

[Lisias]

1 hour ago, Gordon Dry said:

but I also dare to launch the game this way, using a command batch file inside the game root folder which contains

@echo off
cd /d "%~dp0"
cls
start "KSP x64 with MemoryAllocationFix" /NORMAL "KSP MemoryAllocationFix.exe"
KSP_x64.exe -force-d3d11 -single-instance
taskkill /IM "KSP MemoryAllocationFix.exe"

There's nothing wrong on your batch, this is (again) my fault, I missed this use case completely - I'm too used to MacOS's walled garden, as it appears - Apple does a lot of crappy things nowadays, but on the early times of MacOS they really scored some good moves, normalising all the pathnames on program launch being one of them (not that I agree with every move of them, by they were trying to do the Right Thing™ at that time).

Now that I know how this happened, I can reproduce it on my test bed and so I can be sure when I had fixed it.

Thank you! Assuming I had nailed it already (apparently, I did - I'm going to reproduce the problem with your instructions now and then try the fix to check on a real test case), a interim release of KSP-Recall will be released more or less at sunset GMT-3 (I do not use clocks anymore - what's the point? ).

— — POST EDIT — — 

Found a way to reproduce the problem easily. It is, again, another huge sheet-storm from Windows on handling things, this crapness makes me miss the CP/M era.

When you start a executable from inside a batch file using only the executable name on the current directory everything works fine:

@echo off
cls
KSP_x64.exe -force-d3d11 -single-instance

But if you use a full pathname for the executable (no matter it is on the same directory or not), KSP will use whatever you typed there, fully trusting the Operating System (and this is the problem):

@echo off
cls
c:\users\lisias\desktop\gordondry\KSP_x64.exe -force-d3d11 -single-instance

(the directory is named as "C:\Users\Lisias\Desktop\GordonDry") and, yeah, I reproduced the problem doing this!

[LOG 16:32:36.217] [KSP-Recall] Version 0.3.0.7 /L running on KSP 1.12.4
[LOG 16:32:36.630] [KSP-Recall] DETAIL: KSPe is not installed. Checking `pwd`.
[LOG 16:32:36.632] [KSP-Recall] DETAIL: origin is  c:\users\lisias\desktop\gordondry\
[LOG 16:32:36.632] [KSP-Recall] DETAIL: approot is c:\users\lisias\desktop\gordondry\
[LOG 16:32:36.632] [KSP-Recall] DETAIL: pwd is     C:\Users\lisias\Desktop\GordonDry\
[LOG 16:32:36.633] [KSP-Recall] ERROR: pwd != Application Root! -- pwd=C:\Users\lisias\Desktop\GordonDry\ ; AppRot=c:\users\lisias\desktop\gordondry\ at error:0
[LOG 16:32:36.638] [KSP-Recall] "Houston, we have a problem!" about "Your 'pwd' doesn't match KSP's 'Application Root'!" was displayed

Checking now the fix.

Edited November 26, 2022 by Lisias
post edit

[pmborg]

Hello @Lisias!,

I am using this add-on in my new setup please let me know if you need any test from my side I will be gald to do it and contribute to find something.

Be my guest I visit this post, what do you think? Your favorite plane/mod

https://forum.kerbalspaceprogram.com/index.php?/topic/210392-kerbal-space-program-1124-releasing-11-2-2022/&do=findComment&comment=4202879