[WIP] Loading textures only as required

[Green Skull]

Firstly all of my mods are placed into Game Data. They all work perfectly.

Second with ATM aggressive installed AND LOD it loads fine and plays fine with no reduction in RAM - says in window 0/0 textures. Nothing in log file except time of starting ksp. I have ATM v3.1 and configs for all mods.

Third with the appropriate ATM files removed I now have 0/1888 textures on the loading screen. Game loads to menu, each time counts to 54 textures / 1888 and crashes. Already tried a few times with the same result - game crashes as out of memory in logs.

Fourth if I have ATM installed but only set to compress not scale textures the game crashes purely out of memory and LOD if I remember correctly says 0/0 again.

Fifth if I use an earlier or later version of TextureReplacer (v1.22 or the latest 1.41) with or without compression / mipmaps enabled the results are the same in the states listed above.

For future reference what types of image format does LOD not like?

[Faark]

Already tried a few times with the same result - game crashes as out of memory in logs.

It would really be helpful to get those logs (KSP.log, KSP_data\output_log.txt & GameData\LoadOnDemand\LoadOnDemand.log), to analyze what exactly LOD is doing in this situation to better understand what & how i should improve this mod.

Firstly all of my mods are placed into Game Data. They all work perfectly.

Well, those 0/0 is a pretty strong indication that ATM is loaded first. And to be even more specific, most addons are not directly placed into GameData but instead into a subdirectory. That usually works fine, unless you have dependencies to other mods & need a specific load order. Because of that, LOD work best with LoadOnDemand.dll placed directly within GameData, just like ModuleManager or on this screen.

Fifth if I use an earlier or later version of TextureReplacer (v1.22 or the latest 1.41) with or without compression / mipmaps enabled the results are the same in the states listed above.

Hm, not having set isCompressionEnabled to "never" in GameData\TextureReplacer\@Default.cfg would have been the easiest explanation for a crash at a specific texture. Maybe removing both ATM and TextureReplacer would be worth a try. Either way, if it still doesn't work should those log give us a clue on what exactly is going wrong, if you can share them with us/me.

For future reference what types of image format does LOD not like?

Well, the usual stuff + everything i encountered in KSP so far: png, bmp, emf, gif, ico, jpeg, jpg, tiff, tif, wmf, tga, mbm

But i'm pretty sure that one day, someone will find a format or sth within those formats that Unity does but the stuff this mod uses doesn't support^^

[Green Skull]

My add ons are placed in appropriate sub directories in game data same as in your pic - sorry I wasn't very clear and computer terminology is not a strong point for me

I will add back to game data run both scenarios and drop box you the files shortly.

*EDIT* re-ran some scenarios and p.m'd across link.

Edited April 14, 2014 by Green Skull

[MK3424]

I wonder if it's possible to cooperate with partcatalog and only load the textures from the mod you have selected in partcatalog?

Example: i build much of the time with Stock parts and the KSO shuttle, so wouldn't it be helpful if only those parts are loaded?

[SpacedInvader]

Just checking in to see if there has been any new movement on this? I've been using 2.2 for a while now with almost no crashes, so I think it's still ready for release, but I'm still really hoping for global texture management rather than just parts so I can run all of the high-res planet textures I've been working on for RSS without having to worry as much about the memory cap.

[BananaDealer]

Just checking in to see if there has been any new movement on this? I've been using 2.2 for a while now with almost no crashes, so I think it's still ready for release, but I'm still really hoping for global texture management rather than just parts so I can run all of the high-res planet textures I've been working on for RSS without having to worry as much about the memory cap.

I don't know, I'm still getting a lot of loading and crash issues with 2.2. Granted, I'm on 2204 textures as of this post, so I'm guessing it's expected...

Most of the issues are when I'm in editor, the game runs pretty solid otherwise.

I'm currently experiencing some nonsense trying to load my main station (the game just outright refuses to load and freezes, even on the lowest res settings) but I can't say for sure that's due to LOD or something else...

[raidernick]

I also noticed that once I get over 2000 textures the game seems to start crashing frequently, especially in the editor(even with slow menu switching). The RAM climbs and climbs without going back down. For instance with under 2000 textures i load at 2.4gb and when going to the editor the ram usage will climb the more parts I have. When going into flight or back to the menu te ram usage drops back to around 2.4gb.

If I have over 2000 textures it never seems to drop. The game loads at 2.4gb but when I go into the VAB and back to the menu the RAM usage will stay around 3.2gb so the next time I load anything the game will crash, with an out of memory error in LOD log.

[BananaDealer]

I also noticed that once I get over 2000 textures the game seems to start crashing frequently, especially in the editor(even with slow menu switching). The RAM climbs and climbs without going back down. For instance with under 2000 textures i load at 2.4gb and when going to the editor the ram usage will climb the more parts I have. When going into flight or back to the menu te ram usage drops back to around 2.4gb.

If I have over 2000 textures it never seems to drop. The game loads at 2.4gb but when I go into the VAB and back to the menu the RAM usage will stay around 3.2gb so the next time I load anything the game will crash, with an out of memory error in LOD log.

Yeah, I've noticed that as well. Currently that's pretty much preventing me from loading my main station. It's chugs at about 3GB RAM usage and then KSP just freaks out and crashes...

[raidernick]

It also seems to have a major issue with Kerbin Shuttle Orbiter. I can load it up in VAB just fine memory usage at 2.4gb. When I load it on the launch pad the game crashes 3/4 through loading the 34 or so textures on the ship. This makes no sense, even uncompressed the textures come nowhere near 1gb. I watched the memory just as soon as I loaded the ship. I can load much larger ships/station with 400+ parts without a crash but this mod has a major issue for some reason.

I want to make a side note that this mod does not work properly with TGA files. Very often it will crash on loading a tga texture for no reason. The error log shows the file as the reason. I have had to convert all of my tga textures to png and it no longer crashes because of this.

[Faark]

Sorry guys, i didn't jet had enough time due to real life stuff to make any mentionable changes.

Yes, the current loading stuff can be quite resource intensive and in bad cases easily takes up 500mb and more just for "internal data processing". The TGA loader is especially bad, thus i'm not really surprised it likes to crash when loading those files. I haven't seen any "random" crash that wasn't related to missing memory, but it can look quite different in a logs (ParameterException, AccessViolation or the usual OutOfMemoryException).

I wonder if it's possible to cooperate with partcatalog and only load the textures from the mod you have selected in partcatalog?

Example: i build much of the time with Stock parts and the KSO shuttle, so wouldn't it be helpful if only those parts are loaded?

Have you confirmed that it doesn't? I just had a very brief look into its code, but it will likely work just fine with PartCatalog, unless KSP for some reasons runs GreyOutFilters in before ExcludeFilters (i'll might test that some day).

@BananaDealer

Could you verify that your freeze is indeed caused by LOD? Does it work without LOD?

If I have over 2000 textures it never seems to drop. The game loads at 2.4gb but when I go into the VAB and back to the menu the RAM usage will stay around 3.2gb so the next time I load anything the game will crash, with an out of memory error in LOD log.

Still reproducible? Did you maybe add a specific mod to get above 2k textures or have you also tried to remove some other mod? My extensively modded test install does include KSO so i hope to find any compatibility issues that might exist, but that test install currently has that many mods that it just can't finished preparing images on the main menu anyway, so that will be my priority for now...

[BananaDealer]

@BananaDealer

Could you verify that your freeze is indeed caused by LOD? Does it work without LOD?

My game outright doesn't want to load without LOD due to the 2210 textures I got on, even on Eight Res and with Aggressive Texture Management. And I'm not actually sure it has trouble loading due to LOD. It might be something to do with the craft in question getting somehow broken. I've got a thread about it, here:

http://forum.kerbalspaceprogram.com/threads/77313-Station-failing-to-load

Crashes (in VAB) however are definitely due to LOD.

[Guest]

Nope, it's not 2210 textures that break LOD. It's ATM that does. Remove it, clean the folder up and it should load just fine. And get rid of TextureReplacer if you have it.

[BananaDealer]

Nope, it's not 2210 textures that break LOD. It's ATM that does. Remove it, clean the folder up and it should load just fine. And get rid of TextureReplacer if you have it.

I thought LOD was compatible with the latest ATM...

[Guest]

Update everything then, I see the compatibility was recently added.

[BananaDealer]

Update everything then, I see the compatibility was recently added.

I have... And as I said, I'm not actually sure that it's LOD causing the loading issue- might just be something gone wrong with the vessel that's freaking out KSP...

[raidernick]

I'm sure it's not a specific mod causing the issue. I can swap out any of the mods I have for anything else and when over 2000 textures it still crashes. It just happens to be with KSO because that's the last mod I added.

[rottielover]

I just downloaded to test this, but I'm getting the error about "Requested feature not implemented". I even tried a clean KSP install and just put the loadondemand.dll into \GameData.

I tried the suggestion several pages back about installed the Visual C++ redistrib, did not help.

What am I missing?

EDIT: I found the debug version, but the log output doesn't help much I don't think:

Logger started at 4/24/2014 9:59:36 AM (Sync)
4/24/2014 9:59:36 AM: No config found, starting with defaults.
4/24/2014 9:59:36 AM: Todo19: Consider a mechanism to remove old/unsued cached files! (both dead refs and unused cfgs)
4/24/2014 9:59:36 AM: Device: Direct3D 9.0c [nvd3dum.dll 9.18.13.3165]

Here is the KSP output log as well:

Kerbal Space Program - 0.23.5.464 (WindowsPlayer) Steam


OS: Windows 8.1  (6.3.9600) 64bit
CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz (8)
RAM: 16352
GPU: NVIDIA GeForce GTX 580 (1482MB)
SM: 30 (Direct3D 9.0c [nvd3dum.dll 9.18.13.3165])


Log started: Thu, Apr 24, 2014 09:59:34


[LOG 09:59:35.969] Load(Assembly): /LoadOnDemand
[LOG 09:59:35.970] AssemblyLoader: Loading assembly at D:\Program Files (x86)\Steam\SteamApps\common\Kerbal Space Program\GameData\LoadOnDemand.dll
[LOG 09:59:35.992] AssemblyLoader: Loading assemblies
[LOG 09:59:36.001] AddonLoader: Instantiating addon 'ActivityGUI' from assembly 'LoadOnDemand'
[LOG 09:59:36.002] AddonLoader: Instantiating addon 'DevStuff' from assembly 'LoadOnDemand'
[LOG 09:59:36.003] AddonLoader: Instantiating addon 'Startup' from assembly 'LoadOnDemand'
[B][EXC 09:59:36.090] NotImplementedException: The requested feature is not implemented.[/B]
[LOG 09:59:36.092] AddonLoader: Instantiating addon 'WorkQueue' from assembly 'LoadOnDemand'
[LOG 09:59:36.094] Load(Audio): Squad/Parts/Utility/LandingLeg/sounds/sound_servomotor
[LOG 09:59:37.277] Load(Audio): Squad/Sounds/editorLoop01
[LOG 09:59:37.434] Load(Audio): Squad/Sounds/sound_ambience_nature
[LOG 09:59:37.460] Load(Audio): Squad/Sounds/sound_click_flick
...

And I cut this one off cause it's just showing loading the Squad parts

Edited April 24, 2014 by rottielover

[BananaDealer]

The .dll is supposed to be in the root /GameData folder of your install.

Make sure you have the correct Visual C++ redist. Your system may be 64bit but you still need the 86bit one.

Other than that, make sure a firewall or anti-virus isn't blocking either the .dll or KSP altogether. Win8 is a bit picky when it comes to running "unknown" things...

Also make sure you have the latest .NET Framework version.

[rottielover]

Make sure you have the correct Visual C++ redist. Your system may be 64bit but you still need the 86bit one.

Installed both, rebooted. No effect.

Other than that, make sure a firewall or anti-virus isn't blocking either the .dll or KSP altogether. Win8 is a bit picky when it comes to running "unknown" things...

I'm using AVG free and have my KSP folder in the exception list so it doesn't scan it, nor does resident shield scan it.

Also make sure you have the latest .NET Framework version.

Check.

Any other thoughts?

[BananaDealer]

You can't have "both" Visual C++ versions. Either one or the other. Otherwise things get wonky. Uninstall both and reinstall the x86 one.

[rottielover]

You can't have "both" Visual C++ versions. Either one or the other. Otherwise things get wonky. Uninstall both and reinstall the x86 one.

According to my Add/Remove I actually have 8 total C++ Redistrib's. 4x2008, 2x 2010, and 2x 2013 (both x64 and x86 as well as sep version numbers). Should I be getting rid of all of them in favor of the x86 2013 one?

[Faark]

No, as long as x86 2013 is installed it shouldn't be the cause. The stack trace trace of that exception would be nice to know, you can find it in KSP_Data\output_log.txt. Your log indicates that you tested this on a total new KSP installation without any other mods? That kinda worrying, since at least on my dev version the most likely cause for such an exception would be other mods interfering with LOD...

[rottielover]

My only idea at this point is that it could be the Visual Studio redistribs that AVG installed. So I am currently in the process of removing AVG completely, and I'm just about done cleaning out all the redistribs. I'm going to reboot, install the 2013 C++ redistrib cleanly, then try again.

yes this is a totally clean KSP install (wiped out the folder, let Steam put it back.

EDIT:

Stack Trace:

NotImplementedException: The requested feature is not implemented.
  at System.Runtime.InteropServices.Marshal.GetHRForLastWin32Error () [0x00000] in <filename unknown>:0 

  at LoadOnDemand.UnmanagedLibrary..ctor (System.String fileName) [0x00000] in <filename unknown>:0 

  at LoadOnDemand.NativeBridge.SetupNativeBridge () [0x00000] in <filename unknown>:0 
UnityEngine.GameObject:Internal_AddComponentWithType(Type)
UnityEngine.GameObject:AddComponent(Type)
AddonLoader:StartAddon(LoadedAssembly, Type, KSPAddon, Startup)
AddonLoader:StartAddons(Startup)
:MoveNext()
UnityEngine.MonoBehaviour:StartCoroutine_Auto(IEnumerator)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
:MoveNext()
UnityEngine.MonoBehaviour:StartCoroutine_Auto(IEnumerator)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
GameDatabase:StartLoad()
:MoveNext()
UnityEngine.MonoBehaviour:StartCoroutine_Auto(IEnumerator)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
LoadingScreen:Start()

Edited April 24, 2014 by rottielover

[rottielover]

Ok I've finished the clean out and fresh install. Same error as before. I put the snippet of the stack trace as requested in the above post. If you would like I'm more than happy to provide all the log files etc via dropbox, heck I'll get on skype with ya, anything to help the community!

Does that stack trace help out at all?

Edited April 24, 2014 by rottielover

[Faark]

Okay, it is indeed the native library that your computer cant load for some reason. The current error message is kinda crap thanks to mono. I'll improve that, but that doesn't tell us why it cant load... and this is the hard part. Could you run DependencyWalker (x86 version), drag & drop GameData\LoadOnDemand\NetWrapper.native into it and save & send me its .dwi report?