Jump to content

[WIP] Loading textures only as required

Faark

By Faark
in KSP1 Mod Development

 Share

Recommended Posts

NathanKell

NathanKell

Posted
Posted
I just now thought of something. What about unloading the body textures while in the VAB/SPH/KSP (anything not Flight or TrackingStation)? It would increase loading times certainly, but should free up memory in those scenes.

This is actually supremely important for RSS, since ideally RSS would have 3x8192 per body.

Link to comment
Share on other sites
Alewx

Alewx

Posted
Posted

And got one more bug Crash

Looks like LoD does not like "native" .tga files?


14.05.2014 21:16:04: Unhandled exception!
14.05.2014 21:16:04: System.Exception: Failed to load TGA file: D:\Games\Kerbal Space Program Interstellar X\GameData\ThunderAerospace\TacLifeSupportHexCans\HexCanBreathingOxygen\Texture.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

Link to comment
Share on other sites
ShadyAct

ShadyAct

Posted
Posted (edited)

Bug/Crash report:

https://drive.google.com/file/d/0B61E0ok2QLl6RTRYMlJmbEJ0TzA/edit?usp=sharing

Link to a zip file with the output_log, LOD's log, and an mmp file that wasn't recording (?) from the absolute very beginning, and a "Detailed Description.txt"

When using Romfarer's Lazor System the window for flight control had parts missing - there's a screenshot in the zip of what I saw and what it normally looks like. Also, clicking the button for Action Group Manager in the Toolbar didn't bring up a window. Maybe it's possible for LOD to wipe out a window if it can hide parts of a different one.

Would it be difficult to off-load textures into a different process (to circumvent the memory limit)? The more modern enb binarys for Skyrim do that, but KSP is obviously different so maybe it's completely impratical, I'm not a modder.

Edited by ShadyAct
Link to comment
Share on other sites
SpacedInvader

SpacedInvader

Posted
Posted
I just now thought of something. What about unloading the body textures while in the VAB/SPH/KSP (anything not Flight or TrackingStation)? It would increase loading times certainly, but should free up memory in those scenes.
This is actually supremely important for RSS, since ideally RSS would have 3x8192 per body.

This, though I really still think it should be expanded out to scaled space meshes as well, at least when not being viewed up close. I think Faark has hinted that if this happens at all, it won't be for a long time yet.

Link to comment
Share on other sites
dRuPpI

dRuPpI

Posted
Posted
And got one more bug Crash

Looks like LoD does not like "native" .tga files?


14.05.2014 21:16:04: Unhandled exception!
14.05.2014 21:16:04: System.Exception: Failed to load TGA file: D:\Games\Kerbal Space Program Interstellar X\GameData\ThunderAerospace\TacLifeSupportHexCans\HexCanBreathingOxygen\Texture.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

I have TAC too and it works fine !?

Link to comment
Share on other sites
Faark

Faark

Posted
  • Author
Posted
What about making LoD loading only unlocked parts in SPH/VAB editor in career mode?

It should. But afaik KSPs filters kick in a little bit delayed, so it initially starts loading the entire first page.

unloading the body textures while in the VAB/SPH/KSP

Sure, there are a lot of textures that aren't needed in a lot of situations and loading / unloading them as required would be great and is what this mod aims for. Planets are a great example since even in flight you usually are far away from them. But that need a reliable framework and i feel like LoD has not quite archived this, yet. So that is what i am currently focusing on, especially since ATM can take care of the remaining memory problem.

[Crashes]

That is one of the error pattern that seems to be caused by KSP running out of memory.

Would it be difficult to off-load textures into a different process (to circumvent the memory limit)? The more modern enb binarys for Skyrim do that, but KSP is obviously different so maybe it's completely impratical, I'm not a modder.

I don't think its possible with KSP modding techniques available / allowed.

Your zip doesn't contain any images. output_log.txt does contain a large number of exceptions caused by a number of addons. This isn't just annoying when searching for other problems but usually also hurts your gameplay experience. This is sth you should forward to / work out with the developer of those addons, since i cant support them... unless ofc they are caused by an incompatibility with LOD, in what case they should disappear once you uninstall LOD. But i don't really expect that. Also check for updates of those mods before contacting their developers.

LoadOnDemand.log does contain a final crash reason. But its the same one as Alewx has and usually indicates KSP running out of memory. But your mmp seems not even near the 4gb mark. When exactly did you take it? I might should have mentioned that you have to attach (or at least refresh) vmmap directly after the "game has crashed" dialog appears.

Link to comment
Share on other sites
Alewx

Alewx

Posted
Posted

That is one of the error pattern that seems to be caused by KSP running out of memory.

But that was about 5 minutes after starting the game only in the VAB,RAM was at 2,7GB.

I did nothing special, just scrolling a bit fast through the part lists.

Anything I could give, to provide a help at it?

Link to comment
Share on other sites
Kolago

Kolago

Posted
Posted

Here are some crash data while in VAB:

17.05.2014 18:58:54: Unhandled exception!

17.05.2014 18:58:54: System.Exception: Failed to load TGA file: C:\Kerbal Space Program 235\GameData\R&SCapsuledyne\Parts\TaurusHCV\bump_final_NRM.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

17.05.2014 21:00:54: System.Exception: Failed to load TGA file: C:\Kerbal Space Program 235\GameData\Klockheed_Martian\Parts\Refinery\Refinery-complete.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

17.05.2014 18:15:37: System.Exception: Failed to load TGA file: C:\Kerbal Space Program 235\GameData\Fr3aKyPartPack\Parts\Vinci\extendableNozzel.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

Full reports can be supplied.

Chewie

Link to comment
Share on other sites
Faark

Faark

Posted
  • Author
Posted (edited)

@Franklin: Your TextureReplacer is compressing textures. This has to be disabled in your TR-configs (it might be multiple) since the current version of LOD is unable to stand it.

@Kolago: It looks like "out of memory". I'm still trying to improve this behavior of LOD, especially in the VAB when switch pages fast.

@phoenix_ca: Probably. Your output_log.txt doesn't contain much helpful data, it probably would have been in GameData\LoadOnDemand\LoadOnDemand.log.

@Alewx: I'm trying, ofc. It would be nice if you would either verify it isn't because of memory via vmmap or wait for the next version of LOD and then try it again.

Edited by Faark
Link to comment
Share on other sites
Alewx

Alewx

Posted
Posted (edited)
@Alewx: I'm trying, ofc. It would be nice if you would either verify it isn't because of memory via vmmap or wait for the next version of LOD and then try it again.

I will have a look at vmmap.

Could it be that VMMAP is not liking that KSP is in focus and fullscreen?

Update:

Does not look good, was able to reproduce the crash without reaching RAM Limit, but not only KSP crashed also VMMap does, it could not save its file.

Update2:

Sorry, this is the maximum I was able to reach, the mmap file still causes it to crash.

9cqK4oO.jpg

18.05.2014 10:06:10: Unhandled exception!
18.05.2014 10:06:10: System.Exception: Failed to load TGA file: D:\Games\Kerbal Space Program Interstellar X\GameData\B9_Aerospace\Parts\Cockpit_S2_Body\model000.tga ---> System.ArgumentException: Ungültiger Parameter.
   bei System.Drawing.Image.get_RawFormat()
   bei System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   bei System.Drawing.Bitmap..ctor(Image original)
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei FormatDatabase.RecognizeTGA(FileInfo file, Byte[] data)
   bei FormatDatabase.AddRecognitionScope`1.Recognize(FileInfo file, Byte[] bytes)
   bei FormatDatabase.Recognize(String file, Byte[] data)
   bei TextureManager.StartLoadHighResTextureScope.ProcessLoadedData(Byte[] loaded_data)
   bei Disk.RequestFileScope.RunProcessor()
   bei Work.WorkThread()
   bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   bei System.Threading.ThreadHelper.ThreadStart()

Edited by Alewx
Update2
Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted (edited)

Ran into a crash. Looking over the exception in the LoD log, looks like something wacky happened handling a Near Future Propulsion texture.

https://dl.dropboxusercontent.com/u/5296507/NearFuturePropulsion-LoD-CrashLogs.7z

See! I'm doing better now! MOAR LOGS!

I should also mention that this doesn't happen all the time. In fact that was very much a random occurrence.

Edited by phoenix_ca
Link to comment
Share on other sites
phoenix_ca

phoenix_ca

Posted
Posted

Just how fast are you jumping to the flight scene? The mod needs time to prepare mipmaps, and if they aren't done you'll see that.

You need to give LoD a lot of breathing room. Jumping around between scenes quickly or flipping pages in the VAB too fast is going to cause issues (ie crashes).

Link to comment
Share on other sites
Spartan-S63

Spartan-S63

Posted
Posted
Just how fast are you jumping to the flight scene? The mod needs time to prepare mipmaps, and if they aren't done you'll see that.

You need to give LoD a lot of breathing room. Jumping around between scenes quickly or flipping pages in the VAB too fast is going to cause issues (ie crashes).

I waited to see the message that said all textures were prepared, if that's what you mean.

Link to comment
Share on other sites
Spartan-S63

Spartan-S63

Posted
Posted
Cool, I didn't try to run KSP on all my 3 monitors, was afrait the FPS would go through the floor.

It really depends on your hardware on how well it'll run on three monitors. It runs pretty well for me on three, quite a great experience too. I am running SLI GTX 670s though.

Link to comment
Share on other sites
Alewx

Alewx

Posted
Posted
It really depends on your hardware on how well it'll run on three monitors. It runs pretty well for me on three, quite a great experience too. I am running SLI GTX 670s though.

Nice I only have GTX 580 SLI while one of them is crouching ist vent is broken.

Link to comment
Share on other sites
Kolago

Kolago

Posted
Posted
Make a ramdisk. Put KSP on it. Presto.

Ok, this is not bad idea. Fewer crashes for my feeling and it loads a little faster! But I was loading from a SSD already (thanks to the computer god for these) and my i3 CPU is the limiting factor now.

Is a extension of RAM outside the 32bit memory space of the game with a DLL within the game even possible? This remembers me of the middle age of computing with Intel 286 CPUs and 640kb of memory and memory extensions.

If this is possible, you can map out textures to the extension RAM in the format you have in the game memory, without the load overhead of the file system.

Link to comment
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...