[WIP] Loading textures only as required

[raxo2222]

The actual amount of RAM you have doesn't matter as KSP can only ever use about 3GB at most.

By the looks of your GameData folder, I'm guesstimating that you're hitting about 2000 textures, probably more. The mod in general has issues with over-loading textures in VAB. Having more than 2000 textures makes crashes even more often. I don't know of a work-around to this unfortunately... We'll just have to wait until the mod is made to handle loading and unloading of textures better...

Well, it crashed at 1500 MB used, and I heard it can go up to 2048 MB on 32 bit systems, so it can be something else - can't even get to main menu.

Also LoD said, that there were around 1400 textures.

Will try to load without KW/Firesplatter/Novapunch/B9.

The next version will be able to disable loading texture in VAB inventory, which should fix your issue (VAB icons will look ugly, though). It has a problem with flipping through pages in VAB now.

And SPH too?

Edit: turning off memory hogs managed to get to main menu, but it still crashed at ~1500 MB

Only 720 textures to load.

Memory usage suddenly jumps by 500 MB.

And then it crashes.

Here are crash logs: http://www24.zippyshare.com/v/51060895/file.html

Edit:

Removed Better Atmosphere and Enchanted Visuals mods (that make ksp look better).

Now it takes 1400 MB of space on startup, but there are compability issues with B9 mod.

Here is my crash logs (it crashed in main menu):

http://www66.zippyshare.com/v/90851453/file.html

Also I use active texture management in aggresive version, and I set max texture resolution to 256.

Also why my KSP can use only 1600-1800 MB before it crashes? I heard it can use over 2048 MB up to 3500 MB or something.

Is there any trick to make it large address aware? I searched for this stuff and no dice. So I think I shroud use to 64 bit windows.

Edit:

Moved out Firesplitter/KW/B9/Novapunch and game loaded fine (although there was sudden 300 MB bump during loading).

Here is logs after crash on purpose from messing in VAB:

http://www51.zippyshare.com/v/74186721/file.html

Edit:

I did memory stress test on my computer - apparently my 32bit windows starts acting up, when there is less than 1000MB of free RAM memory.

So technically I ran into in system limitation, not game limitation.

Soo.. I should get a copy of 64 bit Windows 7. Then I could squeeze out around 1000 -1500MB more of RAM for game.

I have 4096 MB of ram, but my system can use only 3500 MB.

Also Windows uses 1000 GB of ram for itself.

So that would leave around 2500-3000 MB for me, hopefully allowing me to run this many mods at once (I heard, that 32bit application can use up to 3500 MB of RAM on 64 bit systems)

Edited May 11, 2014 by raxo2222

[phoenix_ca]

Even with 64-bit Windows, the OS is going to eat a substantial chunk of memory for itself (can be upwards of 1GB on Windows), and it'll cause any program that goes past what's left to crash. You can't get around that unless you have more RAM to give Windows its...rather substantial amount of breathing room required (seriously it eats more memory than any OS I've ever used).

Technically on a 64-bit system a 32-bit process should be able to address 4Gb (that's gigabits not bytes) of system memory. Derp derp. See Faark's post below.

Edited May 12, 2014 by phoenix_ca

[dRuPpI]

The actual amount of RAM you have doesn't matter as KSP can only ever use about 3GB at most.

By the looks of your GameData folder, I'm guesstimating that you're hitting about 2000 textures, probably more. The mod in general has issues with over-loading textures in VAB. Having more than 2000 textures makes crashes even more often. I don't know of a work-around to this unfortunately... We'll just have to wait until the mod is made to handle loading and unloading of textures better...

with this littel tool you can use up to 3.7 gb for ksp.exe but I think it oly works for 64 bit systems

http://www.ntcore.com/4gb_patch.php

[dRuPpI]

So this mod would take advantage of a SSD?

I think yes as it loads Textures from HDD in to Ram as when they are needed, it should cut loading times.

I have SSD and I can't fell any legs or delay when I click on a part or load a vessel.

[dRuPpI]

The next version will be able to disable loading texture in VAB inventory, which should fix your issue (VAB icons will look ugly, though). It has a problem with flipping through pages in VAB now.

and when ca we expect the next version? cant wait .... go and buy some more redbull

[likke_A_boss]

with this littel tool you can use up to 3.7 gb for ksp.exe but I think it oly works for 64 bit systems

http://www.ntcore.com/4gb_patch.php

This is poorly documented and makes me think twice about downloading. Do you know anything further about it?

[Faark]

Technically on a 64-bit system a 32-bit process should be able to address 4Gb (that's gigabits not bytes) of system memory.

Nah, it is bytes.

@raxo2222

Interesting. VMMapis a nice tool to collect some data about memory usage. You might want to check it out and upload its mmap files. But i don't think there is much we can do. LOD's mem usage can spike quite heavily, especially on v2. The next version might help, but i can't promise anything ofc. Yes, even on Win32 it should be possible to do some LargeAddressAware stuff, but its most likely not easy and probably requires messing around with OS. Did other programs ever consume more than ~2gb memory on your PC?

@PizzaMore

Yes, unfortunately the statement regarding TR was narrowed down quite a bit later. It can work, but only for some replacements. Anyway, don't waste to much time getting it to work with the current version of lod. Upcoming releases will improve it!

@raxo2222

Out of memory. Can't really say more

@dRuPpI / Bloodbunny

Added "check compatibility with Stretchy" to my todo list.

@Franklin

Yea, maybe one day LoD will handle planetary textures. But the todo list is long and rarely gets shorter...

@Alewx

Thats kinda the point of this mod.

@PolecatEZ

Thanks for sharing your findings. I knew that LoD breaks some Alpha stuff because of a crappy compression API that caused other issues, but thought this only affects compressed previews. I didn't yet had time to investigate, though.

@dRuPpI

Made a lot of progress this weekend, but less than i hoped to. And I have a deadline on tuesday for some real life related stuff. "When its done", hopefully before the next weekend.

Also that utility most likely just sets largeadressaware, sth that was already done for KSP.

[BananaDealer]

Hmmm, just a quick suggestion/tip/whatever for the next version- maybe have the name/file path for the texture that's getting loaded show up above/below the counter.

I'm currently experiencing issues with the thing crashing on initial texture load when it reaches about 500 textures (out of 2207). Having a thing that tells you which texture is currently being loaded/prepared would come in handy for tracking down the problem...

[Faark]

@BananaDealer

Its either memory (can't really do anything except optimizing LoD) or a problematic texture (check LOD.log). So i don't really think you can "fix" this. Admittedly especially on v2 with a fast HDD, multiple large files in a row might be especially problematic, but this is already solved for the next update. And "manually" working around sth like this isn't even sth i would do.

Edited May 12, 2014 by Faark

[BananaDealer]

@BananaDealer

Its either memory (can't really do anything except optimizing LoD) or a problematic texture (check LOD.log). So i don't really think you can "fix" this. Admittedly especially on v2 with a fast HDD, multiple large files in a row might be especially problematic, but this is already solved for the next update. And "manually" working around sth like this isn't even sth i would do.

"Manually" working around this would be removing the part/texture causing the problem. And it's not memory, I think- as in at that point the KSP.exe is only at about 1GB.

In any case, I'll be off KSP until I find some solution to this...

[dRuPpI]

Nah, it is bytes.

@raxo2222

Interesting. VMMapis a nice tool to collect some data about memory usage. You might want to check it out and upload its mmap files. But i don't think there is much we can do. LOD's mem usage can spike quite heavily, especially on v2. The next version might help, but i can't promise anything ofc. Yes, even on Win32 it should be possible to do some LargeAddressAware stuff, but its most likely not easy and probably requires messing around with OS. Did other programs ever consume more than ~2gb memory on your PC?

@PizzaMore

Yes, unfortunately the statement regarding TR was narrowed down quite a bit later. It can work, but only for some replacements. Anyway, don't waste to much time getting it to work with the current version of lod. Upcoming releases will improve it!

@raxo2222

Out of memory. Can't really say more

@dRuPpI / Bloodbunny

Added "check compatibility with Stretchy" to my todo list.

@Franklin

Yea, maybe one day LoD will handle planetary textures. But the todo list is long and rarely gets shorter...

@Alewx

Thats kinda the point of this mod.

@PolecatEZ

Thanks for sharing your findings. I knew that LoD breaks some Alpha stuff because of a crappy compression API that caused other issues, but thought this only affects compressed previews. I didn't yet had time to investigate, though.

@dRuPpI

Made a lot of progress this weekend, but less than i hoped to. And I have a deadline on tuesday for some real life related stuff. "When its done", hopefully before the next weekend.

Also that utility most likely just sets largeadressaware, sth that was already done for KSP.

..... That with the stretch tanks seams to be a fault by me it was just the "normal" VHB issue I think

[BananaDealer]

..... That with the stretch tanks seams to be a fault by me it was just the "normal" VHB issue I think

Stretchy tanks and this mod don't really seem to like each other...

Try not to flip through the multiple textures, otherwise LoD will throw a fit on the floor and start drooling all over the carpet...

[Faark]

"Manually" working around this would be removing the part/texture causing the problem. And it's not memory, I think- as in at that point the KSP.exe is only at about 1GB.

In any case, I'll be off KSP until I find some solution to this...

If its not OutOfMemory it should already be logged, though maybe no one but me understands what to search for in both KSP_Data\output_log.txt + GameData\LoadOnDemand\LoadOnDemand.log ^^

@dRuPpI

K, if it keeps crashing please report this issue again once the next version is out.

[NathanKell]

Stretchy (and Procedural Parts) will dynamically change the texture assigned to the tank, by looking through GDB for textures that match given URLs, on keypress in the VAB.

FStextureSwitcher does the same thing generally (not sure if that's also an issue for LoD).

I'm not immediately sure how you could go about setting an exclusion in these cases, since you won't know what textures will need to be kept in the DB except by reading the appropriate cfgs....

[SpacedInvader]

Stretchy tanks and this mod don't really seem to like each other...

Try not to flip through the multiple textures, otherwise LoD will throw a fit on the floor and start drooling all over the carpet...

I've yet to have any troubles with stretchy tanks with LOD... I usually rapid-fire 'T' to get to the texture I want, sometimes needing to cycle a couple of times because I'm doing it too fast, and I still don't get crashes from this...

[phoenix_ca]

FStextureSwitcher does the same thing generally (not sure if that's also an issue for LoD).

It doesn't seem to be. I tried texture switching with my FusTek parts and they did fine.

----

Is there any way to get the LOD window back up after I close it? Having it as a permanent resident on my screen is somewhat trying at times (and it's a 27" screen...sooooo many addon windows...).

Compared to ATM with quasi-aggressive settings I'm getting about 200MB less memory use with this thing. Quite impressive. I'd expect even more if you start using some judicious DXT compression.

But hell, I can enjoy full textures with far less chance of crashing. That's pretty impressive, and I'd say worth the trade-off in some texture-pop-in (though I'm fairly sure getting rid of texture pop-in means one hell of an advanced/smart bit of streaming...at least I've never heard it referred to as an easy task >.> ). Seriously, I'm impressed. I would've thought that to do this with any sort of efficiency it'd have to be done on Squad's end and have them actually put Unity's asset streaming to work. But you did it as a freaking addon.

In short: Such savings. Much textures. W...*gets killed by someone who hates doge meme*

Edited May 12, 2014 by phoenix_ca

[Alewx]

Nah, it is bytes.

@raxo2222

Interesting. VMMapis a nice tool to collect some data about memory usage. You might want to check it out and upload its mmap files. But i don't think there is much we can do. LOD's mem usage can spike quite heavily, especially on v2. The next version might help, but i can't promise anything ofc. Yes, even on Win32 it should be possible to do some LargeAddressAware stuff, but its most likely not easy and probably requires messing around with OS. Did other programs ever consume more than ~2gb memory on your PC?

@PizzaMore

Yes, unfortunately the statement regarding TR was narrowed down quite a bit later. It can work, but only for some replacements. Anyway, don't waste to much time getting it to work with the current version of lod. Upcoming releases will improve it!

@raxo2222

Out of memory. Can't really say more

@dRuPpI / Bloodbunny

Added "check compatibility with Stretchy" to my todo list.

@Franklin

Yea, maybe one day LoD will handle planetary textures. But the todo list is long and rarely gets shorter...

@Alewx

Thats kinda the point of this mod.

@PolecatEZ

Thanks for sharing your findings. I knew that LoD breaks some Alpha stuff because of a crappy compression API that caused other issues, but thought this only affects compressed previews. I didn't yet had time to investigate, though.

@dRuPpI

Made a lot of progress this weekend, but less than i hoped to. And I have a deadline on tuesday for some real life related stuff. "When its done", hopefully before the next weekend.

Also that utility most likely just sets largeadressaware, sth that was already done for KSP.

That is sooo cool, and I just read it is compatible with ATM that means I will try it this evening with my Scott Manley Package of too many mods.

[raxo2222]

Nah, it is bytes.

@raxo2222

Interesting. VMMapis a nice tool to collect some data about memory usage. You might want to check it out and upload its mmap files. But i don't think there is much we can do. LOD's mem usage can spike quite heavily, especially on v2. The next version might help, but i can't promise anything ofc. Yes, even on Win32 it should be possible to do some LargeAddressAware stuff, but its most likely not easy and probably requires messing around with OS. Did other programs ever consume more than ~2gb memory on your PC?

@PizzaMore

Yes, unfortunately the statement regarding TR was narrowed down quite a bit later. It can work, but only for some replacements. Anyway, don't waste to much time getting it to work with the current version of lod. Upcoming releases will improve it!

@raxo2222

Out of memory. Can't really say more

@dRuPpI / Bloodbunny

Added "check compatibility with Stretchy" to my todo list.

@Franklin

Yea, maybe one day LoD will handle planetary textures. But the todo list is long and rarely gets shorter...

@Alewx

Thats kinda the point of this mod.

@PolecatEZ

Thanks for sharing your findings. I knew that LoD breaks some Alpha stuff because of a crappy compression API that caused other issues, but thought this only affects compressed previews. I didn't yet had time to investigate, though.

@dRuPpI

Made a lot of progress this weekend, but less than i hoped to. And I have a deadline on tuesday for some real life related stuff. "When its done", hopefully before the next weekend.

Also that utility most likely just sets largeadressaware, sth that was already done for KSP.

Hmm so KSP can use only 2GB on my 32 bit system. Looks like I need upgrade of system.

Also this heap memory seems to be very bloated:

Before crash (it didn't even go to main menu).

After crash:

Here is mod list:

And here are logs: http://www8.zippyshare.com/v/37500494/file.html

I don't think, that here were applications using more than 2 gb memory.

Well Civilization 4 with Caveman2Cosmos was memory hungry as kethiopia guy.

Edit:

Hmmm reduced memory load very hard this time by loading these mods only:

Hmm I think EVE and Atmospheric Enhancer mods are hogs to. Crashed, when I reverted my flight to runaway (after getting epicly disassembled by FAR in clouds)

Here are logs: http://www65.zippyshare.com/v/70136701/file.html

Also this VMM just freezes, when I tried to save its data in .vmm file.

Edited May 12, 2014 by raxo2222

[dRuPpI]

This is poorly documented and makes me think twice about downloading. Do you know anything further about it?

I only installed it and run taskmanger on my second monitor and noticed that with out it ksp usualy crashes at reachning ~3,3 gb and with the patch it crashes around ~3,7 gb

Edit: a virus was not found, also I have found this tool some time ago someweher in the KSP forum or a other offical ksp related forum

Edited May 13, 2014 by dRuPpI

[dRuPpI]

If its not OutOfMemory it should already be logged, though maybe no one but me understands what to search for in both KSP_Data\output_log.txt + GameData\LoadOnDemand\LoadOnDemand.log ^^

@dRuPpI

K, if it keeps crashing please report this issue again once the next version is out.

I think it is clear that the changinge size of the texturs while you stretch a part could caus some troubel..... but for real i think this is detail work for later you should concentrate on the basic functionality first

[Faark]

I think it is clear that the changinge size of the texturs while you stretch a part could caus some troubel..... but for real i think this is detail work for later you should concentrate on the basic functionality first

I have never seen any code where they (Stretch & Procedural Parts) actually modify any existing textures. They only change what and how textures are used, and LoD doesn't care about it. LoD did break the way they search for textures, since they don't use KSPs GameDatabase for some reason. But that was fixed. Well see whats now the problem once i find the time to investigate.

@raxo2222

That kinda makes me happy i don't have a 32bit os anymore^^

I don't think there is much LoD or I could do for you and probably just makes it worse. As mentioned earlier, LoD does require quite some memory for "processing" and 2GB isn't very much to begin with, since we can't change that KSP itself requires a large chunk of it. I'm afraid your better of with a very aggressive ATM, instead.

[raxo2222]

I have never seen any code where they (Stretch & Procedural Parts) actually modify any existing textures. They only change what and how textures are used, and LoD doesn't care about it. LoD did break the way they search for textures, since they don't use KSPs GameDatabase for some reason. But that was fixed. Well see whats now the problem once i find the time to investigate.

@raxo2222

That kinda makes me happy i don't have a 32bit os anymore^^

I don't think there is much LoD or I could do for you and probably just makes it worse. As mentioned earlier, LoD does require quite some memory for "processing" and 2GB isn't very much to begin with, since we can't change that KSP itself requires a large chunk of it. I'm afraid your better of with a very aggressive ATM, instead.

I have Aggressive ATM installed

Also I got disc with 64 bit windows now. I will upgrade my system in Friday, when I have a lot of time to reinstall stuff

[dRuPpI]

I have Aggressive ATM installed

Also I got disc with 64 bit windows now. I will upgrade my system in Friday, when I have a lot of time to reinstall stuff

The problem with ATM is that I have much mods and I have to use so hard comprising that my KSP looks really ugly and it reduces the fun of playing and even with ATM Im reaching the Ram limit soon if I don't want even more ugliness, don't get my wrong ATM is a great tool but I think LOD has the potential to be the better solution in future, I don't know maybe rbray89 will join the development of LOD He knows a lot about texture's and his Visual Mod looks great uncompressed .

Also I'm now thinking about (since LOD is out) installing HD texture's for everything possible with LOD it could be work in the near future hopefully...... and by the way I noticed performance boost with LOD compeering it with ATM, the whole game runs smother, when in VAB or flying (after main texture loading is finished)

Edited May 13, 2014 by dRuPpI

[dRuPpI]

... But another question how is LOD compatible with ATM? Because as many I understand ATM generates it's own texture's and don't changes the original files,

so does LOD detect ATM and loads it's files or does it just bypass ATM and loads the original files?

[Guest]

If the new version fixes the transparency issues and allows disabling loading VAB inventory textures, I'll really be in heaven. Full screen, full HD for everything. No more tradeoffs, downscaling or anything like this.