Jump to content

[1.0][Release-5-0][April 28, 2015] Active Texture Management - Save RAM!

rbray89

By rbray89
in KSP1 Mod Releases

 Share

Recommended Posts

Nachocuban

Nachocuban

Posted
Posted

After updating to the 2-12 version(basic)

My game will sometimes crash on load, I use a wide variety of mods, but never had this issue before this update.

I've attached my output log of the 2 most recent crashes. It's not always the same part that stalls the loading.

https://dl.dropboxusercontent.com/u/20498327/output_log6.txt

https://dl.dropboxusercontent.com/u/20498327/output_log7.txt

Link to comment
Share on other sites
Nachocuban

Nachocuban

Posted
Posted
After updating to the 2-12 version(basic)

My game will sometimes crash on load, I use a wide variety of mods, but never had this issue before this update.

I've attached my output log of the 2 most recent crashes. It's not always the same part that stalls the loading.

https://dl.dropboxusercontent.com/u/20498327/output_log6.txt

https://dl.dropboxusercontent.com/u/20498327/output_log7.txt

After Reverting to 2-11(Basic) no crashes in over 10 repeated startups.

Link to comment
Share on other sites
TheDude

TheDude

Posted
Posted (edited)

Greetings,

After a day of cleaning up, configuring and tweaking mods and config files, I've finally got my texture compression configs where I want them. There is however, one single directory that refuses to play nice. I have several FusTek parts that I can't get set properly no matter what I do. They keep showing up as Not Readable.

Here are the errors:

[LOG 16:06:22.655] Load(Model): FusTek/Parts/KarmonyMkII/model

[ERR 16:06:22.678] GameDatabase: Texture 'FusTek/Parts/KarmonyMkII/model001' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.680] Load(Model): FusTek/Parts/KarmonyMkIIAdapter/model

[ERR 16:06:22.708] GameDatabase: Texture 'FusTek/Parts/KarmonyMkIIAdapter/model001' requested as normal map but texture is not a normal map and is not readable

[ERR 16:06:22.709] GameDatabase: Texture 'FusTek/Parts/KarmonyMkIIAdapter/model003' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.713] Load(Model): FusTek/Parts/StaticPanelXL/model

[ERR 16:06:22.715] GameDatabase: Texture 'FusTek/Parts/StaticPanelXL/model003' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.946] Load(Model): FusTek/Station Parts Expansion/Internals/Spaces/UniKarmony/model

[ERR 16:06:22.954] GameDatabase: Texture 'FusTek/Station Parts Expansion/Internals/Spaces/UniKarmony/model002' requested as normal map but texture is not a normal map and is not readable

Here is my Fustek.CFG

NORMAL_LIST

{

texture = Fustek/Parts/KarmonyMkII/model001

texture = Fustek/Parts/KarmonyMkIIAdapter/model001

texture = Fustek/Parts/KarmonyMkIIAdapter/model003

texture = Fustek/Parts/StaticPanelXL/model003

texture = Fustek/Station Parts Expansion/Internals/Spaces/UniKarmony/model002

}

LEAVE_READABLE

{

}

OVERRIDES

{

}

OVERRIDES_FOLDERS

{

}

I'm running 2.12 basic. Can anyone help me with these last few textures?

best regards,

The Dude

Edited by TheDude
Link to comment
Share on other sites
TheDude

TheDude

Posted
Posted

Greetings,

Nope. In or out of the CFG makes no difference. The same errors always show up in the KSP.log. It was my understanding that textures throwing the "Not Readable" error should be put in the Normals List. This was the case with about 30 other texture errors that I fixed by following that very process.

Best regards,

The Dude

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
Greetings,

After a day of cleaning up, configuring and tweaking mods and config files, I've finally got my texture compression configs where I want them. There is however, one single directory that refuses to play nice. I have several FusTek parts that I can't get set properly no matter what I do. They keep showing up as Not Readable.

Here are the errors:

[LOG 16:06:22.655] Load(Model): FusTek/Parts/KarmonyMkII/model

[ERR 16:06:22.678] GameDatabase: Texture 'FusTek/Parts/KarmonyMkII/model001' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.680] Load(Model): FusTek/Parts/KarmonyMkIIAdapter/model

[ERR 16:06:22.708] GameDatabase: Texture 'FusTek/Parts/KarmonyMkIIAdapter/model001' requested as normal map but texture is not a normal map and is not readable

[ERR 16:06:22.709] GameDatabase: Texture 'FusTek/Parts/KarmonyMkIIAdapter/model003' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.713] Load(Model): FusTek/Parts/StaticPanelXL/model

[ERR 16:06:22.715] GameDatabase: Texture 'FusTek/Parts/StaticPanelXL/model003' requested as normal map but texture is not a normal map and is not readable

[LOG 16:06:22.946] Load(Model): FusTek/Station Parts Expansion/Internals/Spaces/UniKarmony/model

[ERR 16:06:22.954] GameDatabase: Texture 'FusTek/Station Parts Expansion/Internals/Spaces/UniKarmony/model002' requested as normal map but texture is not a normal map and is not readable

Here is my Fustek.CFG

NORMAL_LIST

{

texture = Fustek/Parts/KarmonyMkII/model001

texture = Fustek/Parts/KarmonyMkIIAdapter/model001

texture = Fustek/Parts/KarmonyMkIIAdapter/model003

texture = Fustek/Parts/StaticPanelXL/model003

texture = Fustek/Station Parts Expansion/Internals/Spaces/UniKarmony/model002

}

LEAVE_READABLE

{

}

OVERRIDES

{

}

OVERRIDES_FOLDERS

{

}

I'm running 2.12 basic. Can anyone help me with these last few textures?

best regards,

The Dude

the problem is case sensivity. Fustek != FusTek. I am used to finding these in code :)

Link to comment
Share on other sites
Mihara

Mihara

Posted
Posted
Windows is not case sensitive. Linux can be. Dunno about OS X.

Linux always is. Since OS X is BSD Unix deep down inside, I expect it also is.

So how did they manage to make a program that's case sensitive for file names on Windows??????

Simple. KSP plugins do not work with actual files as much as they work with GameDatabase entries, and this is the case here. File names are case insensitive only when you're opening the file directly, but GameDatabase makes a list of files as strings, basically, and it also converts slashes in paths to OSX/Linux convention while at it. If you're comparing one string to another, the comparison will be case sensitive.

Link to comment
Share on other sites
Galane

Galane

Posted
Posted

So on Windows, if I change the case of a letter in a part's file or folder name, it'll break the game even though Windows doesn't care?

Just one of those things showing Windows has a 33 year legacy VS those "newbies" Linux and BSD. ;) Back then, DOS didn't even do subdirectories, let alone such fancy stuff as treating txtfile.txt and TxtFile.TXT as distinct files and allowing them to be in the same location. Adding case sensitivity to Windows would not only confuse the users, it'd also break every program ever written for Windows that might still be able to run on any version that added case sensitivity. (*sigh* If only MS had drop kicked 16 bit completely back in 1995 and added case sensitivity.)

Hmmm, there *might* be a possible way to hack it in and make it not break "legacy" software. The product so cleverly named "Long File Names for Windows 3.1 and 95" added support to Windows 3.1x to read and write Windows 95's long file names (for those who dual booted 3.1x and 95), and also added LFN support to 16 bit software running on Windows 95. Unlike any other LFN hack, that used the genuine article. The 'exactly what it says on the tin' name made it extremely hard to find, but someone did locate a copy and it's somewhere on the MDGx website. Dunno if it also works in Win98/Me or other versions that still support 16bit software. Could be it only works on FAT16 with LFN.

I'd bet a similar kind of hack could add case sensitivity to Windows, but it would be much more involved because without Microsoft doing it themselves, it would require major modifications to the filesystem that has plenty of murky areas in the publicly available documentation. Handling programs that don't use the stock file dialogs (such as nearly everything from Adobe and Microsoft Office) would be a right pain because they don't play by the rules. Would be easier to do it as an addon to allow Winderz to read/write Linux and BSD storage in much the same way as that old product made 3.1x able to use that 'foreign' version of the directory structure 95 introduced.

Link to comment
Share on other sites
Mihara

Mihara

Posted
Posted
So on Windows, if I change the case of a letter in a part's file or folder name, it'll break the game even though Windows doesn't care?

Depends on whether the file name is ever compared to another string explicitly. It might.

Just one of those things showing Windows has a 33 year legacy VS those "newbies" Linux and BSD. ;)

Are you trolling or are you actually unaware that both Linux and BSD implement Unix standards, which date back to 1969, far earlier than Windows and even DOS (which dates back to 1981) existed?

I'd bet a similar kind of hack could add case sensitivity to Windows, but it would be much more involved because without Microsoft doing it themselves, it would require major modifications to the filesystem that has plenty of murky areas in the publicly available documentation. Handling programs that don't use the stock file dialogs (such as nearly everything from Adobe and Microsoft Office) would be a right pain because they don't play by the rules. Would be easier to do it as an addon to allow Winderz to read/write Linux and BSD storage in much the same way as that old product made 3.1x able to use that 'foreign' version of the directory structure 95 introduced.

http://support.microsoft.com/kb/100625

I.e. the file system has long since been case sensitive, the file dialogs and possibly some API functions explicitly behave like they aren't.

But we should stop polluting the thread.

Link to comment
Share on other sites
Neutrinovore

Neutrinovore

Posted
Posted

Yadda yadda, ratcheta ratcheta...

-Snip-

But we should stop polluting the thread.

Exactly, yes, thank you! I was just about to suggest that this discussion should certainly be moved somewhere else, thank you Mihara. :)

I'm sure this is all fascinating to those who are interested, though. I'm just not one of them. :P

Link to comment
Share on other sites
Mihara

Mihara

Posted
Posted
I'm sure this is all fascinating to those who are interested, though. I'm just not one of them. :P

It's bad not to be curious. :)

Coming back to the topic, though, I have a suggestion.

Currently, config files are only loaded from one location. I.e. if I'm a new modder, who released a large pack of parts and wishes to specify how small the textures in it are allowed to go, I have no option to distribute a config file with it without complicating the directory structure, I'd have to place it in a specific location.

But GameDatabase actually loads all *.cfg files into memory at once, regardless if the blocks in them are recognised or not. You don't even have to use System.IO to get at them -- for example, here's a piece from RPM:


                        foreach (ConfigNode node in GameDatabase.Instance.GetConfigNodes ("JSIGRAPHSET")) {
                                if (node.HasValue("name") && node.GetValue("name") == graphSet)
                                        foreach (ConfigNode graphNode in node.GetNodes("GRAPH"))
                                                graphs.Add(new GraphLine(graphNode, xGraphSpan, ySpan, interval, comp));
                        }

If you enclose the current .cfg structure in an extra block, you'll be able to search for it by name instead and process it regardless of where it's located.

Link to comment
Share on other sites
TheDude

TheDude

Posted
Posted
the problem is case sensivity. Fustek != FusTek. I am used to finding these in code :)

Gads... yep. That was it.

I do have another question.

Why is it that when there is a problem with the game hanging/pausing/crashing on load it's because of something in either SQUAD or INTERSTELLAR? I'm running 2.12 Basic and out of 30 attempts to start the game it'll only launch one time. The other 29 times it'll hang on something in the SQUAD or INTERSTELLAR folder. The KSP log will show one of thee thing:

1) Nothing. It jus stops.

2) Out of memory even though kSP was only using 1.5 GB at the time.

3) I'll get a "NullReferenceException: Object reference not set to an instance of an object" (See log entry below)

[LOG 06:43:05.302] PartLoader: Compiling Part 'Squad/Parts/Command/cupola/part/cupola'

[LOG 06:43:05.311] Tac.LifeSupportModule[FFFC77CE][54.02]: OnAwake

[LOG 06:43:05.321] Tac.LifeSupportModule[FFFC77AA][54.02]: OnAwake

[LOG 06:43:05.323] PartLoader: Compiling Part 'Squad/Parts/Command/Mk1-2Pod/part/Mark1-2Pod'

[LOG 06:43:05.337] Tac.LifeSupportModule[FFFC7712][54.02]: OnAwake

[LOG 06:43:05.349] Tac.LifeSupportModule[FFFC76EE][54.02]: OnAwake

[LOG 06:43:05.352] PartLoader: Compiling Part 'Squad/Parts/Command/mk1pod/part/mk1pod'

[LOG 06:43:05.365] Tac.LifeSupportModule[FFFC7632][54.02]: OnAwake

[EXC 06:43:05.366] NullReferenceException: Object reference not set to an instance of an object

[EXC 06:43:05.379] NullReferenceException: Object reference not set to an instance of an object

[ERR 06:44:08.643] MechJeb module MechJebModuleMenuToolbar threw an exception in OnDestroy: System.NullReferenceException: Object reference not set to an instance of an object

at MuMech.MechJebModuleMenuToolbar.OnDestroy () [0x00000] in <filename unknown>:0

at MuMech.MechJebCore.OnDestroy () [0x00000] in <filename unknown>:0

[ERR 06:44:08.659] MechJeb module MechJebModuleMenuToolbar threw an exception in OnDestroy: System.NullReferenceException: Object reference not set to an instance of an object

at MuMech.MechJebModuleMenuToolbar.OnDestroy () [0x00000] in <filename unknown>:0

at MuMech.MechJebCore.OnDestroy () [0x00000] in <filename unknown>:0

[WRN 06:44:08.668] [Kethane] Saving settings

I deleted Interstellar which has helped but I'm still getting lots and lots of hangs on the SQUAD folder. What is it about these two mods that just doesn't play well with the texture compressor?

Best regards,

The Dude

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
Gads... yep. That was it.

I do have another question.

Why is it that when there is a problem with the game hanging/pausing/crashing on load it's because of something in either SQUAD or INTERSTELLAR? I'm running 2.12 Basic and out of 30 attempts to start the game it'll only launch one time. The other 29 times it'll hang on something in the SQUAD or INTERSTELLAR folder. The KSP log will show one of thee thing:

1) Nothing. It jus stops.

2) Out of memory even though kSP was only using 1.5 GB at the time.

3) I'll get a "NullReferenceException: Object reference not set to an instance of an object" (See log entry below)

[LOG 06:43:05.302] PartLoader: Compiling Part 'Squad/Parts/Command/cupola/part/cupola'

[LOG 06:43:05.311] Tac.LifeSupportModule[FFFC77CE][54.02]: OnAwake

[LOG 06:43:05.321] Tac.LifeSupportModule[FFFC77AA][54.02]: OnAwake

[LOG 06:43:05.323] PartLoader: Compiling Part 'Squad/Parts/Command/Mk1-2Pod/part/Mark1-2Pod'

[LOG 06:43:05.337] Tac.LifeSupportModule[FFFC7712][54.02]: OnAwake

[LOG 06:43:05.349] Tac.LifeSupportModule[FFFC76EE][54.02]: OnAwake

[LOG 06:43:05.352] PartLoader: Compiling Part 'Squad/Parts/Command/mk1pod/part/mk1pod'

[LOG 06:43:05.365] Tac.LifeSupportModule[FFFC7632][54.02]: OnAwake

[EXC 06:43:05.366] NullReferenceException: Object reference not set to an instance of an object

[EXC 06:43:05.379] NullReferenceException: Object reference not set to an instance of an object

[ERR 06:44:08.643] MechJeb module MechJebModuleMenuToolbar threw an exception in OnDestroy: System.NullReferenceException: Object reference not set to an instance of an object

at MuMech.MechJebModuleMenuToolbar.OnDestroy () [0x00000] in <filename unknown>:0

at MuMech.MechJebCore.OnDestroy () [0x00000] in <filename unknown>:0

[ERR 06:44:08.659] MechJeb module MechJebModuleMenuToolbar threw an exception in OnDestroy: System.NullReferenceException: Object reference not set to an instance of an object

at MuMech.MechJebModuleMenuToolbar.OnDestroy () [0x00000] in <filename unknown>:0

at MuMech.MechJebCore.OnDestroy () [0x00000] in <filename unknown>:0

[WRN 06:44:08.668] [Kethane] Saving settings

I deleted Interstellar which has helped but I'm still getting lots and lots of hangs on the SQUAD folder. What is it about these two mods that just doesn't play well with the texture compressor?

Best regards,

The Dude

The system is running out of memory. I'm looking into why right now. I think garbage collection may need to be forced. I have a heavily modded game folder that runs at the main menu at just 1.2GB, but it can get up to 3.xx and crash just as easily.

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
It's bad not to be curious. :)

Coming back to the topic, though, I have a suggestion.

Currently, config files are only loaded from one location. I.e. if I'm a new modder, who released a large pack of parts and wishes to specify how small the textures in it are allowed to go, I have no option to distribute a config file with it without complicating the directory structure, I'd have to place it in a specific location.

But GameDatabase actually loads all *.cfg files into memory at once, regardless if the blocks in them are recognised or not. You don't even have to use System.IO to get at them -- for example, here's a piece from RPM:


                        foreach (ConfigNode node in GameDatabase.Instance.GetConfigNodes ("JSIGRAPHSET")) {
                                if (node.HasValue("name") && node.GetValue("name") == graphSet)
                                        foreach (ConfigNode graphNode in node.GetNodes("GRAPH"))
                                                graphs.Add(new GraphLine(graphNode, xGraphSpan, ySpan, interval, comp));
                        }

If you enclose the current .cfg structure in an extra block, you'll be able to search for it by name instead and process it regardless of where it's located.

hmm... i'll look into this, but with the number of problems people are currently having still, I like the idea of all the configs being easy to find, rather than to check in each mod's folder to ensure that they have a config or not.

Link to comment
Share on other sites
sarbian

sarbian

Posted
Posted
The system is running out of memory. I'm looking into why right now. I think garbage collection may need to be forced. I have a heavily modded game folder that runs at the main menu at just 1.2GB, but it can get up to 3.xx and crash just as easily.

Forcing garbage is seldom a good idea and shouldn't change much, except get you a "real" info on the memory usage. Unless mono GC is badly written.

Link to comment
Share on other sites
rbray89

rbray89

Posted
  • Author
Posted
Forcing garbage is seldom a good idea and shouldn't change much, except get you a "real" info on the memory usage. Unless mono GC is badly written.

That's what I thought... until I just tried it out. No crash at all, and I'm running with a ridiculous number of mods. During loading never gets above 2.1GB. Before it would usually keep increasing until the main menu, at which point it would go down. I'll be uploading this release today.

Link to comment
Share on other sites
team.leit

team.leit

Posted
Posted
That's what I thought... until I just tried it out. No crash at all, and I'm running with a ridiculous number of mods. During loading never gets above 2.1GB. Before it would usually keep increasing until the main menu, at which point it would go down. I'll be uploading this release today.

If it works as well as you are implying, I think I might just grab it

Link to comment
Share on other sites
TheDude

TheDude

Posted
Posted

Greetings,

Even though task manager says KSP is using 1.x GB of memory it's still running out of memory? Is it trying to allocate more than than the 3.7 GB limit which is causing the crash? Now that I think of it I need to go into task manager and view the commit size column. (I'm not a coder by any stretch of the imagation.)

I've gone in and deleted half of my mods and still have the same issue. Even so I can always count on things going pear shaped at the SQUAD folder. Would it help to apply the texture reduction pack or would that potentially make things worse?

Best regards,

The Dude

Link to comment
Share on other sites
Bloodbunny

Bloodbunny

Posted
Posted
Greetings,

Even though task manager says KSP is using 1.x GB of memory it's still running out of memory? Is it trying to allocate more than than the 3.7 GB limit which is causing the crash? Now that I think of it I need to go into task manager and view the commit size column. (I'm not a coder by any stretch of the imagation.)

I've gone in and deleted half of my mods and still have the same issue. Even so I can always count on things going pear shaped at the SQUAD folder. Would it help to apply the texture reduction pack or would that potentially make things worse?

Best regards,

The Dude

It may make it slightly better to use texture reduction, but not much, and it may result in graphical oddities too; dark parts, shiny parts, etc.

Link to comment
Share on other sites
Kass

Kass

Posted
Posted

Loving the extreme reduction pack. I've literally not installed any of the custom reduction packs and KSP with all the usual culprits ( AIES, KW, B9, NP, Kethane, EPLP, City & Clouds, Alternis etc) run with no probs at about 2.5GB.

Thanks for your work on this rbray, have some rep :)

Link to comment
Share on other sites
Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...