[Script] AutoPruner v1.1 - Prune those parts that suck up your RAM! (2015/018)

[Felger]

AutoPruner

AutoPruner is a mod that is designed to go along with the various Procedurals, and remove parts from the game without removing their data.

How to use it:

Drop the files inside the zip into a folder in your root KSP directory (outside GameData).

Open a command line, or run StartThis.bat

To prune a set of parts run:

pruner -prune RO_Global_Tanks.prnl

Or to revert the pruning run:

pruner -unprune RO_Global_Tanks.prnl

And you're done! The files pruned will be ignored by the game when you launch it up next time. Note that any vessels with these parts will not function.

Where to get it:

Download the latest release

Source

How it works:

It works by recursing through all of the directories in your GameData directory, comparing filenames to those in the prunelist (.prnl file), and renaming those that match the criteria in the prunelist.

By renaming the files to an unknown filetype (.pruned) the loader ignores them, and you save the RAM they would have otherwise used.

Changelog:

v1.1 ---- \/

• Added Ariane 6 to prunelists
  
• Added stock wings to prunelists
  

v1.0 ---- \/

• Fixed some bugs, incorporated PJF's performance fixes
  
• Added AIES to the prunelists
  
• Added -list option, lists the available prunelists
  

Edited January 19, 2015 by Felger
Updated to 1.1

[lextacy]

to be clear, this mod physically removes the parts from game saving performance, but does not remove the reference from the part if a vessel is using it?

[Felger]

to be clear, this mod physically removes the parts from game saving performance, but does not remove the reference from the part if a vessel is using it?

Not exactly. The parts remain in the GameData folder, renamed so the game ignores them. It doesn't do any editing of craft files, and since the game doesn't load the parts craft files with those parts will not be loaded by the game.

[Gaalidas]

The model still exists for other parts to reference, as do the textures, but the part would not load or compile. It might improve performance in the part lists, but that's about it really. The loader will skip compiling the parts because it doesn't know they are there, but the model and texture (the really big hit for performance) will still load as an unreferenced asset. Also, while the crafts might not load correctly into the flight-scene, the game will still show them in the editor list with warnings over locked parts detected.

So, basically this is a load-time saver. And a slightly less complicated version of an editor-part-list pruner.

[Felger]

The model still exists for other parts to reference, as do the textures, but the part would not load or compile. It might improve performance in the part lists, but that's about it really. The loader will skip compiling the parts because it doesn't know they are there, but the model and texture (the really big hit for performance) will still load as an unreferenced asset. Also, while the crafts might not load correctly into the flight-scene, the game will still show them in the editor list with warnings over locked parts detected.

So, basically this is a load-time saver. And a slightly less complicated version of an editor-part-list pruner.

My understanding is that the game will not load files into memory that aren't recognized filetypes. I'll test that and come back here with results.

[Felger]

To verify, I loaded up with KW's parts pruned, and without. Without the pruning the game used about 2.7GB of ram, with the pruning it used 2.5GB. Nothing else changed. Conclusion - Part pruning using my method prevents the game from loading parts into memory, and saves you the ram of the unused parts.

[Gaalidas]

My understanding is that the game will not load files into memory that aren't recognized filetypes. I'll test that and come back here with results.

Well, yeah, if the "part" file is unrecognized, then it will not load the part. As for the model and texture, I am less certain. I suppose without the part file, if the model references dummy textures used in texture sharing, then it would not load the full scale textures. The models themselves (except when using static mods like KerbinSide) are relatively small. iT is possible that it still loads all the textures initially, but doesn't commit a copy towards the game if the part compiler deems it unnecessary. I just know that when I watch the loading process, it loads the models and textures before it compiles the parts, but i am unsure when it actually loads the configs themselves. If those are loaded before any of the assets, then you might be correct in that the assets are also skipped.

Yeah, logically that all makes sense. I stand corrected.

[Dust]

Nice mod for automating the process. I recently removed a bunch of fuel tanks to save RAM by using procedural parts instead.

Regarding loading textures: The way I understand it the game will find all cfg files, parse them for assets (textures, models, etc.) and load them into the game. If there are assets that are not referred to by any cfg file then they won't get loaded. If cfg files have a model but refer to non-existent textures the part will have a placeholder blue texture in game. If the model doesn't exist it won't be available in the VAB and any craft using it will not be loaded. Also, existing crafts in flight that use parts that are pruned will not be loaded, and you'll lose them permanently in the next autosave. Also, craft files that used missing parts can't be loaded in the VAB or SPH.

[The-Bean]

Does this not work well with PartCatalog? The bar next to the parts list disappeared. It was working fine over the past few days but just disappeared while playing a while after having installed this.

http://forum.kerbalspaceprogram.com/threads/35018-0-25-PartCatalog-3-0-RC8-%282014-10-08%29

[NathanKell]

Let's clear up some misinformation. Here's how GameDatabase Loader works.

First it processes all files in GameData in alphabetical order. If the file is a dll, try to load it. If the file is a cfg, load it into GameDatabase's confignode tree. If the file is an mu, load it as a model. If the file is an mbm / tga / png / jpg, load it, decompress it, compress it to DXT.

Then process all folders recursively, again in alphabetical order. If the folder is named PluginData, however, do not process any files or folders within it.

Once all that is done, *then* parts, interals, etc are compiled, by linking up the model reference and the texture references as stated in the cfg.

If you rename a file to be not have a recognized extension, it will not be loaded. If, on compiling a part, any of the referenced assets are not found, compilation will fail.

[Felger]

Does this not work well with PartCatalog? The bar next to the parts list disappeared. It was working fine over the past few days but just disappeared while playing a while after having installed this.

http://forum.kerbalspaceprogram.com/threads/35018-0-25-PartCatalog-3-0-RC8-%282014-10-08%29

I have part catalog installed, pruned my folders, and it works just fine:

[The-Bean]

You're right. Looks like I must have accidentally scrolled down.

Thanks for the mod, by the way. Works great.

[Zeenobit]

This is a great idea. It would be even better if it worked alongside Addon Controller.

If this script could automatically read a savefile generated by Addon Controller, and prune all the parts disabled by it, then it would be icing on the cake.

If I'm not mistaken though, Addon Controller saves a whitelist of all the parts enabled in game. This script would have to generate a blacklist from that, I imagine.

[AlmightyR]

I'm receiving "Access denied" when trying to apply any of the prune lists...Any ideas?

[EDIT]

Nevermind...It was my annoying Anti-Virus disabling "remote execution" of .exes (cmd.exe trying to execute pruner.exe...totally local...)

Edited December 4, 2014 by AlmightyR

[Explorator]

Just gonna leave a comment here...

I like how the title is Autopruner... Very clever op

[Felger]

Just wait til you see the fancy GUI version AlmightyR is making!

And I'm not that clever, what am I unintentionally referencing?

[Malah]

A script to easy delete MiniAVC, THANKS!

MiniAVC.prnl:

.*/MiniAVC.dll

[Felger]

A script to easy delete MiniAVC, THANKS!

MiniAVC.prnl:

.*/MiniAVC.dll

That doesn't really save you anything, since plugins don't take up much memory at all. But okay.

[Malah]

This is not for memory saving ... I just don't like to have something that I don't want/need/use, something that I'm forced to install on each update ...

[pjf]

So, I made the code much faster, and also stopped it from mucking around in places like the ATM cache where it has no business being in. We can also show the user which files will be pruned before doing the operation (although right now we just display a summary.) The side-effect is that all rules are now taken to be *prefixes* into GameData, and are no longer counted as regular expressions, avoiding all the future bugs where folks think that '.' is a dot, but is actually a wildcard.

For future versions it might be worth having different types of rules (eg: FILE: Thumbs.db or FILE: MiniAVC.dll to find and remove all files with a given name, along with PREFIX: FASA/Mercury/Fairings to prune everything with a certain prefix), but I think my current changes (if you like them) provide more predictable behaviour and faster operation.

~ Paul

[Felger]

Updated to 1.0, included pjf's fixes, and added some additional functionality.

[mecki]

great!

If I may express a wish: Support of Vens Stock Revamp would be great!

Thanks for your efforts!

- - - Updated - - -

I have a proposal for Vens Stock Parts Revamp yet I'm not sure if this is how AutoPruner works – maybe it can be simplified or it isn't even needed for Vens.

Sorry for not making a pull request – I don't really know how to use Git.

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR10.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR25.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankR1.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankRadial.m

VenStockRevamp/Squad/Parts/FuelTank/RCStankRadialLong.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/T100Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T200Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T400Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T800Tank.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank.m

VenStockRevamp/Squad/Parts/FuelTank/X200-8.m

VenStockRevamp/Squad/Parts/FuelTank/X200-16.m

VenStockRevamp/Squad/Parts/FuelTank/X200-32.m

VenStockRevamp/Squad/Parts/FuelTank/X200-64.m

VenStockRevamp/Squad/SPP/Wings/Wings.m

Edited January 20, 2015 by mecki

[Felger]

great!

If I may express a wish: Support of Vens Stock Revamp would be great!

Thanks for your efforts!

- - - Updated - - -

I have a proposal for Vens Stock Parts Revamp yet I'm not sure if this is how AutoPruner works – maybe it can be simplified or it isn't even needed for Vens.

Sorry for not making a pull request – I don't really know how to use Git.

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR10.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR25.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankR1.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankRadial.m

VenStockRevamp/Squad/Parts/FuelTank/RCStankRadialLong.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/T100Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T200Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T400Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T800Tank.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank.m

VenStockRevamp/Squad/Parts/FuelTank/X200-8.m

VenStockRevamp/Squad/Parts/FuelTank/X200-16.m

VenStockRevamp/Squad/Parts/FuelTank/X200-32.m

VenStockRevamp/Squad/Parts/FuelTank/X200-64.m

VenStockRevamp/Squad/SPP/Wings/Wings.m

Actually, it's a bit simpler than that. Since AutoPruner works off of regular expressions, it'll match anything in the GameData path. So we could very easily reduce that list to:

VenStockRevamp/Squad/Parts/FuelTank/

I'll add that into the next release!

[AlmightyR]

Felger, I just want to let you know that I'm resuming work on the autopruner GUI...Which I had stopped during the holidays and vacation.

Also, do you know of any in-game debug which I can use to know if files are being used or not after removing the CFGs with autopruner? (since it's removing the models and textures that frees up memory, and so they should be autopruned too when something else doesn't use them) (I'm making a list of stuff that comes with the recommended mods and that isn't in any of the current prunelists, and which I don't use and assume many don't either...Some are tanks and fairings, and some I'd say some belong to a separate global list...and some more belong to specialized lists just to give the users the option of easily pruning them)

(PS: Can someone make a kerbal version of this? ^^^^ That would be awesome! )

Edited January 24, 2015 by AlmightyR

[Felger]

Felger, I just want to let you know that I'm resuming work on the autopruner GUI...Which I had stopped during the holidays and vacation.

Also, do you know of any in-game debug which I can use to know if files are being used or not after removing the CFGs with autopruner? (since it's removing the models and textures that frees up memory, and so they should be autopruned too when something else doesn't use them) (I'm making a list of stuff that comes with the recommended mods and that isn't in any of the current prunelists, and which I don't use and assume many don't either...Some are tanks and fairings, and some I'd say some belong to a separate global list...and some more belong to specialized lists just to give the users the option of easily pruning them)

https://hauntedbyhumans.files.wordpress.com/2014/05/821138238_1381248483.jpg

(PS: Can someone make a kerbal version of this? ^^^^ That would be awesome! )

Awesome stuff! Yeah, if you hit alt+f12 it'll open up the debug console, pretty sure it's got a list of all loaded parts in there somewhere. Otherwise you might just have to check the outputlog...