Felger

[Script] AutoPruner v1.1 - Prune those parts that suck up your RAM! (2015/018)

Recommended Posts

AutoPruner

AutoPruner is a mod that is designed to go along with the various Procedurals, and remove parts from the game without removing their data.

How to use it:

Drop the files inside the zip into a folder in your root KSP directory (outside GameData).

Open a command line, or run StartThis.bat

To prune a set of parts run:

pruner -prune RO_Global_Tanks.prnl

Or to revert the pruning run:

pruner -unprune RO_Global_Tanks.prnl

And you're done! The files pruned will be ignored by the game when you launch it up next time. Note that any vessels with these parts will not function.

Where to get it:

Download the latest release

Source

How it works:

It works by recursing through all of the directories in your GameData directory, comparing filenames to those in the prunelist (.prnl file), and renaming those that match the criteria in the prunelist.

By renaming the files to an unknown filetype (.pruned) the loader ignores them, and you save the RAM they would have otherwise used.

Changelog:

v1.1 ---- \/

  • Added Ariane 6 to prunelists
  • Added stock wings to prunelists

v1.0 ---- \/

  • Fixed some bugs, incorporated PJF's performance fixes
  • Added AIES to the prunelists
  • Added -list option, lists the available prunelists

Edited by Felger
Updated to 1.1

Share this post


Link to post
Share on other sites

to be clear, this mod physically removes the parts from game saving performance, but does not remove the reference from the part if a vessel is using it?

Share this post


Link to post
Share on other sites
to be clear, this mod physically removes the parts from game saving performance, but does not remove the reference from the part if a vessel is using it?

Not exactly. The parts remain in the GameData folder, renamed so the game ignores them. It doesn't do any editing of craft files, and since the game doesn't load the parts craft files with those parts will not be loaded by the game.

Share this post


Link to post
Share on other sites

The model still exists for other parts to reference, as do the textures, but the part would not load or compile. It might improve performance in the part lists, but that's about it really. The loader will skip compiling the parts because it doesn't know they are there, but the model and texture (the really big hit for performance) will still load as an unreferenced asset. Also, while the crafts might not load correctly into the flight-scene, the game will still show them in the editor list with warnings over locked parts detected.

So, basically this is a load-time saver. And a slightly less complicated version of an editor-part-list pruner.

Share this post


Link to post
Share on other sites
The model still exists for other parts to reference, as do the textures, but the part would not load or compile. It might improve performance in the part lists, but that's about it really. The loader will skip compiling the parts because it doesn't know they are there, but the model and texture (the really big hit for performance) will still load as an unreferenced asset. Also, while the crafts might not load correctly into the flight-scene, the game will still show them in the editor list with warnings over locked parts detected.

So, basically this is a load-time saver. And a slightly less complicated version of an editor-part-list pruner.

My understanding is that the game will not load files into memory that aren't recognized filetypes. I'll test that and come back here with results.

Share this post


Link to post
Share on other sites

To verify, I loaded up with KW's parts pruned, and without. Without the pruning the game used about 2.7GB of ram, with the pruning it used 2.5GB. Nothing else changed. Conclusion - Part pruning using my method prevents the game from loading parts into memory, and saves you the ram of the unused parts.

Share this post


Link to post
Share on other sites
My understanding is that the game will not load files into memory that aren't recognized filetypes. I'll test that and come back here with results.

Well, yeah, if the "part" file is unrecognized, then it will not load the part. As for the model and texture, I am less certain. I suppose without the part file, if the model references dummy textures used in texture sharing, then it would not load the full scale textures. The models themselves (except when using static mods like KerbinSide) are relatively small. iT is possible that it still loads all the textures initially, but doesn't commit a copy towards the game if the part compiler deems it unnecessary. I just know that when I watch the loading process, it loads the models and textures before it compiles the parts, but i am unsure when it actually loads the configs themselves. If those are loaded before any of the assets, then you might be correct in that the assets are also skipped.

Yeah, logically that all makes sense. I stand corrected.

Share this post


Link to post
Share on other sites

Nice mod for automating the process. I recently removed a bunch of fuel tanks to save RAM by using procedural parts instead.

Regarding loading textures: The way I understand it the game will find all cfg files, parse them for assets (textures, models, etc.) and load them into the game. If there are assets that are not referred to by any cfg file then they won't get loaded. If cfg files have a model but refer to non-existent textures the part will have a placeholder blue texture in game. If the model doesn't exist it won't be available in the VAB and any craft using it will not be loaded. Also, existing crafts in flight that use parts that are pruned will not be loaded, and you'll lose them permanently in the next autosave. Also, craft files that used missing parts can't be loaded in the VAB or SPH.

Share this post


Link to post
Share on other sites

Let's clear up some misinformation. Here's how GameDatabase Loader works.

First it processes all files in GameData in alphabetical order. If the file is a dll, try to load it. If the file is a cfg, load it into GameDatabase's confignode tree. If the file is an mu, load it as a model. If the file is an mbm / tga / png / jpg, load it, decompress it, compress it to DXT.

Then process all folders recursively, again in alphabetical order. If the folder is named PluginData, however, do not process any files or folders within it.

Once all that is done, *then* parts, interals, etc are compiled, by linking up the model reference and the texture references as stated in the cfg.

If you rename a file to be not have a recognized extension, it will not be loaded. If, on compiling a part, any of the referenced assets are not found, compilation will fail.

Share this post


Link to post
Share on other sites
Does this not work well with PartCatalog? The bar next to the parts list disappeared. It was working fine over the past few days but just disappeared while playing a while after having installed this.

http://forum.kerbalspaceprogram.com/threads/35018-0-25-PartCatalog-3-0-RC8-%282014-10-08%29

I have part catalog installed, pruned my folders, and it works just fine:

CzDoFOi.jpg

Share this post


Link to post
Share on other sites

You're right. Looks like I must have accidentally scrolled down.

Thanks for the mod, by the way. Works great.

Share this post


Link to post
Share on other sites

This is a great idea. It would be even better if it worked alongside Addon Controller.

If this script could automatically read a savefile generated by Addon Controller, and prune all the parts disabled by it, then it would be icing on the cake. :)

If I'm not mistaken though, Addon Controller saves a whitelist of all the parts enabled in game. This script would have to generate a blacklist from that, I imagine.

Share this post


Link to post
Share on other sites

I'm receiving "Access denied" when trying to apply any of the prune lists...Any ideas?

[EDIT]

Nevermind...It was my annoying Anti-Virus disabling "remote execution" of .exes (cmd.exe trying to execute pruner.exe...totally local...)

Edited by AlmightyR

Share this post


Link to post
Share on other sites

Just wait til you see the fancy GUI version AlmightyR is making!

And I'm not that clever, what am I unintentionally referencing?

Share this post


Link to post
Share on other sites

A script to easy delete MiniAVC, THANKS!

MiniAVC.prnl:

.*/MiniAVC.dll

Share this post


Link to post
Share on other sites
A script to easy delete MiniAVC, THANKS!

MiniAVC.prnl:

.*/MiniAVC.dll

That doesn't really save you anything, since plugins don't take up much memory at all. But okay.

Share this post


Link to post
Share on other sites

This is not for memory saving ... I just don't like to have something that I don't want/need/use, something that I'm forced to install on each update ...

Share this post


Link to post
Share on other sites

So, I made the code much faster, and also stopped it from mucking around in places like the ATM cache where it has no business being in. We can also show the user which files will be pruned before doing the operation (although right now we just display a summary.) The side-effect is that all rules are now taken to be *prefixes* into GameData, and are no longer counted as regular expressions, avoiding all the future bugs where folks think that '.' is a dot, but is actually a wildcard.

For future versions it might be worth having different types of rules (eg: FILE: Thumbs.db or FILE: MiniAVC.dll to find and remove all files with a given name, along with PREFIX: FASA/Mercury/Fairings to prune everything with a certain prefix), but I think my current changes (if you like them) provide more predictable behaviour and faster operation.

~ Paul

Share this post


Link to post
Share on other sites

Updated to 1.0, included pjf's fixes, and added some additional functionality.

Share this post


Link to post
Share on other sites

great!

If I may express a wish: Support of Vens Stock Revamp would be great!

Thanks for your efforts!

- - - Updated - - -

I have a proposal for Vens Stock Parts Revamp yet I'm not sure if this is how AutoPruner works – maybe it can be simplified or it isn't even needed for Vens.

Sorry for not making a pull request – I don't really know how to use Git.

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR10.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR25.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankR1.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankRadial.m

VenStockRevamp/Squad/Parts/FuelTank/RCStankRadialLong.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/T100Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T200Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T400Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T800Tank.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank.m

VenStockRevamp/Squad/Parts/FuelTank/X200-8.m

VenStockRevamp/Squad/Parts/FuelTank/X200-16.m

VenStockRevamp/Squad/Parts/FuelTank/X200-32.m

VenStockRevamp/Squad/Parts/FuelTank/X200-64.m

VenStockRevamp/Squad/SPP/Wings/Wings.m

Edited by mecki

Share this post


Link to post
Share on other sites
great!

If I may express a wish: Support of Vens Stock Revamp would be great!

Thanks for your efforts!

- - - Updated - - -

I have a proposal for Vens Stock Parts Revamp yet I'm not sure if this is how AutoPruner works – maybe it can be simplified or it isn't even needed for Vens.

Sorry for not making a pull request – I don't really know how to use Git.

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/1.25mTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/2.5mRCS_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/Big0Tank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/LargeTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR10.m

VenStockRevamp/Squad/Parts/FuelTank/RCSFuelTankR25.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankR1.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTankRadial.m

VenStockRevamp/Squad/Parts/FuelTank/RCStankRadialLong.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_Clr.m

VenStockRevamp/Squad/Parts/FuelTank/RCSTanks_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/T100Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T200Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T400Tank.m

VenStockRevamp/Squad/Parts/FuelTank/T800Tank.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_CLR.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank_NRM.m

VenStockRevamp/Squad/Parts/FuelTank/ToroidalTank.m

VenStockRevamp/Squad/Parts/FuelTank/X200-8.m

VenStockRevamp/Squad/Parts/FuelTank/X200-16.m

VenStockRevamp/Squad/Parts/FuelTank/X200-32.m

VenStockRevamp/Squad/Parts/FuelTank/X200-64.m

VenStockRevamp/Squad/SPP/Wings/Wings.m

Actually, it's a bit simpler than that. Since AutoPruner works off of regular expressions, it'll match anything in the GameData path. So we could very easily reduce that list to:

VenStockRevamp/Squad/Parts/FuelTank/

I'll add that into the next release!

Share this post


Link to post
Share on other sites

Felger, I just want to let you know that I'm resuming work on the autopruner GUI...Which I had stopped during the holidays and vacation.

Also, do you know of any in-game debug which I can use to know if files are being used or not after removing the CFGs with autopruner? (since it's removing the models and textures that frees up memory, and so they should be autopruned too when something else doesn't use them) (I'm making a list of stuff that comes with the recommended mods and that isn't in any of the current prunelists, and which I don't use and assume many don't either...Some are tanks and fairings, and some I'd say some belong to a separate global list...and some more belong to specialized lists just to give the users the option of easily pruning them)

821138238_1381248483.jpg

(PS: Can someone make a kerbal version of this? ^^^^ That would be awesome! :D)

Edited by AlmightyR

Share this post


Link to post
Share on other sites
Felger, I just want to let you know that I'm resuming work on the autopruner GUI...Which I had stopped during the holidays and vacation.

Also, do you know of any in-game debug which I can use to know if files are being used or not after removing the CFGs with autopruner? (since it's removing the models and textures that frees up memory, and so they should be autopruned too when something else doesn't use them) (I'm making a list of stuff that comes with the recommended mods and that isn't in any of the current prunelists, and which I don't use and assume many don't either...Some are tanks and fairings, and some I'd say some belong to a separate global list...and some more belong to specialized lists just to give the users the option of easily pruning them)

https://hauntedbyhumans.files.wordpress.com/2014/05/821138238_1381248483.jpg

(PS: Can someone make a kerbal version of this? ^^^^ That would be awesome! :D)

Awesome stuff! Yeah, if you hit alt+f12 it'll open up the debug console, pretty sure it's got a list of all loaded parts in there somewhere. Otherwise you might just have to check the outputlog...

Share this post


Link to post
Share on other sites
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.