[1.8.x-1.12.x] Module Manager 4.2.3 (July 03th 2023) - Fireworks season

[sarbian]

Starwaster :

If you want it to apply the patch only if FAR is here then you need both the AFTER and NEED

It you use only after then the patch will apply even if FAR is not here ( but after FAR patch if it's here)

If you use only NEED the patch will apply only if FAR is here but not not after FAR

----------------------------------------------------------------------------------------------------------

Swamp_ig finished the last batch of change to MM and 2.1.1 is ready. I need volunteers to test it before we release it.

First a short version of what change in this :

- the code that make sure the module order stays the same after MM applied its patch is still in

- a new Savegame fixed was added. it makes sure that your module order in the save game is the same as the one in the game memory

- more, but it's 2:39 AM here, and it's the short version.

Just before you get to KSP main menu MM will check all you saves and "fix" them. Since we are careful all file that are changed by that process are backuped before in the KSP\saves_backup\xxxxxxxx-xxxxxx directory. With the x being the date and time.

we need some to test that and make sure MM does not mess up thing (I did some testing myself and some bug were fixed, I am quite sure it works fine now).

If you want to test, and feel up to it please :

- BACKUP your saves. MM does it too, but it's better to do it since you may find a nasty bug related to the save process. ( I am quite sure the backup works fine )

- get version 2.1.1 here : ModuleManager.2.1.1.dll

- play the game

- come back to us if you find a bug you think is related to the new MM

Once more :

A) This is a test version

Backup yours saves

C) Don't expect me to shed a tear if you found a bug and did not backup your game.

Edited May 10, 2014 by sarbian

[Woopert]

sarbian: If I'm seeing no visible "symptoms" of save corruption does that mean it's not corrupted? I have a backup from a week ago but the current save that I had going, and installed 2.1.0 on, is that corrupted?

How will you know when a save is corrupted?

[Starwaster]

sarbian: If I'm seeing no visible "symptoms" of save corruption does that mean it's not corrupted? I have a backup from a week ago but the current save that I had going, and installed 2.1.0 on, is that corrupted?

How will you know when a save is corrupted?

Ooooooh, you'll know.

You'll know when you try to switch to a craft and it apparently explodes for no reason at all. (but probably didn't really explode because if you switch away and then back it's still intact)

You'll know if time freezes. The planet no longer passes by beneath the ship.

You'll know when you're not able to get a context menu from any part of the ship.

When dogs and cats are living together.

Mass hysteria!

(although we say the save is corrupted, the damage is generally confined to specific affected craft. Destruct them from the Space Center)

[regex]

Can someone give me a hand with these configs? I'm trying to remove the Oxidizer propellant from my nuclear engines:

@PART[ltby5000]:Final
{
    @MODULE[ModuleEngines]
    {
	@PROPELLANT[LiquidFuel]
	{
		@ratio = 1.0
	}
	!PROPELLANT[Oxidizer] {}
    }
}

I'm using 2.0.8, should I upgrade to fix or just because? TIA

[phoenix_ca]

Is it possible to use a logical OR function in dependency checking? I'd like to apply a FusTek station parts patch that adds RocketParts to the warehouse (for some reason, it doesn't contain them by default). Doing that for myself is fine, but to share it, it would need to do a check for one of two possible loaded dlls, OrbitalConstruction OR Launchpad. Having only one or both should apply the patch, having neither shouldn't.

As I understand it, the "," character denotes an AND. Perhaps using "|" (borrowed from POSIX regex) would be useful for an OR. Preferably globally for all things, so we could do something like say, "@MODULE[ModuleEngines|ModuleEnginesFX]" and the like.

Or just allow for regular expressions for everything and be done with it.

Edited May 10, 2014 by phoenix_ca

[sarbian]

Woopert : as Starwaster explained corruption can be quite visible, or quite subtle. On my save I have lander unable to undock from their base. While searching for a (fixed) bug in this version I had my station cut it self at each docking port on load. The less mod that adds module you use the less chance anything was corrupted. Using tweakable everything or a mechanic is jeb increase stakes a bit (let me repeat that it is a bug with MM and not with those mod)

I hope your save is fine anyway.

regex : the @ratio is not working here too. Looking into it.

Edit:

Regex : it's was a bug. Either revert to 2.0.7 (bug free) or get the newest 2.1.1.

An new version of 2.1.1 is out, without that bug :

https://ksp.sarbian.com/jenkins/job/ModuleManager/lastSuccessfulBuild/artifact/jenkins-ModuleManager-26/ModuleManager.2.1.1.dll

Edited May 10, 2014 by sarbian

[Alewx]

Looks like some real hard work with the new version, the dll size increased by 50%.

Hope you got some spare time in between, and Thanks for the new version.

[swamp_ig]

Don't rush out and get 2.1.1 because I've done a fair few improvements and bug fixes for:

Release 2.1.2

This has some major new features that you'll never want to be without .

I'll post something more detailed up tomorrow after I've slept.

I'd appreciate anyone willing to take it for a spin and see how badly it crashes how amazingly well it works.

WARNING

This version will mess with your save files - that's saved craft and save games.

It does take backups of everything that it touches, but you might be well advised to keep your own backup too.

[swamp_ig]

Is it possible to use a logical OR function in dependency checking? I'd like to apply a FusTek station parts patch that adds RocketParts to the warehouse (for some reason, it doesn't contain them by default). Doing that for myself is fine, but to share it, it would need to do a check for one of two possible loaded dlls, OrbitalConstruction OR Launchpad. Having only one or both should apply the patch, having neither shouldn't.

As I understand it, the "," character denotes an AND. Perhaps using "|" (borrowed from POSIX regex) would be useful for an OR. Preferably globally for all things, so we could do something like say, "@MODULE[ModuleEngines|ModuleEnginesFX]" and the like.

This is indeed implemented, but only for dependencies (NEEDS) You can use either , (comma) or & for and, and | for or. Note that & has lower precedence, so:

Mod1|Mod2&Mod3|Mod4 => ( Mod1 OR Mod2) AND ( Mod3 OR Mod4 )

Edited May 10, 2014 by swamp_ig

[Alewx]

Don't rush out and get 2.1.1 because I've done a fair few improvements and bug fixes for:

Release 2.1.2

WARNING

This version will mess with your save files - that's saved craft and save games.

It does take backups of everything that it touches, but you might be well advised to keep your own backup too.

Is it a feature of a bug?

[phoenix_ca]

No way of reversing that priority? Just curious. I can't think of a use-case now for such logic being necessary and I doubt it'd ever happen but still. When you have the chance it'd be helpful to add that information about the & and | operators for dependency checks to the second post. It's rather conspicuously missing. >.>

On another note, are you saying that 2.1.2 will mess-up save files started with say, 2.1.1 or 2.1.0? (And how long is this savegame killing going to last? >.< )

[toadicus]

This version will mess with your save files - that's saved craft and save games.

It does take backups of everything that it touches, but you might be well advised to keep your own backup too.

Does "mess with" mean "break forever" or "do changes to avoid breaking forever, but might not be working at 100% yet so be careful because forever-breaking remains within the realm of possibility until suitable testing has occured"?

[sarbian]

He did not mean "mess" in a bad way. It's the same as my previous post : this version edits your save files so they work properly even when you add/remove/"oh crap I started without" a mod.

[SnappingTurtle]

"0 unsatisfied needs"? Is MM taunting me now?

[swamp_ig]

Ok, so here's to put some words behind what the 'messing up saves' is all about.

The Problem

Spaceship parts are built up of multiple modules in KSP. Each module is a unit of functionality - for example an engine has a module to manage thrusting and using fuel, one to manage generation of power, and another to throw off the engine fairing when it's stack attached. The way to add functionality to different parts is to add new modules that do whatever it is you're trying to do - so real fuels for example will add its own engine logic to all engines in the game, and some other engine modifying mod may well do the same.

When you load up or save a part, each module gets saved to the save file (that's a .craft for the VAB, and an .sft file for launched ships) in a particular order. This order is expected to remain constant forever by KSP. If you add a module to this list, then not only will that module not reload from the save file, but all the modules after it won't load up correctly either. Not only does it break existing saves, but it breaks them in a way that if you load up the ship, realize you've stuffed up, and then try and reinstall or uninstall the mod you were using, the save is broken permanently and can't be repaired (unless you happened to keep your own backup).

Because the whole purpose of MM is to mess with existing parts, any mod that adds its own module through MM will potentially mess up the save games, which isn't ideal. Obviously this is a bug in KSP, and really would ideally get fixed, but we won't get into that .

The Fix

MM now contains code to stop this problem from happening. What happens is that after you load up KSP, but before it gets a chance to break all your save games, MM runs through all the existing save files and reorders the modules within them to match whatever order they've been set up as in memory (post MM usual editing steps). This will prevent the issue from happening completely, so you can add mods and remove them at any time, safe in the knowledge that you won't be breaking the save games.

Whenever it does this, MM will store a backup of the .craft or .sfs file that has been touched, along with a log of what it has done to the part. This is saved in the root of your KSP install, under saves_backup/YYYYMMDD-HHMMSS/ (dates and times as of the time of loading KSP). It's pretty aggressive about creating this backup at the moment, but we may be able to back-off on some of the backing up for changes that just reorder things rather than removing stuff in the future.

This backup is also created whenever you have a missing part on a ship. Normally KSP would just destroy the ship and it would be gone forever, so that is handy too.

Another feature of this is that if you ever uninstall a mod that adds modules to parts, the save fixer will store the saved state from that module with the part. If you decide you want to reinstall the mod again, the backup of the config will be restored automatically. This is handy if say you're a mod developer and you like to swap real fuels in and out.

The other problem

Because the old version of MM didn't have this behaviour, it's possible that there might be a bit of weirdness when you first load up your saves. Because there's a huge number of mods out there, we can't possibly test everything. This phase of experimental release is really about seeing what happens.

This functionality is really here to stay, and will be an important feature of MM ongoing. Please let me know if you have any issues.

[swamp_ig]

A primer on how KSP configures itself

KSP uses ConfigNodes extensively for all aspects of configuration. The main way in which modders interact with these is in the configuration of parts. There's a fair bit of documentation about this elsewhere, but for the purposes of this document I'll define a few things:

// This is a top level node. Note the name with curly brackets afterwards.
PART
{
    // This is the name value. It's just a standard name/value pair, but any of these named 'name' 
    // is treated a bit special by MM
    name = myPart

    // This is another value. Note name = value . You can't have a name without a value, 
    // but you can have a value without a name (treated as the empty string for name )
    module = Part

    // This is a node named 'MODULE'. It's a collection of further values and modules
    MODULE
    {
        name = ModuleEngines
    }
}

// This is a patch. Note the prefix - this will be one of  '@' for edit, '+' or '$' for copy, 
// '-' or '!' for delete, '%' for replace. A patch takes some pre-existing top level node, and modifies it
@PART[myPart]
{
    // Edit the value with name 'module'
    @module = PartEnhanced

    // Delete all 'MODULE' nodes.
    -MODULE,* { }
}

Mod dependency checking

You can now put a NEEDS section on any section of your file. This isn't just MM patches, it can be literally anywhere - on values, on nodes, on patches, wherever you like. (this enhanced behaviour is new)

An example:

PART:NEEDS[RealFuels|ModularFuelSystem]
{
    name = myPartName

    description:NEEDS[RealFuels] = This part is used in real fuels
    description:NEEDS[ModularFuelSystem] = This part is used in modular fuel system
}

So what this means is the part will only be defined if RealFuels OR ModularFuelSystem is present.

There's two alternatives for the description field, based on what's present.

This is handy if you're a mod developer and you want to create parts that vary a bit depending on what's installed.

The stuff within the needs section is based on either:

• A plugin .dll with the same name (not including dotted extensions) so you'd always get ModuleManager.
  
• A FOR[blah] defined would allow NEEDS[blah]
  

The old , listed operator is still supported, and is treated as an alias for & (AND). If you combine several | and &, eg NEEDS[Mod1|Mod2&Mod3|Mod4] this is treated as ( Mod1 OR Mod2 ) AND ( Mod3 OR Mod4 ). I won't be implementing brackets, it would make the parser far too complicated. There's always a way to represent what you want in this form, although it might need a few repeated terms, but I'm not sure I can truly see much of a use case for anything super complex.

In the below stuff, I've not put in the NEEDS section for clarity, however you can use it wherever you like.

Operators available in patches - and their ordering

The general form of a patch for values is:

<Op><Name-With-Wildcards>(,<index>)?

So breaking this down:

<Op> : One of: nothing for insert, '@' for edit, '+' or '$' for copy, '-' or '!' for delete, '%' for replace.

<Name-With-Wildcards> : The name of the value you'll be messing with. Wildcards are not always available for every Op.

(,<index>)? : Optional index. Again, not available with every option. Not that these indexes are with respect to all name matches not the total index in the node. I will support negative indexes for running backwards through the list soon. Also * is not yet supported.

Wildcards include ? for any character, and * for any number of chars. Note that only alphanumeric chars are allowed in value names for patches. If you have spaces or any other char, use a ? to match it.

The general form for nodes is:

<Op><NodeType>([<NodeNameWithWildcards>])?(:HAS[<has block>])?(,<index-or-*>)?

<Op> : Operator, as above

<NodeType> : typically MODULE or something like it. Wildcards not allowed (I can't imagine you'd need them)

([<NodeNameWithWildcards>])? : This is a wildcard match for the name = <name> value within the node. Optional.

(:HAS[<has block>])? : Optional has block. You can't (currently) use indexes with HAS. This has been described previously. If this is present then all matches will be matched, there's no indexing available currently.

(,<index-or-*>)? : Index to select particular match zero based. 0 is the first node, -1 is last node. Values 'off the end' will match the end, so large positive matches the end and large negative matches the beginning. Again this matches against everything the wildcard selects in order. * here will match everything.

Insert

Insert is the default operator - it's what you get if you don't specify anything. You can now specify the index at which you'd like to insert your bit of config, otherwise it defaults to the end of the list

Example:

@PART[partName] 
{
    node_stack_atend = 0.0, 7.21461, 0.0, 0.0, 1.0, 0.0
    node_stack_atstart,0 =0.0, -7.21461, 0.0, 0.0, -1.0, 0.0  
    MODULE,0 
    {
        name = ModuleSomething
    }
}

This will add an extra value called node_stack_atend at the end of the list, another value node_stack_atstart at the start, and insert a new module first in the list.

Obviously wildcards, * indexes, and other stuff isn't available. Just the value or node name.

Edit - @

Edits the node or value in place. The order will not change (new since 2.0.9 ish).

For nodes, all options are available to select the node, including indexes, * index, HAS, and wildcards in the name. If there are multiple matches and the index is not supplied, this will default editing all matches.

Example

@MMTEST[nodeEdit]
{
	// edit by name, will select the first one if multiple nodes with same name
	@MODULE[module2]	{ ... }
	// edit by index - will edit the second module (0 based indexing )
	@MODULE,1 { ... }
	// edit by name and index - will edit the second module with name=module2
	@MODULE[module2],1 { ... }
	// edit by wildcard and index - will edit the sixth module with name ending with 2. If there's no fifth, edit the last one defined
	@MODULE[*2],5 { ... }
	// edit by wildcard and index - edit the last module with three letter name starting with c
	@MODULE[c??],-1 { ... }
}

For values you additionally have the option of a replacement based on the existing node value, including standard arithmetic operators plus regular expressions for strings. Note: if no index is specified, only the first match will be edited. Once ,* is implemented, then this behaviour should really change to all matches by default, so maybe would be a good idea for patch maintainers to move to using ,0 explicitly.

Example:

@PART[somePart1]
{
	// Unindexed - edits the first
	@name = somePart2
	// Arithmetic - add 5 and multiply by 20. Unindexed so edits the first one.
	@numeric += 5
	@numeric *= 20
	// Regexp expression. Replaces any instance of "tw" with "mo", so would turn "twotwo" into "moomoo"
        // Note also the index.
	@multiVal,1 ^= :tw:mo:
}

For regexp, the first character in the list is used as the separator. You can use whatever you like, but : is often a good choice. Obviously ensure that this isn't present in the regexp expression.

Please note I will not support questions about regexps on this thread! I assume if you want to use this feature you're well versed with how regexps work, including the various variants. Please refer to the .net documentaton and/or copious other documentation available on the Internet.

Anyhow, here's some useful regexps:

":$: Some Extra Stuff:" $ matches the end of the string, so this is an easy way to add suffixes.

":^: Preamble :" Similar to above. ^ matches the start of a string

":^.*$: Preamble $0 Some extra stuff:" Combining the above. ^.*$ matches the entire input, and $0 will stick it in in the output.

Delete - ! or -

Delete a node or value. The order of everything afterwards will obviously drop back one step.

For nodes, all options are available to select the node, including indexes, * index, HAS, and wildcards in the name. If there are multiple matches and the index is not supplied, this will default to again to all nodes.

For values, again all options available. If no index is specified then all matching values are deleted - This differs in behaviour to edit. I will likely change edit to doing this in the future.

Example:

@PART[nodeDelete]
{
	// Delete the first copy
	-MODULE[module2] {}
	// Indexed delete
	-MODULE,2 { }	
	// Indexed delete from end
	-MODULE,-2 { }
	// Indexed delete off end
	-MODULE,9999 { }
	@MODULE[module1]
	{
		// Unindexed (remove all)
		-multiVal = dummy
		// Indexed
		-multiVal2,0 = dummy
		// Wildcard
		-num*ic = dummy
	}
}

Note that you still need to use { and } for nodes, and a dummy value for values. If you don't do this then the parser doesn't know what it's dealing with.

Copy - + or $

Copy behaves identically to Edit, however rather than editing the node or value it copies it. It will always put the copy at the end of the list.

Note that for parts, you must always give a new name or it's a bit pointless:

+PART[myPartName] 
{
    @name = myNewPartCopy
}

Replace - %

This will edit the value it it exists, otherwise it will create a new value as though this was an insert.

For existing nodes this will be identical to edit if the node exists, otherwise it will create an empty node, named as per the node name, and run it through the patch. Obviously because of the insert part, wildcards aren't allowed because the result wouldn't make sense. (although this is not flagged as an error in the current build for nodes)

For values this is identical to doing a delete and then an insert. Wildcards and indexes are not supported. The delete will delete all matches. This command has quite limited functionality, but it's there.

Test cases / examples

I've created some automated test cases for this build, they're here. These also double as good examples of how it works.

Later on I'll put up another post targeted really towards modders, so they can get the most out of dynamic module adding and removing, but that will do for now.

Edited May 12, 2014 by swamp_ig

[Alewx]

And to the normal guys that have no clue about coding it means 2.1.1 is now safe to use with any mod, and in the unlikly case something breaks it makes a backup and logs the stuff, yeah Need it

But why isn't 2.1.1 then the download in the first post and in the title?

[phoenix_ca]

Probably because it still breaks things.

What I really want to know is if I'm starting a new game, should I use 2.1.2? Or should I expect more changes to be made that'll break things? I assume the answer is no but still...

[Starstrider42]

The general form of a patch for values is:

<Op><Name-With-Wildcards>(,<index>)?

...

The general form for nodes is:

...

Thanks for the detailed post, it's much clearer than the original MM instructions. However, do you think you could define (possibly with examples) what you mean by patch and node? This part is pretty confusing to me, and I imagine it would be worse for people who are just starting with MM and aren't familiar with the file structure.

[Starwaster]

I think my mind is blown now...

[ialdabaoth]

...

Well dayam. Salutes swamp_ig.

[NathanKell]

Hol-eee smokes. Very nice!

Quick question: you have

    node_stack_atend = 0.0, 7.21461, 0.0, 0.0, 1.0, 0.0
    node_stack_atstart =0.0, -7.21461, 0.0, 0.0, -1.0, 0.0  

How does MM know to place one at the end and one at the start? Is the example missing something, or am I?

[Alewx]

Hol-eee smokes. Very nice!

Quick question: you have

    node_stack_atend = 0.0, 7.21461, 0.0, 0.0, 1.0, 0.0
    node_stack_atstart =0.0, -7.21461, 0.0, 0.0, -1.0, 0.0  

How does MM know to place one at the end and one at the start? Is the example missing something, or am I?

That are the attachment nodes and not modules, MM won't touch them as far as I understood it.

[ialdabaoth]

Hol-eee smokes. Very nice!

Quick question: you have

    node_stack_atend = 0.0, 7.21461, 0.0, 0.0, 1.0, 0.0
    node_stack_atstart =0.0, -7.21461, 0.0, 0.0, -1.0, 0.0  

How does MM know to place one at the end and one at the start? Is the example missing something, or am I?

I'm pretty sure his example is missing; it should look like this:

    node_stack_atend = 0.0, 7.21461, 0.0, 0.0, 1.0, 0.0
    node_stack_atstart,0 =0.0, -7.21461, 0.0, 0.0, -1.0, 0.0  

[NathanKell]

That's what I thought. Just wanted confirmation.