[0.90] TestFlight [0.4.6.1][04FEB15] -Configurable, extensible, parts research & reliability system

[Agathorn]

TestFlight

A configurable, extensible, parts research and reliability system for Kerbal Space Program (KSP).

What Is TestFlight?

TestFlight is a persistent, parts based, research and reliability system. It gives you a reason to do test flights of your rockets. Test flights let your engineers generate flight data, which can then be analyzed to further improve the reliability of your rocket parts. Flight data generated on a part is persistent, and carries over to new rockets flown with those parts.

In a nutshell; Fly your parts more, generate flight data, get more reliable parts. But TestFlight is much more!

License

TestFlight is released on the Creative Commons 4.0 by-nc-sa License.

http://creativecommons.org/licenses/by-nc-sa/4.0/

GitHub - Source Code

https://github.com/jwvanderbeck/TestFlight

Bug Tracker

https://github.com/jwvanderbeck/TestFlight/issues

Releases

https://github.com/jwvanderbeck/TestFlight/releases

v0.1.0 Alpha

https://github.com/jwvanderbeck/TestFlight/releases/tag/v0.1.0

v0.2.0 Alpha

https://github.com/jwvanderbeck/TestFlight/releases/tag/v0.2.0

v0.2.1 Alpha

https://github.com/jwvanderbeck/TestFlight/releases/tag/v0.2.1

v0.3.0 Alpha

https://github.com/jwvanderbeck/TestFlight/releases/tag/v0.3.0

v0.4.0 Alpha

https://github.com/jwvanderbeck/TestFlight/releases/tag/v0.4.0

v0.4.5 Alpha

• Stock: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.5-Stock
  
• RealismOverhaul: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.5-RealismOverhaul
  
• 
  

v0.4.6 Alpha

• Stock: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.6-Stock
  
• RealismOverhaul: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.6-RealismOverhaul
  
• 
  

v0.4.6.1 Alpha

• Stock: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.6.1-Stock
  
• RealismOverhaul: https://github.com/jwvanderbeck/TestFlight/releases/tag/0.4.6.1-RealismOverhaul
  
• 
  

v.0.4.6.1 Alpha Release

• ShutdownEngine failure should now also shutdown the visual effects playing with certain mods like RealPlume/Smoekscreen
  
• ShutdownEngine failure now uses the common EngineModuleWrapper
  
• ReducedMaxThrust should now work properly with RF/MFT or any other ModuleEngineConfigs based mod
  
• TestFlight now has AVC .version files. It does not include miniAVC, but if you are using the full AVC mod (which I highly suggest because its awesome) you should now be notified of updates
  
• TestFlight now generates CKAN files so should be on CKAN soon
  
• FIX Fixed a bug in Failure modules that was preventing the `duFail` and `duRepair` properties from working. Unfortunately due to the way KSP works, you will need to create new craft for this fix to take effect. Any existing craft or saved .craft files will not work.
  

v.0.4.6 Alpha Release

• FIX #21 When triggering a random failure, skip ones marked as disabled
  
• FIX #22 Only add positive flight data. Should resolve issue with 0 or negative du showing in MSD
  
• NEW #23 Parts can now gain bonus flight data when they fail, and when failures are repaired. After all, we're supposed to learn from our failures! New properties `duFail` and `duRepair` added to Failure modules and implemented automatically by the base class.
  
• NEW #26 Failures can be "one-shot" which means the bad stuff from the failure happens but the part is not placed into a failed state and continues to operate (if possible). New property `oneShot` added to Failure modules.
  
• NEW #29 Support for EngineIgnitor mod. The IgnitionFail Failure Module can optionally restore a used up ignition when it fails. New property `restoreIgnitionCharge` added to TestFlightFailure_IgnitionFail Module
  
• FIX EngineCycle Reliability was incorrectly modifying Base Failure Rate, and was being enabled when it should have been. Both issues fixed.
  
• NEW #31 Instead of defining a reliabilityCurve for every one of the possible 33 scopes, even when they were the same, a "default" scope curve can now be used and TestFlightReliability will use the default scope if a specific one is not found.
  
• [RealismOverhaul] Removed accidental double definition of Aerobee-150.
  
• [RealismOverhaul] Re-added use of IgnitionFail failure after bugs were fixed.
  
• [RealismOverhaul] Added large chance of ignition failure to X-405 Vanguard.
  
• [RealismOverhaul] WAC-Corporal/Aerobee Line, X-405, and AJ-10-37/42 all use EngineCycle Reliability now, which defines a "bathtub" curve for reliability over the expected operating cycle of the engine, based on real manufacturer specs. This means the engine will have an increase in failure rate for the first few seconds of operation, smooth out to normal Base Failure Rate for the "Rated Burn Time" of the engine, and the the failure rate will slowly start increasing as the engine exceeds the manufacturer's "Rated Burn Time". Rated burn times: WAC/Aerobee: 47 seconds, X-405: 145 seconds, AJ-10-37/42:L 115 seconds.
  

v.0.4.5 Alpha Release

• All TestFlight modules can now use a new property configuration. If specified TestFlight will only enable that module if the part's currently active MFT/RF configuration matches the specified value. This allows configuration of ModuleEngineConfig parts as if they were their own separate parts.
  
• DataRecorder runs off operatingTime now and therefore will not record data if the part is in a failed state
  
• API: Added new methods for retrieving TestFlight modules on a part.
  
• New Module: Added new Failure module, TestFlightFailure_IgnitionFail which is works a bit different than most, and would be considered a FailureTrigger module. It monitors for an engine to change states from not ignited, to ignited, and applies a chance for that ignition to fail. This chance is based on FlightData and the curve can be configured in .cfg file.
  
• Initial configs for RO starting with the WAC-Corporal/Aerobee engine line.
  
• operatingTime is no longer capped at MTBF so parts can still fail after MTBF has passed
  
• Added new class to TestFlightAPI to wrap up the annoying duality of ModuleEngines and ModulesEnginesFX
  
• New property added to all Module Interfaces `TestFlightEnabled` determines if the module is currently running under TestFlight
  
• Fixed bug with "0 time to repair" failures not giving the Repair Button
  
• Fixed bug with failures that had no repair config breaking the system
  
• Added Technology Transfer system that allows partial flight data to transfer from one part to another part that is considered to be a related technology path.
  
• Swapped to using System.Random for random numbers since apparently KSP spews out the same random sequence each time you load a game.
  
• Added instance of System.Random to TestFlightCore and TestFlightManagerScenario for use by other modules.
  
• Fixed issue #16 - parts continue to accrue operating time even when failed
  
• Fixed issue #19 - operating time wasn't respecting IsPartOperating
  
• More adjustments to MSD window size to avoid horizontal scrolling
  
• Updates to build system to allow building Stock and RealismOverhaul configs as needed
  

v0.4.0 Alpha Release

• Rewrote core scenario code. TestFlightManagerScenario is now only a data store for the persistent data, and contains no appreciable game logic
  
• Redesigned TestFlightCore API to accommodate both the new "push" method as well as upcoming changes to the underlying reliability and failure architecture
  
• Flight Data and Flight Time now stored as doubles
  
• Implemented brand new Test Flight API
  
• Changes to ITestFlightCore interface
  
• Changes to ITestFlightDataRecorder interface
  
• Changes to ITestFlightReliability interface
  
• Core system refreshes status more often now that there is less of a performance hit
  
• Changed method of configuring how FlightData converts into Reliability in the config files. This is now done using a FloatCurve which allows the modder to make it as simple or as complex as desired
  
• Fixed a bug that caused reliability to not load properly when going through the VAB with a craft
  
• Fixed a bug causing "sub" modules to load before the Core and therefore never attaching properly
  
• Added new Editor Window to see part's flight data and reliability while building
  
• Defer loading of prefab data to Start() in case KSP is being slothful. Should fix reliability not loading correctly in Flight.
  
• Initial implementation of new MTBF system
  
• MSD now shows MTBF
  
• MSD now shows Failure Rate
  
• Make font in MSD smaller, and adjusted window to accommodate new data
  
• Updated settings pane in MSD to remove no longer used settings, and rename/tweak ones that are left.
  
• New Reflection Interface for other mods to integrate with TestFlight. This is still a WIP and feedback is more than welcome.
  
• Fix issue #11 - Settings Dropdown List hard to read
  
• Fix issues #14 - Data rate multiplier being incorrectly applied and giving zillions of data units (or none!)
  
• New API methods to allow repairs to take time to complete
  
• Expanded Reflection Interface and API
  
• MSD can now show failure rate in addtion to MTBF
  
• failure rate in MSD now properly shows the 'worst' momentary failure rate
  
• DataRecorder modules can now control when a part is considered "operating"
  
• Mission time is now calculated from activation of first stage, not from MET which KSP does not start until you leave the pad. This means two main things.
  
  • Engine test stands are a thing
    
  • The practice of igniting your first stage before releasing the launch clamps is a good one, to ensure you engine(s) are ignited and running properly.
    

  [*] Added new Reliability module that increase the failure rate of an engine for the first 5 seconds after ignition. This is also a sample of the power of the new system to do things like this! Currently enabled on liquid engines.

  [*] Added new API methods to control flow of Flight Data

  • SetDataRateLimit()
    
  • SetDataCap()
    

  [*] Changed failure checks to be a constant chance of failure, rather than increasing towards MTBF

  [*] Added Travis CI integration for continuous integration testing of commits, and automated builds.

  [*] TestFlight configs are now built in JSON and then compiled to standard ModuleManager config format upon build and release. This standardizes the configs, as well as makes them easier to write by removing redundancy and allowing re-use of setups.

v0.3.0 Alpha Release

• Added entirely new Master Status Display (MSD) for TestFlight
  
• Added settings window for configuring TestFlight settings in game
  
• Added ability to independently toggle on or off various pieces of part status in the MSD
  
• Added new compact in flight HUD that can be toggled on/off and positioned as desired
  
• MSD can be either docked, or unlocked and moved around
  
• MSD now scrolls if there are too many parts to fit
  
• MSD height is adjustable
  
• Settings pane broken into multiple pages so as to not require a huge window
  
• Repair Requirements tooltip is now color coded for clarity and quick assimilation of data
  
• Fixed spelling error in settings dialog
  
• Added button to "acknowledge" a failure. This will clear it from the Flight HUD and the MSD if set to show only failed parts, but will not repair the failure. It will still remain in the full MSD list (If the show only failed parts setting is OFF)
  
• Added the failure title to the tooltip of the Flight HUD
  
• Rebuilt ScenarioModule to provide a more reliable data store
  

v0.2.1 Alpha Release

• Fixed Master Status Display not showing some failures
  
• Fixed configs for stock parts
  

v0.2.0 Alpha Release

• New Recorder module, FlightDataRecorder_Resources, only records flight data while the part has stored resources.
  
• New Failure module, TestFlightFailure_ResourceLeak, leaks a named or random resource in the part. Initial leak amount, and amount per second can be configured in config node
  
• Changes to ITestFlightFailure interface to allow mod authors of Failure modules more flexibility
  
• Added Tooltips to UI part status that indicate repair requirements for part
  
• Don't poll any parts on a new vessel until at least 10 seconds after mission start
  
• Made MasterStatusDisplay GUI window a bit wider
  
• Documentation updates
  
• Fix to bug preventing REPAIR{} nodes from loading and persisting properly
  
• Updated all existing failure modules to use new failure API interface
  
• Removed old TestFlightFailure_LiquidFuelLeak Failure module. Use the new TestFlightFailure_ResourceLeak instead
  
• All new configs to apply TestFlight to Stock Engines and Fuel Tanks. Many thanks to JeffreyCor for the help here.
  
• Changed calculation for base reliability based on flight data. Now uses only one variable, reliabilityMultiplier with higher numbers being easier to obtain 100% reliability and lower numbers being easier. Value of 1 puts 100% reliability at 10,000 units of flight data
  
• Implemented repair systems, as well as initial repair requirements. Note that some repairs have a part they require for repair. These parts can't be made, yet, but are not required. They are optional and give a repair bonus if present. These parts will soon be available through my companion AddOn MaterialPrinter.
  

How You Can Help

At this stage while TestFlight is under early development, there are three important ways that other KSP players can help out.

1. Ideas - If you have any ideas for Failure Types or Recorder/Reliability modules please post! By fleshing out the system now with different modules, we can help ensure the API is solid for the future.
   
2. Build Modules. If you are a KSP Modder, I would love some extra help in writing modules for the system. They are very easy to make. If you are interested in helping please let me know.
   
3. Make ModuleManager Configs. If you know how to make config files, I could sure use some help adding configs for both Stock/Stock-alike parts, and Realism Overhaul. Please let me know!
   
4. Testing. As alpha releases are made, you can help out by testing them out and reporting any bugs via the Issue Tracker on GitHub, and any feedback, questions, or discussion on how the system works here in this thread is most welcome.
   

FlightData

The more you fly parts, the more flight data they record, which in turns improves reliability. As stated, this is persistent. That means that if you launch a "Super Rocket 1" with a Mainsail engine, that mainsail generates let's say 10,000 units of flight data during the flight. Now you go back to the VAB and build "Super Rocket 2" which also uses the mainsail. That instance of the mainsail starts with 10,000 units of flight data already from the earlier flight. That means test flights make your rockets more reliable!

Reliability and Failures

TestFlight calculates reliability using the flight data recorded by a part, based on various criteria and mathematical equations. It will periodically make failure checks against this data and if a part is determined to have failed, it will generate failure events.

TestFlight is Configurable and Extensible

The real beauty of TestFlight is that it is extremely configurable by the end user, or by mod authors who want to integrate it into their own mods. TestFlight works on a series of pluggable PartModules that give full control over how reliability is calculated and what failures are possible - all on a per part basis if desired. Further more, all of the various options can be configured by the user or mod authors, such as minimum and maximum reliability ratings, failure rates, repair costs, and more.

On top of all that, TestFlight comes with a public API that allows mod authors to further extend the system! Mod authors can easily add additional failure types, or change how flight data is recorded, or how reliability is calculated. The API for example could allow a mod author to easily extend the system to penalize the reliability of individual parts as they get old.

TestFlight - A First Look

Master Status Display

Introducing the TestFlight Master Status Display (MSD). The MSD is how you track the status of TestFlight enabled parts on your vessel. It lets you see, at a glance, the status of all the parts, their accumulated flight data, and any failures that have occurred.

Javascript is disabled. View full album

Documentation for Modders

If you are a KSP modder interested in extending the TestFlight system to add your own Recorder, Reliability, or Failure modules, please see the additional documentation here: http://forum.kerbalspaceprogram.com/threads/88187-WIP-TestFlight-v0-1-0-Alpha-A-configurable-extensible-parts-research-eliability-system?p=1635489&viewfull=1#post1635489

Sample Part Config

Moved sample config into Modder documentation to save space in main post.

Edited February 5, 2015 by Agathorn

[BrutalRIP]

Sounds interesting will keep my eye on this

[Rakird]

Hello Aghatorn, it will be similar to what the RPL project used on the probes right? I'll follow it closely i loved that data gathering concept, a shame RPL isn't working anymore. Ty and regards.

[Agathorn]

Hello Aghatorn, it will be similar to what the RPL project used on the probes right? I'll follow it closely i loved that data gathering concept, a shame RPL isn't working anymore. Ty and regards.

Eh kind of but not really. I do agree with you though that the way that works is very nice. Actually I have a second project not yet announced that expands upon the Custom Science stuff specifically so you might want to keep an eye out for that

[Rakird]

Alright, let me know when it comes out , if it's tech tree related even better! Btw do you know something similar to RPL? I can't find any progression mod that starts with unmanned flight and then keeps evolving. Ty and Regards!

[Agathorn]

Well you are making me speak ahead of myself, but yes. These AddOns i'm working on now are to build the foundation for an RSS/RO based career campaign that I will be making. RPL is great, and I love it, it just doesn't quite jive with the direction I would prefer to go. So I am working on my own version thereof. TestFlight and the later upcoming tentatively named S.A.M., Scientific Analysis Module, form the core of my future campaign.

[BrutalRIP]

any news on this?

[NathanKell]

Yeah, just want to chime in regarding how awesome this is

[Agathorn]

any news on this?

Patience please I'm on vacation right now so unable to get a lot of work done. The core is in place, but i've been having some turmoil figuring out how to handle the API.

Yeah, just want to chime in regarding how awesome this is

Thanks Nathan!

[Agathorn]

Hey all,

So I just got back from vacation on Sunday. I'm a little distracted because my Oculus Rift came in while I was gone, and its just amazing, especially in Elite: Dangerous

Anyway I did get some time to start digging back into this yesterday. Its always a bit slow getting back into code after a noticeable time away, but I managed to get the API basics working, and tested and it all looks good. Next up I need to get the persistence between instances of parts working, but that shouldn't be too hard. I already did some tests before I started, and i've got the groundwork in place.

[Joeythompson31]

Please tell me this isn't dead!? This seems like it would be great.

[Agathorn]

Not dead exactly, but unfortunately the work I do comes in spurts where sometimes I am not doing a thing and other times I am working my arse off. Unfortunately the last few months have been the latter, so I just haven't had the time to work on this. It is still on my plate for my next bout of free time. Sorry

[Agathorn]

So one thing I didn't consider in my previous reply, is the holidays are coming up and I will have free time between Christmas and New Years, so while I make no promises, I might be able to get some good progress over the break. The main hindrance besides KSP updates since Aug, is the old adage that code you haven't looked at in months might as well have been written by someone else

That said I just took a look at my code -- and thank for for OSX's Time Machine backups because apparently my GitHub wasn't up to date -- and it turns out I was a lot closer to an initial release than I thought. So I will try real hard to get something out here over the break, though it will be only an initial incomplete alpha.

[JeffreyCor]

This is a very interesting idea. I hope it goes well, I'm looking forward to giving it a try when it is ready

[Agathorn]

Hey all,

I have gotten the base persistence system in place and working excellent. I've made a very short, extremely boring, video showing the system in action.

[lextacy]

hi, im looking forward to this as the direction seems to be amazing compared to the Dangit mod. Im hoping as you say "user" configurable that we have control in what parts this can be applied to? For example I install this mod and want to test some new engines that will go on a new launch vehicle (Falcon 9 Heavy) BUT I have a long standing list of launchers that have made many flights. I would not want the reliability tests/failures to apply to these due to them being past successful launchers (pre-testFlight). Is this the type of user configurability we are talking here?

[Agathorn]

hi, im looking forward to this as the direction seems to be amazing compared to the Dangit mod. Im hoping as you say "user" configurable that we have control in what parts this can be applied to? For example I install this mod and want to test some new engines that will go on a new launch vehicle (Falcon 9 Heavy) BUT I have a long standing list of launchers that have made many flights. I would not want the reliability tests/failures to apply to these due to them being past successful launchers (pre-testFlight). Is this the type of user configurability we are talking here?

You have full control over what parts they go on to, and the system is configurable on a per part basis. That said, let me be clear. The entire point is it is persistent per part type not part instance. So if you apply the TestFlight modules to say the Merlin engine (this is done using ModuleManager) then it applies to **all** Merlin Engines. So if some had already flown then those would be affected as well.

If your concern is over applying it to parts that have already flown, then I can keep that in mind and see if there is a way to somehow exclude "pre-TestFlight instances".

[Kibble]

I absolutely love this mod you are making - in fact I have been actively looking for something like it. Reliability of hardware heritage is extremely important to human spaceflight (and human technology in general, I guess). I just hope there are failure modes for all kinds of different parts not just engines and stuff! <3

[Agathorn]

I just hope there are failure modes for all kinds of different parts not just engines and stuff! <3

The beauty of the system is that it itself is moddable. That means that if there isn't a failure type you want, you or another modder could add it to the TestFlight system.

That said I do want to bundle in a decent amount of typical reliability and failure systems, and I am open to suggestions.

[Agathorn]

I am looking to get some user input on the Failure system if anyone wants to chime in.

The way the failure system works is in two stages.

Stage 1: In stage 1, the TestFlight system polls all the TestFlight_Reliability modules on the part to generate a total chance of the part failing during that poll. The part can have one or more TestFlight_Reliability modules attached, which all contribute to the total reliability of the part. A roll is then made against that reliability to determine if a failure occurs during this poll. If no failure occurs, then the system does nothing until the next polling cycle. If a failure is determined to have occurred, then we move on to stage 2.

Stage 2: In stage 2, we know the part has failed. Now we need to determine how it fails. Does it explode? Does it lose power/thrust? Does it freeze up until restarted? Whatever effects are determined by the TestFlight_Failure modules on the part. There can be one or more of these modules on the part, each defining a specific failure, and how it might be repaired. The question though is how to determine which of the available failures to trigger? And this is where I am looking for input.

Off the top of my head I can see three obvious ways to determine which failure module to use, but I am certainly open to other ways of doing this if anyone has some.

Option 1: Each failure module has a defined chance to occur, and we simply work down the list rolling against each until one fails. The big problem here is that the order the modules are defined would be important, and given that you might have multiple third party mods adding failures, the order can't be guaranteed, which would be bad.

Option 2: All the failure modules on the part are combined into a ToFail table and then a single roll is made against that table. For example if five failure modules are defined, and each is defined as having a 20% chance to fail, then the system would build a table where a roll of 1-19 triggers module A, 20-39 triggers module B, etc. If the total percent chance to fail of all mods is less than or greater than 100%, they would be normalized to 100% when the ToFail table is built. The biggest downside here is that as more failure modules get added by mods, the relative chance for a specific failure to occur starts to drop. A mod author may have intended Failure A to happen roughly 50% of the time, but now the user has installed a bunch of mods that adds 10 more failures to the part, and suddenly that 50% chance for Failure A is more like 5%.

Option 3: A bit of both. Each failure module has two things defined. A Severity and a Probability. Two rolls can be done, one to determine the severity of the failure, then a second against the failure mods of that severity. The second roll could be done using either Option 1 or 2. If done using option 2, then while the same inherent issue with #2 applies, the relative lessening of the probability is reduced somewhat by the spreading out of failures in the roll.

Right now I am leaning towards option #3, but I haven't decided. Frankly I am not happy with any of them and feel there must be a better way, one that scales well with more failures added by additional mods.

EDIT: I might be over complicating this. I need to sit down and follow the logic, but I think I might have a solution. If you take a single random number in a range, say 1 to 100, the odds of any number are equal. It isn't until you start combining rolls and modifiers and what not that distribution becomes uneven. So through out the whole % tables and what not. Why not simply have the failure module define a "weight". Then add up all the weights to a total, and roll 1 to total. If one module has a weight of 10 and another as 20, then the second module would be twice as likely to get picked over the first. I like it, it feels right, but I haven't dug into it yet to make sure it is solid.

Edited December 24, 2014 by Agathorn

[JeffreyCor]

The option 4 under the edit sounds the most logical as it give something with a higher chance of failure a higher chance of failing, which conversely translates into a greater reliability for a part really becoming more reliable.

[Agathorn]

The option 4 under the edit sounds the most logical as it give something with a higher chance of failure a higher chance of failing, which conversely translates into a greater reliability for a part really becoming more reliable.

Yeah the more I think about it I agree, and that is the route I am currently taking unless someone chimes in with something better. I hindsight I don't know where I Was going with the first 3 options, as they really don't work at all in an expanding system.

[lextacy]

i agree with option "edit" A weighted average could go like this :

-Weight is based on part reliability. So a lets say the Mainsail has low reliability rating and the Poodle has a high rating. The Mainsail could have a 2% chance of major failure and 1 % of minor failure. The Poodle can then have a 1% change of major and .5% of minor.

[Agathorn]

So just an update on this. Good news is pretty much all of the core TestFlight system is in place and working(ish). Right now I am working on smoothing out as many rough edges as I can and starting on the GUI. I am aiming to get an initial Alpha release soon, but I want to stress that initial releases will be aimed more towards 1) Mod developers who are curious to play with the system and 2) Technically inclined players who are curious to play with the system. The first releases will be very rough, and will only have a few built in modules initially.

[Agathorn]

Getting ready to push out a release. The OP has been edited to show a sample part config to define all the pieces of TestFlight on a part.