[1.8.x] Oh Scrap!- A ScrapYard based Part Failure and Reliability Mod 2.0.1 (07/12/2019)

[severedsolo]

1 hour ago, Tyko said:

I re-downloaded...Now the engineer can repair the parts, but several parts showed 100% chance to repair and the repairs failed. Also, the glow is back for some parts, but other damaged parts still don't glow. 

Good catch.

Hotfix for Beta 9 (0.9.1) Released

• Fixed issue where displayChance updating was being skipped before the figures are passed to EditorWarnings on reloading a craft with existing failures (thanks Tyko)
• Fixed issue where part highlight would default to off on reloading a craft with Failures. (thanks Tyko)
• Further refinement to Reliability Model. Failures will now occur up to a maximum of 6 hours after craft is loaded, but will depend on how reliable the part is (unreliable parts will fail sooner) - Engines are excluded from this change.

 

Note that it probably won't fix vessels which have already encountered this bug, but any new builds will be fixed.

 

Edited July 23, 2017 by severedsolo

[Cheesecake]

Quote

Failures will now occur up to a maximum of 6 hours after craft is loaded

Does this mean that after 6 hours after launch no failures will occur, only engines can fail? That`s a pity because I like the possibility to have a fail after a long turn.

[Tyko]

@severedsolo this is progressing nicely. I like the new dialog box. The new model feels odd though. It seems like you're really focused on modeling re-usable parts, but most parts aren't actually re-used and the new model seems to start parts off with a "new part" failure rate even if I've used the same type of part before. The 50th Swivel engine I build should not have the same initial failure rate as the very first I ever built. I thought your old model for new versions of the same parts - in which subsequent builds became more reliable - felt more correct.

[severedsolo]

32 minutes ago, Cheesecake said:

Does this mean that after 6 hours after launch no failures will occur, only engines can fail? That`s a pity because I like the possibility to have a fail after a long turn.

Under the old system, part failures would occur a maximum of 30 minutes after Start() was called (essentially, when you launch, when you switch back to the craft, or it comes into physics range). Engines shortened that to two minutes so they were more likely to fail when you launched.

The idea was that you would see if you had a problem before the craft left orbit. Good idea in theory. What was actually happening though was that Unity's Random Number generator was rolling failures waaay more than it should have done. (on a craft with all <20% failure rates, I would expect the norm to be no failures, but what I am actually seeing is failures happening on every launch. I extended it to 6 hours so that if you switch away before that 6 hour window happens, the failure won't happen (essentially it is a band aid to stop constant failures).

Having said all that, I am currently experimenting with a new way of rolling the random numbers which seems to be working a whole lot better and is giving results that I would expect. Assuming it pans out it will be included in the next release. (I am testing in my own game at the moment, because you don't really get a feel for these things without actually playing it, rather than using a testing environemt)

24 minutes ago, Tyko said:

The 50th Swivel engine I build should not have the same initial failure rate as the very first I ever built. I thought your old model for new versions of the same parts - in which subsequent builds became more reliable - felt more correct.

The failure rates should generally still trend downwards, but it's a bit random, so if you get a good roll on Part 1 you may not necessarily notice. This is good feedback though, I will have to have a think about some changes, because I agree with you, I want parts your engineers are familiar with to be more reliable than ones where they aren't.

Just for reference, this is the way it currently works:

1) The first time a part is loaded into the editor or SY applies the inventory, if it's never been used UPFM will force SY to allocate it a new ID (to stop the issue where auto-loaded vessels don't get new IDs).

2) UPFM will check it's database to see if we've already allocated this ID a failure penalty. If we have, it will just make a note and wait for Step 3. Otherwise it will generate a random failure chance (this is: (Random number between 0+1/3)/number of times that part has been built before), it will then add that number to the database, so when you use the same part, it always gets the same penalty.

3) UPFM will do it's failure calculations, then add a further failure "penalty" based on the number we got from Step 2.

So, generally speaking, a part that has been built 10 times, will basically have an "additional failure penalty" of 0, of course there is a chance of a first time part getting a good roll and also coming back as 0, which is probably the bit I need to address.

Edited July 23, 2017 by severedsolo

[Cheesecake]

hm, my last craft, flying to Moon (I play Quarter Sized RSS), some parts failed after 10-30 days after launch. Using UPFM 0.8. I have tested 0.9 yesterday. My solar panels had a 82%-chance to fail. I launched this satellite 5 times. The Chance of fail decreases but there where no fails.

[Tyko]

I ran 10 identical launches from a new save. I captured the failure % (below) on the launch pad. Launched each rocket and let the engine burn for 1 minute. I then separated the pod, did a parachute descent and recovered the pod. No parts were re-used, so each rocket was entirely of unused parts. I can see the numbers decreasing and also the effects of the randomization.

	L1	L2	L3	L4	L5	L6	L7	L8	L9	L10
Parachute Failure	21	17	12	10	10	9	7	7	7	7
Capsule Battery	7	15	12	10	9	7	7	7	7	7
Capsule Reaction Wheel	12	20	17	15	14	12	12	12	12	12
Capsule Resource (monoprop)	9	18	15	12	11	10	10	9	9	9
FLT-800 Fuel Tank	19	23	18	15	14	17	11	10	10	9
Swivel Engine	38	15	14	12	12	11	10	10	10	10

 

I think failure chances should drop more rapidly though and I'm surprised they plateau'd so high. I think your previous failure model roughly halved the chance every launch and got down to 1%(ish). This felt more reasonable. Having a 10% failure rate after 10 launches seems really high.

As an interesting side note, I had zero parts failures over my 10 launches. Each flight lasted about 3 minutes with the first minute being the boost phase and the last 2 being descent. 

[severedsolo]

2 hours ago, Cheesecake said:

hm, my last craft, flying to Moon (I play Quarter Sized RSS), some parts failed after 10-30 days after launch

Start() is an internal Unity function, it may get called at other places too (I suspect for instance that it fires on a SOI change).

2 hours ago, Cheesecake said:

I have tested 0.9 yesterday. My solar panels had a 82%-chance to fail. I launched this satellite 5 times. The Chance of fail decreases but there where no fails.

Few possibilities here (in decreasing order of likeliness)

1) You grabbed the messed up release (it was up for about 12 hours before Tyko pointed it out to me) - at the point of part failure an exception would have been thrown, so the failure would never happen. Simplest fix for that is to redownload and reinstall (completely reinstall, delete the entire folder)

2) Your solar panels weren't the retractable kind. At that point UPFM would abort the failure, because there was nothing to fail.

3) REALLY good luck on the RNG.

1 hour ago, Tyko said:

I ran 10 identical launches from a new save. I captured the failure % (below) on the launch pad. Launched each rocket and let the engine burn for 1 minute. I then separated the pod, did a parachute descent and recovered the pod. No parts were re-used, so each rocket was entirely of unused parts. I can see the numbers decreasing and also the effects of the randomization.

That's good data, thanks!

I've got a few ideas about fixing the drop off, I think the old model dropped off too quickly, but I agree that is a bit silly.

Do me a favour, I've just put a pre-release up, see if you think that's better. What I've essentially done, is loosen up the randomisation (so first part failures will be a bit on the high side),  but the drop off seems to be quite sharp, and lowered the base rate, so it will plateau at 1%.

1 hour ago, Tyko said:

As an interesting side note, I had zero parts failures over my 10 launches. Each flight lasted about 3 minutes with the first minute being the boost phase and the last 2 being descent. 

Thats the 6 hour window. I really don't like that, and will be changing it back in the next release, assuming this new randomisation method works out.

 

Edited July 23, 2017 by severedsolo

[Tyko]

I'll do some more testing this morning

One thought about base failure chances. It seems like the two things that NASA has the hardest time with are electrical systems (batteries) and reactions wheels. In your model these are the 2 most reliable types of parts, but in the real world this was the opposite. The ISS, Hubble, Dawn, Kepler - have all had reaction wheel failures, just to name some big ticket projects. Saturn 1 had the biggest electrical failure, but there have been many others and "batteries" are really your indicator for the overall power system and wiring of a craft.

If possible you might want to have the failure window shorter for engines and control surfaces. During a boost most engines run on the average of 2 minutes or so before they're jettisoned (that's a WAG, not an exact number, but in most of my craft that use staging 2 minutes seems like the sweet spot)

Edited July 23, 2017 by Tyko

[severedsolo]

42 minutes ago, Tyko said:

 

If possible you might want to have the failure window shorter for engines and control surfaces. During a boost most engines run on the average of 2 minutes or so before they're jettisoned (that's a WAG, not an exact number, but in most of my craft that use staging 2 minutes seems like the sweet spot)

It's already two minutes for engines (for that very reason).

Regarding the base failure rates - I originally had "realistic" failure rates, based on actual NASA studies and documents. The results were... not fun.

[Tyko]

3 hours ago, severedsolo said:

Do me a favour, I've just put a pre-release up, see if you think that's better. What I've essentially done, is loosen up the randomisation (so first part failures will be a bit on the high side),  but the drop off seems to be quite sharp, and lowered the base rate, so it will plateau at 1%.

Thats the 6 hour window. I really don't like that, and will be changing it back in the next release, assuming this new randomisation method works out.

 

Here are the results of a 10 launch series again - completely new game to avoid any benefits from the previous 10 launches. I added a few more parts to get more data too. I also extended flight times out to 6-8 minutes

	L1	L2	L3	L4	L5	L6	L7	L8	L9	L10
Parachute	21	2	1	1	1	1	0	0	0	0
Probe Core Battery	16	10	7	5	4	3	3	3	2	2
Probe Core Reaction Wheel	16	10	7	5	4	3	3	3	2	2
Probe Core Tank (there is no tank though)	16	10	7	5	4	3	3	3	2	2
Capsule Battery	4	11	7	5	4	4	3	3	2	2
Capsule Reaction Wheel	4	11	7	5	4	4	3	3	2	2
Capsule Tank	4	11	7	5	4	4	3	3	2	2
FLT-200 Tank	3	15	10	7	6	5	4	4	3	3
FLT-800 Tank	16	5	4	3	2	2	2	1	1	1
Swivel Engine	19	0	0	0	0	0	0	0	0	0

NOTE: I had zero failures over the 10 tests - does physics warp prevent UPFM from registering failures because I was using physics warp x4 during many of the slow portions

Watching these numbers and thinking about it, I'm not sure making it all random and stretching out the number of tests before 1% is reached is actually more fun (it is a game after all). I don't think you want to turn this into Kerbal Engineering Program  

1. Randomizing a number that's already a % randomization is kind of unnecessary and makes your formulas and balancing more difficult
2. Giving players the ability to swap out a particularly bad part in the VAB is just adding clicks because who's not going to throw out a bad part? 
3. While you want to introduce a chance to fail and that failure chance should be higher for a new part, it should drop really quickly - Your 1 player is representing a whole team of engineers and QA people, making the player go through 10 launches to get a reliable part is just throwing the fun/grind balance off.
4. Consider this progression: 24%, 12%, 6%, 3%, 1.5%, 1%...stays at 1%  - (you could add a difficulty level function to set this higher or lower)
   1. Starting at 24% supports the idea that your hardware teams are doing a reasonably good job before delivering parts to you. Anything with a 25% + failure rate would have been screened out in bench testing before it made it to the pad
   2. Dropping by half each time means that by launch 5 or 6 you have a very reliable piece of hardware - consider how much your players will be using parts
      1. in a Career or Science game players will be upgrading to new parts fairly rapidly early on so it's good to see these parts become really reliable before they're dust-binned in favor of newer parts.
      2. In a Sandbox game players often already use a smaller portion of the available parts. Making it too time consuming to increase reliability of other parts could just further reduce the pool of parts a player is choosing from.

Edited July 23, 2017 by Tyko

[Tyko]

1 hour ago, severedsolo said:

It's already two minutes for engines (for that very reason).

Regarding the base failure rates - I originally had "realistic" failure rates, based on actual NASA studies and documents. The results were... not fun.

LOL, that would be interesting to see the actual NASA data. I've read about early Mercury and the before that...all the issues with boosters especially. Just my 2 cents, but from a fun/grind ratio perspective I think that it's safe to assume Rockomax or Jeb's Junkyard, etc are doing initial testing and only delivering relatively reliable parts .The chance for failure should be threatening enough to force players to build / plan for it, but not so high that I'm spending hours and hours running test flights before each actual mission.

Edited July 23, 2017 by Tyko

[Micro753]

On 7/23/2017 at 11:34 AM, severedsolo said:

Failures follow the bathtub curve 

 

On 7/23/2017 at 0:08 PM, Tyko said:

Giving players the ability to swap out a particularly bad part in the VAB is just adding clicks because who's not going to throw out a bad part?

 I have an idea. When a part is reused many times, the chance of failure increases, so add an option (either in the difficulty menu of a cfg) for what % chance or # of uses until a part is marked as a "bad part". When it is marked as a bad part, have ScrapYard delete it from the inventory as if it was thrown out. Players can set it higher to save money but have higher risks, or set it lower to spend more money but keep their Kerbals safer.

Edited August 6, 2017 by Micro753

[Tyko]

20 hours ago, Micro753 said:

 

 I have an idea. When a part is reused many times, the chance of failure increases, so add an option (either in the difficulty menu of a cfg) for what % chance or # of uses until a part is marked as a "bad part". When it is marked as a bad part, have ScrapYard delete it from the inventory as if it was thrown out. Players can set it higher to save money but have higher risks, or set it lower to spend more money but keep their Kerbals safer.

That works great for re-used parts and I thing that the latest release offers something like that I the right-click menu. 

In my case I wasn't re-using parts. There's also a randomizer applied to new parts (that have been built before)

[Jade_Falcon]

Has anyone seen how well this plays with Kerbalism? I use it in a KSP 1.2 game, but Ive been wanting to start a new 1.3 game now that KCT and Scrapyard seem to be progressing well, and this mod looks very interesting; but Im hooked on a lot of the Kerbalism gameplay mechanics so it would be a shame to not be able to run it.  

[Cheesecake]

I use it along Kerbalism. No problems yet.

[strudo76]

Is there some form of interface that lists all the parts on a craft, and their associated failure rates? I tried this mod a while back, but couldn't find anything like that, and mousing over every part to see the percentage got tedious. The reason I want this is because I'd like to test flight the parts in unmanned craft first, so I can have lower failure rates on the manned craft. Is this something that can be done and I've just missed it?

Thanks

[severedsolo]

50 minutes ago, strudo76 said:

Is there some form of interface that lists all the parts on a craft, and their associated failure rates? I tried this mod a while back, but couldn't find anything like that, and mousing over every part to see the percentage got tedious. The reason I want this is because I'd like to test flight the parts in unmanned craft first, so I can have lower failure rates on the manned craft. Is this something that can be done and I've just missed it?

Thanks

It's indirectly possible. If you go into the difficulty settings and change the "safety threshold" to 0, the EditorWarnings window will tell you the failure rate of each part.

I need to get back to working on this, as I am really not happy with the last update. I will see if I can knock something that shows it a little better together.

[Jade_Falcon]

On 8/13/2017 at 11:30 AM, Cheesecake said:

I use it along Kerbalism. No problems yet.

Thanks for the heads up.  I did end up loading a new 1.3 install after I saw your post: Kerbalism, KCT, ScrapYard, UPFM, and Monthly Budgets (with quite a few other mods, mostly parts and UI mods).  It all works great, hours of fun! This set of mods should be a stock option: Career, Sandbox, and KKSUM. 

Thanks @severedsolo for UPFM and Monthly Budgets! Keep up the good work!

[Bottle Rocketeer 500]

@severedsolo May I suggest the name "Jeb's Unlucky Day"?

[Jade_Falcon]

@severedsolo Are there any plans to show exactly whats wrong with the failed part in the UPFM Broken Parts List? I.E. If an engine has suffered a fuel leak, and the Engine Fuel Leak warning message has already been dismissed, it would be nice if the Broken Parts list reflected that to keep you from blowing up the engine before youve had a chance to attempt a repair.  With some of the other failure modes the right click menu gives you enough info to know whats wrong with the part (Gimbal failure, Underthrust), but nothing for engine fuel leaks.

[severedsolo]

37 minutes ago, Jade_Falcon said:

 Are there any plans to show exactly whats wrong with the failed part in the UPFM Broken Parts List? I.E. If an engine has suffered a fuel leak, and the Engine Fuel Leak warning message has already been dismissed, it would be nice if the Broken Parts list reflected that to keep you from blowing up the engine before youve had a chance to attempt a repair.  With some of the other failure modes the right click menu gives you enough info to know whats wrong with the part (Gimbal failure, Underthrust), but nothing for engine fuel leaks.

To be honest, I don't like the way fuel leaks are currently implemented anyway. You actually have zero chance of repairing it, even if you do catch it early, as the repair option doesn't appear until the failure actually happens (ie the engine explodes). I am going to either do away with that one completely or (preferably) write some special logic for that failure so you can repair it when it happens.

[Jade_Falcon]

18 hours ago, severedsolo said:

To be honest, I don't like the way fuel leaks are currently implemented anyway. You actually have zero chance of repairing it, even if you do catch it early, as the repair option doesn't appear until the failure actually happens (ie the engine explodes). I am going to either do away with that one completely or (preferably) write some special logic for that failure so you can repair it when it happens.

Well, I do like it as a failure mode because it can happen, so I'm not sure Id like to see it go away completely.  My vote would be that instead of it always resulting in a catastrophic failure, it could go either way; there would be a chance that the leak is detected and the engine shutdown to protect it (then you could repair it or work around it), or there is a leak severe enough that the engine is lost.  

Another way to go is that it would be shutdown, and if you tried to start it again, then it catches fire and explodes. Kind of like the engine fire detection on an aircraft:  You can leave it running, shut it down and see if it goes out, or just pull the fire handle.  Of course, leaving it running is at your own peril.  But either way, I dont mind that particular failure, its made for some interesting launches!  Sometimes the mission can still go on (or the craft can do an alternate objective), sometimes it ends in spectacular failure, it always ends up fun!

[linuxgurugamer]

On 5/21/2017 at 4:31 PM, severedsolo said:

Alright Mr Picky.

"Solid Fuel engines will no longer fail in UPFM because KSP doesn't model the failures very well"

I don't know if you've addressed this, but, you can have solid fuel engines fail in a number of ways:

• Reduced thrust
• Overthrust
• Exploding, either immediately or after a reduced/overthrust event

These three are fairly simple to address in KSP, feel free to pull the code from DangIt if you like.

[severedsolo]

On 28/08/2017 at 5:25 PM, linuxgurugamer said:

I don't know if you've addressed this, but, you can have solid fuel engines fail in a number of ways:

• Reduced thrust
• Overthrust
• Exploding, either immediately or after a reduced/overthrust event

These three are fairly simple to address in KSP, feel free to pull the code from DangIt if you like.

It was more "The engine failure module needs an overhaul, so rather than fixing a broken system I'll just re-write it at some point and disable solids for now" - but thank you, TBH this wouldn't have even happened without DangIt - I spent a couple of days just reading the code, figuring out how things are done.

On that note - apologies for the quietness lately, the last Windows 10 Insider build corrupted my OS, so I've had to re-install Windows, which means I lost my mod-dev setup. I'm re-installing Unity and VS now, but these things take time, even on a Fibre connection. This is why Microsoft don't recommend Insider builds on a production machine I guess.

Edited August 30, 2017 by severedsolo

[Pand5461]

Here's me again with some inappropriate suggestions.

Found a nice website listing spacecraft failures: http://claudelafleur.qc.ca/Scfam-failures.html

Looks like tank leakage is not a problem at all, at least not as big as tanks sometimes randomly exploding (Apollo 13, AMOS-6, Vostok-2M explosion at Plesetsk in 1980). And stock tanks are so ridiculously heavy already, it is easy to imagine they are shielded from any micrometeorites and the walls are thick enough to not crack from internal pressure.

Loss of pressurization gas can be an issue however (resource permalock?).

From what did fail or malfunction, engines certainly have the worst record, or just everything else doesn't have a chance to fail after engine explosion followed by RUD. There were loss of performance, emergency shutdowns, turbopump explosions and even some premature starts (Nedelin's catastrophe is a notable example).

Some ideas for more failures:

• A fairly common malfunction is attitude control failure but I have no idea how that can be implemented in-game. IRL it was usually wrong wiring so roll commands went to pitch thrusters or were executed in the opposite direction.
• Antennas fail quite a lot, either not deploying or just losing the signal (sudden drop of antenna range?).
• Reaction wheels are destined to fail eventually, so maybe set failure chance to 100% BUT introduce a minimal operation time (that increases each generation starting from a few hours and capping at maybe decade). Not sure how to deal with turning RWs on and off.
• Increase generation of every part returned from space / upper atmosphere (if it's possible to keep track of that). Cap the generation that can be achieved without returning parts (simulates limited capacity to test things in lab). Increase generation after completing part testing contracts.
• Increase base reliability of all parts each RnD / VAB / SPH upgrade
• Add decoupler failures - one mode is "failed to decouple", another "violent decoupling". The latter is a random damage to one of the attached parts (so a decoupler either damages a tank below it or an engine above it) and only for inline decouplers. Increasing decoupler strength in the right-click menu increases the chance of damage but also increases chances of successful separation, and vice versa.