Loss of ASTRO-H Satellite

[PB666]

31 minutes ago, Frida Space said:

True, but it was the computer who made the mess. At the time of the anomaly, the satellite wasn't communicating with Earth, so the only thing they could have done to prevent the anomaly (given what we know so far) is have uploaded correct software for the IRU and the thrusters and activated the star trackers.

Like I said, I would hate to have been a software engineer on that project. Failsafe checkstatus was not properly programmed. It may be true that the mission could not have succeeded with that failed gyro, anyway satellite is still in one piece, the could have deorbited it. And of course they could have had 3 gyro. and more inertial sensors.

 

[Shpaget]

With the 20/20 hindsight, it's always easy to find ways to fix stuff and even prevent bad things, but if you are in a position where at one moment you have a full control over the satellite and the next it's not responding to your commands, but doing its own maneuvers that are pushing it deeper in the doodoo, then you can just sit back and watch it spin itself to pieces.

Yes, the satellite should not have made attitude corrections without consulting multiple attitude sensors. That is Redundancy 101. However, the situation appears to be a chain of minor issues that just happened to occur at the same time, making the situation unrecoverable. If you've ever watched any "Seconds from disaster" episode or a similar documentary, you'll see striking similarities. Almost every aircraft/ship/train accident in history was a result of multiple relatively minor issues each of which could have been easily fixed and none of which would have been sufficient to cause the accident on its own. Only in combination of all of them did they lead to disaster.

Finally, as they say, every cloud has a silver lining. While the damage is not insignificant, it is mostly just financial - luckily no one was injured or got killed. A new satellite can be launched, new algorithms for redundancy will be written, new procedures put in place. Science is not done just by succeeding. Failing is just as important. This time around, it's computer science that is getting a level up.

[Frida Space]

1 hour ago, Shpaget said:

With the 20/20 hindsight, it's always easy to find ways to fix stuff and even prevent bad things, but if you are in a position where at one moment you have a full control over the satellite and the next it's not responding to your commands, but doing its own maneuvers that are pushing it deeper in the doodoo, then you can just sit back and watch it spin itself to pieces.

Yes, the satellite should not have made attitude corrections without consulting multiple attitude sensors. That is Redundancy 101. However, the situation appears to be a chain of minor issues that just happened to occur at the same time, making the situation unrecoverable. If you've ever watched any "Seconds from disaster" episode or a similar documentary, you'll see striking similarities. Almost every aircraft/ship/train accident in history was a result of multiple relatively minor issues each of which could have been easily fixed and none of which would have been sufficient to cause the accident on its own. Only in combination of all of them did they lead to disaster.

Finally, as they say, every cloud has a silver lining. While the damage is not insignificant, it is mostly just financial - luckily no one was injured or got killed. A new satellite can be launched, new algorithms for redundancy will be written, new procedures put in place. Science is not done just by succeeding. Failing is just as important. This time around, it's computer science that is getting a level up.

It absolutely was a chain of events that took the satellite down. It was a catastrophe of programming errors really. Obviously it's easy to say in hindsight what should have been done, but I'm quite surprised by some of the minor problems that led to the failure -- e.g., how can you miscalculate the centre of mass by so much? But obviously even NASA can mix up metric and imperial values and crash a probe in the Martian atmosphere, so anything's possible really. That just goes to prove how hard rocket science is. But it's in a certain sense easier to accept a failure when it is caused by something almost 100% unpredictable, which is not the case with this satellite.

True, failure is sometimes necessary to succeed, but it's not as easy as you say, "a new satellite can be launched". For example, this is like the sixth time the SXS X-Ray instrument either fails in orbit, explodes, dives into the ocean following a launch mishap or doesn't even make it onto a satellite because of financial problems. Japan has been trying to launch that instrument for a decade now without success. Failure is sometimes useful, but so many failures probably aren't.

Astro-H was indeed the replacement for Astro-E, which dived head first into the Pacific. Nevertheless, I hope Japan decides to launch a "replacement for the replacement" -- its scientific potential is very very high and I couldn't wait to see its first science results. Good luck JAXA!

Edited April 18, 2016 by Frida Space

[magnemoe]

1 hour ago, Frida Space said:

It absolutely was a chain of events that took the satellite down. It was a catastrophe of programming errors really. Obviously it's easy to say in hindsight what should have been done, but I'm quite surprised by some of the minor problems that led to the failure -- e.g., how can you miscalculate the centre of mass by so much? But obviously even NASA can mix up metric and imperial values and crash a probe in the Martian atmosphere, so anything's possible really. That just goes to prove how hard rocket science is. But it's in a certain sense easier to accept a failure when it is caused by something almost 100% unpredictable, which is not the case with this satellite.

No margin of error is hard, it might be an good idea to require base jumping or similar from the management, an useful way to remind them that its no margin of error in some tasks the hard way, its also fun and it make them earn their bonuses and few would complain

And yes the no marin of error is hard, work with this in the oil industry where its an billion dollar business, environmental damage, loss of platforms and so on. 
Gone too then the safety fanatics demand offshore rules in the offices   
 

[PB666]

56 minutes ago, magnemoe said:

No margin of error is hard, it might be an good idea to require base jumping or similar from the management, an useful way to remind them that its no margin of error in some tasks the hard way, its also fun and it make them earn their bonuses and few would complain

And yes the no marin of error is hard, work with this in the oil industry where its an billion dollar business, environmental damage, loss of platforms and so on. 
Gone too then the safety fanatics demand offshore rules in the offices   
 

Also raise there life insurance rates.

[Camacha]

17 hours ago, PB666 said:

If you walk into a mine field, the best thing is not to start jumping around to find out where the mines are. Turn around and step your previous steps backwards until you are out.

That would normally be simple enough, though in this case it is dark, you hit your leg and you are drunk. Good luck getting out. In hindsight it was easy - bring a torch and do not drink. A disaster is always an addition of very preventable and small, but critical errors.

[Glaran K'erman]

18 hours ago, Frida Space said:

e.g., how can you miscalculate the centre of mass by so much?

 

20 hours ago, Shpaget said:

That is Redundancy 101.

Hence my gut-punch comment. I get space is hard, we all get space is hard - which is why they should know to double-check, triple-check, then quadruple-check. While definitely good lessons to learn, a true 'level up' for computer science would have involved the failure of multiple redundancy where some edge case(s) were exposed that lead to increased resiliency of the system going forward. However, in terms of redundancy, this just seems like a "lack thereof" scenario.

[PB666]

Its a goner.

http://www.bbc.com/news/world-asia-36167064

 

[Gojira1000]

http://www.geek.com/science/software-update-causes-286-million-japanese-satellite-to-break-apart-in-orbit-1654675/

Talk about life imitating art

[lukethecoder64]

An intern probably accidentally clicked the "Your free Windows 10 upgrade is ready!" popup.

Edited May 11, 2016 by lukethecoder64
w -> W

[Theysen]

6 minutes ago, lukethecoder64 said:

An intern probably accidentally clicked the "Your free windows 10 upgrade is ready!" popup.

LOL. Made my day.

[Just Jim]

Oh wow, that is so sad.... but also such a Kerbal thing to have happen!!!  

Are they sure one of the technicians wasn't playing KSP at the time???   

[PB666]

this thread is duplicate material we have already discussed.

[Vanamonde]

A pair of threads on the same story have been merged.