For Want Of A Check Valve

My piece on the SpaceX abort is up at Popular Mechanics.

Here’s the bit that got left on the cutting-room floor:

But even as the incident validated the safety of the vehicle, it raises issues about its launch reliability (that is, the ability to launch on schedule). Every rocket design is a compromise of cost, safety and reliability. In the case of the Falcon, it has nine Merlin engines in the first stage because this allows it to use the same engine in both that stage and the upper stage, which only has one (with a larger nozzle for vacuum operation). This saved a great deal in development costs, and provides economies of scale in manufacturing, with a steady production of them 24/7 in the company’s factory in Hawthorne. It also provides a more forgiving design, allowing engines out on ascent, while also allowing the functional capability to “deep throttle” the stage by selectively shutting down engines to maintain gee limits for crew.

But nine engines also means nine times the things that can go wrong and prevent a launch. In fact it’s worse than that. It actually increases the unreliability exponentially. For instance, if the probability of an event like Saturday’s for one engine is one in a thousand, the probability of it not happening on any one of the engines is 0.999 to the ninth power, or .991, which means that there’s about a one in a hundred chance of an abort. If it’s only one in a hundred, that means that there will be an abort every tenth flight or so. And that’s just for Falcon 9. Falcon Heavy will have twenty-seven first-stage engines, which means a probability of abort of almost three in a hundred for a one in a thousand single-engine reliability, and a probability of one in four for one in a hundred.

The company doesn’t have enough experience with this vehicle to know what its true reliability is, but if they continue to have pad aborts, they may decide that they’d like to get bigger, and fewer engines.

But it also raises the issue of the value of a flight-readiness firing (FRF), as SpaceX performed a few days ago, with a hold-down test of the first-stage engines on the launch pad for a few seconds a few days ago, in preparation for this launch. The Space Shuttle also did this each time before the maiden launch of an orbiter, to ensure that all systems were ready to go before the first flight, but it was a reusable vehicle. Max Hunter, developer of the Thor that evolved into the Delta in the sixties, used to say that FRFs caused more problems than they solved, because a clean vehicle from the factory could be damaged or worn in the process, making it less reliable for actual flight. The valve seemed to have worked all right in SpaceX’s FRF, and it’s unclear (though SpaceX may know) whether or not the failure was a result of the FRF (I would bet they’re already reviewing the data to see if they saw any anomalies toward the end of the test, if they haven’t already).

But unlike the Delta, either ancient or modern, the company has a goal of full reusability for the vehicle, including the engines, so it may make sense to do FRF, at least once they start to refly, when they won’t have to do it for every flight. But the incident has no doubt given Mr. Musk and his team quite a bit to think about.

I will be curious to hear what they think the cause of the valve failure was.

32 thoughts on “For Want Of A Check Valve”

  1. A good explanation. The Russians have handled multiple engines for years and might be able to add something to add on that front.

    On another note: I have a question you may be able to answer Rand. I have read about the one second launch window many times. Can this be true? The launch must take place on the second or abort till a new window is available? What about minor throttling in flight? This seems far to exacting for a practical launch time.

    Unless I am missing something a one second window makes no sense. A minute I could go along with though it is still short. What gives?

    1. I seem to recall Ariane 5 launches that had an instantaneous launch window. I’m not sure if I’m misremembering them or not.

    2. Gwen Shotwell answered the ‘instantaneous window’ question in the press conference before the Saturday morning launch attempt. It’s not *actually* an instantaneous window, she said, “like there would be for a planetary mission”, but instead it costs “something like several kilograms [of dragon propellant] per second” delays. At that rate, the dragon doesn’t have enough propellent reserve to delay long. In theory, they should probably handle something like a 30-second delay — but in practice, SpaceX always takes longer than that to recycle, so the window is effectively instantaneous.

      1. Thanks Scott, that makes sense. They could delay the launch for a few seconds but once the engines start there is no time to abort and do a restart and still launch in the same window.

  2. I object to your sentence “In fact it’s worse than that. It actually increases the unreliability exponentially.”

    While literally correct, this appears to imply that an exponential increase is worse — probably far worse — than a linear increase.

    In fact 1 – (1-x)^y is LESS than x*y, no matter whether x is 0.01 or 0.001, or whether y is 9 or 27. Not much less, but less.

    For example with 27 engines with a one in a hundred chance of failure, multiplication would suggest a 27% chance of at least one failing while exponentiation gives “only” a 23.8% chance.

    Also note that as you have more engines the effect of any one engine failing is less important, as long as it doesn’t take others out with it.

    As re-use becomes the norm and reliability increases maintainence costs will dominate and the number of engines will decrease, just as the 747 has four engines but the 777 has only two.

    1. Bruce, you’re confusing the probability of on-pad abort (and associated launch reliability) with probability of launch failure. I was discussing the former, under current SpaceX guidelines.

  3. A very well written dissertation on the issues involved, it is a shame it got left “on the cutting-room floor”.

    I would point out one additional fact. While two launches and one launch attempt is far too small a sample to do statistical analysis, there have been two similar problems (detected over pressure conditions in engine number 5) in those three occasions. With 30 engines involved that is an actual (not statistical) problem occurring in 1 in 15 engines.

      1. If you say something I agree with I will note it. If you say something I disagree with I will note that as well.

        It is a shame that you cannot take a complement without making a point of contention out of it.

          1. On the other hand saying “Bullshit” to someone or calling them a “douche” is the height of proper etiquette.

            Good night.

          2. That is always an appropriate response to either bullshit or insults.

            Sorry you don’t seem to be able to understand the distinctions.

            Why don’t you go away and try to understand the difference between intelligent discourse and trolling before you troll here any more? You do seem occasionally (but only that) of doing so. You actually even seemed to manage it, albeit momentarily, in this very thread. Perhaps you could improve.

          3. Guess what, joey, it’s ok to be a douche once and a while when you have a long published history of being enthusiastic, expressing that enthusiasm and occasionally being right about the subject. How many decades do you have? How many publications do you have? How many rockets have you blown up on the pad?

    1. Flight #1 of the Falcon 9 aborted with “PC pressure high” on engine #5 according to Shotwell at the post-abort briefing Saturday. They recycled the countdown and launched that rocket without needing to replace anything.

      Though it was the same engine position, it didn’t appear to be the same condition.

      SpaceX has started talking about Falcon 9 v1.1 (after flight #5), which is rumored to have a different engine layout, so I wouldn’t think they are going to put too much effort into statistical analysis by engine position. Also, Falcon 9 v1.1 will be using the new Merlin 1D engines, which may or may not have a high commonality with the 1C version on the pad today.

      The Merlin 1D is supposed to be more robust, since that is the engine that will be used on the reusable version of the Falcon 9. However like any new version of a product, there could be new problems that creep up.

      For those that think this type of stuff is drama (and enjoy the drama), there is more drama to come…

  4. The Russians have several combustion chambers but often they share the same pumps (Glushko did it for the R-7 and Energia booster engines). When they tried otherwise (N1) they had a lot of problems with plumbing shoddy quality control and whatnot. If this valve was in the pumps like Rand said, a similar design would have like one quarter of the number of these kinds of valves (only one set of pumps for each 4 chambers).

    1. From the books I have read, the real culprit with the N1 was that they didn’t have the budget. They couldn’t afford to do test firings on all engines, for instance, so they did unit testing on a couple from each batch. That turned out to be not nearly good enough.

  5. I would think the criteria changes as the number of engines gets very large because you would likely shift from a standard of all engines firing to a certain percentage of engines firing or allowable available thrust. With one engine out on a Falcon 9, 88.9% of full thrust is available, and with one out on a Falcon Heavy, you’d still have 96.3% of full thrust.

    For a hypothetical Falcon 100, given 99.9% engine reliability, you’ll have 99% of full thrust (one engine out) 0.99% of the time, 98% of full thrust (two engines out) 0.0046% of the time, and 97% of full thrust 0.000016% of the time.

    If you designed such a rocket to normally run the engines at 98% of maximum thrust, a double engine failure wouldn’t even cause it to deviate from nomimal operations, and a triple failure would still give you 99% of design thrust, well within any rational mission margin.

    1. I was happy to realize that 8x Merlin 1D will produce more thrust than 9x Merlin 1C. Although there are other factors, this alone should ensure engine out performance from the moment of release. No weasel wording required.

    2. ^^^ Oops! Ignore my above statistics. I had an extra 9 on my percentage of success per engine (99.99% instead of 99.9%), which threw everything else way off. For 99.9% and 100 engines, all 100 fire 90.48% of the time, only 99 fire 9.06% of the time, 98 fire 0.45% of the time, and 97 fire 0.014% of the time, which is unlikely to occur during the life of the launch system (50% probability of occuring in 3,400 launches).

      This type of thing becomes interesting once you have a reusable launch system, since the engine life will in likelihood dramatically decrease the reliability unless you’re doing very expensive tear-downs between flights, as was done with the Shuttle.

      It would be cheaper to do only trivial maintenance and run the engines to the end of their useful lives, say 100 launches each. In that case the engine reliability drops to 99% (for engines of random ages) and the statistics for a 100 engine rocket breaks down as:

      36.6% of the time all engines fire (100% thrust)
      36.9% of the time, 99 out of 100
      18.5% of the time, 98 out of 100
      6.1% of the time, 97 out of 100
      1.5% of the time, 96 out of 100
      0.29% of the time, 95 out of 100
      0.046% of the time, 94 out of 100 (50% probability in 1000 launches)
      0.006% of the time, 93 out of 100 (50% probability in 8000 launches)

        1. Yeah, and the last couple of shuttle flights were delayed at least 6 months. That was just those 2, which had the advantage of learning from a hundred previous flights to work out the kinks.

          Congratulations SpaceX!

  6. Flight Readiness Firings. Takes me back to my days in USAF, trying to keep F106 fighters flying. The 106 was a delicate beast with the enormous MA-1 vacuum tube firecontrol system. Ground aborts for MA-1 failure were pretty common. To reduce the ground abort rate, the pilots wanted us to go out, and power up each aircraft a few hours before takeoff time, just to make sure MA-1 was working. We in maintenance were appalled. MA-1 broke mostly when powering up all those vacuum tubes. Subjecting them to an extra power up and power down cycle would just break more of them. An aircraft was just as likely to have MA-1 break on powerup whether it had powered up successfully an hour before. So we didn’t power up MA-1 just to see if it was working.
    On the other hand, after heavy maintenance, say an engine swap, we ALWAYS ran the engine up to full power on the ground, just to make SURE all the electrical connectors were in fact connected, the plumbing was not leaking under pressure, and no one had broken a wire tramping around inside the empty engine bay. So a Flight Readiness Firing on new hardware ( the hardware is always new if it’s a throw away booster) once, before takeoff sounds like a good idea. Realistic, much more so than checking over QC paperwork.

  7. Rand,
    This sentence:
    “The choice of nine engines for the first stage was made with reliability in mind: From the moment of liftoff, Falcon 9 can suffer an engine shutdown and keep flying; after about 90 seconds, it can tolerate a second engine shutdown.”
    appeared in:
    “Is SpaceX changing the rocket equation?”
    By Andrew Chaikin
    Air & Space magazine, January 2012
    http://www.airspacemag.com/space-exploration/Visionary-Launchers-Employees.html

    I have read that the one-second launch window is due to not having enough fuel to make up for an off-time launch. I suspect that is due to the many maneuvers scheduled in this test flight — they are trying to test so many things. A small fuel cushion might let them work with an engine failure. Besides, who wants to risk that on a test flight at this critical point in the program?

  8. My swiss cheese brain came through for me. Thanks for finding that link. I know I’d read that in another article and it didn’t have to wait for the Merlin 1D to be true. There’s no ambiguity in ‘from the moment of liftoff.”

Comments are closed.