You are on page 1of 39

I was a Lead Sneak Circuit Analyst for the Boeing Aerospace Company.

Sneak Circuit Analysis was developed by Boeing under NASA Contract No. NASW-1650 for
the Apollo Technical Integration & Evaluation effort following the Apollo launch pad test
fire event that killed astronauts Grissom, Chaffee, and White.

1
The Redstone was the first large liquid rocket developed in the US using German V-2
technology. Redstones later launched the first US satellite and the first American astronaut
into space.

The booster for this flight successfully fired, lifted a few centimeters off the pad, turned its
engine off, and settled back on the pad becoming a 28,440 kg, 21 m long, free-standing
bomb loaded with liquid oxygen and alcohol fuel!

2
The batteries were allowed to drain, then the fuel and oxidizer were carefully removed.
Then came the hard work of figuring out what went wrong.

3
There are literally 100s of schematics, wire lists, etc. for this system. Here is the greatly
simplified circuitry from the Redstone booster.

4
from the control center.

5
and from the cabling at the launch pad.
It should now be intuitively obvious to the most casual observer here what the problem
was. Right?
I promise you that you now have all the necessary information to explain this event.

6
Pun intended.

I want to use this talk to also reinforce the INCOSE SE Handbook required knowledge for
ASEP / CSEP certification and its development lifecycle processes.
Please forgive the labor associated with this very, very simple system I will try to expedite
going through many of the slides, but I wanted them available for your reference later.

7
For SEP certification, KNOW this!
In the interest of time, we want to focus in this box tonight, further limited as shown.

8
Each process has a context diagram like this one with inputs, activities, and outputs. The
activities also have controls and enablers.
Just want to deal with the Stakeholder Requirements and Validation Criteria outputs
tonight.

9
Stakeholder Requirements are maybe called a Customer Requirements Specification or
some other name for the SEP certification exam, be exact! In real life, you can be flexible.

Many more requirements, of course, but these serve our needs tonight.
Question: Are these well formed?
A key is a HOW not a WHAT, but if the customer insists
How is the key to be used?
What does the customer mean when they say when I brake?
Well, dont want to go as deep as the Clinton-ism: what the meaning of is is, but we
need to arrive at clarity with our customer as to what they really want. It is an iterative
process.

10
11
Focus on System Requirements and Verification Criteria tonight.
Again, for the SEP exam need to follow exactly. In real life, some of these may be done at
different times for example, Verification Criteria tonight will be a little later.

12
Next we take the input, Stakeholder Requirements, and turn them into a well-formed
Systems Requirements Specification.
Some new phrases and interpretations:
radio has an on/off function, presumably we dont want the radio on all the time we are
using the car.
Note that this could just be our desire/thought and not the customers! I.e., we
could be adding additional scope when not needed, and not reimbursed. Or, it may
be a clarification that we can only provide radios with on/off function and the
customer needs to know that they will have to turn the radio itself on if they want
to use it.
key must be engaged what does that mean?
when I brake  brakes are engaged

13
Focus on System Architecture.

14
From the System Requirements we have so far, we define a System Architecture and have
to make some choices.
The key and power could have been a windup key like on a toy car and perhaps met the
customers requirements.

Architectural decisions have consequences derived sub-system requirements or


additional system requirements.
Loopback to an earlier process!

15
Again, there are many more and these will be deeper, such as battery voltage range,
capacity, charging requirements, etc. Good enough for tonight.

16
This is a Unified Modeling Language (UML) State Diagram and is a clear description of how
the system is intended to operate. Good for communication among all parties: the
customer, developers, testers, etc..
System Modeling Language (SysML) is the result of an INCOSE collaboration with OMG to
modify / extend UML. It is highlighted as being central to SE practices in INCOSEs SE Vision
2020. Prepare for it, because future engineers entering college now will expect this visual
capability.
This could have been done in the earlier Stakeholder Requirements Definition Process, but
not to this degree since the architecture had not yet been selected.

17
Now we can finish the Verification Criteria output of the Requirements Analysis Process.
Could have done so earlier, but again, not to this depth without the Systems Architecture
having been done first.

18
Before implementation, you had better go back to the customer and make sure they agree
with the SyRS, SyAR, SSyRS, SSyAR, Concept Documents, V&V Plans, etc.
Best to have had frequent contact / reviews! Corrections of interpretations and (sub-)
system choices are fair game for discussion; however, be careful that the Stakeholder
Requirements dont change  change orders w/ schedule & cost considerations.

19
Finally, we have a design and implementation we can test! Lots of work to get to this point,
but I wanted to ensure we have a good design based upon a solid lifecycle process.

20
21
I have now built and verified that each of the three sub-systems work as specified. We
would integrate them together and test to ensure they function properly.

22
When we are done and know that everything works as intended, we would turn it over to
the customer for their validation and acceptance testing.

23
I dont want just anyone to be able to use the car, so I want a key.
Ensure the key is present.
I want a radio.
Ensure the radio is present.
With the key, turn the radio control on and off, and ensure the radio does turn on and off.
Remove the key, turn the radio control on and off, and ensure the radio remains off.
I want brake lights that come on when I brake.
Ensure the brake lights are present.
With the key and when braking, ensure the brake lights come on.
With the key and when not braking, ensure the brake lights remain off.
Remove the key and when braking, ensure the brake lights do not come on.
Remove the key and when not braking, ensure the brake lights remain off.
Everything cool? No problems?
Sneak Indication what if the brakes fail? Monitoring the request here, not the action.

24
Later, an Emergency Flasher was added to the 1960s vintage car, adding to the Stakeholder
Requirements.

In the interest of time, I wont go through the entire design process again left as an
exercise for the reader.

25
26
27
I have now built and verified that each of the three sub-systems work as specified. Again,
we integrate and satisfy ourselves via inspection, analysis, and testing that the system
performs as desired. We then turn it over to the customer for their validation and
acceptance testing.

28
Everything cool? No problems?
Sneak Indication still, of course.
With flashing, lights are no longer related just to brakes so should probably be called tail
lights.
But, there is something meatier here than just a name on an indicator. Maybe you see it
already?

29
Let me redraw and walk through the operation.
Without the flasher, the cars operation is as before.

Without the key, the flasher works as intended.

With the flasher and the key, braking overrides the flasher, as desired.
Done right?

30
Please note that there are no component failures here. The electrical path from the Flasher
through the Brake Pedal to the Radio was designed in, albeit unintentionally. It is a latent
condition that initiates an unintended actionthe Radio flashing on and off. This circuit
might have been operational for many years without being discovered for the following
reasons:
1. Each of the (assumed) independent operations works as expected.
2. If you usually turned your Radio switch off when you turned your Key Switch off, the
sneak circuit would not have been apparent.
3. It takes the strange combination of Key Switch off, Radio switch on, Brake Pedal
pressed, and Flasher on to observe this sneak circuit.
You might say, well, I would have included all of the possible states in my testing. If so, I
will give you right here a check for $1M if, in return, you take a checker or chess board, put
1 on the first square, double that (2) on the next, and so on doubling for each next
square, and then you agree to pay me the amount on the last square  more money than
has ever been printed or stamped! Cant test all combinations and need to focus on the
truly independent ones.

This sneak circuit is benign, but sneak circuits can be catastrophic! The potential to be
catastrophic and their nature to occur in apparently random fashion in spite of rigid design
reviews, testing, and even years of operation prompted NASA to fund development of a
method of identifying sneak circuits before they caused system problems.

31
32
33
We have seen simple examples of a sneak path and a sneak indication tonight. Sneak
timing often happens in relay or other switching circuits so called relay races.

34
It is possible to reduce a complex electrical system to a set of topological patterns that can
be readily analyzed by the application of topological criteria, clues, to determine whether
or not sneaks are present. It is possible to findand then eliminateall sneak circuits
before they cause a problem.
The H Pattern is responsible for about 50% of sneaks usually by current flowing through
the center bar in a direction or at a time not intended.
In recognition of the importance of preventing sneak circuits, the U.S. military issued a
standard requiring sneak circuit analysis to be conducted on all new projects (MIL-STD-
785B Reliability Program for Systems and Equipment Development and Production, Task
205, Sneak Circuit Analysis).

35
Back to rocket science
When the Redstone booster left the pad, the tail plug umbilical pulled out 29 milliseconds prior to
the control plug umbilical. Electrically, these connectors are just switches. The ignition indicator
lights lost their normal ground. This caused current to flow through a suppressor diode and reach
the engine cut-off coil to abort the launch. This was a designed in capability. Prior to this specific
launch, there was an accident causing damage to the tail umbilical cable. It was spliced back
together, but slightly shorter this time, so it pulled out first. No component had failed. A prior
failure contributed to the activation of the sneak circuit, but was not its root cause. It was just luck
that the tail umbilical had originally been made longer. There was no stated requirement for doing
so.
But why was this latent capability not recognized during the system design and review? In complex
electrical systems, there are usually a number of different people designing different subsystems.
Sneak circuits are possible in each because the designers were not trained to preclude them;
however, the possibility for sneak circuits greatly increases across interfaces. Assumptions are often
made about the other end or the circuits at the other end are changed with the assumption that
there will be no effect elsewhere. When the subsystems are assembled into the total system, the
sneaks are enabled. In reviews, it is very difficult to see such details amidst the forest of data.
Also, modifications to existing subsystems can often introduce sneak circuits because the designer
fails to recognize the true overall effects of the subsystem change, due to the complexity of
visualization described above.
Why was this latent capability not recognized during the system testing? In complex electrical
systems, the number of possible combinations for all input device settings can be huge, so it
becomes unfeasible to test every scenario. Instead, testers usually test the expected (assumed
independent) functions. But, as we have seen with the automotive sneak circuit, elements thought
to be independent may not be so.
Assumptions, biases, and complexity can render all our best efforts at following a good
development lifecycle less than 100% effective. Simplifying and having clear, accurate visibility into
complex systems; identifying all the truly independent functions for analysis and testing are the
main outcomes of sneak circuit analysis.

36
The topological trees can also be used for failure analyses.
Ive used them for FMEA analyses. For example, what if the one daisy-chain wire breaks or
a connection comes loose here? The radio *might* turn on both brake lights at least
dimly.
Ive also used the topological trees for common-cause analyses in one case, analyzing
what happens if water breaches a connector and shorts two or more adjacent pins.
Understanding the roots of sneak circuits and such failure analyses, one can design systems
that dont have such problems.

37
Here is a reasonable, high level document you can easily review. Dave Buratti and Sylvia
Godoy are referenced at the end. I worked with both while at Boeing.

I have never been able to find the actual circuitry of the car sneak circuit; however, I did
find a similar sneak circuit in the Dodge turn signal circuitry.

38
39

You might also like