An Investigation of the Therac-25 Accidents Nancy G. Leveson, University of Washington Clark S. Turner, University of California, Irvine A thorough account of the Therac-25 medical electron accelerator accidents reveals previously unknown details and suggests ways to reduce risk in the future. Snbasaconsequence-have beeninvolvedinacidets Some of he most day ctodsotwareelsted aecdonsinsucty-rtia esters involved a computerized radiation thertyy machine aed te Therse-2, Between June 1983 and January 1987, ss Known ace ivolved maine overdones bythe Therac25~ wath esulant dente ad seriou nts, They have eon dese {the wortserievof radiation aciJents inthe 38 yar history of medial acceler ‘With information fortis rise taken rom puis avaiable documents. we eset detailed acide vestigation ofthe Factor nwsved in he overdoses Sn the avtempt by the were, manufacturer. and he US an Canadian govern inofs io dcal with them. Our El steers earn fom his expertense nok toasts the equipments manatacturer or anyone sve The mistakes hat were tmade are no unset thismanufacturer bt are unortnately fay common Siersafeh-citieal stems. As Fank Howston tthe US Food and By Adin iration (FDA) si." ipniiunt amount of star foie rial sptems om toma nepal inthe media device duty fim that tthe rofl ofthoveresiant or uninformed ofthe pinsplsofthersytem sets Fortermorethve problems ace not nite tothe medica industry. x silla common Blt tat any good engineer can uid software, egal of whether he or shes trained dateat the-art softeme-engincting procedures. Many Companies ulin safety rial sftmare are not using proper procedes 0) ‘Voftware-enginceing and safety-enaincsring perspective Moa accidents ae tom acl: that hy stem from complex interac tions between vaiouscomponentsand aves attibut angle cous oan {chen uals serous mistake. In hs article, me hope to demonstate the cmpiex nature of acidents amd the need to investigate ll apts of stern evelopment and operation to understand what has happened and to prevent ftrare acedens Despite whatcan be earned from schinvestgations.feasof potential ability Gc ‘omputers are increasingly being introduced into safety-critical systems are-susnnonaa sone 0 ie COMPUTER‘or loss of business make it dificult to find out the details ehind serious engi neering mistakes. When the equipment fs regulated by government agencies, some information may be availabe, Oc «asionally. major acidents draw the at- tentionof the US Congressor President tnd result in formal acident investiga tions (for istance the Rogers comm sion investigation of the Challenger a ident and the Kemeny commission investigation of the Three Mile Island incident) The Therac-25accidentsare the most serious computer-related accidents t0 Gate (at least nonmiltary and admit ted) and have even drawn the atention fof the poplar press (Stories about the Therac-25 have appeared in trade jout nals newspapers, People Mogazine.and fon television's 20/20 and MeNeil/ Lehrer News Hour) Untortunatelythe proviousaccountsof the Therae:25 prob ems have heen oversimpliied, with misleading omissions In an effort to remedy this, we have obtained information from a wide vari- fy of sources, including lawsuits and theUSandCanadian government agen- cies responsible for regulating such fquipment. We have tried to be very careful to present only what we could document from original sources, but there no guarantee that the documen: tation itself s correct, When possible, ‘welooked for multiple contimingsoure {2 forthe more important facts ‘We have tried not tobias our des tion of the aceidents, but iis dificult ‘ot a filter unintentionally what ede scribed, Also, we were unable to inves: tigate Hrsthandorget information about some aspects of the accidents that may be very relevant, For example, detailed information about the manufacturer's software development, management, Aandqvalitycontrolwasunavalable, We had to infer most information about these from statementsincorrespondence lor other sources. [As a result, our analysis ofthe aci donts may omit some factors, But the facts availabe support previoushypoth ‘ses about the proper development and tne of software to control dangerous processes and suggest hypotheses that need further evaluation, Following our sccount of the accidents and the re sponses of the manufacturer, over: ment agencies, and users, we present What we believe are the most compe! Tinglessons tobe learned in the context July 1998, ofsoftwarcengineering.saety engineer ‘ng. and government anduserstandards and oversight Genesis of the ‘Therac-25 Medical linear accelerators (linacs) sccelerate electtons 10 ceeate high- tenergy beams that can destroy tumors with minimalimpacton he surrounding healthy tise, Relately shallow tissue Isteeated withthe accelerated electrons toreach deepertissue,theelectronbeam is converted into X-ray photons. Inthe early 1970s, Atomic Energy of ‘Canada Limited (AECL) and a French company called CGR collaborated 10 bill linear accelerators, (AECL is an arms-length entity. called a crown cor poration, ofthe Canadian government. Sincethe vimeofthetncidentsrelatedin this article, AECL Medical, a diision of AECL, is in the process of being privatized andi now called Theratron- kes Internationsl Limited, Currently ‘AECL 's primary busines is the design snd installation of nuclear reactors.) ‘The products of AECL and CGR's co- operation were (1) the Therae-, 2 6 millionelecironvolt(MeV) accelerator capable of producing X rays only and, er. (2) the Therac-20,420-MeV dual mode (X rays o eleetrons) accelerator. Both were versions of older CGR ma chines, the Neptune and Sagitaire. re spectively, which were augmented with Disolay ~ we ‘min fete ‘computer contro using a DEC PDP 11 ‘minicomputer. ‘Software funcionakty was Himited in both machines: The computer merely auddedconvenience tothe existinghard> ware, which was capable of standing lone. Industry standard hardware sae ty features and interlocks inthe under- lying machines were retained. We know that some old Therae-6 software rou tines were used in the Therae-20 and that CGR developed the initial sot “The business relationship between AECLandCGR falteredatter the Ther 446-20 effort, Citing competitive pres Sires the two companies did nat renew their cooperative agreement when scheduled in 1981. In the mid-1970s, [AECL developed a radical new *dou- ‘e-pass” concept for electron acceler tion, A double-pass accelerator nec ruthless space to develop comparable nergy levels Because it folds the long physical mechanism required to accel trate the electrons, and it is more cco fnomic to proiice (since it uses a ma fetron rather than a klysteon a6 the ‘energy source) Using this double-pass concept AECL designed the Therac-25, a dua: ‘mode linear accelerator that can deliver tither photons at 25 MeV or electrons ft various energy levels (ce Figute 1) ‘Comparedwith the Therae-20,the Thet: ac25 fs notably more compact, more Yersatile, and arguably easier to use The higher energy takes advantage of ‘the phenomenon of “depth dose”: As Figure 1 Typical Therae-25 fathe energy inereases, the depth in the body at which maximum dose buildup ‘occu also increases, sparing the tissue shove the target ares, Economicadvan- tages also come into play for the cus. tomer. since only one machine is re fired for both treatment modalities (electrons and photons), Several features ofthe Therae-28 are Jmportant im understanding the acc dents, First like the Therac-6 and the ‘Therac20, the Thorse-25 is controlled byaPDP IL, However, AECL designed the Therac-25totake advantage of com- puter control from the outset: AECL tid pot buldonastand-slone machine The Thera and Therae-20 had been designed around machines that already hhadhistoriesfclincaluse withoutcom puter control Ih addition, the Therac:25 software has more responsibilty for maintaining safety than the software the previous machines, The Therae-20 has indepen: hardware reliability. dent protective circuits for monitoring tleetron-beam scanning, plus mechani ‘a interlocks for policing the machine and ensuring safe operation. The Ther 16-25 relies more on software for these functions. AECL took advantage of the computer abilities tocontrol and mon= itor the hardware and decided not to duplicate all the existing hardware sae ty mechanisms and interlocks. Ths ap proach is becoming more common 35 Companies decide that hardware inter Tocks and backups are not worth the expense, or they put more Lait (per haps misplaced) om software than on Finally, some software for the ma chines was interrelated oF teused. Ina Jeter to a Therae-25 user, the AECL quality assurance manager said "The same Therae-6 package was use ‘AECL software peop when they start ced the Therac25 software. The Therac 20 and Therac.28 software programs were done independently, starting rom 4 common base.” Reuse of Theracé ‘design features or modules may explain Some ofthe problematic aspects ofthe ‘Therse-28 software (see the sidebar Therae25 software development and sign’). Tae quality assurance manag: fer was apparently unaware that some Therac-20 routines were also used in the Therac-25:this was discoveredafter fa bug related to one ofthe Therae-25 tecients was found in the Therae-20 soltware [AECL produced the fist hardwired prototype ofthe Therac-25 in 1976, and the completely computerized commer cial version was avaiable in late 1982 {The sidebars provide details about the machine's design and controlling sot ‘ware, important in understanding the secidents) Tn March 1983, AECL performed a safety analysis on the Therae-28. This analysis wae in the form of 2 fault tree ‘Therac-25 software development and design ‘Wo know that the software fr the Therac-25 was dev coped bya single person, using PDP 11 assembly language, ‘over a period of several years. The sotware “evolved” from the Therac-6 software, which was started in 1972. According to alter from AECL to the FDA, the “program structure and cerain subroutines were carried ove 1 the Thorac 25 ‘around 1976." “Apparently, very tte sofware documentation was pro vcd during dovelopment. In @ 1986 internal FDA memo, 8 reviewor lamented, “Unfortunately, the AECL response also ‘sams to point out an apparent lack of documentation on Software spectications and a sofware test plan.” “The manufacturer said that the hardware and sofware wore “ested and exercised separately or together over ‘many yeas.” In his deposition for one ofthe lawsuits, the ‘ually assurance manager explained tat testing was done ‘nwo parts. A“emall amount of software testing was done ‘ona simulator, but most testing was done as a system. I ‘appears that unt and software testing was minimal with most effort directed atthe integrated system tect. Ata Thor- 8¢-25 usergroup meeting, the same qualty assurance ‘ger ald thatthe Therae-25 software was tasted for 2,700 hours. Under questioning by the users, ho clariied tis as ‘moaning "2,700 hours of use "The programma ot AECL in 1986. In a lawsuit connected with one of the accidents, the lawyers were unable to obtain Information about the programmer from AECL. In the depo ‘tions connected with that ease, none ofthe AECL empioy- ‘288 questioned could provide any information about his edu ‘cational background of experience. Although an attempt was, ‘made to obtain @ deposition trom the programmer, te law- ‘ult was sottod before this was accomplishod. We have ‘boon unabio to leam anything bout his background. ‘AECL calms proprietary rights to its software design. However rom voluminaue documentation regarding the ac- ‘ident, the repairs, and the eventual design changes, we ‘can build a rough picture of “The software ie responsibe for monitoring the machine status, accepting input about the treatment desires, and set- ting the machine up for ti treatment. turns the Beam on in responce to an operator command (assuming that certain ‘operational checks on the status of he physical machine are ‘atefe) and also tums the beam off when treatment is ‘completed, when an operator commands tof whien a mal- function fs detected, The operator can printout hard-copy versions ofthe CRT dieplay of machine setup paramore. “The treatment unt has an inlarock system designed to re- rove power to the unit when thre Isa hardware malfunc tion. The computer monitors this ntertock system and pro- ‘vides diagnostic messages. Depending on the faut the ‘computer ether prevent a treatment from being started or ifthe treatment i in progress, creatos a pause ora suspen- ‘10 ofthe treatment “The manufacturer describes the Therac-25 sofware as having a stand-alone, realsime treatment operating system, ‘The system is nt bul using a standard operating system or ‘executive. Rather, ho real-time executive was writen espo- ‘aly forthe Therao:25 and uns on a.32K POP 11/28. A proomptiva scheduler allocates cycles to the rial and honeris! tasks. “The software, writen in POP 11 assembly language, has four major components: stored data, a scheduler, a set of crical and noneetioal tasks, and interrupt services. The ‘stored data includes callbation paramotors forthe accelera- tor setup as well as patient-trestment data. The interrupt rou- tines include COMPUTER