Failure Modes and EffectsAnalysis (FMEA) for RadiationM di iMedicineR. Alfredo C. Siochi, PhD
Outline An Introduction to FMEAFMEA for radiotherapy workflowimprovementReverse FMEA for implementation of newtechnologyFMEA after an accident – what can welearn from the NY Times?
MOC PQI – hope you stay awake!www.dilbert.com
FMEA Failure Modes and Effects AnalysisFM: What could go wrong? And how?E WhatE:Wh t are theth consequences?Analyze: Probability of Occurrence,Detectability, Severity
Types of FMEA Process FMEADesign FMEAS tSystemFMEAProduct FMEABasic Methodology is the same
Failure Modes What could go wrong? And how! RequiresRiBBrainstormingi ti TTeam– familiar with the subject of their analysis((process,system,tproduct)d t)– Identify everything at this stage– (even seemingly trivial or improbable items)
Murphy’sMurphys LawIf anything can go wrong, it will!
Effects For each failure modemode, identify theeffect(s) These can be effects that happen to– Patients– StaffSt ff– Other processes or workflows (e.g. the effectmay nott bbe a problembliin andd off ititselflf bbutt if it iisallowed to propagate it could becomesignificant)
Analyze What is the Severity of the effect?– No harm 1, Lethal 10 What is the probability of Occurrence?– not likely 1, certainty 10 What is the likelihood that the failure modewill escape Detection before it causes aneffect?– Always detected 1, undetectable 10
Risk Priority Number RPN Severity x Occurrence x DetectionRanges from 1 to 1000Hi h numbersHigherbhhave greatert priorityi itMultiple failure modes exist in a system,which one is the most critical to address? Risk managementgshould considerregulatory issues
Proposed TG100 Rating Scales[RACS1]Ahigh value for detectability actually means that it is less likely to be detected. This can be confusing for a novice.
Risk Management Reduce the RPN Re-design the product or ImproveProcesses in order to:– Remove the failure mode, or– IncreaseItheth detectabilityd t t bilit off ththe ffailureilmode,dor– ReduceR dththe severityit bby changinghi ththe effectff t
Risk Management by Signage?www.classicalvalues.com/archives/2009 10.html
A more serious example example Failure Mode: HDR Door Interlock FailsEffect: Unintended radiation exposure Severity:y ? Dependspon source Occurrence: ? Depends on interlockreliability Detection: ? Depends on system design RiskRi k MManagement:t DDailyil QA off ddoorinterlock and all emergency switches
Multiple Fault Tolerance Many backup systems in caseone fails redundant in purpose Mayy be redundant in designg Examples– Signageg g– Emergency stop button– Emergencygy Power off button
Part II: FMEA for RT WorkflowImprovement A well run clinic has well establishedestablished,understood, and implemented processes Processes affect the total environment ofthe clinic: business, technical, clinicalaspects FOCUS here is on the safety of the clinicalprocess
Process FMEA Process Map or Process treeInclude Control PointsA lAnalyzesubb processesCreate Fault treesMitigate Hazards
Process Hazard Mitigationhttp://safety.lovetoknow.com/Funny Safety Pictures 1
Process Mapping Flowcharts to follow a product frombeginning (“raw materials”) to end (productin the hands of consumer) Radiotherapy: Very Data Driven OneO method:th d followf llththe ddatat tot createt thethprocess map
DataFlowin RO*Fig. 11.1 fromSiochi, Informationresources forradiation oncology,Ch. 11 of aforthcoming book:Informatics inRadiationOncology, G.Starkschall, B.Curran, editors.
Clinical siciansTherapistsIn-House SoftwareAdapted from Fig 5. Siochi, et al.Radiation therapy plan checks in apaperless clinic,clinic JJ. App. Clin.Clin Med.MedPhys., 10(1):43-62.
Understand Your Process You cancan’tt determine failure modes if yourprocess is a black box Break down process into single actions Identify interfaces between actions Identify resources for each action Determine failure modes Mitigate Hazards
Failure Modes: Device vs Process Example: Radiosurgery Ring Placement– Device: Plastic Support Snaps– Process: Pin was over-tightened Device Failure Mode:– Intrinsic Device Design Problem– May be mitigated by processes Process Failure Mode:––––Sequence not followedSt ForgottenStepFttStep done incorrectlySequenceqpproduces undesirable side effects
Process Failure Modehttp://www.darwinawards.com/
Example: IMRT Plan PreparationProcess Example Process for FMEA Sub process of the IMRT treatmentprocess Each clinic has to evaluate their ownprocess
TG 100 IMRT Process Tree- Draft
Where do I begin?http://safety.lovetoknow.com/Funny Safety Pictures 14http://safety.lovetoknow.com/Funny Safety Pictures14TAKE IT ONE STEP AT A TIME- WORK WITH SUBPROCESSES
Sub Process – plan preparationWhat are the failuremodes for each of thesteps in eachsubprocess?What are their effects?HHowddo ththey propagate?t ?How do they interact?How do we mitigatethem?
plan preparation failure modesWrong patientWrong coordinate systemWrong Isocenter for DRR ,e.g. calc point was chosenTypographical errorsCourse change without re-planre planMissing data
plan preparation effectsPatient receives wrongtreatmentDose distribution changes,Dose to wrong siteDose to wrong siteDepends on which elementwas a typoRadiobiological EffectsDepends on what ismissing
plan preparation - analysisS 10, O 1, D 10?Systematic error, O 10,S 10, D 10?S 1010, O 7 (many cases,casesiso calc), D 7(verification day – imageslook strange?)Depends on which elementwas a typoS 6?,6? O 55 (protocols arewell established), D 10Depends on what ismissinggAssessing Detectability means you know the whole process. Are there othersub processes that will catch the error before it affects the patient?
plan preparation - RPN1001000490Depends on which elementwas a typo, could be 1000300Depends on what ismissing,g, could be 1000
plan preparation – Risk ManagementPhase 1100Implement plan checkprocess490Implement plan checkprocess300Implement plan checkprocessReduce the value of D for the highest RPN processes, i.e.Make the failure mode more detectable
plan pppreparationp– Risk ManagementgPhase 2, 3, etc.D 11, S 10S 10, O 10O 10.New RPN 100.Modify transfer softwareconfiguration, O 1RPN 10Implement IGRT checkpprocessMitigate the next highest RPN values. Adjust the RPN values of mitigated items.Consider other mitigation steps to reduce D or O. S will not change for the giveneffect.
Overwhelmed?Ask ForHELPhttp://safety lovetoknow com/Funny Safety Pictures 6http://safety.lovetoknow.com/Funny Safety Pictures6
Part III – Reverse FMEA fori limplementationi off new technologyh l New–New unfamiliar – hard to know failuremodes Start with “Effects”Effects Prioritize by effects – no need for RPN Then use fault tree analysis.analysis– requires learning more about failure modes,but the learning is now guided. Examine fault tree to build in mitigations– Process design– Device Modification
Generic RT Effects Wrong PatientWrong SiteWWrongDoseDDistributionDi t ib tiWhich of these top level elements doesyour new technology affect directly? Developp that element in ggreater detail
Fault TreeOR Gate - All inputfaults must bemitigated to avoidthe output ogySTARTHERESTARTHERE
Learn about your deviceAsk yourself questions about how your device works and how it will be integratedwith other devices in your clinical workflowHow does thenew ubsystems?How does thenew technologydeliver dose?What are the special considerationsfor modeling the device or treatmenttechnique in the planning system ?
Learn about your device
Example: Moduleaf - Hardware Add on mini-MLC40 leaff pairsLeaf width 2.5 mmLeaves move from – 6 cm to 6 cmMax field size is 12 x 10Leaf position tolerance 0.5 mmClosed leaves parked 5.5 cm away from centralaxis Rounded leaf tips Slight tilt from divergence on leaf side
Moduleaf Dose Delivery ErrorRememberthis for later
USING THE FAULT TREE Device configuration decision: givenoptions, which one presents the least risk? Is the fault true for the device? Test procedures: should be generalenoughh tto testt t allll possibilitiesibiliti ffor ththe error Clinical Workflow Design: write proceduresthat reduce occurrence of error or increasedetection of error
DECISIONS? SLOW DOWN
Jaw Configuration DecisionELIMINATETHISAND GATE:Mitigateeither inputKEEP THISANDANALYZEWHATSIZE DOWE USE?We decided to keep the Jaws fixed since we have no control over the jaw toleranceof 2 mm. For small fields, a 10% or greater error can occur due to positioninginaccuracy The errors from using a fixed jaw can be reduced to a much lower valueinaccuracy.(dose uncertainty due to leakage modeling in TPS).
MODULEAF field size Decided on 1010.44 x 1010.44 With jaw tolerance this means jaws rangein position from 5 to 55.44 Closed leaves at 5.5 cm are blocked Jaws don’t invade mMLC fields up to10x10 Output factor change minimal
Leakage vs X field size
Leakage measurement method? DonDon’tt assume anything– “Gafchromic is expensive, maybe I can justtest the 10x12 area”area Go back to the fault treeRRememberb ththe ititem ““missingi i shielding”?hi ldi ”?That could be anywhereTest a full field, not just the MMLC field
Missing ShieldingReduced field size10.4x10.4Lead addedY1 sideManufacturerConfiguration10.4 x 12.4Lead added Y1 side &Linac MLC closedlleavesbbehindhi d JJawsLead bothsides & LinacMLC closedleaves behindJaws
Never Assume Anything
V&R – Data Transfer Error Separate fault tree Several items were mitigated related todata integrity Most significant change we adopted was aprocess
Data TransferError LANTIS sends blockcode to LINAC onDMIP Cosmic listens toDMIP Cosmic sets leafpositions from therecord with thecorresponding blockcode BLOCK CODE iscrucial
Field shape communication Problem: Lantis block code does not have to beunique Lantis field IDs are uniqueq Moduleaf block codes in separate files for samepatient can be the samep Potential error: wrong Moduleaf shape is chosent gat o Mitigation:– block code to Lantis field id mapping– One file per patient in Cosmic at a time
Documentingg the FMEAUIHC Rad Onc Department WIKI:Moduleaf Project, FMEA section:The Effects are listed first, with the faults beneathththem.MitigationsMiti tiare ddescribedib d iin eachh sectionti fforeach fault, with links to the clinical procedures,design changes, and configuration decisions.
Documentation - IILink to ourfieldnamingconventionand blockcodemappingLink toprocedureth tthatinvolvesthisconvention
Follow your m/Funny Safety Pictures 3
Block Code MappingOur convention fornaming Lantisfield IDs makes itpossible to keepfield IDs uniquecaveatst notedt d forfnumber of beams,segments, RxMapping a uniqueLantis field ID tothe Moduleafblock code makesthe block codesuniqueMapping scheme
Segue to NY Times What if Moduleaf block codes were notsent? What if we did not check it? A 10x10 field opening with high MU!– Fractionated IMRT (350 – 500 MU)– SRS (2000 to 5000 MU) NY Times article: from descriptions, it ispIMRT without MLC shapes
IV: FMEA after an accident Reported Effects are extremely severe– (OR they wouldn’t get so much attention!) High Priority We should analyze the Failure Modes How does this relate to our practice?– Do we mitigategthis FM?– Is the mitigation effective?
PulitzerPiPrizeWinnerreportson /reference/timestopics/people/b/walt bogdanich/index.html
Example 1: Failure ModesReported“InIn another case,case an unnamed medicalfacility told federal officials in 2008 thatPhilips Healthcare made treatment planningsoftware with an obscure, automatic defaultsetting causing a patient with tonsil cancersetting,to be mistakenly irradiated 31 times in theoptic nerve.nerve ”Is this IGRT related? What was the failure mode? Wrong isocenter chosen?
Some clues as to what happened?“Many of these mistakes could have beencaughth hadh d basicb i checkingh ki protocolsl beenbfollowed, accident reports show. But there isalsol a growingi realizationli ti among ththose whohwork with this new technology that some safetyproceduresdare outdated.”td t d ”
Is your safety procedure effective?
Example 2: Effect and FM“ hishis fatal radiation overdose — which left himdeaf, struggling to see, unable to swallow,burned, with his teeth fallingg out, with ulcers inhis mouth and throat, nauseated, in severe painand finally unable to breathe A New York Cityhhospitalit l ttreatingti hihim ffor ttongue cancer hhaddfailed to detect a computer error that directed alinear accelerator to blast his brain stem andneck with errant beams of radiation. Not once,but on three consecutive days.”y
Examplep 2: more FM“The Times found that on 133 occasions, devices used to shape or modulateradiation beams were left out, wrongly positioned or otherwise misused.”“ I.M.R.T. The unit was made by Varian The first four had been deliveredas prescribed. Now Dr. wanted the plan reworked to give more protection to[his] teeth. Shortly after 11 a.m the computer began seizing up, displaying anerror message system crashes ‘are‘not uncommon with the Varian software,fand these issues have been communicated to Varian on numerous occasions.’ at 12:24 p.m., Dr. approved the new plan At 12:57 p.m. — six minutes after yetanother computer crash — the first of several radioactive beams was turned on. several hours after [he] received his third treatment under the modifiedplan she ran a test the multileaf collimat