Outcome Analysis Process (OAP)
Introduction:
The intent is to describe a modern approach to safety investigations for events that do not require the rigour and horsepower of tools such as ICAM. It pulls heavily from the works of Sidney Dekker (Safety Differently, Behind Human Error, The Field Guide to Understanding Human Error, Just Culture et cetera), Erik Hollnagel (Safety I and II, and his excellent work on Resilience), Dylan Evans (Risk Intelligence), Daniel Kahneman (Thinking, Fast and Slow) and Todd Conklin (Pre-accident investigations).
I have termed it an ‘outcome’ analysis process rather than an ‘incident’ analysis process as we really are trying to understand an unintended outcome related to someone undertaking a task.
The basis of this process is driven from a definition of Safety as described by Dekker in Just Culture – Balancing Safety and Accountability: “Defining ‘Safety’as the presence of positive capabilities, capacities and competencies that make things go right and not as the absence of things that go wrong”. Hollnagel also describes an almost identical definition.
Contemplation of this definition drives us to explore what normally goes right to create safety and not just what went wrong in an event. It drives us to focus on the gaps between the way the work was done on the day of the event (Work-As-Done) and the way the work was intended to be done by the procedure, work instruction, THA et cetera (Work-As-Intended). And it drives us to consider the question “What is responsible for his incident?” rather than “Who is responsible for his incident?” On a day-to-day basis when we have not had an event, it also helps us strive for getting as many things right as possible, rather than minimising the number of things that go wrong.
As a starting point for any incident investigation, we need to put ourselves into the shoes of those involved at the time and not dwell in the land of hindsight bias that we all love so much. As Jens Rasmussen is purported to have said “If you don’t understand why it made sense for people to do what they did, it is not because they were behaving really strangely, bizarrely, or erroneously, It is because your perspective is wrong.” We need to understand what was going on in the world of those involved and see through their eyes at the time of the incident.
Details of the Outcome Analysis Process:
- Step 1 – Get the team together
o Three or four as a maximum.
- Step 2 – Timelines (focus on WAD vs WAI)
o Whiteboard “how do we normally do this work and what was different in the event?”
- Step 3 – Exploration of the explanations
- Step 4 – Actions
- Step 5 – Report – One page
o Overview of the event
o Work-As-Done vs Work-As-Intended gaps identified (Not the entire time line)
o Summary of explanations for the gaps
o Actions
Details:
Keep it simple. Bring together a small team. It should be led by the Supervisor (or Superintendent) of the area. Include the person involved and a couple of their peers. One at least from another shift or area who is very familiar with the work but not directly involved.
Set up a conversation that talks about how the work is normally done. Do not start with any analysis of the event itself but build a common story about “How we normally do this work”. Only then start exploring which parts of the task were different than ‘normal’ and build a simple time line based solely on those steps where the Work-As-Done related to the event differed from the Work-As-Intended or how the work is normally done.
Once these differences have been identified, have a conversation about what was driving those differences. These explanations may look like a simple 5-whys but is not as structured. Think about topics such as:
Resilience
Procedural / task / situational complexity
Task planning, including WYSIATI.
Task assignment / Answering a different question
Task completion / Plan Continuation / Intense Task Focus
Core competencies
Once you have explored the explanations for the differences, talk about what we can do so that it is harder to get wrong next time, and easier to get right. These become the actions. Write up a simple report and complete the actions.
Training Requirements – for investigation leaders (Supervisors and Superintendents):
½ day workshop. A solid mixture of theory and practice. Including:
- Interviewing,
o Asking questions,
o Listening,
o Body language
- Work-As-Done versus Work-As-Intended,
o Exploration of explanations of differences.
- SMART actions
Some questions to help explore explanations and drivers of the differences between Work-As-Done and Work-As-Intended:
- What did the individuals in the team understand by the risks they had described in their THA / Take 5 / “Stop and Think”?
- Was the THA done as a group, by an individual, or by the supervisor?
- Does the THA or Procedure encourage people to think about which bits of a task people needed to really pay attention to?
- Does it encourage them to plan what they ‘could’ do if something starts to go wrong?
- Looking at the THA or Procedure, is it written in a way that makes it easy to do correctly and hard to do incorrectly?
- How many procedures or standards or work instructions needed to be followed to get the task completed?
- Was the task complex or complicated?
- Were there other activities in the area that impacted the task?
- Looking back over the years or months, has the way we have done this task changed?
- Is the way the task was done the same way for all crews and shifts?
- Did the task require so much attention that other things were not seen?
- What is the perspective of what is happening of the people involved? What do/did they see and make of it? How do you see it?
- Is the perspective/view of the situation the same for all team members?
- How realistic are the hazards and hazard controls in the THA?
- What decisions were made during the event (or when creating the THA) that made sense at the time, but seem not quite so clear now?
- Did the way those involved receive the task assignment match the intent of the supervisor who assigned the task?
- Was everybody involved trained and competent?