Some may argue that just enough reliability data is just the right amount. Too much may lead to confusion, too little doesn’t inform well. The reliability work we do helps others make decisions, and recent work in how humans make decisions may help us prepare and present our results effectively.
If preparing reliability data-based recommendations, consider using less information. Ed O’Brien and Nadav Klein have found decision-makers tend to use much less data or information to make a decision than they think they will need.
If using data and the derived information to make a decision consider the situation carefully to know when to use a structured decision-making approach or to simply go with your gut. Daniel Kahneman and Gary Klein provide some insights and basic guidelines for decision making.
Information and Decision Making
Does your team carefully review all available information before making important reliability-related decision? Probably not. Yet, they probably want the risk analysis, reliability modelling, test results, and more before sitting down to make a decision.
In a series of experiments, O’Brien and Klein found people used less information to make a decision than they expected they would. The experiments and findings are published in the Proceedings of the National Academy of Sciences.
While we think we are rational decision-makers and willing to view all relevant information before deciding, we don’t. Our minds react to information as it appears often leading to forming a decision well before we consider all the information. This behaviour serves us well in many situations, yet not all.
So, if preparing a recommendation or report, really do focus on perfecting the executive summary. Get the most important information upfront and make it clear. Include the details, yet know that it maybe be skimmed after the decision has been formed.
In a volatile situation, where each analysis or experiment reveals new or contradictory information, then more information will be necessary to form a decision. If all the results are consistent, then very little information is necessary. Thus, make sure the information you provide highlights the degree of volatility in the data to help the decision making consider an appropriate amount of information.
Intuition and Decision Making
As with the speed of decision making, the method we use to form a decision varies from intuition or gut feel to rigorous and structured consideration. Of course, how we make decisions varies, yet when making an important decision it may worth your time to think about how you think.
Kahneman and Klein published in a September 2009 American Psychology article titled “Conditions for intuitive expertise: A failure to disagree,” describing the circumstances when intuition may result in good decisions. In March 2010 the two authors were interviewed by the folks at McKinsey & Company.
Every decision-maker has some experience to draw upon to aid in assessing a situation. In some cases, such as a person’s 12th product launch decision, they have 11 prior development life-cycles to in part base a decision this time. If the current situation is consistent with the previous patterns of development, then the gut feel may be very informative.
In other situations, especially under time-pressure, we have to rely on intuition as we do not have the luxury to review all the data.
The issue here is a reliance on intuition, which may be just overconfidence in one’s ability to assess the situation and make the right decision. In situations that are stable, predictable, and consistent, then intuition probably has it right.
Yet, we rarely face such decisions when designing a new product or system. We invent, improve, coax, and explore the capabilities of our teams, materials, and structures. We regularly face decisions that have many variables and interactions that are all turbulent. Decisions in a turbulent situation require a structured method to help form a decision.
The structured approach guards decision-makers from making decisions on fragments of information before considering the full complexity of the task.
For complex situations here are two ways to help structure the information to base a decision.
One method involves doing a premortem. A premortem is a short process to think through and articulate the essential a postmortem done before the project or major decision. The idea is to consider what would lead to a poor decision, what ‘did’ we not do that leads to failure? In short, don’t wait to experience the failed or poor decision to consider what went wrong. Do this upfront to help identify the essential steps, bits of information, and necessary data that will help avoid failure.
Another approach, which aids in the delay of premature intuition-based decisions is what Kahneman, Lavollo and Sibony call “Mediating Assessments Protocol” (The paper is published in the MIT Sloan Management Review). This idea is to first create 6 or 7 attributes necessary to consider before making a decision.
Consider each attribute individually and decide just on that element. Then consider all the attribute decisions collectively. Like a decision matrix, which may even include weighting factors, the essential element here is to determine the salient elements or attributes necessary to inform the overall decision. Then consider and decide on each attribute separately.
When Preparing to Decide Summary
Even if we can prepare a 200-page reliability report, the first page or two are the most important – so, spend your time there. Also, if you know the pre-determined (or help create them) attributes be sure to address each clearly.
Finally, keep track. When addressing a novel failure mode, how did the decision form and did it work? How often did the team rely on intuition or a structured root cause analysis process? With each decision record the situation characteristics (level of volatility and complexity) and later capture if the decision went well or not.
What I gleaned from these articles on decision making is we can improve our ability to make the right decision when we think about how decisions are made. For reliability work having the right information at the right time is essential if we consider the needs of the decisions being made.
Nice review plus your contributions were constructive.
It seemed to me that “.. decision-makers tend to use much less data or information to make a decision…” because that’s easier and because of confirmation bias.
After I publish the “User Manual for Credible Reliability Prediction,” I will resume work on “Field Reliability Estimation without Life Data and Random-Tandem Queues.” The latter compares the (Shannon) information in life data vs. ships and returns counts (data rquired by GAAP).
Shewhart Rule 1. Original data should be presented in a way that will preserve the evidence in the original data for all the predictions assumed to be useful.
If managers use less information…, then why don’t reliability engineers use ships and returns counts to make nonparametric estimates of field reliability? Ships and returns contain less information than life data. Distribution assumptions are spurious info.