How Good is MTBF as a Metric?
A bolted hanger along a rock climbing route is often a very welcome site. It provides the climber with safety (by clipping the rope to the bolt); with direction (this is the way); and, with confidence. Does MTBF as a metric do the same for your organization?
As climbers we count on the bolts to provide support in case something goes wrong or we need to rest along the route.
A reliability metric is often used in the same way as a climbing bolt. The measure, whether MTBF or Reliability or Failure Rate, provides a sense of assurance that the product in question is performing as expected.
The organizations profits are or will be safe. The development team uses the measures as a guide for design and supply chain decisions. And, the measure provides confidence to the organization related to meeting customer expectations around reliability.
Strength of Metric or Anchor
To extend the analogy a bit further, consider the image of the bolt closely. The strength relies on the bolt attachment within the rock. It is not visible. Once the bolt is set, the climber trusts the integrity of the attachment.
A reliability metric, likewise, often hides the underlying data. The strength of the measure or summary of the data relies on the design of the metric, the analysis, the assumptions, and the underlying data. The more accurately the measure conveys the data the better the ‘attachment to the rock’.
What do we want in a Metric?
In conversation we often desire to be true, beneficial and timely. With metrics the same is often at play. Metrics that are false, harmful or late are of little value. I suspect you agree.
Then why are we still clipping our reliability discussions on an MTBF bolt?
True? Rarely is the underlying data accurately modeled using the constant failure rate assumption.
Beneficial? As you undoubtedly experienced, the metric itself leads to confusion and misunderstanding.
Timely? While it is possible to make predictions quickly, that’s only 1 of 3 of criteria.
Designing and manufacturing a product requires strong and helpful metrics. MTBF is like a loose bolt that is off route. Take a look at your use of MTBF and critically assess the truth it conveys, and how it is understood among your team.
The Reliability Metric : A Quick and Valuable Improvement Over MTBF (book)
Exploring Alternatives to MTBF (recorded webinar)
When to Use MTBF as a Metric? (article)
Is MTBF a beginners metric? (article)
MTBF as a Metric (article)
Arjan van Druten says
Good Post, Nice comparison…
We also might add that when one of the bolts slowly cracks and fractures, bents or suddenly completely falls out, it does not only impact that part of the rope, but might induce failure in other bolts / connections / the rope and may result in different sorts of failures… (other MTBFs are or may be impacted – in different ways). The whole reliability problem is mostly about chains of lower level events, combined failures or partial failures, interfaces, systems level effects and the total top level requirements. It is about physics of failure and scenario modelling, classifying and structuring.
So, we should question if it is really possible to easily divide the problems in just a few numbers of independent measures (MTBF numbers) with often unclear definitions of failure. This relates to the good habbit to always try to very clearly define the F in the term MTBF, together with the other Letters….(or better not to use it for other reasons as shown on this nice site!). In safety engineering this is already often common practise.
Fred Schenkelberg says
Thanks for the comment. I agree we need to focus on the failure mechanism in many cases. That is too often lost with the use of MTBF.