Sample Size and Duration and MTBF

If you have been a reliability engineer for a week or more, or worked with a reliability engineer for a day or more, someone has asked about testing planning. The conversation may have started with “How many samples and how long will the test take?”

You have heard the sample size question.

What I continue to hear in these discussions is the mistaken idea that adding another sample extends the effective time the testing represents in normal use. If I have a 1,000-hour test and add another unit, that does not mean the results represent reliability for an additional 1,000 hours of use time.

The Legacy of the Exponential Distribution

The problem stems from the exponential distribution, where the chance of failure each hour for each unit is the same. There is no change to the hazard rate over time; therefore, accumulating more individual hours provides additional information about how the system will behave over an hour of use.

This is rarely if ever is true. Hazard rates change as different failure mechanisms evolve, as materials settle or wear, as damage accumulates, or environmental contains change. If you check, you will find that assuming a content failure rate is invalid for your product or system.

Yet, you know that.

Then why do so many, when planning a reliability test, continue to rely on the total time of testing (number of units in test times the number of hours under test)?

• It’s in the textbooks describing how to plan a life test
• It’s what we have always done
• It’s in XYZ standard
• It’s what the customer requested

Maybe it’s time to step back and consider what you are trying to accomplish with the life test?

Is the Goal Reliability of MTBF?

In general terms, we conduct testing to learn something. For product life testing, we generally want to know what will fail or when it will fail. We want to understand if the current design and assembly process will create items that will work as expected for as long as expected within our customers’ use conditions. Basically, how many should survive for 2 years (for example).

Learning what will fail is often an exploratory or discovery-style testing. HALT or STRIFE, or HAST, or margin testing, or similar test is the deliberate work to reveal failure mechanisms. Life testing, accelerated life testing, ALT, duration testing, endurance testing, or similar tends to focus on will the units survive long enough.

The HALT approach involves finding failure mechanisms and designing them out of the product, thus improving the reliability performance of the product. The ALT approach involves understanding the time to failure behavior of failure mechanisms in order to characterize the reliability performance of the product.

Or, are you just after estimating MTBF?

Realize that MTBF is not reliability. It is neither a suitable nor complete description of your product’s probability of survival over a duration in your customer’s set of use conditions. It’s just a representation of the average time to failure, or the inverse of the average failure rate. Not very informative when trying to identify or eliminate failure mechanisms or make a decision on reliability performance.

If you only want MTBF (and I’m not able to dissuade you from this folly 😉 then you can use the idea that adding another unit to your test improves your ability to estimate MTBF. If you are assuming the constant hazard rate, then you are fine focusing only on MTBF.

The results of MTBF-based testing will provide some information on the actual reliability performance of your product. If you run your test for 1,000 hours, then you will learn about the reliability performance of the first 1,000 hours of you product’s use. If you run more samples, you will improve your understanding of the performance over those 1,000 hours.

In the vast majority of cases where the assumption of a constant hazard rate is invalid, running 10 units for 1,000 hours does not imply any meaningful information about the reliability performance out to 10,000 hours. The MTBF value may say you have achieved a 10,000 MTBF, yet you have really only learned something about the initial 1,000-hour operation for your product.

Be Prepared for Proper Test Planning

In summary, when you hear this concept of test planning based on a constant hazard rate assumption, help your team understand the error. Help them develop and conduct reliability testing that actually provides meaningful information. Help them create results they can use to make decisions.