Getting the Right Information from Your Reliability Testing
You cannot test in reliability any more than you can test in quality. Often reliability testing is done though, and knowing the range of testing approaches and their associated results will help you get the most information from each test conducted.
Let’s explore the types of testing that generate information useful as you develop a reliable product. There are 4 different types of reliability testing:
Within each type there are many variations to the testing details and the specific results generated. Understanding the questions each type of testing has the capability to resolve is a good first step to implementing the right set of tests for your project.
1 — Discovery Testing
Better named a discovery or exploratory process, this type of testing involved running experiments, applying stresses, and doing ‘what if?’ type probing.
Margin testing, HALT, and ‘playing with the prototype’ are all variations of discovery testing.
The primary purpose is to determine boundaries for giving inputs or stresses. For HALT we are seeking the operating and destruct limits, yet mostly after learning what will fail.
With discovery testing, we are looking for information on what fails. Manifesting failures permits us to understand, mitigate or eliminate the potential failures from occurring for customers. This type of testing rarely provides information on when a failure may occur in the field, yet understanding the failure mechanism is a key piece of information necessary for life testing.
2 — Life Testing
When we need to understand how long an item will operate without failure, the appropriate test approach is called life testing. Estimating when something will fail is a common question when considering warranty policy, setting customer expectations, budgeting for spares or maintenance actions, etc.
Life testing may occur with one or more applied stresses, generally focused on one failure mechanism. Understanding the failure mechanism first allows you to apply stress that excites the failure mechanism of interest.
Life testing may occur at nominal or expected stress levels, at elevated stress levels (accelerated life testing), or at various stress levels. Some testing directly measures the time to failure or may measure performance or an items change in a characteristic value over time using repeated measures.
There are many approaches to life testing providing many options when attempting to estimate an item’s expected time to failure behavior.
If interested in when a solder joint will fail due to metal fatigue, using a high-temperature oven will not cause the metal to fatigue. Thermal cycling causes the relative motion thus strain across the solder leading to fatigue, for example.
If thermal cycling an item that has solder joints and other components and assemblies, of course, the solder will eventually fail, yet the other failure mechanisms that exist may also exhibit failures. Take care with analysis of such data with mixed failure mechanisms as the acceleration factors may differ significantly.
An essential part of life testing is translating the testing results to the set of use conditions and stress levels when in use by a customer. Running an item at an elevated temperature for 1,000 hours is of little value unless there is a way to estimate the time to failure behavior when the item operates at nominal or expected use temperatures.
Thus, a part of life testing is understanding the environmental and use conditions for a product during normal use. Environmental testing also needs this information.
3 — Environmental Testing
Your customers use your product somewhere and somehow. That implies there will be a set of environmental and use conditions or stresses. Your product should operate as expected by your customer where and how your customer expects the product to work.
There are a few ways to approach evaluating if your product will work within the expected customer environment and under the customer’s use conditions. First, you could simulate customer environments and use them as close as possible.
Second, you can select specific failure mechanisms of interest (high risk, low margin or robustness, or uncertainty concerning response) and select stress conditions related to those specific failure mechanisms. This is similar to life testing.
Third, you can evaluate one or more stresses as with discovery testing to determine margins or new failure mechanisms.
Of course, a mix of these approaches is common. Another common approach is to sequence the application of different stresses in order to control and evaluate the effects of aging on performance given the same or different stresses.
For example, you may use a high-temperature exposure to age polymer-based seals, then apply vibration or applied loads to determine if the aged seals continue to function as expected.
The primary question environmental testing answers is ‘will the product function in the customer’s environment?’
4 — Regulatory Testing
Regulatory testing is not often associated with reliability testing. It is a set of required tests often with some form of applied stress that often checks for compliance with safety or specific networked or integration performance characteristics.
The primary focus of regulatory testing is to prove to the regulatory agency the product meets the requirements.
For reliability work, failing to meet regulatory requirements may mean the loss of compliance and removal from the market. While the product may not functional fail, if it doesn’t meet compliance requirements it may lead to unwanted results, similar to a failure.
At times the proscribed regulatory testing has elements similar to life or environmental testing and may provide useful information about the product’s performance under those conditions.
Other time, you may be able to conduct the specific regulatory tests, then use the same prototypes or units for additional discovery, life, or environmental testing effectively leveraging the regulatory testing conditions to improve or enhance other reliability testing objectives.
There are many ways to evaluate the reliability of a product. The 4 types discussed in this article provide a rough framework as select the appropriate approach to meet your objectives. In short, be deliberate with your testing right from the planning stage. Also, for any type of evaluation work being done on the design, consider how that is of use in answering the reliability-related questions of what will fail or when will it fail.
Each test is expensive and should provide meaningful results leading to informed decisions. Testing just for the sake of testing provides little value, therefore for each proposed test check, there is a clear objective, a concise question, and meaningful purpose or use for the results of the test.
What kind of testing do you conduct? How do you determine if the testing is worth conducting? Add your insights and comments in the comments box below.
Ricardo de Arriba says
Very straightforward article, congrats!
During the development of new equipment, performing “Feasibility testing” is common practice in industry. This kind of testing refers to functional and environmental testing leading to verify that the item works as specified and it is able to withstand internal stresses and environmental conditions (temperature, humidity, vibration, impact, etc.). From a Reliability perspective, they are not enough to provide valuable information (qualitative and quantitative) and they are not intended for this objective. So, from my point of view, this “Feasibility Testing”, although related with Reliability secondarily, its primary objective is not focus on that. I would include in this category the “Environmental Testing” and “Regulatory Testing” of your article as Reliability is not primary objective (although very related).
From my point of view, I would classify Reliability Testing in three categories depending on the purpose followed:
1. Reliability Discovery Testing: the purpose is the qualitative knowledge of failures, causes, mechanisms, etc. in order to extend the useful life of the product and increase its robustness. This category is the same as your “Discovery Testing”. The example would be HALT.
2. Reliability Demonstration Testing: the objective is to verify under a certain level of confidence a certain reliability objective. It can be in nominal or accelerated conditions. In this category will lay “Life Testing” of your article, Accelerated Life Testing (ALT) or a more descriptive name: “Quantitative Life Testing” (accelerated or not). I totally agree with you with the idea that “Discovery Testing” is the mandatory previous stage.
3. Screening Testing: the objective is the purge of infant mortality, manufacturing defects, etc. The example would be HASS and HASA.
What do you think of this perspective, Fred, is this approach fair enough to you?
Fred Schenkelberg says
Another good way to separate the various ways we conduct testing.
I’m not sure that the Feasibility isn’t a reliability test, as if the function doesn’t work, that is a failure, if it occurs too early or too often it is a reliability problem to solve. Getting the functions to work is essentially in my way of thinking a reliability task. Keep in mind that I believe firmly that the reliability performance is determined during design.
Anytime we evaluate a design it is really looking either for does it behave as expected and can we optimize performance. Both relate to reliability in the long run. By everyone in the organization recognizes they are actually doing reliability testing it improves the decision making that impact reliability performance.
Kylie Dotts says
It’s interesting how you said that life testing is essentially an accelerated way to see how long an item should last. This kind of mechanical testing would be really important to ensure that something will be durable enough to have a reasonable time that it could be used. That would also allow you to give a good expectation to your customers of about how long it will last for them.
Fred Schenkelberg says
Thanks Kylie, for the comment. Yes accelerated life testing can help answer a number of questions and provide valuable information. What is too common is running the life test without any specific objective. cheers, Fred
ian james says
How can we have a test described as accelerated life testing (HALT) that doesn’t actually assess life ? This has always confused me.
Fred Schenkelberg says
I agree the name of the process does not lend itself to understanding the process. Hence I have deemed it the second worse four-letter acronym we use.
Aggrey Obonyo says
Very informative article