Accendo Reliability

Your Reliability Engineering Professional Development Site

  • Home
  • About
    • Contributors
  • Reliability.fm
    • Speaking Of Reliability
    • Rooted in Reliability: The Plant Performance Podcast
    • Quality during Design
    • Critical Talks
    • Dare to Know
    • Maintenance Disrupted
    • The Leadership Connection
    • Practical Reliability Podcast
    • Reliability Matters
    • Reliability it Matters
    • Maintenance Mavericks Podcast
    • Women in Maintenance
    • Accendo Reliability Webinar Series
    • Asset Reliability @ Work
  • Articles
    • CRE Preparation Notes
    • on Leadership & Career
      • Advanced Engineering Culture
      • Engineering Leadership
      • Managing in the 2000s
      • Product Development and Process Improvement
    • on Maintenance Reliability
      • Aasan Asset Management
      • CMMS and Reliability
      • Conscious Asset
      • EAM & CMMS
      • Everyday RCM
      • History of Maintenance Management
      • Life Cycle Asset Management
      • Maintenance and Reliability
      • Maintenance Management
      • Plant Maintenance
      • Process Plant Reliability Engineering
      • ReliabilityXperience
      • RCM Blitz®
      • Rob’s Reliability Project
      • The Intelligent Transformer Blog
    • on Product Reliability
      • Accelerated Reliability
      • Achieving the Benefits of Reliability
      • Apex Ridge
      • Metals Engineering and Product Reliability
      • Musings on Reliability and Maintenance Topics
      • Reliability Engineering Insights
      • Reliability in Emerging Technology
    • on Risk & Safety
      • CERM® Risk Insights
      • Equipment Risk and Reliability in Downhole Applications
      • Operational Risk Process Safety
    • on Systems Thinking
      • Communicating with FINESSE
      • The RCA
    • on Tools & Techniques
      • Big Data & Analytics
      • Experimental Design for NPD
      • Innovative Thinking in Reliability and Durability
      • Inside and Beyond HALT
      • Inside FMEA
      • Integral Concepts
      • Learning from Failures
      • Progress in Field Reliability?
      • Reliability Engineering Using Python
      • Reliability Reflections
      • Testing 1 2 3
      • The Manufacturing Academy
  • eBooks
    • Reliability Engineering Management DRAFT
  • Resources
    • Accendo Authors
    • FMEA Resources
    • Feed Forward Publications
    • Openings
    • Books
    • Webinars
    • Journals
    • Higher Education
    • Podcasts
  • Groups
    • Reliability Integration
    • Mastermind
    • Study Groups
  • Courses
    • 14 Ways to Acquire Reliability Engineering Knowledge
    • Reliability Analysis Methods online course
    • Measurement System Assessment
    • SPC-Process Capability Course
    • Design of Experiments
    • Foundations of RCM Online Course
    • Reliability Engineering Statistics
    • An Introduction to Reliability Engineering
    • Process Capability Analysis course
    • Root Cause Analysis and the 8D Corrective Action Process course
    • Return on Investment online course
    • CRE Preparation Online Course
    • Quondam Courses
  • Webinars
    • Upcoming Live Events
  • Calendar
    • Call for Papers Listing
    • Upcoming Webinars
    • Webinar Calendar
  • Login
    • Member Home

by Dianna Deeney Leave a Comment

QDD 045 Choosing a Confidence Level for Test using FMEA

Choosing a Confidence Level for Test using FMEA

We’re developing requirements for our product, including setting reliability requirements. Or we’re setting acceptance criteria for our test plans.

What confidence levels do we choose? We don’t have to blindly set them – we can base it off the risks of failure, using our FMEA (failure mode effects analysis).

 

 

View the Episode Transcript

 

We should choose a confidence level for our requirements or their test plans. We can associate that confidence level with the level of risk associated with our product. FMEA is a great tool for us to refer to, to help us choose a relevant confidence level by basing our decision on one or more metrics that the FMEA can provide.

Citations

Previous episodes that are related to this topic:

Episode 27: How many controls do we need to reduce risk? 

Episode 31: 5 Aspects of Good Reliability Goals and Requirements 

 

Episode Transcript

We are developing requirements for our product, including setting reliability requirements and its confidence levels or we’re setting acceptance criteria for our test plans. What confidence levels do we choose? We don’t have to blindly set them. We can base it off the risks of failure. I’ll tell you how after this brief intro.

Hello and welcome to Quality during Design, the place to use quality thinking to create products others love for less. My name is Dianna. I’m a senior level quality professional and engineer with over 20 years of experience in manufacturing and design. Listen in and then join the conversation at qualityduringdesign.com.

Before testing anything, we need to choose what confidence level we want to have in the results. We need to do this because there’s variation in everything in the way that we measure and test. The way that we manufacture introduces variation, including the raw materials that we, and the tools we’re using to make it. Setting a confidence level accounts for the variability we’re going to see in our test data. A confidence level is used in determining the sample size to test. If we want to make statements about how the design will perform in the field, then we need to test a sample size that statistically relevant, where we can use statistics to help us predict the performance in the field from a few tested in the lab.

Usually confidence levels are 90%, 95% or 99%. Why don’t we take the most conservative approach and just pick a 99% confidence level where that may save us time and having to think about it, it wastes a lot of time and resources later. The higher, the confidence level, the more likely we’ll need to test lots of samples. And that means making units for test, testing them all in the lab, and then having a more complex analysis.

Instead, one way we can choose a confidence level that we want for test is to correlate it with the risk of failure associated with it. Our product requirement is likely a control for a potential failure. What was the origin of our requirement? Why did we set it in the first place? What performance or characteristic of the final design is it controlling? If our product doesn’t meet this requirement, what are the ways that it can fail? If we have an FMEA, we can find the place in the table where our requirement is a control or where it’s associated with a potential failure mode and cause. When we find it, then we have a lot of metrics we can use to help us decide on a level of confidence to test based on risk. And if we’ve done our FMEA earlier, then we would’ve had it populated with information from our cross-functional team in a time of cool heads, without the pressures of project management. It will be an objective input into what confidence level we should require for our test.

What are the potential effects of this failure mode? In other words, what type of harm to the user, environment, or performance of the product is possible? Are there many effects listed or just one? If there are many effects, we may want a higher confidence level. What is the severity ranking of the effect? Is it high or is it low? The higher, the severity ranking, the more likely we should choose a higher confidence level. What other controls are in place besides our requirement? And what is the detection ranking? If this requirement is the only control, or if it’s the strongest control, then we may want to choose a higher confidence level. We could also use this information to justify a lower confidence level. If we have a requirement that’s associated with a failure that has one effect, that effect is not severe, and there are two other controls associated with that same cause, then maybe we’ll choose a lower confidence level.
What’s today’s insight to action, we should choose a confidence level for our requirements or their test plans. We can associate that confidence level with the level of risk of our product. FMEA is a great tool to refer to, to help us choose a relevant confidence level for our tests.

If this episode is helpful to you, I recommend two other previous Quality during Design episodes. Episode 27, How many Controls do we Need to Reduce Risk talks more about the controls that we use in an FMEA to control a risk. Episode 31, 5 Aspects of Good Reliability Goals and Requirements, talks about why we want a confidence level associated with our requirement.

Please go to my website at quality during design.com. You can visit me there and it also has a catalog of resources, including all the podcasts and their transcripts. Use the subscribe forms to join the weekly newsletter, where I share more insights and links in your podcast app. Make sure you subscribe or follow quality during design to get all the episodes and get notified when new ones are posted. This has been a production of Deeney Enterprises. Thanks for listening!

 

Filed Under: Quality during Design, The Reliability FM network

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Quality during Design podcast logo

Tips for using quality tools and methods to help you design products others love, for less.


by Dianna Deeney
Quality during Design,
Hosted on Buzzsprout.com
Subscribe and enjoy every episode
Google
Apple
Spotify

Recent Episodes

QDD 062 How Does Reliability Engineering Affect (Not Just Assess) Design?

QDD 061 How to use FMEA for Complaint Investigation

QDD 060 3 Tips for Planning Design Reviews

QDD 059 Product Design from a Marketing Viewpoint, with Laura Krick (A Chat with Cross Functional Experts)

QDD 058 UFMEA vs. DFMEA

QDD 057 Design Input & Specs vs. Test & Measure Capability

QDD 056 ALT vs. HALT

QDD 055 Quality as a Strategic Asset vs. Quality as a Control

QDD 054 Design Specs vs. Process Control, Capability, and SPC

QDD 053 Internal Customers vs. External Customers

QDD 052 Discrete Data vs. Continuous Data

QDD 051 Prevention Controls vs. Detection Controls

QDD 050 Try this Method to Help with Complex Decisions (DMRCS)

QDD 049 Overlapping Ideas: Quality, Reliability, and Safety

QDD 048 Using SIPOC to Get Started

QDD 047 Risk Barriers as Swiss Cheese?

QDD 046 Environmental Stress Testing for Robust Designs

QDD 045 Choosing a Confidence Level for Test using FMEA

QDD 044 Getting Started with FMEA – It All Begins with a Plan

QDD 043 How can 8D help Solve my Recurring Problem?

QDD 042 Mistake-Proofing – The Poka-Yoke of Usability

QDD 041 Getting Comfortable with using Reliability Results

QDD 040 How to Self-Advocate for More Customer Face Time (and why it’s important)

QDD 039 Choosing Quality Tools (Mind Map vs. Flowchart vs. Spaghetti Diagram)

QDD 038 The DFE Part of DFX (Design For Environment and eXcellence)

QDD 037 Results-Driven Decisions, Faster: Accelerated Stress Testing as a Reliability Life Test

QDD 036 When to use DOE (Design of Experiments)?

QDD 035 Design for User Tasks using an Urgent/Important Matrix

QDD 034 Statistical vs. Practical Significance

QDD 033 How Many Do We Need To Test?

QDD 032 Life Cycle Costing for Product Design Choices

QDD 031 5 Aspects of Good Reliability Goals and Requirements

QDD 030 Using Failure Rate Functions to Drive Early Design Decisions

QDD 029 Types of Design Analyses possible with User Process Flowcharts

QDD 028 Design Tolerances Based on Economics (Using the Taguchi Loss Function)

QDD 027 How Many Controls do we Need to Reduce Risk?

QDD 026 Solving Symptoms Instead of Causes?

QDD 025 Do you have SMART ACORN objectives?

QDD 024 Why Look to Standards

QDD 023 Getting the Voice of the Customer

QDD 022 The Way We Test Matters

QDD 021 Designing Specs for QA

QDD 020 Every Failure is a Gift

QDD 019 Understanding the Purposes behind Kaizen

QDD 018 Fishbone Diagram: A Supertool to Understand Problems, Potential Solutions, and Goals

QDD 017 What is ‘Production Equivalent’ and Why Does it Matter?

QDD 016 About Visual Quality Standards

QDD 015 Using the Pareto Principle and Avoiding Common Pitfalls

QDD 014 The Who’s Who of your Quality Team

QDD 013 When it’s Not Normal: How to Choose from a Library of Distributions

QDD 012 What are TQM, QFD, Six Sigma, and Lean?

QDD 011 The Designer’s Important Influence on Monitoring After Launch

QDD 010 How to Handle Competing Failure Modes

QDD 009 About Using Slide Decks for Technical Design Reviews

QDD 008 Remaking Risk-Based Decisions: Allowing Ourselves to Change our Minds.

QDD 007 Need to innovate? Stop brainstorming and try a systematic approach.

QDD 006 HALT! Watch out for that weakest link

QDD 005 The Designer’s Risk Analysis affects Business, Projects, and Suppliers

QDD 004 A big failure and too many causes? Try this analysis.

QDD 003 Why Your Design Inputs Need to Include Quality & Reliability

QDD 002 My product works. Why don’t they want it?

QDD 001 How to Choose the Right Improvement Model

© 2022 FMS Reliability · Privacy Policy · Terms of Service · Cookies Policy

This site uses cookies to give you a better experience, analyze site traffic, and gain insight to products or offers that may interest you. By continuing, you consent to the use of cookies. Learn how we use cookies, how they work, and how to set your browser preferences by reading our Cookies Policy.