Accendo Reliability

Your Reliability Engineering Professional Development Site

  • Home
  • About
    • Contributors
  • Reliability.fm
    • Speaking Of Reliability
    • Rooted in Reliability: The Plant Performance Podcast
    • Quality during Design
    • Critical Talks
    • Dare to Know
    • Maintenance Disrupted
    • Metal Conversations
    • The Leadership Connection
    • Practical Reliability Podcast
    • Reliability Matters
    • Reliability it Matters
    • Maintenance Mavericks Podcast
    • Women in Maintenance
    • Accendo Reliability Webinar Series
    • Asset Reliability @ Work
  • Articles
    • CRE Preparation Notes
    • on Leadership & Career
      • Advanced Engineering Culture
      • Engineering Leadership
      • Managing in the 2000s
      • Product Development and Process Improvement
    • on Maintenance Reliability
      • Aasan Asset Management
      • CMMS and Reliability
      • Conscious Asset
      • EAM & CMMS
      • Everyday RCM
      • History of Maintenance Management
      • Life Cycle Asset Management
      • Maintenance and Reliability
      • Maintenance Management
      • Plant Maintenance
      • Process Plant Reliability Engineering
      • ReliabilityXperience
      • RCM Blitz®
      • Rob’s Reliability Project
      • The Intelligent Transformer Blog
    • on Product Reliability
      • Accelerated Reliability
      • Achieving the Benefits of Reliability
      • Apex Ridge
      • Metals Engineering and Product Reliability
      • Musings on Reliability and Maintenance Topics
      • Product Validation
      • Reliability Engineering Insights
      • Reliability in Emerging Technology
    • on Risk & Safety
      • CERM® Risk Insights
      • Equipment Risk and Reliability in Downhole Applications
      • Operational Risk Process Safety
    • on Systems Thinking
      • Communicating with FINESSE
      • The RCA
    • on Tools & Techniques
      • Big Data & Analytics
      • Experimental Design for NPD
      • Innovative Thinking in Reliability and Durability
      • Inside and Beyond HALT
      • Inside FMEA
      • Integral Concepts
      • Learning from Failures
      • Progress in Field Reliability?
      • Reliability Engineering Using Python
      • Reliability Reflections
      • Testing 1 2 3
      • The Manufacturing Academy
  • eBooks
  • Resources
    • Accendo Authors
    • FMEA Resources
    • Feed Forward Publications
    • Openings
    • Books
    • Webinars
    • Journals
    • Higher Education
    • Podcasts
  • Courses
    • 14 Ways to Acquire Reliability Engineering Knowledge
    • Reliability Analysis Methods online course
    • Measurement System Assessment
    • SPC-Process Capability Course
    • Design of Experiments
    • Foundations of RCM online course
    • Quality during Design Journey
    • Reliability Engineering Statistics
    • Quality Engineering Statistics
    • An Introduction to Reliability Engineering
    • An Introduction to Quality Engineering
    • Process Capability Analysis course
    • Root Cause Analysis and the 8D Corrective Action Process course
    • Return on Investment online course
    • CRE Preparation Online Course
    • Quondam Courses
  • Webinars
    • Upcoming Live Events
  • Calendar
    • Call for Papers Listing
    • Upcoming Webinars
    • Webinar Calendar
  • Login
    • Member Home

by Fred Schenkelberg 1 Comment

Beware of the Type III Error

Beware of the Type III Error

There is a type of error when conducting statistical testing that is to work very hard to correctly answer the wrong question. This error occurs during the formation of the experiment.

Despite creating a perfect null and alternative hypothesis, sometimes we are investigating the wrong question.

Example of a Type III Error

Let’s say we really want to select the best vendor for a critical component of our design. We define the best vendor as their solution or component is the most durable. Ok, we can set up an experiment to determine which vendor provides a solution that is the most durable.

We set up and conduct a flawless hypothesis test to compare the two leading solutions. We can very clear results. Vendor A’s solution is statistically significantly more durable than Vendor B’s solution.

Yet, neither solution is durable enough. We should have been evaluating if either solution could meet our reliability requirements instead.

Oops.

Even if we perfectly answer a question in our work, if it’s not the right question, then the work is for naught.

Short History of Type III Errors

Neyman and Pearson used the terminology for type I and II errors as “error of the first kind” and “errors of the second kind”, respectively. This led others to consider other types of errors, naming them “errors of the third kind” and so forth.

An occasional college of Neyman and Pearson, F.N. David, in a paper published in 1947, suggested she may have a need to extend the Neyman and Pearson’s sources of error to a third source by possibly “choosing the test falsely to suit the significance of the sample.”

Mosteller in 1948 defined Type III error as “correctly rejecting the null hypothesis for the wrong reason.”

Extending Mosteller’s definition, Kaiser in 1966, by defining such a Type III error as the coming to an “incorrect decision of direction following a rejected two-tailed test of hypothesis”.

Kimball, in 1957, suggested a definition close to how I consider a Type III error, as “the error committed by giving the right answer to the wrong problem”.

and so on…

There is no one widely accepted definition for an error of the third kind nor for Type III errors. Yet, for any of the above definitions, the error is one to guard against making by careful consideration when designing, conducting, and analyzing statistical tests.

A Few Pitfall Situations that Lead to Type III Errors

An obvious situation, in hindsight, is the experimenter solving the wrong problem or asking the wrong question. The cause here could be simple ignorance of sufficient information to recognize the error. Another cause could be focusing on the first or most interesting question to investigate.

Another set of situations may be the deliberate or unconscious effort to connect the experimental results to an expected outcome. This sometimes occurs when reinterpreting the results when the results do not agree with the desired outcome.

Another set includes the process of just doing what we always have done. In this case, the experimenter may not even have a connection between the experiment and a suitable hypothesis that would enable analysis. We can do a test or experiment perfectly well, yet it has no meaningful result or influence on any future work.

Yet other situations exist. If you spot one or more that I missed, please add your thoughts in the comment section below.

Filed Under: Articles, CRE Preparation Notes, Probability and Statistics for Reliability

« Don’t Healthcare Workers Fatigue Like Anyone Else?
To Achieve Your Equipment Reliability Goals, Begin at the Beginning… »

Comments

  1. Larry George says

    February 14, 2021 at 8:17 PM

    Prove any null hypothesis!
    Restrict the alternative hypotheses to unlikely possibilities and you can prove practically any null hypothesis. A typical application of this cheat that I learned in school is to test constant failure rate against the alternative of an increasing failure rate. That alternative hypothesis leaves out failure rates that may increase for some ages and decrease for others, a more likely alternative hypothesis than a monotonically increasing failure rate.
    This is an example of Type III error, population misspecification. Ronald Fisher recognized the job of a statistician as:
    1. specification of the kind of population that the data came from
    2. estimation
    3. distribution specification.
    By specification, Fisher meant that the statistical distribution(s) involved should encompass both the null and the alternative hypotheses.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

CRE Preparation Notes

Article by Fred Schenkelberg

Join Accendo

Join our members-only community for full access to exclusive eBooks, webinars, training, and more.

It’s free and only takes a minute.

Get Full Site Access

Not ready to join?
Stay current on new articles, podcasts, webinars, courses and more added to the Accendo Reliability website each week.
No membership required to subscribe.

  • CRE Preparation Notes
  • CRE Prep
  • Reliability Management
  • Probability and Statistics for Reliability
  • Reliability in Design and Development
  • Reliability Modeling and Predictions
  • Reliability Testing
  • Maintainability and Availability
  • Data Collection and Use

© 2023 FMS Reliability · Privacy Policy · Terms of Service · Cookies Policy

This site uses cookies to give you a better experience, analyze site traffic, and gain insight to products or offers that may interest you. By continuing, you consent to the use of cookies. Learn how we use cookies, how they work, and how to set your browser preferences by reading our Cookies Policy.