Steven Wachs — Active Contributor

As an industrial statistics consultant for the past 25 years, I have frequently fielded questions related to sample size determination.  Unfortunately, I have encountered many instances where simple rules of thumb were used for any purpose (like always use 30).  Sample size guidance really depends on what the goal of the study is, the type of data we are dealing with, what statistical method we are using and some other factors as well.  Common activities which typically require sample size determination include:

• Hypothesis Testing (including Equivalence Testing)
• Estimation of statistics like means, standard deviations, proportions
• Calculation of Tolerance Intervals (range of data a process uses)
• Designed Experiments (number of replicates)
• Statistical Process Control Charts (e.g. X-bar charts)
• Acceptance Sampling (to disposition lots or batches of raw materials or finished products)
• Reliability Testing to estimate Reliability performance
• Reliability Testing to demonstrate Reliability performance

All these applications require different assumptions and calculations to determine an appropriate sample size.  In this article, we focus on Sample Size determination for Hypothesis Testing.  It is assumed that the reader is already familiar with Hypothesis Testing.

Stability studies are used to understand and model the degradation of key product characteristics over time.  They are often used to determine the product’s shelf life (the length of time a product may be stored without becoming unfit for use or consumption).

Shelf-Life studies should identify the potential “failure modes” and how they will assessed/ measured.  Examples of characteristics that are measured often include appearance attributes, texture, taste, microbial counts, and product effectiveness/performance.

(and Why Design of Experiments is so Superior)

In my 30-year career as an Industrial Statistics consultant, I have frequently been told by clients that they have performed Design of Experiments (DOEs), to try and resolve design or manufacturing issues.  What has become clear is that many engineers and scientists apply a rather liberal definition to DOE and include any type of experimentation in what they deem to be “DOE”.  

The reality is, simplistic or haphazard “experiments” rarely are effective in solving problems, especially complex ones.  Statistically based DOE provides several advantages over more simplistic approaches such “trial and error” or “one-factor-at-a-time” experimentation.  These advantages include:

• The use of statistical methodology (hypothesis testing) to determine which factors have a statistically significant effect on the response(s)
• Balanced experimental designs to allow stronger conclusions with respect to cause-and-effect relationships (as opposed to just finding correlations)
• The ability to understand and estimate interactions between factors
• The development of predictive models that are used to find optimal solutions for one or more responses

Each of these advantages are discussed in a bit more detail below.

Many statistical tests and procedures assume that data follows a normal (bell-shaped) distribution.

For example, all of the following statistical tests, statistics, or methods assume that data is normally distributed:

• Hypothesis tests such as t-tests, Chi-Square tests, F tests
• Analysis of Variance (ANOVA)
• Least Squares Regression
• Control Charts of Individuals with 3-sigma limits
• Common formulas for process capability indices such as C_p and C_pk

[Read more…]

Most manufacturers would rate product quality as a key driver of their overall ability to satisfy customers and compete in a global market.  Poor quality is simply not tolerated. It follows that manufacturers require objective measures of their product quality.  While many companies still think of quality as “being in specification,” progressive companies focus on reducing variation to minimize waste and produce products that perform consistently well over time.  Quality may be thought of as inversely proportional to variation–that is, as variation increases, product quality decreases. [Read more…]

Most quality professionals are familiar with basic hypothesis tests such as the 2-sample t test.  However, depending on the goals of the study, another type of test, called an equivalence test, may be utilized instead of traditional hypothesis tests.  This article will review statistical hypothesis testing in general and then introduce equivalence testing and its application.  To illustrate the differences between traditional hypothesis tests and equivalence tests, we will focus on the case of comparing 2 independent samples.  The concepts may be easily extended to other situations (such a comparing a sample to a target or paired comparisons).   [Read more…]

SPC and Reliability

We often think of Statistical Process Control as a tool to help drive product quality by informing us when process changes occur.  By systematically detecting (and rectifying) sources of special cause variation upstream in the process, the important process outcomes become predictable.  Furthermore, a focus on reducing common cause variation drives higher levels of process capability and more consistent product performance. [Read more…]

In order to maximize profitability while complying with government regulations regarding net package contents, food manufacturers and packagers must achieve an optimal balance.  Consistent overfilling to minimize risk is inefficient and sacrifices profitability, while aggressive filling practices result in significant risks of non-compliance with net contents regulations leading to potential penalties, loss of reputation, and impaired customer relations.  Statistical process control and process capability methods may be utilized to determine optimal targets for product fill weights or volumes for a given process.  Subsequent focused efforts to minimize variation will allow the target to be further optimized, resulting in less waste without compromising risk. [Read more…]

The purpose of control charting is to regularly monitor a process so that significant process changes may be detected.  These process changes may be a shift in the process average (Xbar) or a change in the amount of variation in the process.  The variation observed when the process is operating normally is called common cause variation.  When a process change occurs, then special cause variation occurs. [Read more…]

Why Measurement Systems Assessment (MSA)?

Effective use of data to drive decision making requires adequate measurement systems. For example, when implementing statistical process control charts, we assume that a signal represents a significant change in the process and we react as such. However, inadequate measurement systems may result in inappropriate signals or even worse, charts that fail to detect important process changes. Thus, it is incumbent upon us to ensure that measurement systems are adequate for their intended use via proper assessments prior to their use. Only capable measurement systems should be utilized in data based methods such as Statistical Process Control, Design of Experiments, Inspection activities, etc.  [Read more…]

Background

More companies are leveraging high speed vision systems to inspect multiple quality characteristics on their products.  

For example, in a high volume baking operation, a vision system can test for bun height, bun length, slice thickness, topping distribution, surface color, and more.  This happens automatically on the line at high speeds.  In bottling or other plastic manufacturing, a vision system may inspect multiple dimensions and surface properties.   [Read more…]

In Part I of this article, we introduced the concept of utilizing Deviation from Nominal (DNOM) control charts for short production runs.  These charts allow us to monitor process characteristics over time even when the units being controlled have varying nominal values.  DNOM charts assume that the process variability (i.e. standard deviation) does not vary significantly by part type.  However, often this assumption does not hold.  Characteristics with larger nominal values tend to have more variation than characteristics with smaller nominal values.  In Part II we discuss how to test whether or not significant differences in variability exist and if so, how to modify the DNOM methods and charts to handle this situation. [Read more…]

In the last Article, we explored the use of contour plots and other tools (such as a response optimizer) to help us quickly find solutions to our models.  In this article, we will look at the uncertainty in these predictions.  We will also discuss model validation to ensure that technical assumptions that are inherent in the modeling process is satisfied. [Read more…]

In the last Article, we learned how to work with predictive models to find solutions that solve for desired responses.  We used some basic algebra to solve for solutions and looked at the use of contour plots to quickly visualize many solutions at a glance.

In this article, we further explore the use of contour plots and other tools to help us quickly find solutions to our models.  We start by revisiting the battery life DOE example that was discussed in the previous article.  The statistical output below shows the coded model that contains only the statistically significant (main and interaction) effects. [Read more…]

Traditional SPC methods were developed to support high volume production and long production runs.  However, with the trend toward product specialization, product diversity, and flexible manufacturing, short production runs have become more common.  Applying SPC in the traditional manner presents challenges in short production runs, because by the time enough data is collected to establish valid control charts, the production run may be over! [Read more…]