Module 11 Logistic Regression (Discrete Responses)

Lesson M11-07

Duration: 17 minutes

Finally, the Nominal Logistic Regression model is discussed.  Here we have a categorical output with no natural ordering.  We can develop a series of logit equations for each outcome relative to a reference level of the response. We present an example to illustrate the Nominal model.  

Advanced Hypothesis Testing & Regression Reference Textbook – Section 3 pages 51-60

Module 11 Logistic Regression (Discrete Responses)

Lesson M11-05

Duration: 19 minutes

In Lesson 5, we extend the model to handle an Ordinal outcome. These are multiple categories that have a natural order to them (like a 1-5 survey response).  In the ordinal model, we develop a series of logit expressions that are similar to the binary approach.  Again, these logit expressions may be manipulated to develop expressions for the probability of falling into a specific category or less.  

Advanced Hypothesis Testing & Regression Reference Textbook – Section 3 pages 36-39

Module 11 Logistic Regression (Discrete Responses)

Lesson M11-04

Duration: 44 minutes

Measures of association are useful for quantifying the predictive ability of the model.  By forming pairs of rows with differing responses, we can classify pairs as concordant or discordant.  A concordant pair results when the model predicts a higher probability of having the response category that it actually had.  A discordant pair is the reverse.  The higher the percentage of concordant pairs, the better the model predicts the data that was observed.  Several statistics are defined that are based on the number of concordant and discordant pairs.  Deviance residuals are also discussed and we illustrate how to interpret the residual plots (this is different from simple and multiple linear regression).  We complete the lesson with couple of exercises.

 Advanced Hypothesis Testing & Regression Reference Textbook – Section 3 pages 31-35

Module 11 Logistic Regression (Discrete Responses)

Lesson M11-03

Duration: 26 minutes

Here, we focus on building a binary logistic regression model. We present an example and show how to analyze the data in Minitab and understand the output. As with the earlier regression models we covered, we need to determine which predictors are statistically significant and specify a reduced model as appropriate. Main effects plots may be used to help visualize how the significant predictors influence the response.  We also discuss the “Goodness of Fit” tests to determine any issues with lack of fit of the model to the data.

Advanced Hypothesis Testing & Regression Reference Textbook – Section 3 pages 17-30

Module 11 Logistic Regression (Discrete Responses)

Lesson M11-02

Duration: 30 minutes

Lesson 2 focuses on the Binary Logistic Regression Model.  The response is the Log(odds) of a response being in the category of interest (e.g. failure).  The log(odds) allow us to map a probability value (0-1) into the real number line which is what the right hand side of the regression model can produce.  Before learning how to build the model, we focus on how to interpret the model coefficients.  Specifically, we define and interpret the odds ratio, which makes the interpretation more natural. From the regression equation we can use algebra to produce expressions for the probability of an event occurring.  This allows us to visualize the model results in a meaningful way.  Several examples are provided.

Advanced Hypothesis Testing & Regression Reference Textbook – Section 3 pages 5-16

Module 11 Logistic Regression (Discrete Responses)

Lesson M11-01

Duration: 17 minutes

In the first lesson, we cover some basic concepts and terminology used in Logistic Regression.  In this type of model, the response is a function of the probability of “success” The link function is discussed and several examples of possible link functions are highlighted.

Advanced Hypothesis Testing & Regression Reference Textbook Section 3 pages 1-5

Module 10 Multiple Linear Regression (Multiple Factors)

Lesson M10-04

Duration: 43 minutes

Finally, we learn how to incorporate discrete predictors into the model.  These could be binary, ordinal, or nominal variables that work differently than continuous predictors.  Minitab can create the necessary indicator variables in the model to incorporate these types of predictors.  We provide an example to illustrate the process.  Finally, we review the whole module through a comprehensive exercise. 

Statistics, Hypothesis Testing, & Regression Reference Textbook – Section 9 pages 41-50

Module 10 Multiple Linear Regression (Multiple Factors)

Lesson M10-03

Duration: 24 minutes

Multi-collnearity occurs when we include multiple correlated predictors in the model.  This can produce confusing results and unreliable p-values.  We discuss checks that can be done to avoid this issue and as discussed previously, the use of Stepwise regression will avoid this.  It is also important to be aware of highly influential data points that significantly influence the regression model.  Anytime a single point has undue influence, there is a risk that the results can be misleading due to errors, unusual results, etc.  We clarify the difference between outliers (high residuals) and influential data points.

Statistics, Hypothesis Testing, & Regression Reference Textbook – Section 9 pages 31-40

Module 10 Multiple Linear Regression (Multiple Factors)

Lesson M10-02

Duration: 26 minutes

In Lesson 2, we discuss the use of Best Subsets Regression and Forward/Backward/Stepwise regression procedures.  These algorithms are very helpful when we are considering many predictors for inclusion in the model.  They also allow us to avoid Multi-Collinearity which is discussed in the next lesson. As in the previous module, we discuss the prediction of average responses and individual responses from a specific combination of predictor values. 

Statistics, Hypothesis Testing, & Regression Reference Textbook – Section 9 pages 21-30

Module 10 Multiple Linear Regression (Multiple Factors)

Lesson M10-01

Duration: 22 minutes

Here we focus on developing model with multiple predictors.  As before once we have a model, we can use it to predict outcomes. We learn how to use the computer output to test the overall significance of the regression model as well as determine which specific terms are significant (to be left in the model).  Model validation (residual analysis) is also covered as in the Simple Linear Regression case. 

Statistics, Hypothesis Testing, & Regression Reference Textbook – Section 9 pages 1-20

Module 9 Simple Linear Regression (Single Factor)

Lesson M09-06

Duration: 36 minutes

The last Lesson includes a useful application of Simple Linear Regression.  Here, we use the modeling methodology to determine an appropriate shelf life for the product (assuming we have a product that can change over time). We have to identify a specification limit as well as the percentage of the population that we allow to not be in specification at the shelf life time.  One difference is that we will typically use a regression line other than the mean value, based on what proportion we can tolerate not meeting specification.  Perhaps we will allow 5% or 1% of the products to not meet specification at the shelf-life time.  We end with an example and an approach for handling a non-linear degradation over time.  

Module 9 Simple Linear Regression (Single Factor)

Lesson M09-05

Duration: 56 minutes

Another Simple Linear Regression example is presented.  We also learn how to make model predictions by using the model equation.  We can get a confidence interval for the predicted mean (of a normal distribution) which is what the model is actually predicting.  Furthermore, we can obtain a Prediction Interval which predicts the range over which we expect a high percentage of the individual responsed will fall for a given predictor value.  The prediction interval will be wider because it reflects the expected variation among individual response as compared to the uncertainty in the average value (which has less variation).  The lesson ends with a comprehensive exercise to develop, validate, and predict using a simple linear regression model.

Statistics, Hypothesis Testing, & Regression Reference Textbook Section 8 pages 29-39

Module 9 Simple Linear Regression (Single Factor)

Lesson M09-04

Duration: 8 minutes

We illustrate how to obtain the key Residual plots in Minitab.  We also show some examples that highlight cases where the residual pattern violates the modeling assumptions. 

Statistics, Hypothesis Testing, & Regression Reference Textbook – Section 8 pages 20-28