What is MLE?
Abstract
Chris and Fred discuss the three-letter acronym ‘MLE’ stands for? Well, it stands for ‘maximum likelihood estimate.’ Ever heard of it? Do you know what it means?
Key Points
Join Chris and Fred as they discuss what the MLE or ‘maximum likelihood estimate’ means … usually when using software to conduct data analysis.
Topics include:
- What does ‘likelihood’ mean? Most models (like the bell curve, lognormal distribution, and so on) are defined by parameters. For example, the two parameters that define the bell curve are (1) the mean and (2) standard deviation. So these two parameters entirely describe the shape of the bell curve. Now let’s say you have 20 data points from some random process. What is the ‘likelihood’ that a bell curve with a mean of 10 and a standard deviation of 2 ‘fits’ your data? It has been found that if we plot out all 20 data points under this bell curve we think might fit the data, and then draw lines up from each data point to the bell curve shape … and then multiply those heights – we get the ‘likelihood’ that the bell curve fits your data. Or perhaps more correctly, we get a number that represents the ‘likelihood’ that the bell curve models the random process that gave us those 2o data points.
- What does ‘maximum likelihood estimate’ mean? Let’s say we are not sure if the bell curve above, or another bell curve with a mean of 9.5 and standard deviation of 1.5 is a good fit. Well … we can go through the same process for the other bell cruve, and come up with another number that represents its ‘likelihood.’ Now we can try all possible combinations of means and standard deviations and come up with likelihoods for each. Computers are good at this, and they can (for example) work out that a bell curve with a mean of 9.76234… and standard deviation of 1.8986… will have the highest likelihood of all possible bell curves. This is the ‘maximum likelihood estimate.’
- It’s just another way of coming up with our ‘best fit’ or ‘best guess.’ Other approaches involve ‘regression analysis’ or ‘root mean square analysis’ where we find a line that minimizes the squares of the ‘residual’ distances between candidate lines of best fit and the data points.
- So my software package allows me to choose from lots of different options … including MLE … which should we use? WHAT DECISION ARE YOU TRYING TO MAKE? There are no absolute guidelines that work for every scenario. The first question you should ask is … do you need the ‘best guess’ of something or the region within which you are ‘confident’ that something lies? For example, the ‘best guess’ at your item’s time to failure might be 5.4 years. But you might only be 95 % confident that the time to failure will exceed 1.9 years. If you are trying to understand the likelihood that your item will fail within a 2-year warranty period, you might not be interested in your best guess, but instead be interested in a 95 % confidence region.
Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.
SOR 991 What is MLE?Christopher Jackson
- Social:
- Link:
- Embed:
I believe that the choice between RRX and RRY should follow where the uncertainty lies. If the time to failure is random (failure is always random!), then the uncertainty lies on the x-axis and RRX should be chosen. If the time is known, maybe in left censored data(?), then RRY should be used since you are considering those failures at a specific time. Never used it myself! Always MLE and RRX.
How I always made the decision between MLE and RRX is based on the page from Reliasoft (https://help.reliasoft.com/weibull20/weibull_standard_folio_control_panel_analysis_page.htm). Essentially, if you have suspensions or interval data, then MLE tends to be a better choice (as was mentioned in the talk). However, MLE requires a decent sample size since it becomes biased.
On the distribution selection, I do struggle with choosing between the various distributions and feeling confident in the selection. I’ve heard about various physics of failure being associated with a distribution (such as lognormal being associated with mechanical fatigue, knowledge I’m guessing was gathered empirically over time) but without that history… I tend to bias towards the Weibull distribution since it can accommodate so much. If you use Weibull distribution for everything however, it can give you a sense of “well both these samples fit the Weibull distribution so they must be comparable” but really the scale and shape parameters are vastly different.
Hi Maxamllion
Some good points here! However I will need to push back on one. Regarding your assertion that the choice between RRX and RRY should follow where the uncertainty lies … there is always uncertainty in ‘both directions’ when it comes to reliability data. For example, you say that time to failure is random (I agree) so therefore the uncertainty lies on the x-axis (happy with that). However, the y-axis always has uncertainty as well. Why? Because the y-axis represents failure probability (CDF) or reliability. And we can only ever estimate this based on the number of data points that have failed. We often use the ‘median rank’ approximation to calculate a CDF estimate that defines where on the y-axis a data point lies. And the word ‘median’ in ‘median rank’ refers to the point (based on the Binomial distribution) where we are 50 % confident that the true CDF value is above that point, and 50 % confident that the true CDF value is below that point. So long story short … there is uncertainty in each direction.
And regarding bias (no one talks about this much … except in the New Weibull handbook) RRX and RRY is all about minimizing the respective distances between the line of best fit and the data points. If you look at a Weibull plot (for example) you will see that in the bottom left hand corner, the scales are ‘tiny’. That is a ‘step taken up’ in the bottom left-hand corner might move about 1 % ‘up’ the CDF values. That same step in the middle of the Weibull plot (around a CDF value of 50 %) will see that same step account for a 10 to 20 % move ‘up’ the CDF. Same for time to failure due to the logarithmic scale. A ‘step taken right’ in the bottom left-hand corner might have a move up that is orders of magnitude smaller than the same ‘step taken right’ on the other side of the plot. What does that mean? It means that the line of best fit under RRX and RRY is biased to be way more sensitive to our data points in the bottom left hand corner.
And I hear you regarding getting the right distribution! What you can do is hedge your bets (sort of). What that means is continue using the Weibull distribution, but know (in advance) what shape parameters you should be expecting. Infant mortality? … 0.3 to 0.5. Fatigue? … 2.0 to 2.5. So if you see something violently different to what you expect … something might be wrong!
Keep the comments coming!
The bias explanation really makes sense. I’ve experienced this behavior when low flier is present. When removed, the distribution shifts significantly and the reliability prediction / calculation dramatically improves. Thanks Chris!
Glad I could help!