edited by John Healy
This theorem is a fundamental element of probability theory. The law is basically that if one conducts the same experiment a large number of times the average of the results should be close to the expected value. Furthermore, the more trails conducted the closer the resulting average will be to the expected value.
This is why casinos win in the long term. Even with a slight benefit of the odds in the game, in the long term, the results of all the bets and chances will reflect the odds. Sure there is variation, sometimes you win, yet on average with many attempts, the house wins.
The experiments or trials are from the same setup or game. The same underlying phenomena creates the results for each trial. If using a fair 6-sided die on average the number of times 6 spots will appear on top is the same as any other side of the die. Calculating the average of a large number of die rolls, we will eventually converge to a result of 3.5, which is the expected value.
If we use a different unfair die for each roll, then all bets are off and the law of large numbers will not apply.
If observing a series of random events, say the flipping of a fair coin, and we note that the last two tosses resulted in heads, we may expect (incorrectly due to a misinterpretation of the law of large numbers) that the next flip of the coin will result in tails. As we know a fair coin has a 50% chance on each flip to land on heads or tails, therefore, for any given set of trials the results should be evenly split between the two outcomes.
The issue is, no one told the coin. The coin flip result is the result of the many small variations that occur to find it’s landing position. It has no memory or is not considering the tally of previous flips. It is just flying through the air and landing on one side or the other based on this one trial.
With a run of say 5 tosses resulting in heads, there is a 1/32 chance of that occurring. The chance of the next toss being heads or tails remains unchanged, it is 1/2 heads and 1/2 tails. We may wish the result to be a tails given the run of 5 heads, yet that is only our belief, not the probability.
If we repeat the experiment of tossing a coin five times many times (many being a large number – say thousands or millions of 5 toss experiments), the result on average would be 2.5 heads per group of five tosses. For any single toss, the result is unknown with equal chance of a heads or tails, whether the first toss or the 1 millionth. None of the many trials provide any information about the results of sixth toss other than it remains 50/50.
Run Test for Randomness (article)
Statistical Terms (article)
binomial probability density function (article)