Title: The idea of a 'limit'
Course: UGQuants (SSPS) > Section: Probability

The idea of a 'limit'

Probability can be calculated theoretically as well as measured empirically. If we can know all possible outcomes of a trial in advance, we can calculate the limit to which we expect the result of a number of repeated trials to approach. As the number of trials increases, so the ratio of successes to failures tends to a limit.

For example, think about tossing a coin. This apparently simple act contains within it almost all we need to know about probability. It is an event with only two possible outcomes which occur randomly: that is, we cannot predict whether we will get heads or tails in advance. We know that (if the coin is unbiased) the probability of success equals the probability of failure. However, although we expect the chance of a head or tail to be the same (i.e. p = 0.5) we can no more obtain half a head or tail than find .43 of a man! On any individual coin toss we can only get either one entire head or one entire tail.

However, if we were to conduct the trial many times we would find that the proportion of heads to tails would gradually equalise; i.e. it would tend to a limit of p = 0.5. We do not actually have to conduct the trial to reach this result: we can get there simply (but tediously) by theoretical logical reflection, based on calculating the possible sequences of heads and tails that could result from a series of tosses of a given length. For example, if we toss the coin FOUR times there are 16 possible sequences, as follows (H = heads; T = tails):

HHHH | THHH | THHT | HTTT
HHHT | HHTT | HTHT | THTT
HHTH | HTTH | THTH | TTHT
HTHH | TTHH | TTTH | TTTT

Since any individual toss of the coin has an equal chance of leading to a head or a tail, each possible sequence of tosses must also be equally likely. We can therefore calculate the probability of obtaining 0, 1, 2, 3 or 4 heads in a series of 4 tosses by counting how many equally likely sequences would lead to this result:

0 heads = 1/16       [i.e. one of 16 possible, equally likely, outcomes]

1 head = 4/16         [i.e. four of 16 possible, equally likely, outcomes]

2 heads = 6/16

3 heads = 4/ 16

4 heads = 1/16

While the most likely result is thus an equal number of heads and tails, we actually have a less than 0.5 (6 out of 16 = 0.375) probability of getting exactly such a result in any one sequence of 4 tosses. If we repeat this theoretical exercise we would soon become aware that as the length of the series of tosses increases, so too does the chances of obtaining a result ever closer to an equal number of heads and tails, but not necessarily exactly equal to this: indeed every time we have an uneven number of tosses we cannot obtain an equal number of heads and tails! This is an example of what statisticians refer to as the central limit theorem: as we increase the number of trials, so the result tends towards, but does not necessarily actually reach, a limit, in this case a proportion of exactly 0.5 heads.

By contrast, empirically, we can carry out repeated trials by actually tossing a coin and observing and recording the results. We will find the same result: the more times we toss the coin, the closer the value of p will get to 0.5. However, no matter how long our sequence of tosses, the chances of p being exactly 0.5 will be less than 1.0. There will always be other sequences of tosses with unequal numbers of heads and tails. However our chances of getting a result very close indeed to 0.5 will get steadily greater.

How can we possibly know, however, how small or large the chance of a particular event happening is? We know that a coin tossed in the air has a roughly equal chance of landing on one face or the other: 'heads' or 'tails'. We know that if we repeat the trial many times, the value of p will get closer to 0.5. But how many is many, how close is closer? The answer lies in a naturally occurring phenomenon: the ‘Normal Curve’ (also known as the ‘Gaussian Distribution’).