Title: Odds and Logits
Course: UGQuants (SSPS) > Section: Logistic Regression

Odds and Logits

We saw that using odds removes the upper limit of 1 for probability, but not the lower limit of 0.  We can get rid of the lower limit of zero by using the natural log of the odds, referred to as the logit.  The logit of 1 (or 'evens') is zero (corresponding to a probability of 0.5), while a logit below 1 (corresponding to probabilities below 0.5) is a negative number and a logit above 1 is a positive number.  Not only that, but, as Table 2 shows, it now has a symmetrical distribution such that the same difference above or below the probability of 0.5 is shown with the same absolute difference in log odds; e.g. probabilities of 0.7 and 0.3 both differ from 0.5 by 0.2, which is reflected in the logit of plus or minus 0.847, respectively.

Table 2: Probabilities, Odds and Logits compared

P | 0.1 | 0.1 | 0.3 | 0.5 | 0.7 | 0.9 | 0.99
(1-p) | 0.99 | 0.9 | 0.7 | 0.5 | 0.3 | 0.1 | .01
Odds (p/(1-p)) | 0.01 | 0.111 | 0.429 | 1 | 233 | 9 | 99
Logit (log odds) | -4.61 | -2.20 | -0.847 | 0 | 0.847 | 2.20 | 4.601

What is more, a unit change in the logit translates into a greater change in probabilities near the midpoint than at the extremes giving rise to the shape of the curve we saw illustrated in Figure 2.

This is exactly why we want to 'linearise' the inherently non-linear relationship between independent variables and a dichotomous dependent variable: the values predicted by our model will cluster close to two values of interest (zero and one) and the model will no longer be able to predict 'impossible' probabilities below zero or above one.  This is the good news.

The bad news is that, while modelling the probabilities or even the odds of something happening (e.g.  Y taking the value 1) is reasonably intuitive, (e.g.  it is 90% likely or odds of nine to one that Y=1), modelling the logits is anything but intuitive.

In linear regression we were able to produce readily understandable results such as: 'a unit change in variable X, controlling for variables A, B and C, is associated with a 1.6 unit change in variable Y'; or 'each extra year of university education is linked to a 67p per month rise in income, other things being equal'.

But if we say that 'each extra year of university education is linked to a 0.7 unit increase in the logged odds of working in the public sector' you would be hard pressed to have much idea of what this meant.  If we were to translate this back into the language of probabilities, we'd face the problem that the change in probabilities would be different for different values of X, as it is not a linear relationship.  This also means that in logistic regression there is no direct equivalent to Beta or standardized coefficients.

We can make comparisons between the coefficients of different independent variables for the same dependent variable, but not across different dependent variables. This is because we do not have any meaningful equivalent to the standard deviation (our standardising mechanism via z-scores) for a variable which can take only two values.  We therefore need to pay close attention to the kind of results we can obtain from logistic regression and how we report these results.

While you may find it easier to understand the process by working through an example, as we will do shortly, this is a good place to describe the kind of results logistic regression produces, comparing them to their linear regression equivalents.