Title: Logistic regression results
Course: UGQuants (SSPS) > Section: Logistic Regression

Logistic regression results

In multiple regression we obtain coefficients which describe the change in the value of the dependent variable associated with a unit change in the value of the independent variable, controlling for all the other variables present in the model.  For example, we might find that whilst controlling for other variables such as gender or age, each extra year spent in higher education, on average, added £32 a month to someone's income.   Thus people with eight years in higher education would on average earn £256 (= 8 x 32) per month more than someone who had not attended higher education, and so on.

In logistic regression our dependent variable takes only two values, so it can make little sense to investigate how much change in Y might be associated with a 'unit' change in the value of the independent variable.  Rather we are interested in the probability that the dependent variable takes one as opposed to the other of its only two possible values (0 or 1).

Thus, in logistic regression, we measure the odds ratio of the probability that the dependent variable equals 1 to the probability that the dependent variable equals 0 associated with a unit change in our independent variable of interest, controlling for all the other variables in the model.  The model itself is of the logits, the natural log of this odds ratio, but SPSS handily reconverts these logs back into unlogged or 'exponentiated' odds ratios.  This appears as Exp(B) in the output.

Thus for our purposes here, an odds ratio greater than 1 will mean that per unit change in the independent variable we will have an increase in the probability that the dependent variable will take the value 1, while an odds ratio less than 1 will represent a decrease.  An odds ratio of 1 will indicate that the independent variable has no effect.  For example, an odds ratio of 4 for an independent variable would mean that, controlling for other variables in the model, an increase of 1 unit in the independent variable increases the odds fourfold that the dependent variable will take the value 1.

This means that we can compare the effect sizes of different independent variables by comparing the relative sizes of Exp(B) for the same dependent variable.  When we are working with samples, SPSS can also calculate 95% confidence intervals, which are a quick and convenient way of checking whether a particular coefficient is statistically significant (but always bear in mind the limitations of this measure).  If the value 1 lies within the range of the confidence interval for Exp(B), we cannot reject a null hypothesis that the independent variable is unlikely to affect the dependent variable, controlling for other variables; i.e. there is insufficient evidence to claim there is an effect.

There is a further complication that we should note.  In order to calculate the 'fit' of the logistic curve to the data, the ordinary least squares (OLS) technique cannot be used.  Rather than trying to minimise the sum of the squared errors from the mean, we use a technique known as 'maximum likelihood estimation' (MLE), which is a procedure whereby the computer programme tries to calculate coefficients for each independent variable that minimise the errors (residuals) and maximise the logits that the dependent variable will take a given value rather than the other value (e.g. take the value 1 rather than 0 on the variable about past voting behaviour).

Starting with a best guess, this technique is iterative and in rare cases it proves impossible to fit variables even after dozens of iterations.  For the same reason there is no direct equivalent in logistic regression to the R squared measure in linear regression to describe the amount of variance accounted for by different models.  There is a measure called pseudo-R squared, which calculates the proportional change in error with the model compared to the null model (i.e. without the independent variables).  Based on this, the most commonly used equivalents that look at the strength of the relationship, rather than the goodness of fit, are Cox and Snell's R squared, which produces maximum values below 1, even with a perfect model, and Nagelkerke's R squared, which standardises Cox and Snell’s R squared to lie within 0 and 1.  All these measures attempt to mimic the kind of measure that R squared produces in linear regression, but none can be interpreted in exactly the same way.  Classification tables are often used to show the proportion of cases correctly classified by the model, but they cannot be entirely relied upon to determine the effect size.