Title: Introduction: Logistic regression
Course: UGQuants (SSPS) > Section: Logistic Regression

Introduction to module 10.

Modelling with multiple (linear) regression can be powerful, allowing us to test our beliefs about which independent variables might offer useful insights into a particular dependent variable.  Nonetheless, it must be used carefully, with attention paid to which variables to include, how to define those variables, what the errors ('residuals') in the model tell us, and whether there is substantial correlation between the independent variables (multi-collinearity). This helps us build a robust model in both a statistical and substantive sense.

However, despite its potential, multiple regression has one big drawback for social scientific work: to work well, it needs a dependent variable at the interval or ratio level of measurement with continuous, as opposed to discrete, values, whose upper and lower limits are not bounded (from minus to plus infinity).  When we are studying collective institutions or groups (e.g. nation states, organisations, voluntary and community groups) such variables are relatively abundant, but when we study individuals, relatively little of interest can be measured at this level. An individual , for example, cannot be 23% male, or half pregnant or one quarter dead - although we could talk about a population in these terms. The most common continuous measures on the individual level are income, wealth and physical aspects such as age, height, weight, blood pressure, etc. Almost everything else (attitudes, aspects of behaviour, economic status, sexual orientation, sense of humour, etc.) is nominal or ordinal.

The relationship of such nominal or ordinal dependent variables with any independent variables cannot be linear in form; that is to say it cannot be represented accurately by a straight line.  This means that linear regression cannot give us an adequate model of such relationships. However, we can apply two ingenuous transformations to the values that a nominal dichotomous (i.e. yes/no, true/false) dependent variable takes to enable us to use linear techniques; the result is called (binary) logistic regression.

Like linear regression, logistic regression has its traps for the unwary, and, unfortunately, unlike linear regression, most of the textbook guides require considerable patience and stamina to work through.  The book by Fred Pampel (2000) Logistic regression: A primer (Thousand Oaks CA: Sage) is thorough but not really a primer.  The standard work is Scott Menard (2002) Applied Logistic Regression Analysis, 2nd edition, (Thousand Oaks CA: Sage) which is rather heavy going if you do not already have a good training in statistics. This is a pity as logistic regression is not much more difficult than its linear counterpart and has all kinds of promising applications in the social sciences, even if some of the background statistics are rather complex.

This module takes you through the basics. Do not worry too much if the explanation of how we 'linearise' the expression for a dichotomous dependent variable is difficult to follow.  Its purpose is to clarify why the kind of results that we get from logistic regression are different from its linear counterpart and why we have to be careful to interpret them in a rather different way.