Title: Multiple Regression and causation
Course: UGQuants (SSPS) > Section: Multiple Regression

Multiple Regression and causation

We can treat evidence of correlation as partial evidence of a causal effect only if all prior and intervening variables have been controlled.  In an experiment with experimental and control groups obtained by random allocation, we can be reasonably sure that this is the case, because the selection procedures aim to produce two (or more) groups for the study which are alike in every respect except for the application of the experimental condition.  In such a situation we can be sure that any difference in the outcome of the experiment across the two groups is because of the experimental condition and not because of the existence of other differences between the two groups.

In the social sciences, when we are dealing with data obtained by observation, rather than experiment, we cannot assume that groups which are different in one way are not also different in other ways.  For example, we might be interested in exploring the influence of gender upon earnings.  We could compare the earnings of a group of men and a group of women and we would almost certainly discover that there was an association: male earnings would probably be substantially higher.  However, we could not conclude from the results of such observation that it was gender alone, or necessarily that it was the most important variable, that caused these difference in earnings since the group of men and group of women that we studied would almost certainly be different in a number of other ways:

 | Perhaps the women were more likely to have spent periods out of the workforce around motherhood;
 | Perhaps there were differences in the level of educational qualifications of the two groups;
 | Perhaps they tended to work in different occupations or sectors of the economy.

Thus our original correlation between gender and earnings might in part be accounted for by these other differences.  In the language of variables, part of the correlation between our independent variable of interest (gender) and our dependent variable (earnings) would be accounted for, or 'explained', by the correlation between our dependent, independent and relevant prior and intervening variables.  In the absence of randomisation to gender groups, we need to control for these other variables statistically.  To the extent that we can hold these other variables constant for the two different values of our independent variable (men and women), we would get a much better measure of its effect on earnings.

One way to control would be to try to select cases in our data that had the same values for each of the variables for which we wished to control.  We might look at the earnings of men and women with the same educational qualifications, who had spent the same amount of time in the labour force, worked in the same occupation and economic sector, and so on.  We would then need to repeat this for each of the values of our different relevant and prior variables.  It is pretty clear that we would very soon run out of cases.  And we'd face another problem. The 'effect' of gender would almost certainly be very different in the myriad of different groups of cases that we would end up with and the groups themselves would be of different sizes.  We would need to calculate some kind of average effect, taking into account the sizes of all the different groups.  It is not hard to see that such a procedure would be cumbersome at best.  Once we tried to include more than a small number of variables, or variables with a wide range of different values, it would soon become impossible.

Multiple linear regression can be thought of as a procedure that produces the same end result of the exercise we have just described, but does so in a much easier way, given the use of computing software to support us.  If we have not one but two independent variables, we could visualise the relationship between these variables and our dependent variable, not as coordinates on a flat plane (described by Y and X), but as coordinates in 3-dimensional space (defined by Y, X_1 and X_2).  We could still plot Y on the vertical axis, and X_1 on the horizontal axis, with X_2 plotted on the depth axis.  We would now have a three dimensional cloud of points representing our data observations and, in order to model Y, we would still be trying to minimise the size of the squared residuals.

Once we move beyond two independent variables, we can no longer easily represent the process visually since we would be working in hyperspace, but the mathematics become no more complex.  We will still have an intercept: the estimated value of Y when each of our independent variables is zero, and a series of b coefficients, each of which will now represent the change in Y associated with a unit change in that X variable, whilst holding all the other variables in the regression equation constant.