Title: Alternatives to experimentation
Course: UGQuants (SSPS) > Section: Causality

Alternatives to experimentation

Social scientists rarely have the opportunity to conduct experiments. The alternative is to compare groups with different values on one variable (e.g. compare parents with two children to parents with one child; compare men to women; compare Labour voters with Conservative voters etc.) to explore whether they also take different values for another variable.

However, the different values for one variable - let's call it variable 'X' - occur naturally in our population and have not been deliberately manipulated by the investigator (as they would have been in an experiment). Therefore, we cannot know to what extent our two groups, apart from having different values for variable X, also take different values for a whole range of other variables, variables which may also be correlated with the values of X or Y, or both. In other words, we may not be (in fact are almost certainly not going to be) comparing two groups who differ only in terms of the values of X.

Since we cannot carry out a controlled experiment to manipulate the values of only one variable, we have to control using statistics. We examine the relationship between our variables of interest, controlling the values of other variables we think might be relevant. We control by examining what happens to our relationship for different values taken separately for our third control variable.

Thus, say we compared the educational qualifications of older men and women in the UK, we would find that on average men had more qualifications. Were we to conclude that their sex as such 'caused' their superior qualifications and that, in this sense, men are more intelligent than women, we would be rightly criticised. Almost certainly the men had more opportunities to study and encouragement to do so in traditional role model settings when they were in education. That is to say, rather than being otherwise alike except for their sex, they were also different in terms of another relevant variable. Had we included this variable in our model, we would have seen that this also differed between the two groups. We might support this argument by referring to the way in which the relative educational qualifications of younger men and women have changed as this variable (opportunities and encouragement to study) has changed. We can think of this as a situation where our two groups did not differ only in terms of the values of X, but also in terms of other variables such as access to education or encouragement to undertake it. It is not that our association between X (sex) and Y (qualifications) is false. It is that another variable intervenes between the two and makes better sense of the relationship.