Title: When can we use regression?
Course: UGQuants (SSPS) > Section: Correlation and Regression

When can we use regression?

As with most tests in quantitative analysis, regression has certain assumptions and violating any one of these may mean that regression is inappropriate.

1. The model is linear.

For our purposes, this means, first, that the dependent variable is at the interval or ratio level of measurement.  There are different types of regression when the nature of the dependent variable is different (e.g. logistic regression when the dependent variable is nominal or ordinal). It also means that the independent variable is interval or ratio and in a linear relationship with the dependent variable.  It is never appropriate to regress a nominal variable on its own against a dependent variable because such variables have no mathematical properties associated with them.

Let us take, for example, region of residence. We might code respondents as 1 if they live in England, 2 if they live in Wales, 3 if they live in Northern Ireland and 4 if they live in Scotland.  In this example the numbers are meaningless except as labels.  We could reorder them (1 for Scotland, 2 for Northern Ireland etc) without changing the meaning of the responses. We also know that living in Scotland (coded originally as 4) is not 'worth' four times living in England (coded originally as 1).  If we wish to use nominal variables in regression we must recode them and create a separate variable for each of the response categories except one.  We code each variable as 1 if the trait is present and 0 if it isn’t.

We could, therefore, create a variable called 'England', and code everyone who lives in England as 1 and everyone who lives in the rest of the UK as 0.  Similarly, we could code all women in our dataset as 1 and all men as 0.  If we have nominal variables with lots of categories we don’t create a separate binary variable for each category. If we have 5 categories we only create 4 variables. Why do we do this? So that we don’t create mirrored versions of the same piece of information.  If we create a variable that has all women coded as 1 and all men coded as 0, we have adequately captured the gender of respondents in our dataset. We don’t need to create a second variable with all men coded as 1 and all women coded as 0. The same is true when we have 3 or 4 or more categories in our nominal variables.

Our other assumption of the independent variables is that they are neither fixed nor entirely random. In other words we don’t have a situation where all the values for x are the same (everyone in the dataset received a score of 4 for political knowledge) or randomly assigned. We could randomly create groups within our survey respondents, and assign them each different numbers, but we wouldn’t assume that such randomly assigned predict their attitudes or behaviour.  .

2. Errors are normally distributed.

Many of the assumptions for Ordinary Least Squares are more relevant to working with more than one independent variable at the same time (e.g. multiple regression), but for bivariate regression we are also concerned with the nature of errors. We assume that the errors in our regression (the way all the cases deviate from the regression line) are normally distributed, rather than consistently higher at one end of the line. If we were to graph the distance between each datapoint and the regression line we would want to see an even distribution, like a tunnel around the line, rather than a funnel at either end. This notion of even distribution of errors (or the normal distribution of errors) is referred to as homoscedasticity. If the errors are not normally distributed (which we call heteroscedasticity) it suggests that there is something wrong with our model. Perhaps we have identified the wrong independent variable (or too few independent variables).