Title: Multiple regression: a checklist
Course: UGQuants (SSPS) > Section: Multiple Regression

Multiple regression: a checklist

As there are quite a few steps involved in building and checking your model, the following checklist is aimed at helping to keep you right.  You can download this checklist as a pdf file here, as well as consulting this page.

1

Select your dependent variable.  What is it that you are trying to understand?  How does your dependent variable measure this?  How was the data for it collected?  Are there any likely sources of measurement error in the data source that make the data weak, or suggest you should be cautious in using it?  Are there missing cases?  Linear regression assumes that your dependent variable is at the interval or ratio level of measurement.  If you use it for an ordinal level variable the results will only make any sense if you can make a reasonable assumption that the distance between the different rank values of the variable are similar and if you have a reasonably large number of cases.

2

Consider what independent variables might be included in your model and sketch the possible causal path models that might operate (based on general knowledge, or previous research).  Remember that OLS MLR will give you values for the direct effect of an independent upon the dependent controlling for all other independents in the model, whether they are prior or intervening variables.  Think about the purpose of your model.  You may be simply trying to identify the variables associated with variation in the dependent (e.g. what are the correlates of high income amongst different earners; what determines the extent of literacy across different countries).  Or you may wish to discover if a strong association between two variables remains when controlling for other variables.  Be clear about the purpose of your model (what am I trying to find out?) and how you would interpret the results, before you obtain them.  Doing this will help you imagine a clearer causal path diagram between your variables.  Of course, once you have your results you can modify the path diagram, or change the variables you use and so on.

3

Note any likely prior or intervening variables that are not included in the model because no information is available (e.g. no data is available in the dataset used).  Think about how they might influence the results if you did have such information.  Remember that to be a relevant prior or intervening variable, the variable must have an association with both the dependent and at least one of your independent variables.  If the independent variable does not have an association with the dependent variable, the variable is not relevant to your model because it cannot affect the dependent.  If it is not associated with any of the independent variables, you will not need multiple regression to describe its association with the dependent variable, because control will not be an issue.  However, you might still want to include such a variable in a model that aimed to explain the maximum amount of variation in a dependent variable.

4

Note any causal relationships between independent and dependent variables in the model that are ambiguous; i.e. the direction of cause and effect might be reciprocal, that X may influence Y and Y may influence X.  Where there is reciprocal causation, the beta coefficients may be biased.  For example, you might be examining the extent of malnutrition in countries and decide that the amount of development aid a country receives would influence its extent.  However, you would face the problem that the extent of malnutrition might also influence the amount of aid a country receives.  If you found a positive effect of aid on malnutrition (more aid, more malnutrition), controlling for other variables, it would be rash to conclude that aid caused malnutrition!  It would be more probable that the extent of malnutrition influenced the amount of aid the country received.

5

Produce scatterplots of each of the interval or ratio level independent variables with the dependent variable to check whether the associations are approximately linear.  If not, consider transforming the variables to achieve this.  You may also wish to produce a model for only a limited range of values of your dependent variable if the nature of the relationship appears to change for different value ranges of the dependent variable, or for the main independent variable(s) of interest.

6

Run bivariate correlations of the independents with each other, to check for possible problems of multicollinearity. Correlations greater than around +/- 0.8 may cause problems.

7

Recode any ordinal or nominal independent variables into a set of dummies each taking the values 0/1 with an omitted reference category.

8

Check the proportion of missing cases in each independent variable and consider whether to retain any with many missing cases from the model, or to substitute an alternative variable with fewer missing cases that might provide an alternative measure.

9

Run the regression and examine the output, including collinearity diagnostics and analysis of the residuals.  Aim to avoid ‘tolerance’ values of less than 0.4.  Aim to have residuals that are randomly distributed; i.e. an approximately ‘normal’ distribution with few (no more than 5%) of observations more than 2 standard deviations away from the mean, and very few outlying residuals (more than 3 standard deviations away).  Try also to avoid any individual cases that exert undue influence on the position of the regression line (use Cook’s distance to check for this).  If you identify such cases, examine them to check whether they are the result of an error in the recording of the data, or a case that is unusual and which we may not expect to be well explained by the model.  Consider excluding any such case from the model, but noting the reasons for their exclusion.

10

Make adjustments as necessary; e.g. dropping variables that do not add a lot to the explanatory power of the model, or cause problems with collinearity; dropping or including new variables to get a model with a better distribution of residuals.

If considering a model based on a sample, check the null hypotheses that:

(A) the model as a whole does not improve our ability to predict the value of the independent (from the p value associated for the F statistics for the entire model)

(B) that each beta coefficient is equal to zero (from the p value associated with the t statistic for each variable).

11

Report the model(s), including the regression equation, and the unstandardised and standardised coefficients as necessary, and the results of any tests run for multicollinearity or the distribution of residuals.  You will need to report unstandardised statistics if you wish to report the size of effects (the unit change in Y associated with a unit change in X in terms of the original units in which they were measured) and standardised (beta) coefficients if you wish to compare the relative size of effects across the independent variables in the model.  If you wish to estimate the direct and indirect effect of an independent variable that operates through intervening variables in your model, you will need two models: one without the intervening variables to estimate the total effect of this independent, and then a model with the intervening variables that divide this effect between the direct effect of the independent (controlling for these variables) and the indirect effect operating through the intervening variables (the effect of these variables controlling for the independent variable).

12

Interpret your model.  Explain what conclusions can be drawn from it, any important qualifications to these conclusions and any important sources of error that may have affected them.  Try to do this in a language that is precise, but not so technical that only an expert in regression could understand and evaluate it.