Title: Outliers and influential cases
Course: UGQuants (SSPS) > Section: Logistic Regression

Outliers and influential cases

As in multiple linear regression we can ask SPSS to produce a list of outliers, and save values for each case, such as where there are cases that our model fits particularly badly, or which exert undue influence.

When checking for outliers, standardised residuals (residuals divided by the estimate of their overall standard deviation) or studentized residuals (residuals divided by the estimate of a case’s standard deviation) are helpful in identifying when cases are poorly represented by the model. In a good model we should not expect more than 5% of standardised/studentized residuals to be below -1.96 or above +1.96 (indicating a weak model) and 1% of them below -2.58 or above +2.58 (indicating an unacceptable model). In our model, 1.5% of the cases have standard residuals below -1.96 but 5.6% have studentized residuals below that value (none of the cases overstep the 1.96 in the positive range).

Influential cases are outliers that distort the model, giving predictions of the dependent variable that are quite inaccurate. This is often known as leverage and has the effect of changing the direction of the linear relationship between independent variables and the logit of the dependent variable, a bit like the proverbial tail wagging the dog. One of the most common diagnostics used for this is Cook’s distance statistic (which should be below 1), although the option is provided in SPSS to look at one called the leverage statistic (which should be closer to 0 than the other extreme of 1, with an expected value of (k+1)/n, where k is the number of independent variables and n is the sample size) and also DFBeta (which should ideally be less than the absolute value of 1).

You can find all these statistics by clicking on 'Save...' – this will create a new variable providing a value for each single case that you then need to check. To get a first overview, you can use a histogram or frequency table. If you want to identify a specific case, a scatterplot is one way to do so.

Avoiding multi-collinearity

High correlations between independent variables can have the same effect in logistic as in multiple linear regression. The best way to avoid this is simply to check the correlations between independents when building your model. The easiest way to check is to run the variables through multiple regression and check the VIF or Tolerance statistics, which show how big a problem any intercorrelations are between independent variables.