Title: Self test: Multiple regression
Course: UGQuants (SSPS) > Section: Multiple Regression

Exercise 1

This is the first out of 15 exercises.

Do the exercise on running a multiple regression analysis using the WDI 2011 data set.

Imagine we are interested in the determinants of the birth rate (children born per 1000 people) in a country  (measured by the variable birthrate in WDI 2011). We develop a theory that the following factors will influence the birth rate:

- We expect that the higher the level of women’s employment (measured by the variable femlab in WDI 2011), the lower will be the birth rate.
- We expect that the higher the level of women’s enrollment in secondary education (measured by the variable fem2ed in WDI 2011), the lower will be the birth rate.
- We expect that the lower the level of poverty (measured by the variable dollday in WDI 2011), the lower will be the birth rate.
- Finally, we expect that the higher the incidence of contraceptive use (measured by the variable contra in WDI 2011), the lower will be the fertility rate.

First run scatterplots of these variables to confirm that their relationship to the fertility rate is what we would expect, and that the form of the relationship is approximately linear. You should find that each of the variables except femlab is strongly correlated with the birth rate.

The strongest correlation appears to be with education and the incidence of contraception, while women’s employment appears to be much more weakly correlated.

However these correlations do not take account of how our variables may work together, (less poverty will be associated with more extensive education, education may be related to knowledge of contraception and so on).

Thus we can use linear regression to estimate the effects of each of our variables, controlling for each of the others.

Let us decide to investigate what the effect of contraception is, once we have controlled for the effects of employment, poverty and education. We will thus run two regression models. One with employment poverty and education variables only, and then one which also includes the variable for contraception. To do this select birthrate as your dependent variable, and using enter as the method of selection select the variables femlab, dollday and fem2ed as your independent variables. Then choose ‘next’ to create a second block of variable(s) to enter and select contra as the independent variable. SPSS will then run two regression models. One for the first three variables and then one using all four variables. This will allow us to examine the effects of contraception prevalence.

Using the unstandardized coefficients and the constant from the first model, examine the effects of each of the three independent variables on birth rates. Estimate the birth rate for each of the following two countries

- 35% of the workforce is female;  25% of people live on less than a dollar a day and 20% of women go to secondary school
- 50% of the workforce is female;  0% of people live on less than a dollar a day and 100% of women go to secondary school

Examining the standardized coefficients from the second model and comparing them to the first model, what would you conclude about the role of contraception, controlling for the impact of employment, education and poverty?

How much of an improvement in our model is obtained by adding in the contraception variable?

Finally run a normal probability plot of the residuals, to check that whether their distribution is reasonably normal.

What else do you notice about the model?

Exercise 1 - results

Let us decide to investigate what the effect of contraception is, once we have controlled for the effects of employment, poverty and education. We will thus run two regression models. One with employment poverty and education variables only, and then one which also includes the variable for contraception. To do this select birthrate as your dependent variable, and using enter as the method of selection select the variables femlab, dollday and fem2ed as your independent variables. Then choose ‘next’ to create a second block of variable(s) to enter and select contra as the independent variable. SPSS will then run two regression models. One for the first three variables and then one using all four variables. This will allow us to examine the effects of contraception prevalence.

Using the unstandardized coefficients and the constant from the first model, examine the effects of each of the three independent variables on birth rates. Estimate the birth rate for each of the following two countries

- 35% of the workforce is female;  25% of people live on less than a dollar a day and 20% of women go to secondary school
- 50% of the workforce is female;  0% of people live on less than a dollar a day and 100% of women go to secondary school

Examining the standardized coefficients from the second model and comparing them to the first model, what would you conclude about the role of contraception, controlling for the impact of employment, education and poverty?

How much of an improvement in our model is obtained by adding in the contraception variable?

Finally run a normal probability plot of the residuals, to check that whether their distribution is reasonably normal.

What else do you notice about the model?

Coefficients(a)

We can use the unstandardized coefficients to tell what each of the effects of these three variables is on the birth rate, controlling for the other two variables. Remember that we must do this in terms of the units in which each variable is measured. Thus a 1% increase in the female labour force ratio is associated with a decrease in the birth rate per 1000 people of .078. A 1% increase in the proportion of people living on less than 1.25 dollars a day is associated with an increase in the birth rate of .091. A 1% increase in the % of women enrolled in secondary education is associated with a decrease in the birth rate of .267.

We can also use these coefficients to estimate what the birth rate is likely to be in a country with different characteristics.
Suppose a country has a labour force which is 35% female, has 25% of its people living on less than a dollar a day and in which only 20% of women go to secondary school. Our estimate for the birth rate would be:

42.235 (the constant) + (35 * -.078) + (25* .091) + (20* -.267)
= 42.235 - 2.73 + 2.275 – 5.34
= 36.44

Suppose a country has a labour force which is 50% female, has 0% of its people living on less than 1.25 dollars a day and in which 100% of women go to secondary school. Our estimate for the birth rate would be:

42.235 + (50*-.078) + 0 + (100*-.267)
= 42.235 - 3.9 - 26.7
= 11.635

The standardized coefficients tell us the relative size of effect of the variables in our model (by expressing the relationship between the variables in terms of units of standard deviation). From these we can see that once we control for education and poverty, employment has more of an effect than was suggested by its simple correlation with the birth rate. In contrast, the other two indicators lose in explanatory strength in the model.
The adjusted R squared for our first model, at .70 is high. With just these three variables we can account for over two thirds of the variation of the birth rate across countries.

Now let us add in the variable for contraception and see its effect. It improves our model as the adjusted R squared rises to .72. Note, too, how it changes the size of the coefficients for the other variables, as we can now see their effect, controlling for the impact of contraception prevalence. The biggest impact is on the poverty head count: unsurprisingly, contraceptives, such as the pill or condoms, will not be affordable for everyone. The size of the effect of education is also affected. This is what we might expect, since there is likely to be an association between the extent of education and the prevalence of contraception. Looking at the Beta coefficients, we can see that strongest influence on the birth rate still comes from education, with contraception having an effect half that size and female employment ratio and poverty rate together explaining only about 9% in the variation of the birthrate.

We can see from our plot of the residuals that they are fairly evenly distributed, but that there are some deviations from the straight line. This means that there are other variables that we could include which would improve our model. One candidate would be the level of economic development (represented by the log of gdp as the relationship with GDP itself will almost certainly not be linear). Another would be the rate of urbanisation and yet another would be life expectancy. Try adding in these or other variables to your model to see if you can improve the fit as measured by R squared.

Don't forget to also check the significance levels of the explanatory variables in the model. In fact, you will find that these are rather poor. A careful check of all variables in advance would have revealed that we have a lot of missing values, which reduces our overall sample size. We simply don't have sufficient data to build a reliable model in this dataset.

Also, keep in mind the nature of inferential statistics: we aim to learn something about a whole population by only using data from a random sample. Is that true here?

-No. The World Development Indicators (if they indeed had access to all the information they aim to gather) collect data about the whole population of all countries in the world. One could thus question the use of linear regression with this dataset.

Question 2

We can control for many variables at once

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 3

We want to maximise the number of variables in the model

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 4

We can only use interval level dependent variables

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 5

We can only use interval level independent variables

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 6

We assume that a straight line is a good summary of the relationship between each of the variable in the analysis

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 7

We can tell the direction of cause and effect

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 8

The coefficient of determination tells us the amount of variance of the dependent variable explained by the model

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 9

Outliers in a model should be removed

- False
Feedback: Correct
- True
Feedback: Incorrect

Question 10

The correlation coefficient between a pair of variables tell us how close to the regression line the data points cluster

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 11

A dummy variable takes two values: 0 and 1

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 12

Correlation coefficients below 0.7 are a potential source of multi-collinearity

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 13

Standardized beta coefficients tell us the size of impact of each variable

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 14

The unstandardized coefficients are used for calculating the regression equation

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 15

Residuals in a good model are normally distributed

- True
Feedback: Correct!
- False
Feedback: Incorrect