Title: Step 2: Exploring the shape of our data
Course: UGQuants (SSPS) > Section: Multiple Regression

Step 2:   Exploring the shape of our data

In data analysis of any kind there is no substitute for getting a 'feel' for the characteristics of the data, exploring its dimensions by means of tables or graphical representations.  First of all let's produce a histogram of female life expectancy: LE_female.  We can see that it has a substantial negative skew.  Most countries have quite high values for female life expectancy around 70 to 80+ years, with a tail of countries to the left with average values in the 60s, 50s and 40s.

When we produce a histogram of gdp we find that it is even more skewed, this time positively.  This means that there are a lot of countries with low levels of GDP, together with a tail of countries to the right with very high values for per capita GDP.

If we then produce a scatterplot of the variables gdp and LE_female we get a result that shows that a straight line is certainly not a good summary of this association!

We can see that both our variables have rather skewed distributions, one negative (LE_female) and one positive (GDP).  Most values for life expectancy cluster around 70 to 80 years, but with a long tail going down towards some very low values below the 50s.  Similarly, the values for GDP per capita are even more skewed.  A very small number of countries have very high values, while large numbers of countries cluster in the 0 to US$5,000 range.  Because of these non-normal distributions, the scatterplot does not look much like a straight line.

If this is the case we need to change this before we can proceed, since linear regression uses straight lines to model the relationship between the variables.  That is to say, it assumes that there is a constant relationship between the change in value of the X variable and the corresponding change in the value of the Y variable across the entire range of values.  This is clearly not the case here.  At first very small changes in gdp are associated with very large changes in LE_female; then, once gdp passes about US$5000, very large changes in gdp seem to be associated with only small changes in LE_female.  Therefore, to use linear regression we will need to transform at least one of the variables.

If you are interested in theories of why the relationship between longevity and wealth takes this form you may like to look at "The Spirit Level: Why Equality Is Better for Everyone" by Richard Wilkinson and Kate Pickett, Penguin, London, 2010.