Title: Step 3: Transformation
Course: UGQuants (SSPS) > Section: Multiple Regression

Step 3:   Transformation

The purpose of transforming the values in a variable is to make its distribution more normal, or at least more symmetrical.  Typically, in any type of regression analysis, we start by considering whether changing the distribution of the dependent variable (Y) would be sufficient to model the data (for statistical reasons which we don’t need to worry about here).  However, it is well known in many economic analyses that measures such as GDP usually benefit from being transformed in any case.

What we are doing when we transform a distribution is to change the scale values, whilst preserving the order of the original numbers.  It is not a way of trying to fiddle the data to get the result we hope for by nefarious means.  It is designed to help us to see patterns that would otherwise remain hidden from view.  However, we need to remember that the new units will be different from the former ones and these have to be reflected in any final equation.  The question that now arises is how we decide how to transform the numbers.  The key to this is to be found in what is called the ‘ladder of powers’.

Ladder of powers

What we do is to raise the number (our observation) to a power of whichever value would have the most advantageous effect on trying to achieve a normal distribution.  It is unlikely that the transformation will be perfect, but we would hope to improve on the starting position (even if it cannot be guaranteed).  While most untransformed numbers that we work with are recorded as values with no powers being shown, we could think of them as being ‘raised to the power 1’; e.g. 7 = 7^1.

Generally speaking, when the distribution has negative skew (i.e. has a long tail on the left-hand side), we are looking to apply a power greater than 1.  For example, we could square all the numbers; i.e. transformed 7 = 7^2 = 49.

In practice we do not usually choose a power any greater than 3 (cubed).  Similarly, when the distribution has positive skew (i.e. has a long tail on the right-hand side), we choose a power less than 1 (including negative powers).  For example, we might use the square root of a number (e.g. 9^½ = 3), or the reciprocal of a number (e.g. 4^-1 = ¼ = 2).

A special transformation that is commonly used, especially with economic measures, is the log, which can be thought of as being similar to using a power of about zero.  Logs (formally ‘logarithms’) have special properties that allow us to add transformed numbers in order to get the same answer as multiplying untransformed numbers.  They work by converting numbers to a different number base, but it is not necessary to understand why or how.  Suffice it to know that the common forms either use base 10, or base e (which is known as a natural log, because it is based on the ‘natural constant’ e, which equals 2.17828…).  The natural log is usually written as ln.  Even though the resulting numbers will be different, the outcome is the same, whether you use base 10, or base e, or any other base.

If we want to discuss our findings in terms of the original scale values, we would need to ‘untransform’ the values.  Thus, if we had squared the values to transform them, we would need to take the square root to convert them back to the original values.  Similarly, if we had taken the log of a number, we would need to antilog the value to return to the original value.  These calculations, while they might seem complicated, are easy to get the computer to calculate.

In our example, if we take the natural log of the independent variable (ln(X)), which is positively skewed, it has the effect of spreading out the lower values, while bunching up the higher values so that its distribution becomes that shown in the histogram below (the red line shows you what a perfect normal distribution for the data would look like). The table below shows you the effect of taking the natural log for a range of GDP values.

GDP | Natural log of GDP | Change in GDP for one unit change in log of GDP
148 | 5.0
402 | 6.0 | 254
1100 | 7.0 | 698
2990 | 8.0 | 1890
8140 | 9.0 | 5150
22000 | 10.00 | 13860
60000 | 11.00 | 38000

When we produce a scatterplot of female life expectancy with the log of GDP, this will make the lower values become more spread out and the higher values closer together on the X axis, with the effect of pushing the centre of the curved line we saw in the previous histogram downwards, thus making it straighter.  This will mean that, to obtain the same change in value of LE_female, we will now need much more similar changes in the log of gdp because these will equate to quite small changes in actual GDP for low values of GDP, and then much bigger changes for high values of GDP.  This is exactly the sort of pattern we are looking for.

We will now be looking at the relationship between female life expectancy and the natural log of GDP per head.  To get back to what happens with real values of GDP we will have to 'untransform' our results using antilogs at the end of the process.

In the WDI2011 dataset the natural log of GDP is represented by the variable lngdp.  Try drawing a scatterplot of  lngdp by LE_female and examine the result.  The line should be a lot straighter but still slightly curved, as in the scatterplot below.  To obtain the R^2 result and  to draw the regression line on the scatterplot, choose ‘edit chart’ and then from the chart options menu choose 'fit' and 'include Rsq in legend'.

The points cluster reasonably well around the regression line. Our transformation has made a substantial difference.  While the correlation of GDP per head and female life expectancy was only about -0.52, that between female life expectancy and the natural log of GDP is about -0.84.  We could interpret this as meaning that the increase in female life expectancy associated with an increase in GDP per head is much greater at lower levels of GDP than at higher ones.  The correlation tells us how closely our data points cluster around our regression line.  A correlation of -0.84 is quite high.

Coefficient of determination

If we square the correlation coefficient (R-squared) we obtain the coefficient of determination.  This tells us how much of the variance in our dependent variable, LE_female, is accounted for by the variance in our independent variable, lngdp.  The coefficient of determination is analogous to the proportional reduction in error method of calculating a correlation since it tells us how much our ability to predict Y is improved by adding X to our model.  The way to think about what this means is to imagine we had to predict what the value of female life expectancy was in a country, without knowing which country it was.  In the absence of any other knowledge, our best estimate for life expectancy would simply be its mean value across all countries.  Knowing the GDP of the country where we were trying to predict the value of female life expectancy would help us to make a much more accurate prediction.  R-squared compares the sum of the squared distances between our observed data points and the regression line to the sum of the squared distances between our observed data points and the mean for Y.  It takes a value between 0 and 1.  A value of 0 means no improvement in our model (i.e. the absence of any relationship at all between our variables), whilst a value of 1 would mean that our independent variable accounted for all the variation in our dependent variable; i.e. if we knew the value of our independent variable we would also know exactly what would be the value of our dependent variable.

An R-squared of 0.63 is fairly high. In turn this suggests (but does not prove) that standard of living, as measured by GDP per head, does affect life expectancy at birth for women and that this effect is greater at lower levels of GDP, provided there are no important prior or intervening variables.  We shall start to look at this in a moment, but first let us look at how our software gives us the results in its output, including the regression equation for this simple model.