Title: Are there right and wrong answers in quantitative analysis?
Course: UGQuants (SSPS) > Section: Introduction to Quantitative Analysis

This page asks whether there are right and wrong answers in quantitative analysis

Dealing with numbers and statistics can lead to two common misconceptions:

1) There is always a ‘right’ and a ‘wrong’ answer. If we leave aside philosophical questions about the nature of knowledge then, yes, it is true that in quantitative analysis there are (sometimes) right and wrong answers. It is possible to say, e.g., that most people in a group are employed, vote Conservative or are British citizens. But it is equally true that there are often no definitive right and wrong answers. The answers often depend on the questions, and this is probably more often true than it is of other approaches to social and political questions. For example, if we examine people's employment status, voting behaviour or citizenship, then the result depends on how we ask them, what response options we provide and the particular approaches to analysis we decide to take. There is also often room for different interpretations, such that the same information might lead to quite different conclusions.

Look at the video to see an example

2) Numbers ‘speak for themselves’; are straightforward and unproblematic. In social and political science we sometimes say that data are ‘collected’, but it is often better to think of data as being created or constructed. This is because data are not pre-existing entities just waiting for us, the researchers, to ‘pick them up’. They are shaped by our interventions as researchers: the questions we decide to ask; the way we decide to ask them; the sources we consult; the approaches to analysis we decide to take, etc. Take a statement such as ‘female life expectancy in Sri Lanka is 78 years’. How do we know this? What kind of data sources and calculations might have been used to produce this one number? Once we know more about the processes that lie behind numerical data we will be aware that they are seldom incontrovertible ‘facts’.

The course should make you aware of the many situations where there are no right and wrong answers and the reasons why this is so. You will also learn more about what we call the ‘social construction of data’ because there is an entire module on that topic! We encourage you to go through that module, but you can do so at any stage: near the beginning; later, when you feel you know a bit more about quantitative analysis; or right at the end.

But a very important point is that social construction of data is not unique to quantitative analysis, but is common to all research approaches: the researcher ‘creates’ the data, often in collaboration with those who participate in the research in various ways. The well-known phrase 'there are lies, damned lies and statistics' is useful in encouraging caution in our reading of numerical information because it might be (deliberately or unintentionally) misleading.

But unfortunately the phrase is too often used as the basis for the belief that all numerical data may therefore be similarly mistrusted or dismissed entirely, and that other kinds of data do not need to be subjected to similar scepticism. Neither are true. In fact, one of the strengths of quantitative analysis is that it is more advanced in terms of transparency and self-critique than are most other approaches in social and political research.

Links:

Why you need to be careful when presented with statistics
2012 article in The Guardian discusses how well economic statistics represent reality in African countries | This paper by Gary Klass from 2008 shows a lot of examples from US-American politics of how easily statistics can be misinterpreted (or abused):