Title: Self test: Inference
Course: UGQuants (SSPS) > Section: Inference

Question 1

This is the first out of 19 questions.

The p value associated with the Chi Sq test tells us the probability of obtaining the association observed in our random sample data if no such association existed in the population from which the sample was drawn.

- True
Feedback: Correct! The p value associated with the Chi Sq test tells us the probability of obtaining the association observed in our random sample data if no such association existed in the population from which the sample was drawn. We thus want it to be as small as possible, although there will always at least be a small chance that our sample is not a good representation of the overall population.
- False
Feedback: Incorrect. The p value associated with the Chi Sq test tells us the probability of obtaining the association observed in our random sample data if no such association existed in the population from which the sample was drawn. We thus want it to be as small as possible, although there will always at least be a small chance that our sample is not a good representation of the overall population.

Question 2

The p value associated with the Chi Sq test tells us the strength of the relationship between the variables in the population.

- False
Feedback: Correct! It does not tell us anything about the strength of the relationship but merely how likely it is that what we find in our sample is not true for the whole population we are trying to infer something about.
- True
Feedback: Incorrect. It does not tell us anything about the strength of the relationship but merely how likely it is that what we find in our sample is not true for the whole population we are trying to infer something about.

Question 3

The p value associated with the Chi Sq test tells us the probability of getting a type II error.

- False
Feedback: Correct! The p value associated with the Chi Sq test tells us the probability of getting a type I error: of concluding that there is a relationship when in truth there is none.
- True
Feedback: Incorrect. The p value associated with the Chi Sq test tells us the probability of getting a type I error: of concluding that there is a relationship when in truth there is none.

Question 4

The p value associated with the Chi Sq test tells us the probability of getting a relationship this strong in the population.

- False
Feedback: Correct! The p-value is not concerned with the strength of the relationship.
- True
Feedback: Incorrect. The p-value is not concerned with the strength of the relationship.

Question 5

The p value associated with the Chi Sq test tells us the probability of commiting a type I error should we reject the null hypothesis.

- True
Feedback: Correct! The p value tells you how likely it is that a relationship we find in our sample does not actually exist in our overall population.
- False
Feedback: Incorrect. The p value tells you how likely it is that a relationship we find in our sample does not actually exist in our overall population.

Question 6

If a finding is statistically significant, this means that we accept the null hypothesis for the level of error we have set.

- False
Feedback: Correct! We reject the null hypothesis (which says that there is NO relationship), because our results suggest that there is a relationship in the sample which (because it is statistically significant) is likely also present in the overall population.
- True
Feedback: Incorrect. We reject the null hypothesis (which says that there is NO relationship), because our results suggest that there is a relationship in the sample which (because it is statistically significant) is likely also present in the overall population.

Question 7

If a finding is statistically significant this means that the result is important substantively.

- False
Feedback: Correct! Incorrect. Statistical significance is an indicator that tells how likely it is that our findings are true for the population that we make inferences about. It does not tell us how important our findings are. This is for us (and other researchers) to decide.
- True
Feedback: Incorrect. Statistical significance is an indicator that tells how likely it is that our findings are true for the population that we make inferences about. It does not tell us how important our findings are. This is for us (and other researchers) to decide.

Question 8

If a finding is statistically significant, this means that our sample size is adequate.

- False
Feedback: Correct! While statistical significance is influenced by sample size, that does not mean we should increase the sample size to arrive at a statistically significant result. If a relationship does not exist, you can not 'magic' it into being this way. On the contrary: if you find a statistically significant relationship with a sample of 12,000 units that you did not find with a sample of 1,200 units, you have probably found a very small, unimportant one. It could be thus reasoned that your sample is inadequately large. Surveys working with such large samples usually do this so that the sample can be split into smaller subsamples - e.g. age groups - and still offer enough cases for sound analysis.
- True
Feedback: Incorrect. While statistical significance is influenced by sample size, that does not mean we should increase the sample size to arrive at a statistically significant result. If a relationship does not exist, you can not 'magic' it into being this way. On the contrary: if you find a statistically significant relationship with a sample of 12,000 units that you did not find with a sample of 1,200 units, you have probably found a very small, unimportant one. It could be thus reasoned that your sample is inadequately large. Surveys working with such large samples usually do this so that the sample can be split into smaller subsamples - e.g. age groups - and still offer enough cases for sound analysis.

Question 9

If a finding is statistically significant this means that subject to the level of error we have set, we can infer the existence of a relationship in the population from the pattern of data in our sample.

- True
Feedback: Correct!
- False
Feedback: Incorrect.

Question 10

The confidence interval for a sample characteristic tells us the probability that the value of that characeristic in the sample lies within that range.

- False
Feedback: Correct. The confidence interval for a sample characteristic tells us the probability that the value of that characeristic in the population lies within that range.
- True
Feedback: Incorrect. The confidence interval for a sample characteristic tells us the probability that the value of that characeristic in the population lies within that range.

Question 11

The confidence interval for a sample characteristic tells us the probability that the value of that characteristic in the population lies within that range, given our sample size and variable distribution.

- True
Feedback: Correct!
- False
Feedback: Incorrect.

Question 12

The confidence interval for a sample characteristic tells us the amount of confidence we can have in our sample estimate.

- False
Feedback: Correct!
- True
Feedback: Incorrect.

Question 13

The confidence interval for a sample characteristic tells us the maximum possible range of the value of that characteristic in the population.

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 14

A substantively important relationship will always be significant.

- False
Feedback: Correct! Significance measures are also dependent on the sample size - just as a small, unimportant relationship can be found to be significant with a large enough sample, an important relationship might be overlooked (be described as insignificant) if the sample that the analysis is based on is too small.
- True
Feedback: Incorrect. Significance measures are also dependent on the sample size - just as a small, unimportant relationship can be found to be significant with a large enough sample, an important relationship might be overlooked (be described as insignificant) if the sample that the analysis is based on is too small.

Question 15

A statistically insignificant relationship is never important.

- False
Feedback: Correct!
- True
Feedback: Incorrect

Question 16

The larger the size of the sample, the lower is the level of association between variables necessary to reach statistical significance.

- True
Feedback: Correct!
- False
Feedback: Incorrect

Question 17

You are exploring a dataset to look for possible relationships of interest (rather than testing a previously established null hypothesis). You find a weak to moderate relationship between two variable of interest which fails to reach statistical significance at the 5% level, p = 0.064. What could you do?

- Reflect further on how strong the relationship needs to be in order to have theoretical interest and how capable your random sample was of picking up relationships of such strength.
Feedback: Correct!
- Investigate whether recoding the values the variable takes can bring p below 0.05.
Feedback: Incorrect
- Decide to proceed no further in investigating this relationship
Feedback: Incorrect
- Decide the you need a bigger sample next time
Feedback: Incorrect

Question 18

An expected count is...

- ...The value that would occur if the table marginals stayed the same but no association existed between the variables.
Feedback: Correct
- ...The residual between the observed value and the fit of no association.
Feedback: Incorrect
- The value observed if the sample had been truly random
Feedback: Incorrect
- The number of cases in a table cell falling within the confidence interval
Feedback: Incorrect

Question 21

Using the full European Social Survey 2010 UK dataset, investigate the following relationships. In each case formulate appropriate null and alternative hypotheses, decide on an acceptable level of significance, describe the association you find in your sample data and decide whether to accept or reject your hypothesis. Decide too how you might elaborate on a theory about the relationships in your data, using other variables, and how you might go about this:

- Between sex (gndr)  and self reported general health (health).

- Between marital status (marstat) and self reported general health

- Between sex and whether respondents feel safe walking in the area where they live after dark (aesfdrk)

Question 21 - answers

Between sex (gndr)  and self reported general health (health).

Null hypothesis: there is no difference between the self-reported health of men and women in the population.
Alternative hypothesis: sex influences level of self-reported general health
Level of acceptable error: 5% (we don't wish to exclude the possibility accepted that there is an association by setting to stringent a level, but we do wish to be reasonably sure that we'll exclude associations thrown up by sampling variation). We'll treat gender as our independent variable and place it in the table columns.

Results
The difference in proportions of the dependent variable (health) within the categories of the independent variable is very small. Indeed so small that it hardly seems worth running a measure of association. If we did run a measure of association we would see that the association in our sample data is very weak.
The value for Chi Sq is low and if we decide to set our risk of committing a type I error at the conventional 5% level, we must accept our null hypothesis, since the level of significance associated with this value of Chi Sq is .90 or 90% - nine out of ten chance that we'd get this result even if there were no association at all in the population.

Between marital status (marstat) and self reported general health

Null hypothesis: there is no difference between the level of self-reported health of people of different marital statuses in the population.
Alternative hypothesis: Marital status is associated with the level of self-reported health
Level of acceptable error: 5% (we don't wish to exclude the possibility accepted that there is an association by setting to stringent a level, but we do wish to be reasonably sure that we'll exclude associations thrown up by sampling variation).
We'll treat marstat as our independent variable and place it in the table columns, although we might also be interested in the other potential causal direction: health might influence whether people get married.

Results
The difference in proportions of the dependent variable (health) within the categories of the independent variable are fairly small between the separated and married, but for the other categories they are often more substantial. If we run a measure of association (let's use Cramer's V) we find a weak association at 0.115.
The value for Chi Sq is high and if we decide to set our risk of committing a type I error at the conventional 5% level, we must reject our null hypothesis, since the level of significance associated with this value of Chi Sq is .000 or less than .05%. We'd almost never obtain such a pattern of results in a random sample if there were no association at all in the population. We provisionally accept our alternative hypothesis that there is an association between marital status and self reported health in the population, however we also note that this association is not a strong one.

Between sex and whether respondents feel safe walking in the area where they live after dark (aesfdrk)

Null hypothesis: there is no difference between the level of perceived safety of walking in the area after dark of men and women.
Alternative hypothesis: Perception of safety after dark is associated with sex.
Level of acceptable error: 5% (we don´t wish to exclude the possibility accepted that there is an association by setting to stringent a level, but we do wish to be reasonably sure that we'll exclude associations thrown up by sampling variation). We'll treat gender as our independent variable and place it in the table columns, perceptions of safety cannot influence gender.

Results
The value of Cramers V is more than double what it was for our previous pair of variables, and comparing the column percentages, we can see that three times as many women as men say that they do feel very unsafe walking in their local area after dark, while more than twice as many men as women say they feel ‘very safe'. The value for Chi Sq allows us to reject our null hypothesis and provisionally accept our alternative hypothesis. In this case it looks as if we have evidence of a substantial relationship in the adult population from which our sample was drawn.