Cross Validated Asked by DC_Liv on November 17, 2020

I have a large dataset of 631 people that have been repeatedly measured across five different variables on some continuous parameter. The means (and SDs) for these variables are as follows:

```
Var1: 7.77 (3.8)
Var2: 6.87 (3.48)
Var3: 7.62 (3.83)
Var4: 3.96 (1.92)
Var5: 7.25 (3.62)
```

Having run a one-way repeated measures ANOVA, I have found a significant difference between them. Expecting this to be mainly a consequence of Var4, I ran post-hoc tests with Bonferroni correction. However, to my surprise, I have found a significant difference between each pairwise comparison. I’m obviously missing something but I find this really strange, especially considering the similarity between Vars 1 and 3.

I’m assuming (perhaps incorrectly) that the large size of the sample is the cause (?!?). If this seems plausible, is there a way I can adequately communicate that whilst there is a statistical significance between them, this doesn’t relate to a practical difference? It seems incomplete to remark that there is a statistical difference when the means between them are so similar.

As you wrote, standard errors are very small, which is the reason why you are getting statistically significant results. The test is testing differences between means, not between whole distributions, so even a small numerical difference in means can be highly statistically significant if it's high with respect to standard errors of the mean.

95% confidence interval is roughly +/- 2SE, so the mean of Var1 is estimated to be 7.77 +/- 0.012, which doesn't even come close to Var3 7.62 +/- 0.006

Yes, as you correctly pointed out, statistical significance does not mean practical differences. In your case, even a small difference in means is statistically significant, simply because you have a lot of data.

The correct interpretation of your results is that you are quite sure that all of the means are different; however, you are also quite sure that their difference is small, except for Var 4. In situations with wider standard errors, you cannot make this conclusion.

Correct answer by rep_ho on November 17, 2020

Get help from others!

Recent Answers

- Peter Machado on Why fry rice before boiling?
- haakon.io on Why fry rice before boiling?
- Lex on Does Google Analytics track 404 page responses as valid page views?
- Jon Church on Why fry rice before boiling?
- Joshua Engel on Why fry rice before boiling?

Recent Questions

- How can I transform graph image into a tikzpicture LaTeX code?
- How Do I Get The Ifruit App Off Of Gta 5 / Grand Theft Auto 5
- Iv’e designed a space elevator using a series of lasers. do you know anybody i could submit the designs too that could manufacture the concept and put it to use
- Need help finding a book. Female OP protagonist, magic
- Why is the WWF pending games (“Your turn”) area replaced w/ a column of “Bonus & Reward”gift boxes?

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP