# Calculating Significantly Above Chance for 3-AFC Test

Psychology & Neuroscience Asked by bernice.anders on December 30, 2020

I am trying to figure out which participants who took a 18-item multiple choice test scored significantly above chance. Each item has 3 choices, so the probability of getting each question correct is 0.33. To know how many items correct a person would have to get in order to be significantly above chance, does it make sense to use a binomial probability calculator like this: https://stattrek.com/online-calculator/binomial.aspx and try different numbers of successes until the probability is < p=0.05 (assuming this is my significance criteria)?

For 18 items where the probability correct is 0.33, the probability of getting 9 correct is 0.061 and the probability of getting 10 correct is p=0.027, so does it make sense to use “10 items correct” as my criteria for “performed significantly above chance?”

The hypothesis would normally be expressed as a one-sided test as p(X≥10). The online calculator you have selected is fine and the significant result could be expressed in an article as follows: (p=0.33, q=0.66, K=10, n=18, p-value = 0.043).

Another way to think of this is that 9 or fewer correct answers is more likely to be random chance.

At the threshold of 10, it is unlikely that the result could be explained by random chance.

Answered by Tony Mobbs on December 30, 2020