Which statement best describes the meaning of a p-value in hypothesis testing?

• A. The probability of minimizing Type I error irrespective of data
• B. The probability that the null hypothesis is true
• C. The probability of obtaining the observed results (or more extreme) if the null hypothesis is true
• D. The probability that the alternative hypothesis is true

Answer: (C)

The p-value tells you how compatible the observed data are with the assumption that there is no effect (the null hypothesis). It is the probability, calculated under that assumption, of obtaining the observed result or something more extreme in the direction of interest. A small p-value means the data would be unlikely if the null were true, so they give evidence against the null; a large p-value means the data are not surprising under the null, so there’s no reason to reject it. It’s not the probability that the null is true, nor the probability that the alternative is true, and it doesn’t by itself specify a long-run error rate. Whether “more extreme” is one-sided or two-sided depends on the test you use.