What is a p-value?

When a study finds a difference, there are two possible causes: something real, or luck. The p-value measures how easily luck alone could produce a difference that big, so a small p-value means luck would rarely manage it. It does not tell you the chance the finding is true.

Take a sleep-drug trial where the treated group fell asleep 12 minutes faster than the sugar-pill group. The boring assumption, called the null hypothesis, is that the drug does nothing and the 12 minutes is ordinary luck, since two groups of people never come out exactly equal. The p-value answers one question: if the drug truly did nothing, how often would luck alone open a gap of 12 minutes or more? If luck manages that only 3 times in 100 repeats of the study, the p-value is 0.03, and something real looks likely. Most fields call anything under 0.05 significant, but that cutoff is a convention, and clearing it does not mean the effect is big enough to matter. One caution: test 20 useless drugs at that cutoff and about one will pass by luck alone, so a single lucky pass proves little.

A friend rolls three sixes in a row and calls it luck. A fair die does that about once in 216 tries, and that once-in-216 chance is the p-value of his claim. It is small, so the die being fair stops being the easy explanation, though rare things do happen.