What does statistically significant mean?

Statistical significance means a result was unlikely to arise by chance alone, judged against a cutoff agreed in advance. It is a screening test rather than a verdict: the pattern is probably not luck, and nothing more than that has been established. Plenty of significant findings are far too small to matter.

A company tests two checkout buttons and the green one wins by 2%. The question worth asking before anyone celebrates is how often random variation alone would hand one of two identical buttons a 2% lead. Statistical significance is that question made formal: researchers pick a threshold before looking, almost always 5%, and call a result significant when luck alone would rarely produce something that big. Why fix a line in advance rather than judge each result on its merits? Because without one, everyone talks themselves into believing whatever they hoped to find, so the line works as a commitment made before the data can argue back. The protection is thinner than it looks. That 5% cutoff is a convention rather than a law of nature, which leaves results at 4.9% and 5.1% treated as opposites despite being near twins. Significance also says nothing about size, since with 50,000 users an effect nobody would ever notice clears the bar comfortably.

A friend claims they can tell tap water from bottled, tastes four glasses, and gets all four right. Impressive until you work out that pure guessing wins all four about 6% of the time, so somebody manages it every slow afternoon with no talent at all. Ask for twelve glasses and guessing almost never survives, which makes twelve correct worth something. Notice what the test still cannot tell you: whether your friend tastes a difference that matters at dinner.