Overfitting vs. Underfitting: What's the Difference?

Overfitting is a model learning its training data too specifically, noise included, so it aces training and stumbles on anything new. Underfitting is the opposite: a model too simple to learn the pattern at all, so it fails everywhere.

The tell is the gap between two numbers: the model's score on the data it trained on, and its score on held-back data it has never seen. Underfitting scores poorly on both, because the model never captured the pattern. Overfitting scores excellently on training data and clearly worse on held-back data, having memorized its study material instead of learning the subject. Overfitting comes from too much model complexity, too little data, or training too long; underfitting from a model too simple or fed weak inputs. The fixes mirror the causes: more data, regularization or a simpler model against overfitting; a richer model or better inputs against underfitting. One distinction to keep straight: these are the symptoms in the scores, while high bias and high variance are the diagnoses behind them.

Learning to drive only in your own driveway: every crack and bump memorized, every maneuver flawless, and then the highway is chaos, because you learned the driveway rather than driving. That's overfitting. Underfitting is having had one vague lesson — the car moves when the pedal is pressed — which fails in the driveway and on the highway alike.