Clicked Gallery

What is Stochastic Gradient Descent (SGD)?

Highlighted from a real engineering doc. Explained by Clicked.

Used in a sentence

To optimize the training loop for large datasets, the ML engine relies on stochastic gradient descent.

The reader highlighted one word in the docs. Clicked explained the technical term “stochastic gradient descent” in simple terms:

Explained in three depths

●○○

Overview

▾

SGD is the method most AI models use to learn: make a prediction, measure how wrong it was, nudge every internal setting a tiny step in the direction that reduces the error, and repeat millions of times. The stochastic part is that each step checks a small random batch of examples instead of the entire dataset.

●●○

Detail

▾

A model is a machine with millions of tiny dials, and its performance is one wrongness score called the loss. For each dial, math can work out which way to turn it so the score gets smaller, and that direction is called the gradient. Descent means turning every dial a little that way, measuring the score again, and repeating millions of times. The choice SGD makes is how many examples to check before each turn. Checking all of them, which for a large model can mean millions of images, points in the most accurate direction but is so slow that few turns happen per hour, while a batch of 32 or 64 random examples points in a slightly wrong direction and allows thousands of turns in the same time. Thousands of slightly wrong turns lower the score faster than a handful of perfect ones. Because each batch differs, the direction also wobbles, and that wobble can shake the model off a setting that looked good but wasn't the best available.

●●●

Analogy

▾

Walking down a mountain in thick fog. You can't see the valley, so you feel the slope under your feet and step downhill, over and over — surveying the entire mountain before each step would give a perfect direction and take forever. SGD is reading only the ground under your boots: rougher readings, far more steps, and the occasional stumble that knocks you out of a small dip that wasn't the real bottom.

AI explanations may contain errors · Not professional advice

Same facts, different vibe — Slang mode 😎

●○○

Overview

▾

SGD is how models learn: guess, get told how wrong, adjust a tiny bit, repeat a few million times. The stochastic part is the honest bit — we check a random handful of examples per step, because checking everything every time would take a lifetime. 😎

●●○

Detail

▾

The pieces: loss is the wrongness score, and the gradient is an arrow pointing toward less wrong for every knob in the model. The learning rate is the step size — too big and you faceplant past the target, too small and training takes a geological era. Mini-batches make each step cheap and slightly drunk, and the drunkenness is weirdly useful: a perfectly sober walker settles into the first comfortable rut, while the wobbly one stumbles out and finds something better. Why random batches win: a rough direction you can compute a thousand times an hour beats a perfect direction you compute twice. That's the whole trick: speed over precision, repeated until the score stops falling. And the wobble is not a bug you tolerate, it's the part that keeps the model from settling for the first mediocre answer it finds. 😎

●●●

Analogy

▾

It's fixing your free throw. Shoot, watch the miss, adjust a little, shoot again — you don't rewatch every shot you've ever taken before each tweak, because the last few misses tell you which way to lean. Occasionally a lucky bounce lies to you and you adjust the wrong way, and a thousand shots later the form is dialed in anyway.

AI explanations may contain errors · Not professional advice

Formal definition — The same term, explained the usual way

Stochastic gradient descent is an iterative optimization algorithm that updates model parameters in the direction of the negative gradient of the loss function, where the gradient is estimated from a randomly sampled mini-batch rather than the full dataset. This yields computationally inexpensive, unbiased but noisy updates; the induced noise can aid escape from poor local minima and saddle points. Convergence behavior is governed principally by the learning-rate schedule, with momentum-based and adaptive variants (e.g., Adam) widely used in practice.

Want Clicked to explain terms like “stochastic gradient descent” directly in your browser — including on PDFs?

Add to Chrome — Free

50 free Explanations · No credit card required

More from the gallery

How to actually read a research paper

Skim smart, break the jargon wall, and keep what you learned — one session, done right.

What is Retrieval-Augmented Generation (RAG)?

Make the AI answer from your documents — retrieval first, generation second, receipts included.

What are Vector Embeddings?

Meaning turned into coordinates — how machines find what's similar without matching a single word.

What does an Idempotent API mean?

Why ten clicks charge you once — the property that makes retries safe.

What is Technical Debt?

Shortcuts and aging systems that bill the future — principal, interest, and repayment.

What is a Zero-Knowledge Proof (ZKP)?

Prove something is true without revealing why — the cryptography behind private verification.