Clicked Gallery

What is Inference (in AI)?

Highlighted from a real engineering doc. Explained by Clicked.

Used in a sentence

The chipmaker's newest processors target inference rather than training, where the volume is.

The reader highlighted one word in the docs. Clicked explained the technical term “inference” in simple terms:

Explained in three depths

●○○

Overview

▾

Inference is an AI model doing its actual job: your question goes in, an answer comes out, using what training already taught it. Training is the one-time build; inference is every use after that. It sounds like the cheap half, and per answer it is, but it runs billions of times a day.

●●○

Detail

▾

Inference is everything that happens between your question and the answer, and it is a completely different job from training. Training happens once, burning enormous computing power over weeks to set the model's internal dials. Inference runs those settled dials forward on one new input: numbers flow through and an answer comes out, with nothing inside the model changing. That is why a single reply costs a fraction of a cent while training the same model cost millions. The catch is volume, because a popular assistant answers billions of questions, and a fraction of a cent multiplied by billions is what actually lands on the electricity bill. That arithmetic is why the industry works so hard to shrink models with quantization and distillation, and why chipmakers now design processors aimed at inference rather than training.

●●●

Analogy

▾

Studying for an exam versus sitting the exam. The studying is exhausting, expensive and happens once; the exam is two hours of using what you already know, with no new learning at all. One student sitting one exam is trivial next to the studying. But run that exam for fifty thousand students every day and the exam hall, not the studying, breaks the budget.

AI explanations may contain errors · Not professional advice

Same facts, different vibe — Slang mode 😎

●○○

Overview

▾

Inference is the model doing the actual work: question in, answer out, using dials that training already set. Training is the one giant build; inference is every single use afterward. One answer is pennies, but multiply by billions and that is where the money goes. 😎

●●○

Detail

▾

Training is the mountain you climb once. Inference is every step you take afterward: your question goes in, numbers rip through the already-tuned model, an answer comes out, and nothing inside changes, which is exactly why one reply costs a fraction of a cent. Harmless until you multiply, because a popular assistant fields billions of questions, and fractions of a cent times billions is a genuine electricity bill with genuine chips behind it. So the whole industry sweats inference, shrinking models with quantization and distillation, and chipmakers now ship silicon built for inference instead of training. Training makes the headlines; inference makes the invoice. 😎

●●●

Analogy

▾

Learning to drive versus driving to work. The lessons, the tests, the terrifying first parallel park: expensive, exhausting, over with. The commute just uses what you already know and burns some gas. Do that commute twice a day for ten years and the gas quietly outspends the lessons.

AI explanations may contain errors · Not professional advice

Formal definition — The same term, explained the usual way

Inference denotes the execution of a trained model on new input to produce an output, as distinct from training, which determines the model's parameters. Inference performs no parameter updates but recurs with every request, making its aggregate compute, latency and energy the dominant operational cost of deployed systems, and motivating optimizations such as quantization, distillation and inference-specialized hardware.

Want Clicked to explain terms like “inference” directly in your browser — including on PDFs?

Add to Chrome — Free

50 free Explanations · No credit card required

More from the gallery

What is Quantization?

Storing a model's numbers less precisely — half the memory, barely any loss.

What is Model Distillation?

Training one model on another model's answers: to shrink your own, or to catch up to a rival's.

What is a Mixture of Experts (MoE)?

Huge model, a small slice awake per question, and why that keeps frontier models affordable.

What is a Context Window?

How much text an AI can hold in mind at once — and why long chats forget their start.

What is a Transformer (in AI)?

The design behind the T in GPT: every word attends to every other word, all at once.

What is a Neural Network?

Math that learns the rules from examples instead of having them written, and why that wins.