Clicked Gallery
What is a Transformer (in AI)?
Highlighted from a real engineering doc. Explained by Clicked.
Used in a sentence
Engineering Notes · AI Systems
The paper proposes a leaner transformer that cuts training costs by nearly a third.
The reader highlighted one word in the docs. Clicked explained the technical term “transformer” in simple terms:
Explained in three depths
●○○
Overview
The transformer is the design behind almost all modern AI, the T in GPT, and its breakthrough is reading everything at once instead of word by word. It does this through attention: every word looks at every other word simultaneously and weighs which ones matter for its meaning. That one change is what made today's giant models possible.
●●○
Detail
The transformer is a neural network design whose whole advantage is reading everything at once. Earlier designs read one word at a time while carrying a fading memory forward, so long passages went blurry and, worse, nothing could be computed until the previous word finished. The 2017 paper Attention Is All You Need solved both at once. Its attention mechanism lets every token, roughly every word piece, look directly at every other token and score how much it matters. In the phrase "the dog chased its tail", attention is what lets the word "its" find the dog across the sentence, no chain of memory required. Each layer uses those scores to sharpen what every word means in its context, and stacking many layers is how raw words become understood meaning. Parallel reading is also why models exploded in size: since no word waits on another, training spreads across thousands of chips at once, and only that made internet-scale training practical. The price of everyone-looks-at-everyone is that the work grows steeply as the text gets longer, which is exactly why every model has a context window, a hard cap on how much it can consider at once.
●●●
Analogy
Two ways for a detective team to work a case file. The old way, one detective reads it aloud start to finish while everyone relies on memory of what came before, and details from page two are fuzzy by page ninety. The transformer way, every page is pinned to the corkboard at once and strings are drawn between related clues, this alibi connects to that timestamp, the name on page ninety to the witness on page two. Meaning comes from the strings, and no page is ever fuzzier just because it came early.
AI explanations may contain errors · Not professional advice
Same facts, different vibe — Slang mode 😎
●○○
Overview
The transformer is the blueprint under basically every modern AI, the T in GPT. Its one big trick: every word looks at every other word simultaneously and weighs what matters, so the model reads everything at once instead of in order. That's what let AI get huge. 😎
●●○
Detail
One trick, enormous consequences: read everything at once instead of one word at a time. The old models were whisper chains, each word waiting on the last while the memory leaked, so long documents went blurry at the start. Then 2017's Attention Is All You Need showed up with a flex for a title and actually delivered. Every token gets to stare directly at every other token and score who matters to it, so in the phrase "the dog chased its tail", the word "its" finds the dog across the sentence, no chain, no leak. Each layer sharpens those scores, and enough stacked layers take the model from spelling to meaning. Side effect one: nothing waits on anything, so training splits across thousands of chips, hence models fed the entire internet. Side effect two: everyone staring at everyone gets expensive fast as the text grows, which is literally why your model taps out at a context window. 😎 😎
●●●
Analogy
Planning a trip over a voicemail chain versus in a group chat. Voicemail chain: each friend hears only the previous friend's retelling, and by the eighth call the hotel name is mangled and the dates are fiction. Group chat: every new message can scroll up and read every earlier message directly, quote the exact one that matters, and ignore the rest, so nothing gets mangled by the chain. The one cost: the longer the chat gets, the more scrolling every single reply takes.
AI explanations may contain errors · Not professional advice
Formal definition — The same term, explained the usual way
The transformer is a neural network architecture based on self-attention, in which each token computes weighted relevance over all other tokens in the input, enabling parallel sequence processing without recurrence. Introduced by Vaswani et al. in 2017, it underpins modern large language models; its quadratic attention cost with respect to sequence length motivates fixed context windows and ongoing research into efficient attention variants.
Want Clicked to explain terms like “transformer” directly in your browser — including on PDFs?
Add to Chrome — Free50 free Explanations · No credit card required
More from the gallery
What is Tokenization?
How AI chops text into chunks before reading — and why some languages cost more.
What is a Context Window?
How much text an AI can hold in mind at once — and why long chats forget their start.
What is a Neural Network?
Math that learns the rules from examples instead of having them written, and why that wins.
What is Backpropagation?
How one final error becomes a precise adjustment for millions of settings — Goal Seek at scale.
What is Stochastic Gradient Descent (SGD)?
Millions of dials, one wrongness score — how models actually learn, tiny step by tiny step.
What is Quantization?
Storing a model's numbers less precisely — half the memory, barely any loss.