How generative AI works

▲CONCEPT

Picture the machine that writes your poem as a clerk in a record room. Somewhere inside, in a labelled file, your poem is lying finished, and the clerk's whole job is to find it. Most of us carry some version of this picture — where else would the poem come from?

Test it. Ask for the poem twice. Two different poems come back. A clerk pulling a finished page from a file would bring the same page both times. So there is no file.

Then what is inside? Think of the man at the tea stall you cross every day after school. Thirty years at that counter, and every kind of talk has passed in front of him — results day, match day, bandh day. Has he memorised even one of those conversations? Not one. But say two sentences to him and he can tell where the third is going. Not because he has heard your third sentence before — because thirty years gave him a feel for what follows what.

A generative model holds exactly that kind of feel, learned from everything it trained on — not stored answers, but a statistical sense of what tends to follow what. Give it a prompt, and it draws one plausible reply from that sense, built fresh, word by word. Nothing is fetched from storage — every output is built at the moment you ask for it.

training → distribution → sampling

Training data (large corpus)→Learned distribution (what is plausible?)→Sampling: given a prompt, draw one output

not retrieval by the base model — it holds no answer database of its own; every output is a fresh draw from the distribution (memorisation and RAG are the two real exceptions — see the sibling atom)

retrieval (database lookup) vs generation (distribution sampling)

Weaker. The clerk in the record room. Send him for a page and he walks straight to the file where it has always sat, and brings back that very page, unchanged. Nothing about the file moves between one request and the next. The page existed before you asked; finding it, never making it, is the whole of his job.

Stronger. The tea-stall man. Put a question to him about a conversation he has never actually heard. He does not go quiet — he answers, fluently, out of thirty years of feel for how such talk usually runs. A generative model works the same way. It holds a learned distribution over everything it trained on, and at generation time it samples one plausible answer that fits your prompt. Run the same prompt again and the answer can come back different, and just as plausible — each pass is a fresh draw from that distribution, never a repeat visit to a file.

How to explain the generative mechanism in the exam

Name what the model holds. Not a database of stored answers. A learned statistical distribution over its training data — a sense of what is plausible, built from patterns, not a file of what is true.
Name what happens at generation time. It samples. Given your prompt, it draws one plausible output from that distribution. The output is built the moment you ask — nothing is pulled from before.
Rule out retrieval, directly. Retrieval means the answer already existed somewhere and the machine only had to find it. Generation has no finished page waiting anywhere.
State the consequence. Each run is a fresh draw from a distribution, not a repeat visit to a file — the same prompt does not guarantee the same output. And a plausible answer is not automatically a true one.

Close this page. In your own words — one sentence only — say how a generative model produces its output. Do not use the word database. Do not say look up. Now read your sentence back: does it say the model samples, draws, or builds its answer from a learned distribution? If it still sounds like something being found rather than made, write it again.

"Generative AI retrieves the answer from a large database."

It holds no database. There is no file anywhere holding your answer, ready to be found — that is the record-room picture, and it is the wrong one here. What a generative model holds is a learned distribution: a statistical sense, built from training data, of what tends to follow what. At generation time it samples one plausible output from that distribution, given your prompt — built fresh, not fetched from storage. That is exactly why the same prompt can come back with two different, equally plausible answers. Each run is a new draw, never a repeated visit to a file. A product built on top of a model can still attach a real retrieval step before generation, feeding it documents to draw from. But that is a step wrapped around the model — not the model itself keeping a database.

▲CONCEPT

Generation does not happen in one jump. It happens piece by piece. At each step, the model draws its next piece from a distribution — a spread of options, each carrying its own weight, shaped by everything written so far.

Ask the tea-stall man the same question on two different afternoons. You may get two different, equally fluent replies — his feel for what comes next allows more than one honest answer. A generative model can do the same, for the same reason. Put the same prompt to it twice, and nothing about the model itself changes between the two askings — only the draw.

It is tempting to picture this as reaching into a dictionary and picking a word at random. Test that picture against what actually reads back: the draw is weighted, not random. Some next pieces are far more likely than others, and that weighting is exactly why the output reads as one connected passage, not as nonsense.

A spam filter does not work this way. Put the same email to it twice and the verdict does not move — its boundary stays fixed between askings. A generative model has no fixed boundary like that. It has a distribution, and every run is a fresh sample from it.

same prompt, different outputs — each run is a fresh draw

HELD CONSTANTsampling vs boundary-drawing

↓

↓

Classification (fixed)same email → same verdict every time; a deterministic boundary call

Generation (probabilistic)same prompt → different paragraph each time; each is a fresh draw from the distribution

the draw is weighted, not random — some tokens are far more likely than others; the randomness is structured

↑ Back to top