TL;DR · Key Takeaways
- Generative AI makes new content (text, images, code). Older AI mostly sorted existing content into buckets.
- A text model like GPT-4o or Llama 3.1 has no answer database. It predicts the next token, one piece at a time, then repeats.
- That single trick explains both the magic (it writes a poem it never saw) and the flaw (it can invent a fact just as fluently).
- Quick test: if a tool creates new content, it is generative. If it only scores or labels, it is not.
A common belief is that ChatGPT looks up answers in a giant database. It does not. There is no shelf of stored replies inside it. What it holds is a statistical model of language, and every sentence it writes is a fresh prediction built one piece at a time. The gap between what people assume and what is actually happening is where most confusion about generative AI begins, so that is where this series starts.
One idea: making, not sorting
For most of its history, AI was good at one job: sorting. Show it a photo and it says cat or dog. Show it an email and it labels it spam or not spam. That is discriminative AI. It learns the line between categories. It never has to produce a cat, it only has to recognise one.
Generative AI flips the task. Instead of drawing a line between cat and dog, it learns the pattern of cats well enough to draw a brand new one that no camera ever captured. A discriminative model answers which bucket? A generative model answers what would a plausible new example look like?
Here is the analogy I keep coming back to. A music critic can tell you whether a track is jazz or rock. A musician can sit down and play a new jazz tune that has never existed. Same knowledge of the genre, very different skill. Older AI was the critic. Generative AI is the musician. Concretely: ask a model to write a haiku about Mondays and it produces one on the spot. It did not fetch a stored haiku, it assembled a new one from patterns learned across millions of poems.
How it makes something: one guess at a time
Text models such as GPT-4o and Llama 3.1 work by predicting the next token. A token is a small chunk of text, often a word or part of a word. The model reads everything so far, your prompt plus whatever it has already written, and produces a probability for every possible next token. It picks one, adds it, and repeats with the now slightly longer text as the new input.
Take the phrase “the cat sat on the”. The model scores likely continuations: mat high, rug and floor lower, refrigerator very low. It samples one and moves on. That loop is the entire trick. A system that only ever answers “what comes next?” can still write essays, code, and email replies, because each of those is just a long run of plausible next tokens given the context.
It is bigger than chatbots
Text is the famous case, but the same generate-from-patterns idea runs a whole family. Image models like Stable Diffusion paint pixels from noise. Code models complete functions. Speech models produce voices, and video models stitch frames. Different data, one core move: learn what the data tends to look like, then sample a fresh point from that learned shape.
What it is not
Three myths are worth killing on day one. First, the model does not understand the way you do. It captures statistical relationships between tokens, which is powerful but is not comprehension. Second, it does not look facts up, which is exactly why it can state something false with complete confidence. We give that its own part later in the series. Third, despite the marketing, it is not conscious and it is not reasoning in the human sense. It is a very good pattern machine.
Is it generative AI? A quick test
Here is a test you can apply without any math. Ask what the tool does with content. If it creates something new, it is generative. If it only sorts, scores, or labels what already exists, it is not. A spam filter scores. Face recognition matches. A tool that drafts the email, writes the function, or paints the picture is generative. Same underlying field, very different behaviour, and the distinction tells you what to expect and what to double-check.
▾ Go Deeper (optional, for technical readers)
The split has a precise statistical shape. A discriminative model learns a conditional probability, P(y | x): given input x, how likely is label y. It only needs the decision boundary. A generative model learns the joint distribution P(x, y), or for plain unconditional generation just P(x), the probability of the data itself. Model P(x) well and you can sample new x values from it.
Modern LLMs are trained to maximise the likelihood of each next token given the tokens before it, written P(token_t | token_1 … token_(t-1)). Generation is sampling from that learned distribution, and a setting called temperature scales how sharply you favour the highest-probability tokens. Low temperature is safe and repetitive, higher is more varied and more error-prone. Earlier image methods like GANs and VAEs reach P(x) by different routes, an adversarial game or an encode-decode bottleneck, but the target is the same: model the data closely enough to draw convincing fresh samples. Tokens, embeddings, and temperature each get their own part later.
New here? This is Part 1 of a 30-part walk from zero to the infrastructure behind production AI. Start at the Generative AI Complete Guide for the full map, then carry on to Part 2, where the common jargon gets decoded in plain English.
The Bottom Line
Strip away the hype and generative AI is a pattern model that produces new data by sampling, one piece at a time, from what it learned. That one picture explains both sides of it: the wonder of a poem it never saw, and the risk of a fact it cheerfully invents. Hold that idea steady and the rest of this series will click into place. Next: the jargon, decoded. Which term has tripped you up the most so far?
Frequently Asked Questions
What is generative AI in simple terms?
Generative AI is software that creates new content such as text, images, code, or audio by learning patterns from huge amounts of examples and predicting what comes next. Instead of looking up a stored answer, it generates a plausible one on the fly.
How is generative AI different from traditional AI?
Traditional AI usually classifies or predicts from fixed options, like deciding whether an email is spam. Generative AI produces brand-new content rather than picking a label, which is why it can write an email or draw a picture.
Is generative AI the same as ChatGPT?
No. ChatGPT is one popular product built on a generative AI model. Generative AI is the broader technology behind many tools across text, images, audio, and code.
References
- Discriminative vs. Generative Models: What is the Difference? (Coursera)
- Introducing Llama 3.1: model sizes and the 128k context window (Meta AI)
- Generative AI: an overview of models and methods (arXiv)
Start of the series | Generative AI Complete Guide | Next: Part 2: the GenAI words, decoded »
About The Author
