TL;DR · Key Takeaways
- Generative AI makes new content (text, images, code, audio). Older AI mostly sorted existing content into buckets.
- A text model like GPT-4o or Llama 3.1 holds no answer database. It predicts the next token, one piece at a time, then repeats.
- That single trick explains both the magic (it writes a poem it never saw) and the flaw (it can invent a fact just as fluently).
- It does not understand, look things up, or reason the way a person does. It is a very good pattern machine, and treating it as more than that is where mistakes start.
- Quick test: if a tool creates new content, it is generative. If it only scores or labels, it is not.
A common belief is that ChatGPT looks up answers in a giant database. It does not. There is no shelf of stored replies sitting inside it, no filing cabinet it riffles through when you ask a question. What it holds is a statistical model of language, and every sentence it writes is a fresh prediction, assembled one piece at a time. The gap between what people assume is happening and what actually happens is where most confusion about generative AI begins, so that is where this series starts. Get this one idea right and almost everything else in the next thirty parts will click into place.
One idea: making, not sorting
For most of its history, AI was good at one job: sorting. Show it a photo and it says cat or dog. Show it an email and it labels it spam or not spam. Show it a loan application and it scores the risk. That is discriminative AI. It learns the line between categories, and it is genuinely useful, the quiet engine behind fraud detection, medical image screening, and the recommendations on every app you use. But notice what it never has to do: it never has to produce a cat. It only has to recognise one.
The critic and the musician
Generative AI flips the task. Instead of drawing a line between cat and dog, it learns the pattern of cats well enough to draw a brand new one that no camera ever captured. A discriminative model answers which bucket? A generative model answers what would a plausible new example look like?
The analogy I keep coming back to is the critic and the musician. A music critic can tell you, instantly and accurately, whether a track is jazz or rock. A musician can sit down and play a new jazz tune that has never existed before. Same deep knowledge of the genre, completely different skill. Older AI was the critic, fast and confident at judging. Generative AI is the musician, able to produce. Ask a model to write a haiku about Mondays and it produces one on the spot. It did not fetch a stored haiku, it assembled a new one from patterns learned across millions of poems.
Why the difference actually matters
This is not a pedantic distinction. It changes what you can trust. A discriminative model has a right answer to aim at: the email is spam or it is not, and you can measure how often it is correct. A generative model is producing something new, where there is often no single right answer, only more or less plausible ones. That is exactly why a chatbot can sound authoritative and still be wrong: being plausible is its whole job, and plausible is not the same as true. The moment you internalise that generation is about producing convincing patterns rather than retrieving verified facts, you stop being surprised when a model invents a citation, and you start checking the things that matter.
| Discriminative AI | Generative AI | |
|---|---|---|
| Question it answers | Which bucket does this go in? | What would a new example look like? |
| Output | A label or a score | Brand-new content |
| Example task | Spam or not spam | Write the email |
| Has a single right answer? | Usually yes | Usually no, only more or less plausible |
| What to watch for | Accuracy on known labels | Confident, fluent wrongness |
How it makes something: one guess at a time
Text models such as GPT-4o and Llama 3.1 work by predicting the next token. The model reads everything so far, your prompt plus whatever it has already written, and produces a probability for every possible next token. It picks one, adds it to the end, and repeats with the now slightly longer text as the new input. That loop, run hundreds of times, is how a whole paragraph appears.
What a token really is
A token is a small chunk of text, and it is not always a whole word. Common words like “the” or “cat” are usually one token each, but a longer or rarer word can be split into pieces: “unbelievable” might become “un”, “believ”, and “able”. The model never sees letters or whole sentences directly, it sees this stream of tokens, and almost everything about cost and length later in the series is measured in them. For now, just hold the picture: text in, broken into tokens, and the model’s only move is to guess the next one.
Sampling, and why the same prompt gives different answers
Here is a detail that surprises people: the model does not always pick the single most likely token. It samples from the probabilities, usually with a dash of randomness, which is why asking the same question twice can give two different answers. That randomness is a feature, not a bug. If a model always took the top choice it would be rigid and repetitive, the prose equivalent of a stuck record. The willingness to occasionally pick a less obvious word is what lets it brainstorm, vary its phrasing, and sound human. It is also, as we will see, what lets it wander off the facts. You do not get one without the other.
flowchart LR
P[Prompt and text so far] --> M[Model scores next token]
M --> A[Pick one token and append]
A --> C{Stop token}
C -->|No| P
C -->|Yes| O[Output text]
Worked example
Give the model the phrase “the cat sat on the” and ask for one more word. Internally it scores every possible next token. The numbers might look like this:
mat 41% · floor 22% · sofa 14% · rug 9% · refrigerator 0.1%
It samples one (most often “mat”), appends it, and now the input is “the cat sat on the mat”. Then it runs the whole thing again to choose the word after that. Nothing was planned ahead. The sentence exists only because the loop kept choosing a plausible next token, and that is true whether the output is a haiku, a function, or a five-paragraph email.
It is bigger than chatbots
Text is the famous case, but the same generate-from-patterns idea runs a whole family of systems. Image models like Stable Diffusion paint pixels out of noise. Code models complete functions. Speech models produce voices, music models compose, and video models stitch frames into motion. Different data, one core move: learn what the data tends to look like, then sample a fresh point from that learned shape. When you understand the text case, you understand the others, because the recipe does not change, only the ingredients do.
This is also why the field moved so fast. Once researchers had a recipe that worked for one kind of data, pointing it at another kind was often a matter of engineering rather than a brand new invention. A single idea, “model the data, then sample from it,” turned out to generalise across nearly everything humans create, and that breadth is most of why generative AI feels like it arrived everywhere at once.
What it is not: three myths worth killing on day one
Myth 1: it understands what it is saying
The model captures statistical relationships between tokens with astonishing richness, but that is not comprehension in the human sense. It has no mental model of a cat, no lived sense of what sitting on a mat feels like. It has a deep map of how the words tend to co-occur. That map is capable enough to be genuinely useful, and shallow enough that the model can produce a fluent sentence about something it has no concept of. Holding both of those facts at once is the beginning of using these tools well.
Myth 2: it looks facts up
It does not consult a source when it answers, which is exactly why it can state something false with complete confidence. The knowledge it has is smeared across its parameters as patterns, not stored as retrievable records, so there is no “lookup” step that could fail loudly. When it does not know, it does not go quiet, it generates the most plausible-sounding thing, in the same calm voice it uses for things it has right. This is important enough that hallucination gets its own part later, along with the techniques for grounding answers in real sources.
Myth 3: it is conscious or reasoning
Despite the marketing language, there is no inner experience here and no reasoning in the way you reason. Newer “reasoning” models that show their working are still, underneath, predicting tokens, just trained to produce a longer chain of intermediate steps first. That helps on some problems and it is genuinely useful, but it is not a mind waking up. It is a very good pattern machine getting better at a particular trick.
Is it generative AI? A quick test
You can apply a test without any maths. Ask what the tool does with content. If it creates something new, it is generative. If it only sorts, scores, or labels what already exists, it is not. A spam filter scores. Face recognition matches. A recommendation engine ranks. A tool that drafts the email, writes the function, or paints the picture is generative. Same underlying field, very different behaviour, and the distinction tells you exactly what to expect and what to double-check before you trust the output.
flowchart TD
Q{Does the tool create or only judge content} -->|Create| G[Generative AI]
Q -->|Judge| D[Discriminative AI]
| Tool | What it does | Generative? |
|---|---|---|
| Spam filter | Scores an email as spam or not | No |
| Face recognition | Matches a face to an identity | No |
| Recommendation engine | Ranks items you might like | No |
| ChatGPT / Claude | Writes new text | Yes |
| Midjourney / Stable Diffusion | Paints new images | Yes |
| GitHub Copilot | Completes new code | Yes |
▾ Go Deeper (optional, for technical readers)
The split has a precise statistical shape. A discriminative model learns a conditional probability, P(y | x): given input x, how likely is label y. It only needs the decision boundary. A generative model learns the joint distribution P(x, y), or for plain unconditional generation just P(x), the probability of the data itself. Model P(x) well and you can sample new x values from it.
Modern LLMs are trained to maximise the likelihood of each next token given the tokens before it, written P(token_t | token_1 … token_(t-1)). Generation is sampling from that learned distribution, and a setting called temperature scales how sharply you favour the highest-probability tokens. Low temperature is safe and repetitive, higher is more varied and more error-prone. Earlier image methods like GANs and VAEs reach P(x) by different routes, an adversarial game or an encode-decode bottleneck, but the target is the same: model the data closely enough to draw convincing fresh samples. Tokens, embeddings, sampling, and temperature each get their own part later in the series.
This is Part 1 of a 30-part walk from zero to the infrastructure behind production AI. The full map lives on the Generative AI Complete Guide. Next, the common jargon gets decoded in plain English in Part 2, and the mechanics of the next-token trick continue in Part 4, what a model really is.
So what generative AI actually is
Strip away the hype and generative AI is a pattern model that produces new data by sampling, one piece at a time, from what it learned. That one picture explains both sides of it: the wonder of a poem it never saw, and the risk of a fact it cheerfully invents. It makes rather than sorts, it predicts rather than retrieves, and it is fluent rather than knowledgeable, which is a combination that rewards anyone who keeps the distinction in mind and quietly traps anyone who forgets it.
If you take only one thing from Part 1, make it this: treat fluency as a default, never as proof. The model’s confidence is a writing style, not a measure of truth, and the habit of checking what matters is worth more than any clever prompt. With that foundation set, the next part clears up the vocabulary, the tokens, parameters, inference, and embeddings that everyone uses and few explain, so the rest of the series reads easily.
Frequently Asked Questions
What is generative AI in simple terms?
Generative AI is software that creates new content such as text, images, code, or audio by learning patterns from huge amounts of examples and predicting what comes next. Instead of looking up a stored answer, it generates a plausible one on the fly.
How is generative AI different from traditional AI?
Traditional AI usually classifies or predicts from fixed options, like deciding whether an email is spam. Generative AI produces brand-new content rather than picking a label, which is why it can write an email or draw a picture.
Is generative AI the same as ChatGPT?
No. ChatGPT is one popular product built on a generative AI model. Generative AI is the broader technology behind many tools across text, images, audio, and code.
References
- Discriminative vs. Generative Models: What is the Difference? (Coursera)
- Introducing Llama 3.1: model sizes and the 128k context window (Meta AI)
- Generative AI: an overview of models and methods (arXiv)
Start of the series | Generative AI Complete Guide | Next: Part 2: the GenAI words, decoded »
About The Author
DrJha