How Diffusion Language Models Work: Iterative Denoising vs Autoregressive Generation

The first time I read that a diffusion language model could hit over a thousand tokens per second, I did the lazy math in my head: parallel means faster, faster means better, so this must be the thing that makes ChatGPT feel slow next year. I was wrong on the second and third step, and being wrong turned out to be the useful part.

This post is my attempt to explain how diffusion language models work without a CS degree — the mental model, the honest catch, and whether a working professional should reorganize anything today. I’ll walk through iterative denoising vs autoregressive generation using two everyday pictures, then show you exactly where my “parallel equals faster” assumption broke.

I pay for ChatGPT. I can’t read a research paper’s math. That’s the reader I’m writing for.

The mental model I already had (and why it’s only half the story)

Here’s the model of ChatGPT most of us carry, whether or not we say it out loud. It types one word at a time, left to right. Each word is a bet placed on every word that came before it. Once a word is out, it cannot un-say it. It never goes back to fix the opening sentence after it reads its own ending.

That picture is correct, and it has a name: autoregressive generation. “Auto-regressive” just means each new token is predicted from the tokens already produced. The model factors a sentence into a chain of one-word-at-a-time guesses. GPT, Claude, and Gemini’s main chat models all work this way. Think of a typewriter with no backspace.

A diffusion LLM is the opposite mental picture. It doesn’t type left to right. It dumps a rough, mostly-blank draft of the whole answer, then edits the entire thing over a few passes — filling in blanks, adjusting, filling in more. Less typewriter, more editor revising a full draft at once. Google DeepMind describes its own experimental model as refining noise step-by-step and error-correcting during generation.

That single swap — typewriter to editor — is the whole concept. Everything technical below is just detail hanging off that hook.

How iterative denoising actually works, step by step

The word “diffusion” is borrowed from image generation. Image diffusion models start with a screen of static noise and clean it up into a picture over many steps. Text can’t be blurry the way pixels can, so text diffusion uses a trick called masked diffusion, and this is the part I had to slow down and reread.

Picture your target sentence with every word replaced by a blank tile, like a crossword before you’ve written anything. That fully-masked line is the “noise.” A text diffusion model runs a learned reverse process that fills those blanks in over a fixed number of steps. Each step, it looks at the whole line at once, guesses the most confident blanks, and locks a few of them in. Locked words stay put; the rest stay blank and get another pass.

Here is a 40-word version worth saving:

A diffusion language model starts from a fully masked draft where every token is a blank. Over a fixed set of denoising steps, it predicts and unmasks the most confident tokens while keeping the rest blank, reading the whole sequence at once until nothing is masked.

The technical name for those blanks is the [MASK] token, and the setup is called an “absorbing state.” Absorbing just means once a real word gets masked in training, it stays masked — it can’t flip back mid-process. The model learns to reverse that. Some newer methods even allow remasking, where the model can un-lock a word it locked too early and try again. That’s the “error correction” DeepMind talks about, and it’s the thing a typewriter can never do.

Two properties fall straight out of this design, and they matter more than any benchmark:

• Bidirectional context. Because the model reads the whole line every step, a blank in the middle can be filled using words on both its left and its right. Autoregressive models only ever see the left. The diffusion model gets to peek at the ending before committing to the middle.
• Parallel filling. It can lock several blanks in the same step instead of one word per pass. That’s where the giant tokens-per-second numbers come from — and where my first mistake was hiding.

Iterative denoising vs autoregressive generation, side by side

I find it easier to hold both models in my head as a table than as prose. Here’s how iterative denoising vs autoregressive generation actually compares on the axes a non-developer cares about.

Axis	Autoregressive (ChatGPT-style)	Diffusion LLM
Direction	Left to right, one token at a time	Whole draft at once, over fixed steps
Context each token sees	Only tokens already written	Both sides — bidirectional
Can it revise earlier words?	No, a token is final once emitted	Yes, via remasking during denoising
Where speed comes from	Nothing parallel — sequential by design	Fill many blanks per step, in parallel
Speed-quality control	Roughly fixed per model	Tunable knob: fewer steps = faster/rougher
Best fit	Short answers, tight step-by-step logic	Long outputs where parallel steps pay off

The row I want you to stare at is the last two. The speed of a diffusion model isn’t free — it’s a knob. Fewer denoising steps means a faster, rougher answer. More steps means a slower, cleaner one. Autoregressive models don’t hand you that dial in the same way. And that knob is exactly what my head-math ignored.

Notice this isn’t a “which one wins” table. It’s a “which shape of problem” table. That distinction is the whole point of the next section.

Where I was wrong: parallel does not mean faster

My clean little chain — parallel, therefore faster, therefore better — falls apart at both joints. Here’s what broke when I actually traced it through.

First break: short answers. Every denoising step is a full pass through the entire model. If a diffusion LLM runs, say, a fixed number of steps to produce a short reply, it may do more total compute than an autoregressive model that just emits four tokens and stops. For a one-line answer, the typewriter can finish before the editor has done its second read-through. The speed advantage shows up on long outputs — a page of code, a long draft — where filling many blanks in parallel actually saves passes. On a short “yes, here’s the command,” it can lose.

Second break: quality collapses when you unmask too much at once. This is the part nobody put in plain words for me. When the model locks in many blanks in a single step, it fills them as if each blank were independent of the others. But words in a sentence aren’t independent. “I’ll have the ___ and ___” — pick both blanks at the same time without coordinating, and you get “the soup and dessert” when you meant “the soup and salad.” Researchers call this the conditional independence assumption, and it’s the root tension of the whole approach: the more you parallelize, the more the tokens stop agreeing with each other, and quality drops. The LLaDA paper on large language diffusion models scales this approach to 8 billion parameters precisely to study that trade-off.

So the honest picture is a tug-of-war, not a free lunch: parallelizability versus generation quality. Crank the knob toward speed and the text gets sloppier. Crank it toward quality and you unmask fewer blanks per step — sometimes down to one token at a time, which quietly erases the very speed win that made the headline.

I had turned a trade-off into a straight line. That’s the mistake. “Parallel” is a lever with a cost on the other end, not a switch that only makes things better. The same discipline applies to any model-internals claim — I made a related error reading past a headline parameter count, which is why how sparse Mixture-of-Experts routing works reframes the “bigger number” instinct the same way this reframes the “faster” one.

The speed numbers, said precisely (and who to attribute them to)

Because the whole value of a piece like this is accuracy, let me give you the real figures with their sources attached, and no rounding into hype.

Gemini Diffusion. Google DeepMind announced this experimental text diffusion model at Google I/O in May 2025. DeepMind reported roughly 1,479 tokens per second, described as about 5× faster than Gemini 2.0 Flash-Lite, with coding performance comparable to that faster model. Important caveat: at announcement it was an experimental research model behind a demo and a waitlist — not a general product you could just switch to. Treat availability, pricing, and context window as unconfirmed; check the vendor before you assume anything.

Mercury 2, from Inception Labs. Mercury is a diffusion model family; Mercury 2 launched on February 24, 2026. Inception and The New Stack’s coverage of Mercury 2 reported roughly 1,009 tokens per second on a single NVIDIA Blackwell GPU, framed as around 5× faster than Claude 4.5 Haiku Reasoning (~89 tok/s) and GPT-5 Mini (~71 tok/s). Reported reasoning scores included AIME 2025 at 91.1 and LiveCodeBench at 67.3. These are vendor and press figures, not independently verified in my hands — attribute them to Inception and The New Stack, and read them as “reported,” not “proven.”

LLaDA. A masked diffusion model scaled to 8 billion parameters, pre-trained from scratch on 2.3 trillion tokens, competitive with LLaMA3 8B on in-context learning, with an open implementation. This is the closest thing to a look-under-the-hood source for the masked-diffusion approach, and its open weights make it the natural entry point if you ever want to poke at one the way you might with the largest open-source AI model. Later work like “improved” LLaDA and scaling efforts are recent and evolving — I’d label them as active research, not settled fact.

Notice what all three share: the impressive number is throughput on the right kind of task. None of them claims to beat GPT or Claude on general quality across the board, and I’m not going to invent that claim for them.

Where this falls short for someone like me

I want to be honest about the limits of everything above, because the anti-hype read is the one that ages well.

The tunable speed-quality knob sounds great until you realize you usually don’t get to turn it. In a hosted chat product, the vendor sets the step count. You inherit whatever balance they chose, the same way you don’t tune your car’s fuel map. So the “you can dial it” freedom is mostly a builder’s freedom, not a chat user’s.

There’s also the availability gap. Gemini Diffusion was experimental. Mercury is real but sits in a developer-facing lane most of us won’t touch directly. The open-weight options need hardware and comfort I don’t assume you have. “Watch this space” is a very different instruction from “switch your workflow.”

And the replacement story is oversold. The active research direction isn’t diffusion killing autoregression — it’s hybrids that use an autoregressive backbone with diffusion-style refinement, borrowing strengths from both. That’s a “these two combine” future, which fits how real systems already stitch models together in compound AI systems that combine models. Anyone selling you “diffusion replaces ChatGPT next year” is forecasting, not building. Frameworks, not forecasts.

So should a working professional care yet?

Here’s the rule I actually use now, and it’s boring on purpose. Watch it; don’t reorganize around it.

If your coding assistant or writing tool suddenly feels like it prints a full block instantly instead of typing at you, a diffusion model may be under the hood — and that’s a genuinely nice feeling for long outputs. That’s the payoff you might notice as a user, with zero effort on your part. Enjoy it when it arrives.

What I would not do is chase it. I wouldn’t switch tools, pay for early access, or rebuild a workflow around throughput numbers that only shine on long generations and can quietly cost quality when pushed. The mental model is the asset here, not the product. Knowing why a tool got fast tells you when to trust the speed and when to slow it down — which is worth more than any single model launch.

That’s the same posture I take with every model-internals topic: learn the shape of the thing, then let the products come to you.

FAQ

What is a diffusion language model in simple terms? It’s a text generator that starts from a blank, masked draft and fills it in over a few passes, editing the whole thing at once — instead of typing one word at a time. Think of an editor revising a full draft rather than a typewriter with no backspace.

How is a diffusion language model different from ChatGPT? ChatGPT is autoregressive: it writes left to right, one token at a time, and can’t revise a word once written. A diffusion model reads the whole sequence at once, fills blanks in parallel, and can even correct earlier tokens during generation. Different direction, different context.

Are diffusion language models actually faster? For long outputs, often yes — filling many blanks per step saves passes, which is where numbers like 1,000+ tokens per second come from. For short answers, not necessarily; a fixed step count can mean more total compute than a model that emits a few tokens and stops.

Can I use a diffusion language model right now? Somewhat. Mercury from Inception Labs is real but developer-facing. Google’s Gemini Diffusion was experimental and waitlisted. LLaDA has open weights but needs hardware and comfort. Availability, pricing, and context windows keep shifting — check the vendor rather than trusting any fixed claim.

Will diffusion models replace autoregressive models like GPT? Probably not outright. The active research direction is hybrids — an autoregressive backbone plus diffusion-style refinement — which suggests the two complement each other rather than one winning. Treat any “diffusion kills ChatGPT” headline as a forecast, not a fact.

What is masked (absorbing-state) diffusion? It’s the dominant text-diffusion approach. Training progressively replaces real words with a special [MASK] token; once masked, a token stays masked (“absorbing”). The model learns the reverse: unmasking a subset of positions each step, predicting the original words until nothing is blank.

Do diffusion language models make mistakes differently? Yes. Because they revisit the whole draft each step, they can revise earlier tokens in light of later context — a kind of error correction an autoregressive model can’t do, since it never takes a token back. The trade-off is that unmasking too many tokens at once can make them disagree.

The typewriter was never the point

I started with a straight line — parallel, faster, better — and ended with a knob that costs you quality when you turn it too far. That swap is the real takeaway. A diffusion language model isn’t a faster typewriter; it’s a different job entirely, an editor that trades left-to-right certainty for whole-draft revision, and pays for its speed in coordination. Learn the shape, not the launch date.

Next in this Framework Deep Dive series, I want to put the two side by side on the same task — one autoregressive, one diffusion — and log where each one actually helps, so the trade-off stops being a diagram and starts being something I’ve measured. Watch for the run.

---

seonjae — Korean office worker documenting his transition into AI systems, agents, and vibe coding — without a CS background. Shipping in public.