Learning inside a dream

Jul 29, 2026 · #artificial-intelligence #world-models #reinforcement-learning #robotics

Trial and error is unaffordable in the physical world, so an agent can learn a predictive model of that world and practise inside it instead. The interesting part is not that this works, it is what breaks: compounding error caps how long a dream stays useful, and a policy trained in a flawed dream will find the flaw.

Essay cover: Learning inside a dream, showing an encoder, a dynamics model and a controller wired into an imagined loop

Learning by trial and error requires a great many errors. That is affordable when an error is free: a lost round in a video game costs two seconds of compute. It becomes absurd the moment the world is physical. An arm that learns to drive screws by ruining a thousand screws, a car that learns braking distance by crumpling a thousand bumpers, a drone that learns balance by crashing. The cost per mistake explodes, and worse, real time does not accelerate. You cannot run gravity at a thousand times speed.

The alternative was sketched by Jürgen Schmidhuber in the late 1980s and put back on the table forcefully in 2018. Instead of learning in the world, the agent first learns a model of the world: a machine for answering “if I do this, what happens next”. Then it trains inside that prediction, as fast as the hardware allows, without touching reality at all. Ha and Schmidhuber gave the phase a name that stuck. The agent learns inside its own dream.

That inversion is the whole point, and it is easy to miss. A world model is not a forecasting tool built for a human watching a screen. It is a training ground manufactured by the learner, in which it lives millions of lives that never happened.

Three pieces, and two loops

The canonical decomposition has barely moved in eight years. Only the scale has.

Everything hinges on the ratio between the two loops. The green one costs hours and broken hardware, the yellow one costs milliseconds. The engineering problem is to move learning from green to yellow without the yellow becoming a lie.

The encoder compresses the raw stream, millions of pixels per second, into a compact state, a few hundred numbers, usually called a latent state. The dynamics model learns the rules of the game: given that state and an action, predict the next state and the expected reward. The controller is tiny by comparison, reading a state and picking an action. That is the counter-intuitive part. Nearly all the intelligence lives in the world model, and almost none in the policy that acts.

A world model is not a video generator

The confusion has been everywhere for two years, so it is worth cutting cleanly. A video generator produces a plausible sequence from a prompt. A world model must accept, at every step, an action supplied from outside, and answer in a way that is consistent with it. Action conditioning is the dividing line, and it is much harder to get than visual beauty.

Criterion	Video generator	World model
Input at each step	nothing, the scene unrolls	an action chosen by the agent
Hard constraint	visual plausibility	causal consistency, object permanence
Success criterion	it looks real	a policy trained inside it works outside
Typical failure	an extra finger	the object behind you is gone when you turn back

Permanence is the cruel test. Look left, then look right again: the table must still be there, with the same cup in the same place. A model that regenerates the scene from scratch on every frame manufactures a new cup with every glance. Nothing is more toxic for an agent that plans, because it will learn strategies that exploit a world with no memory.

The error that compounds

This is the central technical fact of the field. When you unroll a dream step by step, each step starts from the state predicted at the previous step. A tiny error therefore does not stay tiny: it displaces the input of the next step, which is then wrong by more, and so on. This is compounding error, and it sets the useful horizon of any learned model.

The classical result, the simulation lemma, says roughly this: if your model is off by some small epsilon at each step and you plan over H steps, the gap between the performance you believe you are getting and the one you actually get can grow like epsilon times H squared.

Forget the exponent and keep the picture. You are walking through a forest with a compass that reads one degree off. Two things cost you, not one. First you drift by a degree at every step. Second, each new step is taken from a place that is already wrong, so your next one-degree drift applies to a trajectory that did not start where you thought. The error gets charged twice, and that is the square. The practical lesson is blunt: a model twice as accurate does not buy you twice the horizon, it buys barely more. The only real defence is to come back and fetch a genuine observation regularly.

Two ways to dream. In red, predicting the full image: gorgeous at first, unusable after a few dozen steps. In green, predicting only an abstract summary of the scene: far less impressive to watch, but the exploitable horizon runs much longer.

What a model refuses to predict

That chart carries the real argument of the field, and it is more philosophical than technical. Predicting the full image forces the model to guess the unguessable: the exact shimmer of leaves, the grain of asphalt, the precise shape of a flame. Those details are irreducibly stochastic, and the capacity spent hallucinating them is capacity stolen from what matters, the causal structure.

The counter-proposal, argued by Yann LeCun since his 2022 position paper, is to predict in representation space rather than pixel space: encode the present, encode the future, and train a predictor to map one representation to the other, never reconstructing the image. No decoder, no obligation to look good. The consequence is elegant. Whatever the model cannot predict, it learns not to care about.

The price is real, and this is where it gets interesting. A model that reconstructs nothing can no longer show you its dream. You lose visual inspection, that enormous convenience of watching the film an agent is telling itself and seeing immediately that it is wrong. You are trading legibility for efficiency, and the trade is not settled. Both families are advancing in parallel.

The agent cheats against its own dream

One last phenomenon, and it is almost comic. Train a policy long enough inside an imperfect model and it will eventually find not the best strategy in the real world, but the flaw in the model. If the learned dynamics have a blind spot, say a badly modelled collision that lets you pass through a wall on a precise timing, the policy will discover it and exploit it with formidable enthusiasm, because it is an infinitely cheaper path to reward than the real one.

Nothing is malfunctioning here: the agent optimises exactly what it was given. This is called model exploitation, and it is the most honest reminder in the field. An agent trained in a simulation does not learn the world, it learns the simulation. Practice consists of preventing that confusion: cap dream length, penalise model uncertainty, re-anchor on real data, train several models and distrust the places where they disagree.

Where this stands in 2026

Three results mark the current frontier, and they happen to sit on different sides of the trade-offs above.

A long-horizon task solved without ever playing. Dreamer 4, a 2-billion-parameter agent from Danijar Hafner, Wilson Yan and Timothy Lillicrap, obtains diamonds in Minecraft, a quest requiring a sequence of more than 20,000 mouse and keyboard actions from raw pixels. The remarkable part is not the diamond, it is the method: the agent learns from a fixed offline dataset and never interacts with the game during training, with the policy trained by reinforcement learning entirely inside the learned world model. The model runs interactively in real time on a single GPU, and most of its knowledge comes from unlabelled video, with a small action-labelled set enough to teach conditioning. Treat the headline “a hundred times less data” as a comparison against one specific baseline rather than a general law. The robust claim is the other one: zero environment interaction.

An agent trained in worlds invented by another model. Genie 3 generates navigable 3D environments in real time from a text prompt, at 720p and 24 frames per second, holding consistency for “a few minutes”. SIMA 2, a Gemini-backed generalist agent announced in November 2025, is dropped into them: it navigates, recognises objects, carries out requested tasks, and improves at them without new human demonstrations. One model builds the arenas, the other learns in them. That “few minutes” figure is the most telling and the most honest number in the announcement, because it is precisely the useful-horizon limit described above.

Betting on what you refuse to draw. V-JEPA 2, released in June 2025, is the latent line: roughly a million hours of internet video plus a small amount of robot trajectory data, producing a model that predicts in representation space and is then used for understanding, anticipation and planning in physical settings. No pixel reconstruction anywhere. The first two results show the visible dream, this one bets on the invisible one, and in 2026 the argument between seeing your dream and dreaming far is still open.