Machines That Imagine: how AI learns to simulate the world before it acts

Jul 12, 2026 · #ai #world-models #machine-learning

An agent that only reacts to what it sees is brittle. To act well, it needs an internal simulator (a world model) that predicts what happens next. That idea, and the 2025–26 race between two ways of building it, may be the real road past today's language models.

Cover: state feeds a world model that imagines possible futures and returns the best action, with the note try it in your head then act for real — The quiet shift in AI: from reacting to what is on screen, to predicting what happens next, and choosing accordingly.

Start with yourself. Someone tosses you a set of keys, and your hand closes in the right place before the keys arrive. You did not compute a trajectory: your brain simulated what would happen next, a fraction of a second ahead, using an internal model of how objects fall. You have had that model since childhood. It is exactly what an AI lacks when it only reacts to the pixels in front of it: with no anticipation it is brittle, blind to what its own actions will cause. The fix has a name: a world model.

A simulator in the head

A world model is a compressed internal simulator: given the current state and a candidate action, it predicts the next state. In 2018, David Ha and Jürgen Schmidhuber published a landmark paper titled, fittingly, “World Models.” Their agent learns a miniature, dreamed version of its environment (a racing game, a shooter) and then, the detail that stuck, it can train inside its own dream, in the environment hallucinated by its model, without touching the real game. It learns in its imagination, then carries the skill back to reality.

The payoff: instead of trying actions in the world (costly, slow, sometimes dangerous), the agent tries them in its head, keeps the best, and only then acts. Planning is imagining the possible continuations, the same move as your hand anticipating the keys.

Two schools: paint the pixels, or grasp the gist

In 2025–2026 the world-model race splits into two opposing philosophies. The first paints the future, pixel by pixel: interactive video generators such as OpenAI’s Sora or DeepMind’s Genie 3 (August 2025), which produce photorealistic interactive worlds in real time: type a description and get a navigable 3D environment at 24 frames per second, with objects that persist and physics that emerges. The second draws nothing: it predicts in an abstract (“latent”) space, the path of LeCun’s JEPA (Meta) and Hafner’s Dreamer: the model does not bother to redraw every fluttering leaf, it predicts only what matters for acting.

Predicting every pixel wastes effort on irrelevant detail (the shimmer of leaves); predicting in a latent space captures the gist, what actually changes for the decision. That is LeCun's bet: to act, better to understand the plot than to repaint the set.

Keep one image for “latent”: the novel versus the film. The pixel model shoots the whole film: every frame, every extra. The latent model writes the plot summary: “the car brakes, turns left, misses the wall.” To decide what to do, the summary is enough, and it costs a thousand times less. Why does any of this matter beyond the spectacle? Because a world model lets an agent plan by imagining, then act in the real world. In 2025, V-JEPA 2 (Meta), trained on over a million hours of internet video, controlled a real robot arm in a lab it had never seen, with no retraining (zero-shot). Giving a machine a body is not enough; it needs a model of what that body will cause in the world. And there is a striking echo here of biology: a dream may be the brain running its own world model offline, replaying and anticipating.

Where this stands in 2026

On the “paint the future” side, DeepMind unveiled Genie 3 in August 2025, billed as the first real-time interactive world model that generates photorealistic environments from a plain text prompt (24 fps, minutes of consistency, object permanence). It blurs the line between a game engine and a machine’s dream, and offers an endless training ground for agents (DeepMind · Genie 3). On the “grasp the gist” side, Meta released V-JEPA 2 with zero-shot robot control, and Yann LeCun left Meta to found a company devoted entirely to world models, the largest bet yet on the thesis that large language models will never reach general intelligence, and that the road forward runs through world models instead; OpenAI, meanwhile, reportedly shifted resources from Sora toward longer-term “world simulation” research (World model overview · Wikipedia). The centre of gravity is sliding from “predict the next word” to “predict the next state of the world”, a close cousin of the predictive brain in neuroscience.

A simulator in the head

Two schools: paint the pixels, or grasp the gist

Where this stands in 2026

Further reading