Google taught AI to build living worlds with sound on the fly

On October 27 Google DeepMind unveiled Genie 4, the next version of its world model. That's a neural network that generates a virtual environment on the fly and works out how it reacts to what you do. You move around with the keyboard like in a game and steer the scene with plain text: ask for rain, move the camera, drop in a person or a car.

The headline feature is sound. It's DeepMind's first world model that generates audio together with the picture: wind, surf, music and even character dialogue. The characters respond to what the user is doing in real time.

Under the hood it leans on the Gemini family. Gemini Omni keeps the world looking and behaving believably, while Gemini 3.8 Flash watches the scene between prompts, breaks complex goals into steps and keeps the simulation from falling apart. Physics got better too: collisions, aerodynamics and human movement look noticeably more natural than in Genie 3. A single scene now holds together for up to two minutes, twice as long as before.

DeepMind is upfront about the weak spots. The longer a simulation runs and the more edits you make, the more details and objects drift, and in-world audio and text still aren't always accurate.

For now Genie 4 is limited to trusted testers from film, journalism, education, game development and engineering. Next in line are Google AI Ultra subscribers in Project Genie (plans start at $100 a month) and early access partners in the Gemini API and Google AI Studio. There's no firm date or developer pricing yet, just a waitlist.

A game engine from a text prompt that lasts two minutes still feels like a demo. For rough takes and prototypes, though, it's already a real tool, if you get in.