Google-spinoff Waymo is in the midst of expanding its self-driving car fleet into new regions. Waymo touts more than 200 million miles of driving that informs how the vehicles navigate roads, but the company’s AI has also driven billions of miles virtually, and there’s a lot more to come with the new Waymo World Model. Based on Google DeepMind’s Genie 3, Waymo says the model can create “hyper-realistic” simulated environments that train the AI on situations that are rarely (or never) encountered in real life — like snow on the Golden Gate Bridge.
Until recently, the autonomous driving industry relied entirely on training data collected from real cars and real situations. That means rare, potentially dangerous events are not well represented in training data. The Waymo World Model aims to address that by allowing engineers to create simulations with simple prompts and driving inputs.
Google revealed Genie 3 last year, positioning it as a significant upgrade over other world models by virtue of its long-horizon memory. In Google’s world model, you can wander away from a given object, and when you look back, the model will still “remember” how that object is supposed to look. In earlier attempts at world models, the simulation would lose that context almost immediately. With Genie 3, the model can remember details for several minutes.
Snow in San Francisco. Snow in San Francisco.
Autoregressive world models like Genie don’t actually create 3D spaces, but instead render video quickly enough that it feels like an explorable world. Naturally, video games are cited as a prime application for world models, so much so that gaming company stocks dropped when Google recently expanded access to the technology as Project Genie. However, the latency and still rather short memory of Genie make gaming uses far from a certainty. Nevertheless, Waymo says Genie 3 is actually ideal for simulating the kind of data it needs to train self-driving cars.