Google Turns Street View Into Interactive World Simulator

Imagine dropping into your favorite street in Paris, then switching the sunshine to snow, or viewing it from a robot's perspective instead of your own.

That's now possible thanks to Google DeepMind's latest breakthrough. The company just connected two decades of Street View imagery (280 billion images across 110 countries) to Project Genie, an AI that generates interactive, customizable environments.

The integration launched at Google's I/O developer conference this week. It opens up possibilities that go far beyond checking out vacation rentals or revisiting childhood neighborhoods.

"You can imagine how potentially powerful it is to combine this rich source of real-world information and data with an ability to simulate worlds," says Jack Parker-Holder, a research scientist at DeepMind. The system lets users take any real street and simulate it under different conditions.

Want to see London in rare sunshine? Done. Need to test how a robot handles a snowy New York block it's never encountered? The AI creates that scenario from existing street data.

The Ripple Effect

The technology is already making waves in practical applications. Waymo, Google's self-driving car division, uses an earlier version of Genie to train vehicles on incredibly rare events like tornadoes or unexpected wildlife encounters.

Adding Street View data means Waymo could prepare to launch in new cities worldwide by simulating those streets under countless conditions. Traditional simulators only show the view from a car's perspective, but Genie with Street View can shift viewpoints to pedestrians, delivery robots, or any other agent navigating real spaces.

The researchers envision uses in education, gaming, and robotics training. Google rolled out access to some Ultra AI subscribers in the United States today, with global expansion coming in the next few weeks.

The technology isn't perfect yet. The simulations look more video game quality than photorealistic, and the AI doesn't fully understand physics. In one demo, a simulated runner passed straight through cacti in Joshua Tree National Park.

But Parker-Holder believes those gaps will close quickly. "I think for this kind of model, it's maybe six to 12 months behind video in terms of the accuracy and quality," he says.

The real breakthrough is spatial continuity. When you turn 360 degrees in a Genie simulation, the AI correctly remembers and recreates what's behind you. From there, it can build entirely new environments on top of real geography.

Jonathan Herbert, director of Google Maps, sees this as the beginning of something bigger. His team has long imagined building richer world models on top of Street View's 20 years of data collection.

The future might let anyone explore, learn from, and interact with simulations of real places in ways we're just beginning to imagine.