How Google built its Gemini robotics models

“We’d trained models to help robots with specific tasks and to understand natural language before, but this was a step change,” Carolina says. “The robot had never seen anything related to basketball, or this specific toy. Yet it understood something complex — ‘slam dunk the ball’ — and performed the action smoothly. On its first try.” This all-rounder robot was powered by a Gemini Robotics model that is part of a new family of multimodal models for robotics. The models build upon Gemini 2.0 through fine-tuning with robot-specific data, adding physical action to Gemini’s multimodal outputs like text, video and audio. "This milestone lays the foundation for the next generation of robotics that can be helpful across a range of applications," said Google CEO Sundar Pichai when announcing the new models on X. The Gemini Robotics models are highly dextrous, interactive and general, meaning they can drive robots to react to new objects, environments and instructions without further trainin ... Read full article.