π€ NVIDIA has unveiled GR00T 2, a new generation of foundation models for robots. The main idea: the robot learns by watching how humans work.
GR00T 2 predicts both the future of video and the robot's actions simultaneously. This means the model not only issues commands for motors but also envisions how the scene should change after an action. Thanks to this, it understands physics from ordinary videos with humans and then transfers skills to robots that have never performed the task.
Training data:
- 402,730 hours of first-person video where people work with their hands
- 10,184 hours of demonstrations with UMI hand grasp on 2,897 tasks
- 23,500 hours of teleoperation with robots: from simple manipulators to the humanoid Unitree G1
NVIDIA has released 100,000 hours of this human video in open access on Hugging Face. The more human video used in training, the better the robot: moving from 0% to 100% of this data increased success on 72 tasks from 50% to 70%.
Results:
- On 50 tasks that the robot saw only performed by humans or other robots, GR00T 2 succeeds in 40% of cases without a single attempt. Five demonstrations and 30 minutes of fine-tuning on one RTX 5090 raise this to 95%
- A robot with two five-fingered hands inserts filter paper into a funnel and wets it in 65% of attempts, even though it only saw humans do it
- From demonstrations with a wearable exoskeleton glove, the robot prepares yogurt with banana and lights a match to ignite a lamp (this is on video)
- A lightweight version, GR00T 2 Turbo, runs directly on the humanoid G1: 124 ms per prediction, and the robot serves popcorn in all 48 attempts
I have already written that I am waiting for a GPT moment in robotics. The main bottleneck has always been data: teleoperation is expensive and slow, while video with humans is an almost infinite resource. If robots truly learn from it, the amount of data will increase by orders of magnitude.
https://research.nvidia.com/labs/gear/gr00t-2/
#nvidia@rvnikita_blog #robotics@rvnikita_blog #ai@rvnikita_blog #gr00t@rvnikita_blog