Nvidia Introduces GR00T 2: AI Model That Teaches Robots Human Skills from Video

Nvidia has introduced GR00T 2, a new artificial intelligence model for robot control whose key distinction is that it learns not only to execute actions but also to predict how the environment will change as a result of those actions. It is precisely this capability that enables skills demonstrated by a human or another robot to be transferred to machines with fundamentally different hardware designs. In a test across 50 tasks, the model succeeded in 40% of attempts without any prior demonstrations on the target robot; a single demonstration per task raised the success rate to 71%, and five demonstrations increased it to 95%. Furthermore, adapting to each task required less than three minutes of demonstration data and about 30 minutes of fine-tuning on a single Nvidia RTX 5090 graphics card.

World-Action Model Architecture: Predicting the World and Actions Simultaneously

At the core of GR00T 2 lies the World-Action Model (WAM) architecture, which simultaneously predicts future video and robot actions. It receives the latest camera frames, a textual instruction, and the machine's state as input, after which it generates a sequence of future images and corresponding movements. This approach allows the model to comprehend physical interactions by observing not only robots but humans as well. The overall architecture is compatible with parallel-jaw manipulators, articulated multi-fingered robotic hands, and humanoid platforms, while small specialized modules translate the high-level plan into motor commands for a specific machine.

Training Dataset: Over 400,000 Hours of Human Video

For pre-training, Nvidia utilized 402,730 hours of first-person video showing humans performing various working tasks, 10,184 hours of demonstrations with UMI manual grippers, and 23,500 hours of robot teleoperation. The data covers YAM and Franka manipulators, Sharpa and Wuji robotic hands, and the Unitree G1 humanoid; human videos make up the bulk of the training dataset. The company specifically tested their impact across 72 tasks for the Dexter robot: increasing the proportion of human material from 0 to 100% raised the success rate in video simulation from 64% to 77%, and on the real robot from 50% to 70%. Nvidia has released a portion of the Human-1M dataset, comprising 100,000 hours of video, on Hugging Face.

Skill Transfer: From Head-Mounted Camera Video to Robotic Hands

One of the primary achievements of GR00T 2 is skill transfer between different types of robots and humans. For instance, the model was trained on laboratory procedures using video captured from head- and wrist-mounted human cameras, and then tested on Dexter equipped with two robotic hands: in all four laboratory tasks, the robot succeeded without receiving any teleoperation demonstrations. In other trials, the model transferred skills demonstrated via UMI manual input to the YAM manipulator, and demonstrations from the DexUMI wearable exoskeleton to the Tianji robot equipped with two ROBOTERA hands. Mastered actions include sorting items on a conveyor belt, assembling toys, handling parts, preparing breakfast, and laboratory procedures.

Collaboration with a Reasoning AI Model

Nvidia also demonstrated how GR00T 2 works in tandem with a reasoning AI model: the latter breaks down complex tasks into steps, monitors progress, and determines subsequent steps, while GR00T 2 is responsible for the physical execution of actions that are difficult to pre-program. In trials featuring randomly scattered parts, this system assembled a toy airplane, tractor, and Hot Wheels car in 98, 97, and 97 out of 100 cases respectively; without the agent's help, the same model succeeded in 71–84 cases, and using only a pre-fine-tuned policy, in 23–69 cases. In another series of trials, the joint system connected 5 network cables to a switch in 19 out of 20 cases and inserted 5 chargers into power outlets in all 20 cases. The agent's visual cues—marking the grasp point and transfer destination on the image—increased the success rate of short pick-and-place tasks from 40% to 92% on YAM and from 30% to 70% on Dexter.

GR00T 2 Turbo: Tens of Milliseconds Latency On-Board the Robot

Simultaneously, Nvidia unveiled GR00T 2 Turbo, a lightweight version of the model designed to run directly on-board a robot. It utilizes a 2-billion-parameter transformer and a more compact video representation to lower computational costs; combined with execution optimizations, this reduced median prediction latency by 66–86% compared to running the standard model. On an RTX 5090 GPU, a single prediction takes 38 ms, while on the Jetson Thor on-board computer it takes 124 ms. When operating on the Unitree G1 humanoid, the model plans whole-body movements in 1.6-second blocks approximately once per second, and the SONIC controller converts them into joint commands at a 50 Hz frequency.