NVIDIA to power Gen AI-based physical AI acceleration with new Omniverse tech, Cosmos, and Isaac GR00T Blueprint

At CES 2025, NVIDIA unveiled its groundbreaking advancements in generative AI applications, showcasing its commitment to physical AI to transform various industries.

Building on the foundation of Omniverse, NVIDIA introduced tools designed to create 3D worlds with physical attributes and photorealistic outputs, enabling AI models to better understand and interact with real-world scenarios. For instance, NVIDIA NIM microservices, such as USD Code and USDSearch, accelerate the development of OpenUSD assets through text prompts. Additionally, the NVIDIA Edify SimReady generative AI model automates the labeling of 3D assets with physics and material attributes, reducing processing time for 1,000 objects from over 40 hours to mere minutes.

NVIDIA also launched Cosmos, an innovative platform equipped with open, pretrained world foundation models for generating physics-aware videos and world states for physical AI development. Trained on an extensive dataset of 18 quadrillion tokens, including robotics, drones, autonomous driving, and synthetic data, Cosmos bridges the simulation-to-reality gap by creating highly accurate 3D simulations. Its integration with Omniverse ensures realistic outputs while minimizing errors like hallucinations typically seen in world models.

To further support physical AI, NVIDIA introduced four OpenUSD-based digital twin blueprints:

1. Mega Blueprint: Simulates and tests robot fleets in digital twin environments.
2. Autonomous Vehicle Simulation Blueprint: Facilitates closed-loop testing, replaying driving data, and generating ground-truth data for AV development.
3. Omniverse Spatial Streaming Blueprint: Enables immersive streaming of large-scale digital twins via Apple Vision Pro.
4. Real-Time Digital Twins for CAE: Offers workflows powered by NVIDIA CUDA-X™ and physics AI for real-time computational fluid dynamics and physics visualization.

The Omniverse Spatial Streaming Blueprint has already been adopted by companies like Katana Studio, which developed custom car configurators for Nissan and Volkswagen, and Innoactive, which enabled high-resolution project reviews for Volkswagen Group and Syntegon.

In the humanoid robotics domain, NVIDIA revealed a suite of tools to accelerate next-generation robot development. The highlight was the Isaac GR00T Blueprint, a synthetic motion generation system designed to train humanoid robots through imitation learning. This approach allows developers to generate extensive datasets using minimal real-world human demonstrations, significantly reducing data collection costs.

The GR00T system includes three workflows:

• GR00T-Teleop: Uses Apple Vision Pro to capture human actions in a digital twin environment for robots to imitate.
• GR00T-Mimic: Expands datasets using simulation-based mimicking.
• GR00T-Gen: Utilizes Omniverse and Cosmos for domain randomization and 3D upscaling to exponentially increase data volumes.

The resulting datasets are used to train robots via NVIDIA Isaac Lab, enabling safe and efficient interaction with their environments.

Industry leaders such as Boston Dynamics and Figure have already adopted the GR00T system, showcasing its transformative potential. Developers and manufacturers interested in early access can apply now.