NVIDIA Expands Cosmos 3 Platform With Four Open-Weight Models

NVIDIA has expanded its Cosmos world model development platform by introducing four new open-weight checkpoints under the Cosmos 3 model family [1]. The release aims to support interactive world simulation and multi-sensor, multi-view driving scenarios [1].

Overview of the New Checkpoints

NVIDIA's latest addition to the Cosmos 3 family includes four post-trained examples based on Cosmos 3 Nano, each featuring 16 billion trainable parameters [1]. The models are made available under the OpenMDW1.1 license terms, which permit both commercial and non-commercial use [1].

The four released model checkpoints are:

• Cosmos3-Nano-Transfer-Auto: Designed for generating multi-view driving data with variable weather, lighting, and time of day [1].
• Cosmos3-Nano-Sim-Auto: A streaming variant that delivers synchronized camera and LiDAR observations for driving simulations [1].
• Cosmos3-Nano-Sim-Transfer: Converts simulator controls and text prompts into photorealistic video outputs [1].
• Cosmos3-Nano-Sim-Bimanual: Predicts upcoming video frames based on robot or camera actions [1].

Application in Driving and Robotics

According to the platform documentation, these models address distinct simulation requirements across industries. For autonomous driving, Cosmos3-Nano-Transfer-Auto acts as a bidirectional model for offline synthetic data generation across up to 11 camera views, including fisheyes, alongside joint RGB-LiDAR generation [1]. Its causally distilled streaming counterpart, Sim-Auto, generates successive small chunks of synchronized camera video and LiDAR in minimal denoising steps to facilitate interactive rollouts [1].

In robotics, Sim-Transfer utilizes spatial controls from a physics simulator alongside text prompts to render color observations that a policy can consume [1]. Meanwhile, Sim-Bimanual simulates environment evolution in response to two-arm robot actions, supporting YAM and AgiBot embodiments controlled via policies or human-operated joysticks [1].

Architecture and Training Recipe

Underpinning Cosmos 3 is a Mixture-of-Transformers backbone that connects text-processing and visual-generation networks through attention [1]. The models operate on compressed video representations known as latents, utilizing temporal-causal attention to read available history and controls while blocking future targets [1].

NVIDIA outlines a four-stage training recipe for these models, starting with bidirectional adaptation in Stage 0 to learn control-to-video relationships from recorded clips [1]. Stage 1 involves teacher forcing to predict chunks from earlier observations, followed by consistency distillation in Stage 2 to train a student model capable of generating chunks in four denoising steps [1]. The final stage, self-forcing, trains the model on its own generated history to mitigate compounding errors caused by mispredicted objects serving as context for later frames [1].

World models in artificial intelligence attempt to simulate physical environments, predicting how visual and spatial parameters evolve in response to various controls and actions. Such models are critical for training autonomous driving systems and robotic agents safely before deploying them in the physical world.

The release of open-weight checkpoints with commercial and non-commercial usability provides researchers and developers in India and globally with standardized foundational architectures. This lowers the barrier to entry for developing complex simulations in autonomous mobility and robotic automation.

The availability of open-weight interactive simulation models could accelerate the development cycles for autonomous vehicle software and robotic dexterity by reducing the need to build foundational simulation engines from scratch. Researchers may increasingly leverage these frameworks to benchmark driving policies and bimanual manipulation tasks.

The platform documentation notes that evaluating where models faithfully support policy learning, understanding their operational limits, and addressing performance shortcomings remain ongoing technical challenges.