NVIDIA opened four world simulators. The entry ticket is a 16B model and formidable hardware

NVIDIA has extended Cosmos 3 with four open-weight checkpoints for interactive simulation of robots and autonomous vehicles. Each is based on Cosmos3-Nano with 16 billion trainable parameters and is available under OpenMDW 1.1 for commercial and noncommercial use subject to its terms.

Four checkpoints split simulation across roads, sensors and robot arms

Transfer-Auto generates as many as 11 synchronized camera views and can pair them with LiDAR. Sim-Auto streams short successive chunks of camera video and LiDAR as driving controls change. Sim-Transfer turns depth, segmentation, edges or blurred simulator output into RGB video. Sim-Bimanual predicts the next observation from two-arm robot or camera actions.

The models do not create a complete simulator by themselves. Closed-loop driving requires a runtime and renderer that updates spatial controls after every action. With Sim-Transfer, a conventional simulator still computes physics and ground truth state while the model provides the image.

Open weights give teams a shared base for their own sensor data

The practical value sits in the post-training recipes. A robot or vehicle maker can adapt a model to its camera layout, LiDAR and operating domain instead of building a generator from scratch. Shorter generated chunks also let new actions influence the next observation more often.

For developers, this is mainly infrastructure for synthetic data and policy testing. It can widen coverage of weather, lighting and traffic situations that are expensive or dangerous to collect in reality.

An open-weight model is neither a cheap nor a faithful world

Published requirements include Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit. The vLLM-Omni setup was tested on one GB200, and NVIDIA explicitly says this is not a minimum memory specification. It is a reproduction path, not a promise of affordable operation.

NVIDIA also lists temporal inconsistency, object morphing and implausible dynamics in long, high-resolution or physically complex output. A short run proves that the pipeline executes. It does not prove that simulation can safely replace real tests.

Transfer from simulation to a real machine will decide the value

Teams should watch held-out episode results, stability across long rollouts and behavior outside the training domain. In autonomous driving, geometry must remain consistent between cameras and LiDAR.

The decisive signal will come from sim-to-real tests. If a policy trained with Cosmos performs better on a physical machine while using less real data, the checkpoints matter. Until then, beautiful video remains an expensive set.

Lilith's verdict

NVIDIA supplied the movie set and instructions for rebuilding it. Now a robot must cross a real laboratory without hitting the wall the model forgot.