[AI]Ai2: Building physical AI with virtual simulation data

[ChatGPT 0]

Virtual simulation data is driving the development of physical AI across corporate environments, led by initiatives like Ai2’s MolmoBot.

Instructing hardware to interact with the real world has historically relied on highly expensive and manually-collected demonstrations. Technology providers building generalist manipulation agents typically frame extensive real-world training as the basis for these systems.

For some context, projects like DROID include 76,000 teleoperated trajectories gathered across 13 institutions, representing roughly 350 hours of human effort.

This is the hidden content, please

• Sign In
• or
• Sign Up

DeepMind’s RT-1 required 130,000 episodes collected over 17 months by human operators. This reliance on proprietary, manual data collection inflates research budgets and concentrates capabilities within a small group of well-resourced industrial laboratories.

“Our mission is to build AI that advances science and expands what humanity can discover,” said Ali Farhadi, CEO of Ai2. “Robotics can become a foundational scientific instrument, helping researchers move faster and explore new questions. To get there, we need systems that generalise in the real world and tools the global research community can build on together. Demonstrating transfer from simulation to reality is a meaningful step in that direction.”

Researchers from the

This is the hidden content, please

• Sign In
• or
• Sign Up

(Ai2) offer a different economic model with MolmoBot, an open robotic manipulation model suite trained entirely on synthetic information. By generating trajectories procedurally within a system called MolmoSpaces, the team bypasses the need for human teleoperation.

The accompanying dataset, MolmoBot-Data, contains 1.8 million expert manipulation trajectories. This collection was produced by combining the MuJoCo physics engine with aggressive domain randomisation, varying objects, viewpoints, lighting, and dynamics.

“Most approaches try to close the sim-to-real gap by adding more real-world data,” said Ranjay Krishna, Director of the PRIOR team at Ai2. “We took the opposite bet: that the gap shrinks when you dramatically expand the diversity of simulated environments, objects, and camera conditions. Our latest advancement shifts the constraint in robotics from collecting manual demonstrations to designing better virtual worlds, and that’s a problem we can solve.”

Generating virtual simulation data for physical AI

Using 100 Nvidia A100 GPUs, the pipeline created roughly 1,024 episodes per GPU-hour, equating to over 130 hours of robot experience for every hour of wall-clock time.

Compared to real-world data collection, this represents nearly four times the data throughput, directly impacting project return on investment by accelerating deployment cycles.

The MolmoBot suite includes three distinct policy classes evaluated on two platforms: the Rainbow Robotics RB-Y1 mobile manipulator, and the Franka FR3 tabletop arm.  The primary model, built on a Molmo2 vision-language backbone, processes multiple timesteps of RGB observations and language instructions to dictate actions.

Hardware flexibility with Ai2’s MolmoBot

For

This is the hidden content, please

• Sign In
• or
• Sign Up

environments where resources are constrained, the researchers provide MolmoBot-SPOC, a lightweight transformer policy with fewer parameters. MolmoBot-Pi0 uses a PaliGemma backbone to match the architecture of Physical Intelligence’s π0 model, permitting direct performance comparisons.

During physical testing, these policies demonstrated zero-shot transfer to real-world tasks involving unseen objects and environments without any fine-tuning.

In tabletop pick-and-place evaluations, the primary MolmoBot model achieved a success rate of 79.2 percent. This outperformed π0.5, a model trained on extensive real-world demonstration data, which achieved a 39.2 percent success rate. For mobile manipulation, the policies successfully executed tasks such as approaching, grasping, and pulling doors through their full range of motion.

Providing these varied architectures allows organisations to integrate capable physical AI systems without being locked into a single proprietary vendor ecosystem or extensive data collection infrastructure.

The open release of the entire MolmoBot stack – including the training data, generation pipelines, and model architectures – permits internal auditing and adaptation. Anyone exploring physical AI can leverage these open tools for the simulation and building of capable systems while controlling costs.

“For AI to truly advance science, progress cannot depend on closed data or isolated systems,” continues Ali Farhadi, CEO of Ai2. “It requires shared infrastructure that researchers everywhere can build on, test, and improve together. This is how we believe physical AI will move forward.”

See also:

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up

Want to learn more about AI and big data from industry leaders? Check out

This is the hidden content, please

• Sign In
• or
• Sign Up

taking place in Amsterdam, California, and London. The comprehensive event is part of

This is the hidden content, please

• Sign In
• or
• Sign Up

and is co-located with other leading technology events including the

This is the hidden content, please

• Sign In
• or
• Sign Up

. Click

This is the hidden content, please

• Sign In
• or
• Sign Up

for more information.

AI News is powered by

This is the hidden content, please

• Sign In
• or
• Sign Up

. Explore other upcoming enterprise technology events and webinars

This is the hidden content, please

• Sign In
• or
• Sign Up

.

The post

This is the hidden content, please

• Sign In
• or
• Sign Up

appeared first on

This is the hidden content, please

• Sign In
• or
• Sign Up

.

This is the hidden content, please

• Sign In
• or
• Sign Up