Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

Learning reliable surgical manipulation policies is bottlenecked by the scarcity of action-labeled demonstrations. Teleoperated trajectories from systems like the da Vinci Research Kit (dVRK) with synchronized kinematics are costly to collect, yet surgical tasks demand precise contact handling, long-horizon reasoning, and bimanual coordination. Endoscopic video, by contrast, is comparatively inexpensive and abundant — a natural resource for learning world models of surgical scenes. However, prior surgical world models have used video mainly for simulation or policy evaluation, rarely translating learned dynamics into closed-loop control.

Core Innovation

Surgical WAM (World-Action Model) is a unified generative model built on Cosmos Policy that jointly predicts future endoscopic observations and executable surgical robot action chunks. It first learns surgical visual dynamics from action-free video, then is fine-tuned on a fixed budget of action-labeled demonstrations. At deployment, it acts as a closed-loop, receding-horizon controller: it executes a short prefix of each predicted action chunk and replans from the resulting observation.

The central question the paper asks is whether, under a fixed budget of action-labeled demonstrations, action-free video pretraining improves closed-loop surgical manipulation — and the design is explicitly built to answer it.

Results

• On a suite of four simulated surgical manipulation tasks, video pretraining improves average success from 63.5% to 77.8%
• An absolute gain of +20 percentage points on PegTransfer, with the largest improvements on contact-rich and bimanual tasks
• Demonstrates that action-free video provides transferable visual dynamics priors for learning surgical control with limited action supervision

Limitations

Evaluation is limited to four simulated tasks; no physical dVRK or in-vivo validation is reported, so sim-to-real transfer for contact-rich surgical skills remains open. The model builds on the Cosmos Policy base, meaning results may inherit its inductive biases. Task diversity — four tasks — is modest relative to the breadth of real surgical procedures.

Industry Implications

Surgical robotics companies face exactly the cost structure this paper targets: teleoperated, kinematics-labeled data is the expensive bottleneck, while endoscopic video is abundant. Surgical WAM sketches a data-efficient path to scale surgical robot learning, potentially reducing the demonstration-collection burden for autonomy features on platforms like dVRK. If video pretraining transfers to real instruments, it could accelerate semi-autonomous assistance in suturing, tissue handling, and other contact-rich subtasks — a high-value direction for the surgical robotics industry.