From Freehand Demonstrations to Autonomous Robotic Ultrasound

· Editorial Team estimated
medical robotics robotic ultrasound imitation learning flow policy force feedback

Ultrasound acquisition is strongly operator-dependent, and expert skill couples anatomical landmarks, live image interpretation and probe-tissue contact. This work proposes an anatomy-anchored imitation-learning framework: a placement module identifies a task-specific anatomical start point from external vision and fixes the initial probe pose as the scanning reference frame, while a flow-based policy generates scanning motion from anchor-relative pose, ultrasound images, an egocentric probe camera and contact force, with a termination head predicting completion. On 16 unseen participants it reaches 96%, 94%, 88% and 94% success for kidney, gallbladder, liver standard planes and radial artery sweeping, beats a fine-tuned pi_0 by 10-42 points on abdominal tasks, and transfers to a second robot without retraining.

Paper · arXiv:2610.32525

Ultrasound is one of the few imaging modalities still entirely dependent on manual craft. The same machine and the same probe can yield diagnostic-quality planes under one sonographer and unusable ones under another. Automating expert scanning is hard less because of motion control than because the skill is tacit: the clinician reads anatomical landmarks, interprets the live image, and adjusts force and angle through the feel of probe-tissue contact, all at once. That coupling resists hand-written control laws.

Core innovation

The authors frame this as a representation problem. The observation and action spaces of an imitation policy should reflect the anatomical structure of scanning rather than delegating everything to a generic end-to-end network. The framework has two layers.

Anatomy-anchored placement. A placement module identifies the task-specific anatomical starting point from external visual observations and establishes the initial probe pose as the reference frame for the entire scan. The policy then operates in anchor-relative coordinates instead of continuously relying on third-person vision, which reduces its dependence on how the robot and patient happen to be arranged.

Anchor-relative scanning policy. A flow-matching policy conditions on anchor-relative probe pose, ultrasound frames, a probe-mounted camera view and contact force to generate scanning motion, and a termination head predicts task completion. Training data comes from a handheld acquisition platform that records multimodal demonstrations during ordinary freehand scanning, with no robot involved at collection time.

Results

  • Evaluated on 16 unseen participants: 96%, 94%, 88% and 94% success for kidney, gallbladder and liver standard-plane acquisition and radial artery sweeping.
  • On abdominal tasks, 10-42 percentage points better than a fine-tuned pi_0, with shorter time-to-target, lower peak contact force and lower trajectory jerk.
  • Ablations confirm that anatomical anchoring, egocentric vision and force feedback play complementary roles; removing any of them costs performance.
  • End-to-end trials including automatic placement reach 90-100% success.
  • Policies transfer to a second robot without retraining, at 83-100% success.

Limitations

The cohort is 16 participants, and the abstract does not say whether pathological anatomy or varying body habitus was included, so generalisation to a real clinic is unproven. Failure modes of the termination head and of automatic placement are not broken out, and no explicit safety threshold is given for contact force, which matters more clinically than mean success rate. Multi-site, multi-vendor prospective evaluation is absent.

Industry implications

Sonographer shortages and inter-operator variability are the two practical brakes on ultrasound screening, and robotic ultrasound is stuck between “it can scan” and “you can trust the scan”. The most portable idea here is not the policy network but the anchored reference frame: it turns anatomy into an explicit coordinate convention, decoupling the policy from a specific arm and bed layout, which is why retraining-free transfer comes almost for free. For device vendors and tele-ultrasound platforms, that means automated acquisition can ship as per-anatomy deployable skills rather than a model retrained for every site. The handheld platform matters too, because it lets data accumulate inside real clinical workflows instead of consuming robot hours.