HOST: Robots Acquire Manipulation Skills in Seconds from a Single Human Video

· Editorial Team estimated
one-shot learning imitation learning skill acquisition human-to-robot transfer manipulation

HOST enables a robot to acquire novel manipulation skills in an average of 29 seconds from a single human video, while retaining previously mastered skills. It achieves 62% average success rate across tasks — exceeding zero-shot baselines by 45% and even surpassing models fine-tuned on 50 robot demonstrations per task, requiring 50x fewer demonstrations and acquiring each skill 507x faster.

Paper · arXiv:2607.20033

Background

The ability to acquire skills rapidly and effortlessly while retaining those already mastered is essential for practical robots. Current methods rely on a cumbersome training-time loop that is costly and slow, while eroding skills already mastered. This makes scaling robot learning to diverse tasks impractical — each new skill demands extensive robot-specific demonstration data and retraining, limiting deployment in dynamic environments.

Core Innovation

HOST (Human-to-robot One-Shot Skill AcquisiTion) resolves skill acquisition through a cascade of self-grounded prediction. It first estimates the robot’s progress within the demonstrated task, then translates the upcoming progression into the robot’s own future observations, and finally derives actions from these predicted observations. This cascade is trained on targets coupled to the video demonstration by mapping the robot trajectory and the video demonstration onto a shared task progress manifold, then redefining each target to align with the future progression of the video. HOST thus enables the robot to actively follow the demonstrated procedure and adapt it to its own embodiment.

Key Results

HOST acquires novel skills at inference time from a single human video in an average of 29 seconds and achieves a 62% average success rate. It exceeds the zero-shot baseline by 45% while retaining previously mastered skills. HOST even exceeds the baseline fine-tuned on 50 robot demonstrations per task while requiring 50 times fewer demonstrations and acquiring each skill 507 times faster. These results demonstrate a dramatic improvement in the data efficiency and speed of robot skill acquisition.

Limitations

The 62% success rate, while significantly better than baselines, leaves room for improvement in robustness. Performance likely depends on the quality and clarity of the human demonstration video, and tasks requiring precise force control or contact-rich interactions may pose additional challenges. The method’s scalability to highly dexterous or multi-step tasks with long horizons remains to be demonstrated.

Industry Implications

HOST’s ability to acquire skills from a single human video in seconds with no erosion of existing capabilities could transform how robot skills are deployed in industry. Instead of weeks of data collection and model retraining, operators could demonstrate a new task once via video and have the robot operational immediately. This dramatically lowers the barrier to deploying robots in dynamic environments like warehouses, fulfillment centers, and domestic settings where task requirements change frequently.