HALO: Humanoid Autonomy Learned in a Thousand Synthetic Homes

Every humanoid demo eventually runs into the same accounting problem. A policy that maps camera pixels and a free-form instruction to joint commands needs paired supervision: this scene, this sentence, this motion. Nobody has a corpus of that at scale, and the two ways of making one are both expensive. Teleoperation and wearable capture cost at least an hour of skilled human time for every hour of data collected, which means a dataset large enough to cover the long tail of household variation is also large enough to be a staffing problem.

HALO, from Apple’s AIML research group with Vladlen Koltun among the authors, attacks the accounting rather than the policy. Instead of recording humans in homes, it synthesizes the paired data inside a thousand procedurally generated homes, and trains a humanoid autonomy system on the result. The corpus grows with compute. That is a different scaling law from the one the field has been living under.

Core innovation

Procedural homes as the data engine. A thousand generated houses is not a rendering trick; it is a coverage argument. Each home can vary layout, furniture, object placement, doors and clutter, and every variation comes with ground-truth geometry and a language-addressable object list. The pipeline turns that into vision-language-motion triples, which is precisely the supervision that teleoperation produces only at human speed.

Skills, not end-to-end magic. HALO acquires a repertoire of discrete physical skills: sitting on a chair, navigating to a named object, stepping sideways through a half-open door, crouching and crawling beneath a table, pushing a box. Each is a policy in its own right. Composite, long-horizon behaviour comes from orchestrating them rather than from hoping a single monolithic policy absorbs every transition.

Perception tied to the language interface. The interesting skill list is a hint about where the difficulty actually lives. Sidling through a half-open door and crawling under a table are not manipulation problems; they are whole-body perception problems where the robot must estimate free space from its own body’s envelope. Navigating to a named object requires grounding language in a scene it has never seen. These are the parts of household autonomy that a tabletop pick-and-place benchmark never touches.

Results

The headline is a zero-shot transfer claim with a real denominator: 91% success across 360 trials in six unmapped real-world environments, with no real home seen during training. Six environments and 360 trials is enough to rule out a single lucky kitchen, and the word unmapped matters — the robot is not replaying a previously scanned floor plan.

That is the strongest form of the claim the paper could make in an abstract, and it is worth being precise about what it does and does not establish. A 91% aggregate across a heterogeneous trial set can hide a skill that never works, if the trial mix is weighted toward the easier skills. Sitting on a chair and crawling under a table are not equally hard, and the abstract does not give a per-skill breakdown.

Limitations

The abstract reports no per-skill success rates, no failure taxonomy, and no compute or training-time figures. Those omissions matter for the central claim, because the whole pitch is that the corpus scales with compute. Readers cannot yet judge whether the thousand-home corpus is the reason for the transfer, or whether the same result would follow from a tenth of the homes with better domain randomisation.

Procedural homes are also a specific hypothesis about the world: that the variability that matters is layout, furniture and object placement. Real homes vary in ways that are harder to sample — lighting, clutter that has no semantic label, pets, humans who move things mid-task, floors that are not flat. The six test environments are real, but they are six.

Finally, the skill set is deliberately bounded to locomotion-plus-simple-contact. Nothing in the abstract covers dexterous manipulation, force-sensitive interaction, or recovery from a fall. A humanoid that can crawl under a table but cannot right itself is not yet a deployed product.

Industry implications

For humanoid companies, the strategic content here is the cost curve. The current industry playbook is teleoperation farms and wearable capture rigs, which convert capital into data at a roughly fixed ratio of human hours to robot hours. A synthesis pipeline that scales with GPU time changes the shape of that investment: the marginal cost of another thousand homes is a compute bill, and the marginal home is a new sampling seed rather than a new operator shift.

That does not make real-world data worthless — it makes it the validation set instead of the training set. Expect this split to become the standard architecture: synthetic pre-training for coverage, small real-world corpora for calibration and for the failure modes simulation cannot sample.

The near-term product question is safety and reliability under distribution shift. A 91% success rate is a research result; a home robot needs the failure cases enumerated. The paper’s own framing — skills that compose — is also the right engineering framing, because it gives a deployment team something to gate, monitor and fall back on skill by skill.