Freeze the Policy, Rewrite the Harness: RoboArchitect Lifts Unseen-Scene Success 4.5x

The same robot policy can perform very differently depending on the harness around it. The harness is not part of the model. It is control code that decides when to call the policy, when to reach for a vision tool, and when to command the arm directly. Today these harnesses are written by hand and rebuilt for every new robot or benchmark. RoboArchitect asks a blunt question: if the harness matters this much, why not search over it?

Core innovation

Separate the policy from the scheduling. The paper’s premise is that in many failures the policy is fine and the calling convention is not. A policy only emits actions at the moments it is queried; whether to query now, whether to reframe the camera first, whether to back the arm out after a missed grasp — all of that belongs to the harness. RoboArchitect freezes every weight and lets a coding model read rollouts, diagnose the failure, and edit the harness.

Optimize on failures, evaluate off them. The harness evolves over a set of episodes and is then scored on held-out scenes and tasks. That split carries the argument. A harness tuned only on the episodes it failed would collapse into hard-coded fixes for those specific scenes; the held-out numbers are where the real gain shows up.

Recover control strategies that look deliberate. The abstract cites two: lifting a jammed arm clear of an obstacle, and approaching horizontally when something blocks the path from above. Neither is a learned motor skill. Both are harness-level decisions to change the conditions before asking the policy again. That is evidence the search space contains genuine manipulation common sense rather than a handful of thresholds.

Give the harness its own memory. The evolved harness maintains persistent memory and carries experience forward, such as effective grasp depths. The system improves across a task family instead of restarting from scratch each episode.

Results

• With no weight updates, average success rises from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x).
• The effect holds across multiple benchmarks and multiple policies, so it is not a single lucky configuration.
• Gains are largest where the benchmark sits furthest from the policy’s training distribution.
• Wrapping a general-purpose policy, the evolved harness often outperforms a specialist finetuned on the target task.
• Discovered strategies include clearing a jammed arm, switching to a horizontal approach under overhead obstruction, and writing grasp-depth experience into persistent memory.

The generalist-beats-finetuned result is the headline. Finetuning needs target-domain data, a training run, and a stored weight set per robot. A harness needs rollouts and a code search. Those are different cost curves, and the second one is much easier to repeat on a customer site.

Limitations

Search cost is absent. The abstract does not say how many rollouts, how many revision rounds, or how many coding-model calls are required, and that is exactly what decides whether this is cheaper than finetuning. A 43.3% success rate on unseen scenes is still far from deployable: the harness closes part of the generalization gap, not all of it. Evaluation is benchmark-centric, with real-robot validation left out of the abstract. The harness is generated code, and its failure modes, readability, and safety properties go undiscussed; once that code commands the arm directly, a bad edit becomes a physical risk. Transfer of persistent memory across robots and tasks is not quantified either.

Industry implications

The paper shifts the cost center of robot deployment from training to integration. When a workcell or an incoming part changes, the usual response is to collect data, finetune, and regression-test. RoboArchitect proposes keeping one frozen policy and having a coding model rewrite the scheduling layer for the new cell. The first path needs ML engineers and a data pipeline; the second looks more like the work of a traditional automation integrator, which is the largest and most customer-adjacent segment of the robotics industry.

There is a benchmarking consequence too. If the harness alone can swing performance by several times, then a policy’s benchmark score is no longer a comparable number unless the harness is reported with it. That matters for reproduction and for procurement.

Finally, memory placement. Storing learned experience such as grasp depth in the harness rather than in weights makes that experience copyable, reviewable, and rollback-able like a config file. For industrial settings that need audit trails, that is a more attractive artifact than an opaque checkpoint.