SafeHarness: Making Safety a Planning Priority for Coding Agents on Robots

Having a language model write the controller as a program, and letting the resulting coding agent drive a robot with no robot-specific training, is no longer a demo curiosity. It is an emerging route to manipulation. Whether that route is safe had simply not been asked.

The authors ask it. Each manipulation goal is paired with an obstacle the robot must not touch. The agent collides with it in most cases. The interesting part is where the failure sits: the agent discusses the obstacle in its reasoning traces, and the prompt already forbids touching it. So the fault is neither perception nor instruction. The constraint never becomes a priority in planning — task completion is the sole objective, and safety is an unenforced clause in the prompt.

Core Innovation

Rather than stop at “the model is unsafe,” the authors split manipulation into two phases to localize the failure:

• Route phase. The model has no notion of a clearing route and no mechanism to replan once a chosen route becomes infeasible, so it cannot let the safety constraint drive route selection.
• Contact-rich moment. The model is unaware that contact execution is bounded by the same constraint, so the contact point itself can land on the obstacle.

That decomposition yields SafeHarness, two obstacle-aware harnesses:

1. Obstacle-aware route planning. Objects are grounded as bounding boxes, and candidate routes are drawn over them as sequences of waypoints. The agent therefore plans a route in advance, verifies it, replans when necessary, and only then executes — collision avoidance turns from a soft hint into a set of checkable intermediate artifacts.
2. Obstacle-aware contact execution. The contact position is selected so that the contact itself avoids the obstacle, instead of being corrected after the fact.

The design intent is plain: do not hope the model becomes better behaved; give the constraint structure, and let the model work inside it.

Results

• 71.9% task success and 87.5% collision avoidance.
• Against the previous state of the art, +6.5 and +27.0 points respectively.
• Against the same agent without the harnesses, 2.3x and 1.5x.

Both numbers move together. Collision avoidance up 27 points while success also rises indicates the gain addresses collision rather than trading task completion for timidity.

Limitations

The abstract does not say whether the experiments run in simulation or on real hardware, nor which base model, how many tasks, or how complex the scenes are, so transferability to a real production line cannot be judged. Obstacles are reduced to bounding boxes, which holds for well-formed convex geometry but is not evidenced for deformable objects, hanging items, or cluttered scenes demanding tight insertion. The harness overhead and inference latency are unreported; if every step needs extra planning and verification, real-time operation may become the deployment threshold. There is also no failure analysis: what accounts for the 28.1% of unfinished tasks and the 12.5% of remaining collisions is left open, so the bottleneck — planning, perception, or execution — is unknown.

Industry Implications

For teams planning to put model-written controllers into shared spaces or production lines, the most valuable output here is not a number but an engineering conclusion: a safety constraint written into a prompt is not a constraint that executes. It has to be demoted into objects the planner can operate on — bounding boxes, waypoints, candidate routes, checkable intermediates — before it has any chance of holding. The work also offers a reusable split: free-space travel goes to a route layer that plans, verifies and then executes, while the highest-risk contact moment goes to dedicated constraint-driven selection. Where safety compliance is itself the precondition for deployment, a constraint harness of this kind is likely to ship earlier than another round of model scaling, and it is far easier to explain to a regulator.

---