Addressing the Orchestration Gap in Generalist Robots via Physical Agency
Pigey introduces a closed-loop physical agent orchestrator that decomposes goals into subgoals, commands low-level VLA policies, tracks outcomes, and recovers from failures — all without additional data or fine-tuning. It achieves 4x SOTA on LIBERO-PRO (12.8%→53.3%) and lifts frozen policies from near-zero to 90% on real-world reasoning-limited tasks, demonstrating the orchestration gap between motor skills and agentic reasoning.
Paper · arXiv:2607.21725Background
Today’s state-of-the-art vision-language-action (VLA) models attempt to combine perception, planning, success detection, recovery, and low-level control into a single learned policy via large-scale pre-training. While these policies acquire competent motor skills, they consistently fail on tasks that require reasoning about subgoal decomposition, outcome verification, or failure recovery — capabilities that are essential for real-world autonomous operation.
Core Innovation
Pigey rejects the monolithic approach and instead decomposes these capabilities into two components: a general language-conditioned policy/control agent and a high-level agent manager/orchestrator. Rather than training policies to reason, Pigey builds a closed-loop physical agent orchestrator that performs high-level planning, decomposes goals into achievable subgoals, commands low-level motor skills, tracks and verifies outcomes from low-level observations, and recovers from failures. Crucially, it can control existing VLA policies as well as parameterized skills without any additional data collection or post-training.
Key Results
On the LIBERO-PRO benchmark, Pigey advances the state-of-the-art by over 4× (12.8% → 53.3%) with no task-specific fine-tuning. On a real robot, it lifts a frozen VLA policy from near-zero success to over 90% on reasoning-limited tasks. The authors define the “orchestration gap” — the difference between what frozen motor skills achieve alone versus inside an agentic loop — and provide a systematic method to close it.
Limitations
Pigey relies on the existence of a competent underlying VLA or skill policy; it cannot improve fundamentally inadequate motor control. The orchestrator’s planning and recovery logic currently uses hand-designed prompts and rules rather than learned components. Scalability to highly unstructured environments with hundreds of possible subgoals remains to be demonstrated.
Industry Implications
This work has immediate practical value for any organization deploying VLA policies. Instead of collecting vast amounts of task-specific data to teach policies to reason, Pigey shows that an agentic wrapper can unlock reasoning capabilities that already exist in the underlying model. This dramatically reduces the data and compute barrier for deploying generalist robots in warehouses, manufacturing, and service settings. The orchestration gap provides a clear diagnostic for when and why a policy fails, guiding targeted improvement.