PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

Background

Physical AI policies require inference throughout their entire lifecycle: model evaluation during development, cloud reinforcement learning rollouts, edge GPU serving, and final onboard deployment. Although all of these settings consume the same checkpoint with the same action semantics, they conventionally rely on separate inference programs, each reimplementing graph execution, kernel selection, and memory management. This fragmentation forces teams to maintain multiple code paths and makes it difficult to reason about real-time behavior consistently across the stack.

Core Innovation

PhyAI provides one runtime that keeps architecture-specific logic — conditioning, solver, cache, and output handling — inside model adapters, while sharing graph execution, kernels, memory management, and parallel services across every deployment target. The same codebase runs VLA models and world-action models on single or multiple GPUs across onboard, edge, and cloud settings. The adapter interface is deliberately lightweight: the authors added MiniCPM-Robot on the day of its release. The paper also introduces the control-time Roofline, a profiling concept that distinguishes inference-bound from environment-bound control loops, giving practitioners a principled way to decide where optimization effort should go.

Results

Across official implementations, PhyAI achieves 1.40x–4.65x speedups for pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot. On Cosmos3-Nano-Policy-DROID it cuts latency from 2.46 s to 1.18 s on eight H20 GPUs (CFG=2, TP=4), a 2.08x speedup. Detailed profiles show why different models need different execution policies: on a Hopper-series GPU at batch size one, the pi0.5 action expert accounts for 8.8% of FLOPs but 57.2% of latency; at batch size 32 that share drops to 13.5% and throughput reaches roughly 100 samples/s. Cosmos3 remains generation-dominated, gaining only 14.3% throughput as batch size rises from 1 to 16. Measured with the control-time Roofline, pi0.5 on four LIBERO suites is environment-bound while Cosmos3 stays inference-bound.

Limitations

The authors are explicit that specialized runtimes remain faster in several configurations; the goal is one runtime with competitive latency rather than the fastest result in every case. The benchmark coverage centers on a specific set of open models, so absolute numbers may shift as new VLA/WAM architectures appear. Real-time claims are based on profiled GPU deployments, and onboard results on constrained robot hardware are less extensively characterized than cloud and edge settings.

Industry Implications

Robot companies currently maintain separate inference stacks for simulation rollouts, edge servers, and onboard computers — a duplication that slows iteration and complicates latency guarantees. A single runtime with pluggable adapters directly attacks that cost: new model releases can be integrated in days, the same code can scale from an 8-GPU cloud pod to an onboard computer, and the control-time Roofline gives teams a shared vocabulary for diagnosing whether a robot is compute-bound or environment-bound. As open VLA and world-action models proliferate, an open, adapter-based inference engine is a credible candidate for the default serving layer of the Physical AI stack.