Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson

Background

Bimanual manipulation policies trained with imitation learning are almost always evaluated on workstation- or datacenter-class GPUs, leaving a practical question unanswered: what does it actually cost to deploy them on embedded hardware? For products that must ship in factories, clinics, or homes, the answer determines whether learned manipulation is commercially viable on entry-level hardware at all. This paper measures that cost precisely, using a bimanual SO-101 system running entirely on an NVIDIA Jetson Orin Nano Super (8 GB) — the entry tier of NVIDIA’s embedded line — with an RTX 3070 used only for offline training.

Core Innovation

The work is a full-stack deployment study with three components. First, a GStreamer capture pipeline backed by NVMM buffers eliminates redundant host–device copies across three cameras. Second, ACT and Diffusion Policy are trained on identical demonstrations at their reference budgets (100k gradient steps for ACT, 200k for Diffusion Policy) to compare convergence under embedded constraints. Third, ACT is converted to TensorRT at FP16 and INT8 precision. The paper also documents two findings not previously reported for ACT: TensorRT’s general-purpose INT8 calibration quantizes the ResNet18 backbone but accepts zero of 145 transformer layers — explaining why INT8 shrinks the model only 0.9% versus FP16 despite a further 28% latency gain — and the need for quantization is conditional on ACT’s action-chunking configuration, with full precision feasible at n_action_steps = 100 but not for the per-step re-prediction that temporal ensembling requires.

Results

The conventional capture path fit the memory budget and dropped no frames; zero-copy sensing’s payoff is CPU headroom (peak single-core utilization falling from 98.0% to 77.0%) and worst-case latency (117.31 ms to 101.52 ms). In training, ACT converged to a task-competent policy (19/20 trials) while Diffusion Policy did not converge to a usable one (0/10) even at twice the step count — attributed to differing convergence costs, not an accuracy ceiling. TensorRT conversion cut mean ACT inference latency from 114.02 ms to 17.93 ms in FP16 (6.4x) and 12.65 ms in INT8 (9.0x), with task success preserved at all three precisions (19/20, 18/20, 19/20).

Limitations

This is a single-task, single-arm-type evaluation: pick-and-place of a deformable beanbag on one SO-101 system. Diffusion Policy’s failure to converge at its reference budget is informative but not a general verdict, since convergence depends on demonstration quality and hyperparameters. The quantization findings are specific to ACT’s architecture and to TensorRT’s calibration pipeline; other VLA architectures or calibration strategies may behave differently.

Industry Implications

The paper effectively redraws the cost floor for deploying learned manipulation: a policy stack that previously presumed a desktop GPU now runs, end to end, on an $249-class embedded module while preserving task success. The INT8-calibration finding matters beyond ACT — teams quantizing transformer-based policies should verify layer-level coverage rather than trusting aggregate model-size numbers. And the action-chunking result gives deployers a concrete rule of thumb: large action-chunk horizons can run in full precision, while temporal-ensembling configurations are the ones that actually need quantization. For robotics startups and integrators, this is directly actionable engineering knowledge.