Packing the Gripper Into the Plan: Closed-Loop Dense Packing of Irregular Objects

· Editorial Team estimated
dense-packing gripper-aware-planning warehouse-automation force-control manipulation perception-loop

Automatic dense packing is what warehouse operations actually want, and it has stayed largely in simulation: prior work treats the object as an isolated rigid body with idealized contact, inserts the gripper as a discrete post-hoc feasibility check if at all, and leaves perception and contact drift during execution unaddressed. This paper delivers a closed-loop pipeline that runs on real hardware, integrating perception, gripper-aware placement optimization, and force-guided execution. The optimizer represents the object together with the gripper as one composite body of hierarchical sphere trees and searches five degrees of freedom on a GPU inside a CMA-ES framework, with the vertical coordinate grounded analytically against the current heightmap. During execution a force-monitored vertical descent stops on first contact, and a post-release consolidation push closes the residual lateral clearance that gripper-aware planning leaves behind, while the container is re-perceived between placements so drift does not accumulate. The system is validated on a Franka Emika Panda packing 3D-printed flat, curved, and concave objects plus a YCB subset, with an ablation isolating gripper-aware optimization, the consolidation push, and mesh-derived geometry against end-to-end success, achieved density, and computational cost, and a benchmark against a heightmap-minimization baseline representative of prior work.

Paper · arXiv:2609.22062

What warehouse operations actually want is not pick-and-place but filling a container as densely as possible with objects that refuse to be tidy. That problem has been worked on in simulation for years and still does not hold up on real robots, for concrete reasons. Prior work usually treats the manipulated object as an isolated rigid body with idealized contact. The gripper shows up, if at all, as a discrete post-hoc feasibility check rather than as part of the plan. And the drift that perception error and real contact introduce during execution is simply left unmanaged. Plans that look dense in the abstract degrade quickly against real clearances, real contact, and real error.

Core Innovation

The contribution is not a new objective function but a pipeline that genuinely closes the loop across three stages: perception, gripper-aware placement optimization, force-guided execution.

  • Put the gripper inside the packing problem. The optimizer stops treating the object as an isolated rigid body and represents the object and gripper as a single composite body built from hierarchical sphere trees. The grasp thus moves from after-the-fact check to optimization variable, so planned placements are ones the robot can actually reach and release with its jaws attached.
  • Five-DoF search on the GPU. The composite pose is searched over five degrees of freedom inside a CMA-ES framework running on a GPU, while the vertical coordinate is excluded from the search and solved analytically against the current heightmap. That division of labor is the key move: the one dimension the heightmap already determines is removed from the search, which saves compute and rules out physically implausible hovering placements.
  • Force-monitored descent, stopped on first contact. Execution switches to a force-monitored vertical descent that halts at first contact instead of pressing to a commanded depth.
  • A consolidation push to close the gap. Here the authors answer their own method’s weakness directly: gripper-aware planning inevitably leaves lateral clearance, so a push after release squeezes it out.
  • Re-perceive the container between placements. Drift is no longer assumed to be zero; it is blocked from accumulating by observing the container afresh each cycle.

Results

  • Platform: a Franka Emika Panda, packing 3D-printed flat, curved, and concave objects plus a YCB subset.
  • An ablation isolates the contributions of gripper-aware optimization, the consolidation push, and mesh-derived geometry to end-to-end success, achieved density, and computational cost.
  • Benchmarking against a heightmap-minimization method, taken as representative of prior irregular-object packing work.

Limitations

The abstract gives evaluation axes but no numbers, so neither the achieved density on the more general YCB subset nor the distribution of failures can be judged. The packed objects are known 3D-printed parts plus a YCB subset, a comparatively controlled shape distribution; there is no evidence for deformable or fragile objects, for surfaces with very different friction, or for the scene that defines real bin picking, where items must be extracted from an unordered pile before they can be placed. Compute cost is measured in the ablation, but absolute planning time is not reported in the abstract, and if a single placement needs seconds of optimization, cycle time becomes the binding constraint on a production line. Finally, the consolidation push means the system deliberately shoves already-placed objects, and whether that is safe for fragile parts or for workpieces that must hold a pose is left unaddressed.

Industry Implications

Dense packing is one of the few manipulation tasks with an unambiguous metric, genuine demand, and a buyer, yet the bottleneck has never been grasping. It sits in the seam between planning with a tool attached and executing under real contact. The value of this work is that it turns both sides of that seam into engineering handles: let the jaws enter the optimization, and accept that planning leaves residual error that a cheap physical action can absorb. For teams doing tote packing, mixed depalletizing, or space-constrained loading, the structure transfers without new hardware and without retraining in a more photorealistic simulator, because it assumes execution returns error and then eats that error with re-perception and a push. The insistence on reporting density beside success rate deserves attention too: in packing, density is the money, and a success-only evaluation cannot support a purchasing decision.