I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry

Moving real objects into simulation now stalls on a preprocessing step few want to discuss: the generated mesh looks fine, but it is not a collision body a physics engine can use. I2CD compresses the reconstruct-then-decompose pipeline into a single forward prediction from image to geometry.

Background

Physics simulators and motion planners require convex collision geometry, but image-to-3D generative models emit dense, frequently non-manifold visual meshes. Bridging the two today means a slow, brittle reconstruct-then-decompose pipeline: reconstruct the mesh, then repair, decimate and approximate convex decomposition (ACD). It is time-consuming, fragile, and often yields fragmented geometry a planner cannot accept.

Core Innovation

I2CD collapses reconstruct-and-then-decompose into one step, predicting a convex decomposition directly from a single RGB image. It does not train a new image-to-3D model: it freezes the pretrained Hunyuan3D-2 image-conditioned diffusion transformer and shape decoder and trains only a lightweight cross-attention head (38M parameters, under ten GPU-hours). That head’s learned convex-slot tokens emit the halfplane parameters of K convex polytopes directly. The output is therefore compact, convex by construction, and loads into physics engines with no post-processing, at roughly 0.5s per image.

Results

• On 227 held-out OmniObject3D and Google Scanned Objects instances, highest volumetric IoU among eight reconstruct-then-decompose pipelines, and 6 to 37 times faster end-to-end.
• In a cross-simulator study across MuJoCo, PyBullet, Genesis and Isaac Sim, every engine uses I2CD geometry as delivered, whereas raw generated meshes load everywhere but are silently replaced by a different collision shape in most cases, or need seconds to minutes of per-object preprocessing.
• On a physical xArm7, planner-ready geometry for a 20-object cluttered scene takes 11s versus 328s for the strongest baseline, at comparable pick-and-place success (85 vs 90 of 100 trials).

Limitations

I2CD’s decomposition quality is still bounded by the frozen Hunyuan3D-2 prior — where the upstream generator reconstructs a class poorly, a lightweight head cannot recover the loss. The abstract reports volumetric IoU and engine usability, not physical fidelity in contact-rich tasks (friction, compliant contact), and offers no dedicated evaluation of hard structures such as thin walls or articulated parts. Because it emits convex pieces by construction, the trade-off between the number of slots K and fidelity for strongly non-convex objects remains to be tested.

Industry Implications

For any team moving real objects into simulation, this pipeline has been a hidden cost: slow, brittle, and prone to failure in the worst way — silently swapping in a different collision shape and undermining the credibility of the results. I2CD turns it into a single forward prediction from image to geometry, roughly half a second each and delivered ready to use, cutting per-object preprocessing from hundreds of seconds to a dozen. That directly lowers the cost of large-scale scene construction and domain randomization, where every new object previously meant re-running the geometry pipeline. For robot vendors and simulation platforms, the practical meaning is that the production rate of usable geometry now depends on images and compute rather than on a reconstruction pipeline’s patience; the key adoption check is whether target object classes fall inside the upstream generator’s comfort zone.