TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation

Robotics rarely lacks algorithms; it lacks real tactile data you can afford to train on. This work pushes the scale up an order of magnitude and adds the quality and annotation machinery around it.

Background

Large-scale robot learning increasingly depends on diverse human demonstrations, yet tactile manipulation data remain difficult to scale: fingertip contact signals either demand specialized hardware or involve cumbersome collection procedures. Meanwhile, existing multimodal datasets often lack consistent quality control and structured supervision, making them large but hard to use. Supporting contact-rich manipulation takes more than adding frames — it needs a scalable, reusable collection and governance system with structure attached to the data.

Core Innovation

The contribution is the collection interface and the data pipeline. TacClaw-Infinity is built with TacClaw, a tactile-enabled, UMI-style handheld capture device that records synchronized visual observations, fingertip tactile signals and end-effector trajectories, and that needs no target robot during capture — decoupling data collection from any specific embodiment. Data come from distributed worldwide collection spanning homes, laboratories, industrial workplaces, retail stores and outdoor environments, including tasks contributed by academic and industrial collaborators. Around the raw data, the team builds a scalable pipeline for trajectory reconstruction, multimodal quality control, embodiment-aware filtering and structured annotation: contact-centric segmentation, subtask description and spatial grounding.

Results

• Scale: more than 10,000 hours, 1 million episodes and 1 billion frames across over 1,000 task types.
• Data level: the collection interface, the action-space compatibility of the curated demonstrations and the structured annotations are all assessed.
• Model level: cross-embodiment transfer across three heterogeneous robots without robot-specific demonstrations.

Limitations

The abstract emphasizes scale and pipeline rather than the downstream ceiling, so how much of this data converts into stronger policies is not yet quantified in absolute terms. Although more than 1,000 task types are claimed, their distribution over difficulty, horizon length and contact complexity is not described, leaving the share of long-horizon or force-sensitive work open. The alignment error between the UMI-style interface’s action space and different robot embodiments directly governs transfer quality, and that point is not examined in depth.

Industry Implications

Touch has long been treated as a “nice to have” modality; large-scale, structured tactile data is the precondition for turning it into a trainable signal. For industry, the value of an infrastructure dataset like this is that it lowers the per-team collection cost: instead of re-collecting for every robot, teams can accumulate data with a shared handheld interface and land it on their own hardware through cross-embodiment transfer. Once data pass the million-episode scale, the bottleneck moves from “do we have data” to “can we do quality control and annotation” — which is exactly the other half this work supplies.