In-Hand Pen Writing for an Anthropomorphic Hand, Learned in Real Time Without Demonstrations

· Editorial Team estimated
dexterous-manipulation in-hand-manipulation anthropomorphic-hand jacobian-estimation learning-free-control data-efficient

The authors present an embodied control approach that estimates the task Jacobian of the combined hand-and-object system in real time on the physical robot. Using only a laptop CPU, the controller begins in-hand pen writing after roughly 18 seconds of initialization and keeps adapting online, with no analytic hand-object kinematic or contact model, no simulation training, and no precollected demonstrations. The same estimator/controller formulation works on three anthropomorphic hands (one physical, two simulated), demonstrating an embodiment-independent method. On the physical robot it reaches sub-millimeter in-plane precision (mean 0.6 mm across runs) for letters and shapes written both in the air and on paper, which the authors describe as the first demonstration of an anthropomorphic hand writing arbitrary single-stroke trajectories with a grasped pen through purely in-hand motion.

Paper · arXiv:2609.11775

Dexterous in-hand manipulation of a grasped object with an anthropomorphic hand remains an unsolved frontier for robot dexterity. The contact-richness and highly dynamic nature of object-hand interaction tend to demand extensive modeling or data collection: simulators used for reinforcement learning cannot fully replicate the required contact complexity, and collecting dexterous demonstrations for imitation learning remains an open problem.

Core Innovation

This work takes a different route: no training, no modeling, no demonstrations. Instead it estimates the task Jacobian of the combined hand-and-object system in real time on the physical robot and uses it for embodied control. The controller runs on a laptop CPU alone, begins in-hand pen writing after about 18 seconds of initialization, and keeps adapting online. It needs no analytic hand-object kinematic or contact model, no simulation training, and no precollected task demonstrations. The same estimator/controller formulation holds across three anthropomorphic robotic hands (one physical, two simulated), indicating an embodiment-independent method.

Results

  • On the physical robot, letters and shapes written both in the air and on paper reach sub-millimeter in-plane precision (mean 0.6 mm across runs).
  • To the authors’ knowledge, this is the first demonstration of an anthropomorphic hand writing arbitrary single-stroke trajectories with a grasped pen through purely in-hand motion.
  • The authors position the work as an alternative to compute- and data-heavy approaches such as RL and IL, achieving dexterous manipulation through computationally simple and data-efficient algorithms.

Limitations

The abstract omits the hand model and degrees of freedom, the number of runs, the full precision distribution (especially tail errors), and writing speed, and it does not explain how Jacobian estimation recovers from slip or contact loss. The demonstrations are limited to single-stroke trajectories, so the boundary of what the method can write — continuous multi-stroke or more complex figures — is unclear; the reported precision is in-plane, with depth and orientation error unaddressed. There are also no quantitative comparisons against RL or IL baselines, so the “alternative” claim is a framing rather than a measured result.

Industry Implications

If “a pure-CPU online estimator buys sub-millimeter in-hand manipulation” holds, the significance lies in bypassing the data barrier: dexterous manipulation has long been gated by simulator fidelity and demonstration-collection cost, and this class of method lowers the entry price to a laptop. For teams building dexterous hands, assembly, or fine-manipulation products, it points to a low-cost, fast-to-deploy route — especially in settings that only require repeated high-precision motion within a bounded workspace.