FasTac: A Curved Multispectral Vision-Based Tactile Sensor for High-Speed High-Precision 3D Shape and Force Perception

Background

Dexterous manipulation places severe demands on tactile sensing: fingertips must resolve fine contact geometry, distinguish normal from tangential loads, and capture transient signals — all in a curved form factor that fits a robotic finger. Existing curved vision-based tactile sensors struggle to combine accurate 3D reconstruction, three-axis force estimation, and high-speed processing in a compact package. Each capability has been demonstrated separately, but integrating them without compromising any single one has proven difficult.

Core Innovation

FasTac is a curved vision-based tactile sensor built around three coordinated design choices. First, single-image-sensor multispectral photometric stereo provides spatially aligned observations for robust surface normal estimation, followed by boundary-prior fast Poisson depth reconstruction. Second, HyperForce, a position-aware dynamic convolution network, models the spatially nonuniform mechanical response of curved elastomers to estimate three-axis contact forces. Third, the entire image-to-normal-force pipeline is deployed on an FPGA, moving the heavy compute off a GPU and into the sensor’s embedded hardware.

Results

The system was evaluated across reconstruction accuracy, force estimation, and speed. Near-infrared illumination combined with the boundary prior reduces depth mean absolute error from 0.2730mm to 0.0415mm. HyperForce achieves normalized MAE of 2.74% for normal and 2.39% for shear forces. FPGA deployment shortens processing latency from 3.26ms on GPU to 1.09ms. Multi-object reconstruction, feedback grasping, and vibration measurement further validate fine geometric perception, stable force feedback, and dynamic contact sensing.

Limitations

The evaluation centers on the sensor’s core perceptual metrics; whole-hand integration and long-duration robustness (wear, thermal drift, calibration stability over months of use) are not addressed in detail. FPGA deployment demonstrates a specific latency budget, but the trade-offs of power consumption and manufacturing cost versus the GPU baseline are not quantified in the abstract.

Industry Implications

Reliable, low-latency tactile feedback is one of the missing pieces for commercial dexterous hands — in logistics picking, assembly, and healthcare. A curved fingertip sensor that delivers sub-50-micron depth accuracy, three-axis force sensing, and millisecond-level latency on embedded hardware is directly relevant to gripper and prosthetic developers. The FPGA-based pipeline points toward a sensor architecture that could be manufactured as a self-contained fingertip module rather than requiring an external GPU, which matters for cost-sensitive robot hands at scale.