Feeling the Pinch: Differentiating Thin Materials with a Soft Vision-based Tactile Sensor

Thin materials are often fragile and difficult to perceive and manipulate. Vision-based tactile sensing with soft contact interfaces could help identify thin sheets within grasps and enable delicate manipulation. This work presents a study in biomimetic sensing of thin materials using the TacTip visuo-tactile sensor. The working principle involves pinching a thin material between two TacTips, capturing the deformation of their biomimetic soft skins and determining material properties via a learned model. By pinching a set of 3D printed thin material samples represented by different Shore hardnesses with a range of forces, a Multi-Input Convolutional Neural Network (CNN) can be trained to differentiate the thin materials with a 96% accuracy. The model was validated with real thin materials, showing correlation in terms of elasticity with the trained materials. By analyzing the physical spread of tactile pins in the tactile images, we show how the sensor material affects the perception range of sensed material properties. The proposed method is demonstrated by sorting plastic and paper via a robot arm, showing a pathway to more sophisticated human-like sensing and manipulation of thin materials.