Skip to content

From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor

Research output: Contribution to journalArticle

Original languageEnglish
Article number8641397
Pages (from-to)2101-2107
Number of pages7
JournalIEEE Robotics and Automation Letters
Issue number2
Early online date13 Feb 2019
DateAccepted/In press - 23 Jan 2019
DateE-pub ahead of print - 13 Feb 2019
DatePublished (current) - 1 Apr 2019


Deep learning has the potential to have same the impact on robot touch as it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this letter, we apply deep learning to an optical biomimetic tactile sensor, the TacTip, which images an array of papillae (pins) inside its sensing surface analogous to structures within human skin. Our main result is that the application of a deep convolutional neural network can give reliable edge perception, and, thus a robust policy for planning contact points to move around object contours. Robustness is demonstrated over several irregular and compliant objects with both tapping and continuous sliding, using a model trained only by tapping onto a disk. These results relied on using techniques to encourage generalization to tasks beyond which the model was trained. We expect this is a generic problem in practical applications of tactile sensing that deep learning will solve.

    Research areas

  • biomimetics, deep learning in robotics and automation, Force and tactile sensing

Download statistics

No data available



  • Full-text PDF (accepted author manuscript)

    Rights statement: This is the accepted author manuscript (AAM). The final published version (version of record) is available online via IEEE at . Please refer to any applicable terms of use of the publisher.

    Accepted author manuscript, 4 MB, PDF document

    Licence: Other


View research connections

Related faculties, schools or groups