Bodily aware soft robots: integration of proprioceptive and exteroceptive sensors

Being aware of our body has great importance in our everyday life. It helps us to complete difficult tasks, such as movement in a dark room or grasping a complex object. These skills are important for robots as well, however, robotic bodily awareness is still an open question, and the nonlinearity of soft robots adds even more complexity. In this paper, we address this problem and present a novel method to implement bodily awareness into a real soft robot by the integration of its exteroceptive and proprioceptive sensors. We use an octopus-inspired arm as an example where the proprioceptive representation is approximated by four bend sensors integrated into the soft body, while a camera records the movement of the arm capturing its exteroceptive representation. The internal sensory signals are mapped to the visual information using a combination of a stacked convolutional autoencoder (CAE) and a recurrent neural network (RNN). As a result, the soft robot can learn to estimate and, therefore, to imagine its motion even when its visual sensor is not available.