A geometric approach for fast affordance determination in 3D

Abstract

Visual perception for robotics aims to provide robots with the relevant information about the environment such that they can interact with it, understand it and accomplish tasks successfully. Thus, agents that need to act on their surroundings can significantly benefit from the perception
of their interaction possibilities or affordances. The concept of affordances calls for an approach to visual perception that is free from complex representations, and that is there to help the agent to interact with the world.
This thesis presents an approach for the determination of affordances in visually perceived 3D environments. The introduced method builds on the hypothesis that geometry on its own provides enough information to enable the detection of significant interaction possibilities in the environment. The motivation behind this is that geometric information is intimately related to
the physical interactions afforded by objects in the world.
The work presented in this thesis introduces a geometrical representation for the interaction between two entities in 3D space. The nature of the approach provides the possibility to generically describe interactions for everyday objects such as a mug or an umbrella, and also for more complex affordances such as Sitting or Riding. Experiments with numerous synthetic and RGB-D scenes show that the representation enables the prediction of affordance candidate locations in novel environments at fast rates and from a single training example, i.e. one-shot learning. Then, it is shown that the one-shot capability of the proposed approach and the abstraction power of state-of-the-art data-driven methods allow to devise a compact and optimised representation for the detection of multiple affordances in any given location. Experiments and evaluations show that the proposed algorithms achieve high precision rates that outperform alternative methods. The evaluations include human validations via crowdsourcing, which show the meaningfulness of the affordance predictions made with the proposed algorithm.

Date of Award	25 Jun 2019
Original language	English
Awarding Institution	University of Bristol
Supervisor	Walterio W Mayol-Cuevas (Supervisor)