Urban Patterns

: Using Spatial Arrangement for Vision-Based Place Recognition and Localisation

Abstract

In this thesis, we investigate the impact of using visual landmarks with spatial knowledge to improve the performance of vision-based localisation. We separate our work into two parts. First, we introduce a new place recognition method based on a novel representation called landmark distribution descriptor (LDD) which combines landmark identification based on CNN features with their spatial distribution across a view. We use the representation to do matching within an image-to-image place recognition framework, which is to compare test images with single images taken at distinct locations in urban environments. Results on large datasets from 10 different cities obtained from Google StreetView and Bing Streetside demonstrate an average precision of around 70% at 100% recall, compared with 58% obtained using whole image CNN features and 50% for a comparable landmark method without spatial information. Second, we investigate the problem of localisation in urban environments using only image data and a 2-D map of the area. We employ binary semantic descriptors (BSD): 4-bit binary descriptors indicating the presence or otherwise of salient landmarks at a given location which are indicated on the 2-D map. On their own, these descriptors are not sufficiently distinctive to allow localisation. However, when combined sequentially over routes, the resulting concatenated descriptors prove to be highly discriminative, enabling robust localisation corresponds to the map. Performance can be further improved by incorporating the turn information along with a route. Landmark presence in 360-degree images taken at a given location is detected using a CNN binary classifier, trained using Google StreetView and OpenStreetMap data. Experiments with over 6,000 locations in an urban area show that the approach can give 95% of accuracy using an average route length of 200 metres. Thus, in both works, we demonstrate that landmarks combined with spatial knowledge provide an effective means of improving vision-based localisation.

Date of Award	25 Jun 2019
Original language	English
Awarding Institution	University of Bristol
Supervisor	Andrew Calway (Supervisor)