Centroids triplet network and temporally-consistent embeddings for in-situ object recognition

Miguel A Lagunes Fortiz, Dima Damen, Walterio W Mayol-Cuevas

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

1 Citation (Scopus)
157 Downloads (Pure)


This work proposes learning to recognize objects from a small number of training examples collected and deployed in-situ. That is, from data collected where the objects are commonly placed or being used, perhaps after first encountering them, the learning algorithm immediately is able to recognize them again. We refer to this method-ology as in-situ learning, and it opposes to the conventional methodology of using complex data acquisition mechanisms, such as rotating tables or synthetic data, to build a large-scale dataset for training convolutional neural networks (ConvNets). To learn in-situ, we propose a novel loss function that generates discriminative features for known and unseen objects, by utilizing a regularization term that reduces the distance between features and their manifold centroid. Additionally, we propose a temporal filter that is particularly useful to quickly react to appearing objects on the scene, which depending on the distance between neighboring video-frame features, it applies a weighted average between the current and the previous frame. Our framework achieves state-of-the-art accuracy for in-situ and on-the-fly learning, for the case of known objects achieves an average increase in accuracy of 3.01%, an increase of 3.3% for novel objects, and an average increase of 7.07% for the combined case, compared with the closest baseline. Utilizing the temporal filtering, led to a further increase in accuracy against nuisances of 7.32% for the known and novels objects case.
Original languageEnglish
Title of host publication2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages7
ISBN (Electronic)978-1-7281-6212-6
ISBN (Print)978-1-7281-6213-3
Publication statusPublished - 10 Feb 2021
Eventconference is IROS 2020 -
Duration: 25 Oct 202029 Oct 2020

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866


Conferenceconference is IROS 2020


  • training
  • manifolds
  • filtering
  • data aquisition
  • object recognition
  • convolutional neural networks
  • intellingent robots


Dive into the research topics of 'Centroids triplet network and temporally-consistent embeddings for in-situ object recognition'. Together they form a unique fingerprint.

Cite this