Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks

Toby Perrett; Dima Damen

doi:10.1109/ICCVW.2017.161

Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks

Toby Perrett, Dima Damen

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

3 Citations (Scopus)

239 Downloads (Pure)

Abstract

In this paper, we investigate whether it is possible to leverage information from multiple datasets when performing frame-based action recognition, which is an essential component of real-time activity monitoring systems. In particular, we investigate whether the training of an LSTM can benefit from pre-training or co-training on multiple datasets of related tasks when it uses non-transferred visual CNN features. A number of label mappings and multi-dataset training techniques are proposed and tested on three challenging kitchen activity datasets - Breakfast, 50 Salads and MPII Cooking 2. We show that transferring, by pre-training on similar datasets using label concatenation, delivers improved frame-based classification accuracy and faster training convergence than random initialisation.

Original language	English
Title of host publication	2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017)
Subtitle of host publication	Proceedings of a meeting held 22-29 October 2017, Venice, Italy
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	1354-1362
Number of pages	9
ISBN (Electronic)	9781538610343
ISBN (Print)	9781538610350
DOIs	https://doi.org/10.1109/ICCVW.2017.161
Publication status	Published - Feb 2018
Event	International Conference on Computer Vision Workshops (ICCVW), - Venice, Italy Duration: 22 Oct 2017 → …

Publication series

Name
ISSN (Print)	2473-9444

Conference

Conference	International Conference on Computer Vision Workshops (ICCVW),
Country	Italy
City	Venice
Period	22/10/17 → …

Access to Document

10.1109/ICCVW.2017.161

Full-text PDF (accepted author manuscript)
This is the author accepted manuscript (AAM). The final published version (version of record) is available online via IEEE at http://ieeexplore.ieee.org/document/8265371/ . Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 2.1 MB

Fingerprint Dive into the research topics of 'Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks'. Together they form a unique fingerprint.

Projects

1 Finished

LOCATE: LOcation adaptive Constrained Activity recognition using Transfer learning

Aldamen, D.

4/07/16 → 3/05/18

Project: Research

Cite this

Perrett, T., & Damen, D. (2018). Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks. In 2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017): Proceedings of a meeting held 22-29 October 2017, Venice, Italy (pp. 1354-1362). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICCVW.2017.161

@inproceedings{74bed7f6882d42ddade78401c7c4aef1,

title = "Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks",

abstract = "In this paper, we investigate whether it is possible to leverage information from multiple datasets when performing frame-based action recognition, which is an essential component of real-time activity monitoring systems. In particular, we investigate whether the training of an LSTM can benefit from pre-training or co-training on multiple datasets of related tasks when it uses non-transferred visual CNN features. A number of label mappings and multi-dataset training techniques are proposed and tested on three challenging kitchen activity datasets - Breakfast, 50 Salads and MPII Cooking 2. We show that transferring, by pre-training on similar datasets using label concatenation, delivers improved frame-based classification accuracy and faster training convergence than random initialisation.",

author = "Toby Perrett and Dima Damen",

year = "2018",

month = feb,

doi = "10.1109/ICCVW.2017.161",

language = "English",

isbn = "9781538610350",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "1354--1362",

booktitle = "2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017)",

address = "United States",

note = "International Conference on Computer Vision Workshops (ICCVW), ; Conference date: 22-10-2017",

}

Perrett, T & Damen, D 2018, Recurrent Assistance: Cross-Dataset Training of LSTMs on Kitchen Tasks. in 2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017): Proceedings of a meeting held 22-29 October 2017, Venice, Italy. Institute of Electrical and Electronics Engineers (IEEE), pp. 1354-1362, International Conference on Computer Vision Workshops (ICCVW), , Venice, Italy, 22/10/17. https://doi.org/10.1109/ICCVW.2017.161

Recurrent Assistance : Cross-Dataset Training of LSTMs on Kitchen Tasks. / Perrett, Toby ; Damen, Dima.

2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017): Proceedings of a meeting held 22-29 October 2017, Venice, Italy. Institute of Electrical and Electronics Engineers (IEEE), 2018. p. 1354-1362.

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Recurrent Assistance

T2 - International Conference on Computer Vision Workshops (ICCVW),

AU - Perrett, Toby

AU - Damen, Dima

PY - 2018/2

Y1 - 2018/2

N2 - In this paper, we investigate whether it is possible to leverage information from multiple datasets when performing frame-based action recognition, which is an essential component of real-time activity monitoring systems. In particular, we investigate whether the training of an LSTM can benefit from pre-training or co-training on multiple datasets of related tasks when it uses non-transferred visual CNN features. A number of label mappings and multi-dataset training techniques are proposed and tested on three challenging kitchen activity datasets - Breakfast, 50 Salads and MPII Cooking 2. We show that transferring, by pre-training on similar datasets using label concatenation, delivers improved frame-based classification accuracy and faster training convergence than random initialisation.

AB - In this paper, we investigate whether it is possible to leverage information from multiple datasets when performing frame-based action recognition, which is an essential component of real-time activity monitoring systems. In particular, we investigate whether the training of an LSTM can benefit from pre-training or co-training on multiple datasets of related tasks when it uses non-transferred visual CNN features. A number of label mappings and multi-dataset training techniques are proposed and tested on three challenging kitchen activity datasets - Breakfast, 50 Salads and MPII Cooking 2. We show that transferring, by pre-training on similar datasets using label concatenation, delivers improved frame-based classification accuracy and faster training convergence than random initialisation.

U2 - 10.1109/ICCVW.2017.161

DO - 10.1109/ICCVW.2017.161

M3 - Conference Contribution (Conference Proceeding)

SN - 9781538610350

SP - 1354

EP - 1362

BT - 2017 IEEE International Conference of Computer Vision Workshop (ICCVW 2017)

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 22 October 2017

ER -