DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

19 Citations (Scopus)
85 Downloads (Pure)


Domain alignment in convolutional networks aims to learn the degree of layer-specific feature alignment beneficial to the joint learning of source and target datasets. While increasingly popular in convolutional networks, there have been no previous attempts to achieve domain alignment in recurrent networks. Similar to spatial features, both source and target domains are likely to exhibit temporal dependencies that can be jointly learnt and aligned. In this paper we introduce Dual-Domain LSTM (DDLSTM), an architecture that is able to learn temporal dependencies from two domains concurrently. It performs cross-contaminated batch normalisation on both input-to hidden and hidden-to-hidden weights, and learns the parameters for cross-contamination, for both single-layer and multi-layer LSTM architectures. We evaluate DDLSTM on frame-level action recognition using three datasets, taking a pair at a time, and report an average increase in accuracy of 3.5%. The proposed DDLSTM architecture outperforms standard, fine-tuned, and batch-normalised LSTMs.
Original languageEnglish
Title of host publication2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages10
ISBN (Electronic)978-1-7281-3293-8
ISBN (Print)978-1-7281-3294-5
Publication statusPublished - 9 Jan 2020
EventComputer Vision and Pattern Recognition (CVPR) - Rhode Island, United States
Duration: 16 Jun 201221 Jun 2012

Publication series

NameConference on Computer Vision and Pattern Recognition (CVPR)
ISSN (Electronic)2575-7075


ConferenceComputer Vision and Pattern Recognition (CVPR)
Country/TerritoryUnited States
CityRhode Island


Dive into the research topics of 'DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognition'. Together they form a unique fingerprint.
  • HPC (High Performance Computing) Facility

    Sadaf R Alam (Manager), Steven A Chapman (Manager), Polly E Eccleston (Other), Simon H Atack (Other) & D A G Williams (Manager)

    Facility/equipment: Facility

Cite this