Play It Back: Iterative Attention For Audio Recognition

Alexandros G Stergiou; Dima  Damen

doi:10.1109/ICASSP49357.2023.10096532

Play It Back: Iterative Attention For Audio Recognition

Alexandros G Stergiou, Dima Damen

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

1 Citation (Scopus)

40 Downloads (Pure)

Abstract

A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the art performance across three audio-classification benchmarks: Audio Set, VGG-Sound, and EPIC-KITCHENS-100.

Original language	English
Title of host publication	ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Number of pages	5
ISBN (Electronic)	9781728163277
ISBN (Print)	9781728163284
DOIs	https://doi.org/10.1109/ICASSP49357.2023.10096532
Publication status	Published - 5 May 2023
Event	IEEE International Conference on Acoustics, Speech and Signal Processing - Rhodes, Greece Duration: 2 Jun 2023 → … https://2023.ieeeicassp.org/

Publication series

Name	IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	IEEE
ISSN (Print)	1520-6149
ISSN (Electronic)	2379-190X

Conference

Conference	IEEE International Conference on Acoustics, Speech and Signal Processing
Abbreviated title	ICASSP
Country/Territory	Greece
City	Rhodes
Period	2/06/23 → …
Internet address	https://2023.ieeeicassp.org/

Access to Document

10.1109/ICASSP49357.2023.10096532

Full-text PDF (accepted manuscript)
This is the accepted author manuscript (AAM) of the article which has been made Open Access under the University of Bristol's Scholarly Works Policy. The final published version (Version of Record) can be found on the publisher's website. The copyright of any third-party content, such as images, remains with the copyright holder.
Accepted author manuscript, 1.42 MBLicence: CC BY

Handle.net

Persistent link

Cite this

Stergiou, A. G., & Damen, D. (2023). Play It Back: Iterative Attention For Audio Recognition. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICASSP49357.2023.10096532

@inproceedings{ae3902aebc3e44dfa691ecce43213f6a,

title = "Play It Back: Iterative Attention For Audio Recognition",

abstract = "A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the art performance across three audio-classification benchmarks: Audio Set, VGG-Sound, and EPIC-KITCHENS-100.",

author = "Stergiou, \{Alexandros G\} and Dima Damen",

year = "2023",

month = may,

day = "5",

doi = "10.1109/ICASSP49357.2023.10096532",

language = "English",

isbn = "9781728163284",

series = "IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

booktitle = "ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "IEEE International Conference on Acoustics, Speech and Signal Processing , ICASSP ; Conference date: 02-06-2023",

url = "https://2023.ieeeicassp.org/",

}

Stergiou, AG & Damen, D 2023, Play It Back: Iterative Attention For Audio Recognition. in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Institute of Electrical and Electronics Engineers (IEEE), IEEE International Conference on Acoustics, Speech and Signal Processing , Rhodes, Greece, 2/06/23. https://doi.org/10.1109/ICASSP49357.2023.10096532

Play It Back: Iterative Attention For Audio Recognition. / Stergiou, Alexandros G; Damen, Dima .
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers (IEEE), 2023. (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Play It Back

T2 - IEEE International Conference on Acoustics, Speech and Signal Processing

AU - Stergiou, Alexandros G

AU - Damen, Dima

PY - 2023/5/5

Y1 - 2023/5/5

N2 - A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the art performance across three audio-classification benchmarks: Audio Set, VGG-Sound, and EPIC-KITCHENS-100.

AB - A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the art performance across three audio-classification benchmarks: Audio Set, VGG-Sound, and EPIC-KITCHENS-100.

UR - https://2023.ieeeicassp.org/program-at-a-glance/

U2 - 10.1109/ICASSP49357.2023.10096532

DO - 10.1109/ICASSP49357.2023.10096532

M3 - Conference Contribution (Conference Proceeding)

SN - 9781728163284

T3 - IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

BT - ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 2 June 2023

ER -

Stergiou AG, Damen D. Play It Back: Iterative Attention For Audio Recognition. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers (IEEE). 2023. (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)). doi: 10.1109/ICASSP49357.2023.10096532

Play It Back: Iterative Attention For Audio Recognition

Abstract

Publication series

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

UMPIRE: United Model for the Perception of Interactions for visual Recognition

EPIC-KITCHENS-100

HPC (High Performance Computing) and HTC (High Throughput Computing) Facilities

Cite this

Play It Back: Iterative Attention For Audio Recognition

Abstract

Publication series

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

Projects

UMPIRE: United Model for the Perception of Interactions for visual Recognition

Datasets

EPIC-KITCHENS-100

Equipment

HPC (High Performance Computing) and HTC (High Throughput Computing) Facilities

Cite this