TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

Zeynel A Samak; Philip L Clatworthy; Majid Mirmehdi

doi:10.1109/ISBI53787.2023.10230576

TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

Zeynel A Samak, Philip L Clatworthy, Majid Mirmehdi

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

7 Citations (Scopus)

20 Downloads (Pure)

Abstract

Acute ischaemic stroke, caused by an interruption in blood flow to brain tissue, is a leading cause of disability and mortality worldwide. The selection of patients for the most optimal ischaemic stroke treatment is a crucial step for a successful outcome, as the effect of treatment highly depends on the time to treatment. We propose a transformer-based multimodal network (TranSOP) for a classification approach that employs clinical metadata and imaging information, acquired on hospital admission, to predict the functional outcome of stroke treatment based on the modified Rankin Scale (mRS). This includes a fusion module to efficiently combine 3D non-contrast computed tomography (NCCT) features and clinical information. In comparative experiments using unimodal and multimodal data on the MRCLEAN dataset, we achieve a state-of-the-art AUC score of 0.85.

Original language	English
Title of host publication	20th IEEE International Symposium on Biomedical Imaging, ISBI 2023
Publisher	IEEE Computer Society
Pages	1-5
Number of pages	5
ISBN (Electronic)	9781665473583
ISBN (Print)	9781665473590
DOIs	https://doi.org/10.1109/ISBI53787.2023.10230576
Publication status	Published - 1 Sept 2023
Event	20th IEEE International Symposium on Biomedical Imaging, ISBI 2023 - Cartagena de Indias Convention Center, Cartagena de Indias, Colombia Duration: 18 Apr 2023 → 21 Apr 2023 http://2023.biomedicalimaging.org/en/

Publication series

Name
ISSN (Print)	1945-7928
ISSN (Electronic)	1945-8452

Conference

Conference	20th IEEE International Symposium on Biomedical Imaging, ISBI 2023
Abbreviated title	ISBI
Country/Territory	Colombia
City	Cartagena de Indias
Period	18/04/23 → 21/04/23
Internet address	http://2023.biomedicalimaging.org/en/

Keywords

Transformer
Multimodal
Stroke
Ischaemic
NCCT
Outcome

Access to Document

10.1109/ISBI53787.2023.10230576Licence: Unspecified

Full-text PDF (accepted author manuscript)
This is the accepted author manuscript (AAM). The final published version (Version of Record) can be found on the publisher's website. The copyright of any third-party content, such as images, remains with the copyright holder.
Accepted author manuscript, 339 KBLicence: CC BY

Handle.net

Persistent link

Cite this

@inproceedings{c291e0043d464d698a45a3e5de853a8b,

title = "TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction",

abstract = "Acute ischaemic stroke, caused by an interruption in blood flow to brain tissue, is a leading cause of disability and mortality worldwide. The selection of patients for the most optimal ischaemic stroke treatment is a crucial step for a successful outcome, as the effect of treatment highly depends on the time to treatment. We propose a transformer-based multimodal network (TranSOP) for a classification approach that employs clinical metadata and imaging information, acquired on hospital admission, to predict the functional outcome of stroke treatment based on the modified Rankin Scale (mRS). This includes a fusion module to efficiently combine 3D non-contrast computed tomography (NCCT) features and clinical information. In comparative experiments using unimodal and multimodal data on the MRCLEAN dataset, we achieve a state-of-the-art AUC score of 0.85.",

keywords = "Transformer, Multimodal, Stroke, Ischaemic, NCCT, Outcome",

author = "Samak, {Zeynel A} and Clatworthy, {Philip L} and Majid Mirmehdi",

year = "2023",

month = sep,

day = "1",

doi = "10.1109/ISBI53787.2023.10230576",

language = "English",

isbn = "9781665473590",

publisher = "IEEE Computer Society",

pages = "1--5",

booktitle = "20th IEEE International Symposium on Biomedical Imaging, ISBI 2023",

address = "United States",

note = "20th IEEE International Symposium on Biomedical Imaging, ISBI 2023, ISBI ; Conference date: 18-04-2023 Through 21-04-2023",

url = "http://2023.biomedicalimaging.org/en/",

}

Samak, ZA, Clatworthy, PL & Mirmehdi, M 2023, TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction. in 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023. IEEE Computer Society, pp. 1-5, 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023, Cartagena de Indias, Colombia, 18/04/23. https://doi.org/10.1109/ISBI53787.2023.10230576

TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction. / Samak, Zeynel A; Clatworthy, Philip L ; Mirmehdi, Majid.
20th IEEE International Symposium on Biomedical Imaging, ISBI 2023. IEEE Computer Society, 2023. p. 1-5.

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - TranSOP

T2 - 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023

AU - Samak, Zeynel A

AU - Clatworthy, Philip L

AU - Mirmehdi, Majid

PY - 2023/9/1

Y1 - 2023/9/1

N2 - Acute ischaemic stroke, caused by an interruption in blood flow to brain tissue, is a leading cause of disability and mortality worldwide. The selection of patients for the most optimal ischaemic stroke treatment is a crucial step for a successful outcome, as the effect of treatment highly depends on the time to treatment. We propose a transformer-based multimodal network (TranSOP) for a classification approach that employs clinical metadata and imaging information, acquired on hospital admission, to predict the functional outcome of stroke treatment based on the modified Rankin Scale (mRS). This includes a fusion module to efficiently combine 3D non-contrast computed tomography (NCCT) features and clinical information. In comparative experiments using unimodal and multimodal data on the MRCLEAN dataset, we achieve a state-of-the-art AUC score of 0.85.

AB - Acute ischaemic stroke, caused by an interruption in blood flow to brain tissue, is a leading cause of disability and mortality worldwide. The selection of patients for the most optimal ischaemic stroke treatment is a crucial step for a successful outcome, as the effect of treatment highly depends on the time to treatment. We propose a transformer-based multimodal network (TranSOP) for a classification approach that employs clinical metadata and imaging information, acquired on hospital admission, to predict the functional outcome of stroke treatment based on the modified Rankin Scale (mRS). This includes a fusion module to efficiently combine 3D non-contrast computed tomography (NCCT) features and clinical information. In comparative experiments using unimodal and multimodal data on the MRCLEAN dataset, we achieve a state-of-the-art AUC score of 0.85.

KW - Transformer

KW - Multimodal

KW - Stroke

KW - Ischaemic

KW - NCCT

KW - Outcome

UR - https://biomedicalimaging.org/2023/

U2 - 10.1109/ISBI53787.2023.10230576

DO - 10.1109/ISBI53787.2023.10230576

M3 - Conference Contribution (Conference Proceeding)

SN - 9781665473590

SP - 1

EP - 5

BT - 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023

PB - IEEE Computer Society

Y2 - 18 April 2023 through 21 April 2023

ER -

TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

Abstract

Publication series

Conference

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Automatic prediction of functional outcome of patients with ischaemic stroke

Cite this

TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

Abstract

Publication series

Conference

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Student theses

Automatic prediction of functional outcome of patients with ischaemic stroke

Cite this