Skip to main navigation Skip to search Skip to main content

Heterogeneous Multi-Task Learning With Expert Diversity

Research output: Contribution to journal › Article (Academic Journal) › peer-review

35 Citations (Scopus)

Abstract

Predicting multiple heterogeneous biological and medical targets is a challenge for traditional deep learning models. In contrast to single-task learning, in which a separate model is trained for each target, multi-task learning (MTL) optimizes a single model to predict multiple related targets simultaneously. To address this challenge, we propose the Multi-gate Mixture-of-Experts with Exclusivity (MMoEEx). Our work aims to tackle the heterogeneous MTL setting, in which the same model optimizes multiple tasks with different characteristics. Such a scenario can overwhelm current MTL approaches due to the challenges in balancing shared and task-specific representations and the need to optimize tasks with competing optimization paths. Our method makes two key contributions: first, we introduce an approach to induce more diversity among experts, thus creating representations more suitable for highly imbalanced and heterogenous MTL learning; second, we adopt a two-step optimization (Finn et al., 2017 and Lee et al., 2020) approach to balancing the tasks at the gradient level. We validate our method on three MTL benchmark datasets, including UCI-Census-income dataset, Medical Information Mart for Intensive Care (MIMIC-III) and PubChem BioAssay (PCBA).
Original languageEnglish
Pages (from-to)3093-3102
Number of pages10
JournalIEEE/ACM Transactions on Computational Biology and Bioinformatics
Volume19
Issue number6
Early online date16 May 2022
DOIs
Publication statusPublished - 1 Nov 2022

Fingerprint

Dive into the research topics of 'Heterogeneous Multi-Task Learning With Expert Diversity'. Together they form a unique fingerprint.

Cite this