Abstract
Background
Interest on the usefulness of machine learning (ML) methods for outcomes prediction has continued to increase in recent years. However, the advantage of advanced ML model over traditional logistic regression (LR) remains controversial. We performed a systematic review and meta-analysis of studies comparing the discrimination accuracy between ML models versus LR in predicting operative mortality following cardiac surgery.
Methods
The present systematic review followed the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) statement. Discrimination ability was assessed using c-statistic. Pooled c-statistics and its 95% credibility interval for ML models and LR were obtained were obtained using a Bayesian framework. Pooled estimates for ML models and LR were compared to inform on difference between the two approaches.
Results
We identified 459 published citations of which 15 studies met inclusion criteria and were used for the quantitative and qualitative analysis. When the best ML model from individual study was used, meta-analytic estimates showed that ML were associated with a significantly higher c-statistic (ML 0.88; 95%CrI 0.83-0.93 vs LR 0.81; 95%CrI 0.77-0.85; P=0.03). When individual ML algorithm were instead selected, we found a non-significant trend toward better prediction with each of ML algorithms. We found no evidence of publication bias (P=0.70).
Conclusions
The present findings suggest that when compared to LR, ML models provide better discrimination in mortality prediction after cardiac surgery . However, the magnitude and clinical impact of such an improvement remains uncertain.
Interest on the usefulness of machine learning (ML) methods for outcomes prediction has continued to increase in recent years. However, the advantage of advanced ML model over traditional logistic regression (LR) remains controversial. We performed a systematic review and meta-analysis of studies comparing the discrimination accuracy between ML models versus LR in predicting operative mortality following cardiac surgery.
Methods
The present systematic review followed the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) statement. Discrimination ability was assessed using c-statistic. Pooled c-statistics and its 95% credibility interval for ML models and LR were obtained were obtained using a Bayesian framework. Pooled estimates for ML models and LR were compared to inform on difference between the two approaches.
Results
We identified 459 published citations of which 15 studies met inclusion criteria and were used for the quantitative and qualitative analysis. When the best ML model from individual study was used, meta-analytic estimates showed that ML were associated with a significantly higher c-statistic (ML 0.88; 95%CrI 0.83-0.93 vs LR 0.81; 95%CrI 0.77-0.85; P=0.03). When individual ML algorithm were instead selected, we found a non-significant trend toward better prediction with each of ML algorithms. We found no evidence of publication bias (P=0.70).
Conclusions
The present findings suggest that when compared to LR, ML models provide better discrimination in mortality prediction after cardiac surgery . However, the magnitude and clinical impact of such an improvement remains uncertain.
| Original language | English |
|---|---|
| Number of pages | 23 |
| Journal | Journal of Thoracic and Cardiovascular Surgery |
| DOIs | |
| Publication status | Published - 10 Aug 2020 |
Research Groups and Themes
- Bristol Heart Institute
- Anaesthesia Pain and Critical Care
Keywords
- risk model
- prediction
- mortality
- machine learning
- logistic regression
- meta-analysis