Machine Learning on MALDI-TOF Spectra Predicts Antimicrobial Resistance but Requires Local Retraining
Using nearly 15,000 MALDI-TOF mass spectra from E. coli, K. pneumoniae, and S. aureus, this validation study achieved good AMR prediction (AUROC 0.81-0.85) with gradient boosting and random forest classifiers. However, performance degraded substantially when models were tested on data from different locations or after 18 months, with AUROC drops of up to 0.25. The findings establish that ML-based AMR prediction from MALDI-TOF is feasible for accelerating susceptibility results by one day, but models must be trained on local data and retrained regularly.
The original study
Prediction of antimicrobial resistance from MALDI-TOF mass spectra using machine learning: a validation study.
- Authors
- Wiesmann N, Enders D, Westendorf A, Koch R, Schaumburg F
- Journal
- Journal of clinical microbiology
- Type
- Journal Article, Validation Study
- PMID
- 41296602
Original abstract
UNLABELLED: Matrix-assisted laser desorption-ionization-time of flight (MALDI-TOF) mass spectra can be used to predict antimicrobial resistance (AMR) using machine learning (ML). This study aimed to validate the performance of ML models for AMR prediction using own and publicly available MALDI-TOF data and to test how these models perform over time. Mass spectra of Escherichia coli (n = 7,897), Klebsiella pneumoniae (n = 2,444), and Staphylococcus aureus (n = 4,664) from routine diagnostics (Germany) and the DRIAMS-A database (Switzerland) were used. Six classification models were benchmarked for AMR prediction using cross-validation (regularized logistic regressions [LR], multilayer perceptrons [MLP], support vector machines [SVM], random forests [RF], gradient boosting machines [LGBM, XGB]). Performance was prospectively observed for 18 months after training. The performance of AMR prediction evaluated by the mean area under the receiver operating characteristic curve (AUROC) was comparable between the DRIAMS-A data set and own data. The best predictive performance (classifier, AUROC) on own data was achieved for oxacillin resistance in S. aureus (RF, 0.85), ciprofloxacin resistance in E. coli (XGB, 0.83), and piperacillin-tazobactam resistance in K. pneumoniae (XGB, 0.81). ML performance was poor if training and test data were unrelated in terms of location and time. Performance (change in AUROC) decreased within 18 months after training for S. aureus (oxacillin resistance, RF: -0.10), E. coli (ciprofloxacin, XGB: -0.19), and K. pneumoniae (piperacillin-tazobactam, XGB: -0.25). The performance of ML for the prediction of AMR based on MALDI-TOF data is good (AUROC ≥ 0.8) but classifiers need to be trained on local data and retrained regularly to maintain the performance level. IMPORTANCE: MALDI-TOF mass spectrometry can be used not only for bacterial species identification but also for the prediction of antimicrobial resistance (AMR) using machine learning (ML). Such an approach would provide antimicrobial susceptibility test results one day earlier than conventional routine diagnostics. This is essential for an early targeted treatment to reduce mortality of severe infections. We show that the performance of ML for the prediction of AMR based on MALDI-TOF data is good (AUROC ≥ 0.8). However, the ML models need to be trained on local data and retrained regularly to maintain a good performance.