Generalist Medical AI: A New Paradigm for Multimodal Clinical Foundation Models
This Nature review proposes the concept of Generalist Medical AI (GMAI) -- foundation models trained through self-supervision on diverse datasets that can flexibly interpret imaging, EHR data, laboratory results, genomics and medical text with minimal task-specific labelling. The authors identify high-impact applications and outline the technical capabilities required to enable them. The framework has direct implications for how diagnostic laboratories will integrate AI across imaging, molecular and clinical chemistry workflows.
The original study
Foundation models for generalist medical artificial intelligence.
- Authors
- Moor M, Banerjee O, Abad ZSH, Krumholz HM, Leskovec J, Topol EJ, et al.
- Journal
- Nature
- Type
- Journal Article, Review, Research Support, Non-U.S. Gov't, Research Support, N.I.H., Extramural, Research Support, U.S. Gov't, P.H.S., Research Support, U.S. Gov't, Non-P.H.S.
- PMID
- 37045921
Original abstract
The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets.