Will AI alone best physician-AI collaboration?

Clinical takeaway: When assessing an AI workflow for clinical practice, consider whether it will be used alone or with clinician review.  

Most clinicians have been handed a reassuring premise about AI: it can be a useful assistant to guide human decision-making. The American Medical Association favors the term "augmented intelligence," and the American College of Physicians holds that AI should stay supportive and not displace physician judgment. But the uncomfortable possibility is that physician supervision may sometimes degrade accuracy rather than protect it. 

Clinicians are sometimes unreliable judges of when to trust a model. Algorithm aversion appears in 75% of studies on algorithmic advice; it's strongest among high-expertise decision-makers. Regular use may also erode the skills supervision depends on. The overall argument draws on AI-in-medicine literature published since January 1, 2024, spanning five cognitive tasks: history taking, differential diagnosis, test selection, guideline-concordant prescribing, and chronic disease management. 

A meta-analysis of 106 human-AI experiments found that pairing helps only in one direction. When humans outperformed the model, collaboration improved results. When the model was better, collaboration made things worse. The clinical version is stark: GPT-4 alone reached a median diagnostic reasoning score of 92% on real patient cases, versus 76% for physicians using the same model. Radiology is the one exception the authors carve out as working best with AI-human collaboration, because AI alone has not reliably beaten physicians there. 

ChatGPT o3 ranked the correct diagnosis first in 60% of 377 complex cases, against 15.9% for internal medicine physicians on a subset. In 159 simulated encounters, Google's Articulate Medical Intelligence Explorer prescribed appropriate treatment in 90% of cases versus 37% for licensed primary care physicians. A separate system reached correct diagnoses more often at roughly 19% lower testing cost. 

The AI models assessed here and in prior work are already largely outdated. Many were also tested straight out of the box, without prompts or setup tailored to the clinical task, which the authors say makes the models look worse than they are. The authors dismiss an 83-study meta-analysis showing no clear AI advantage because 80 of those studies used older ChatGPT versions, and they set aside another study for excluding the strongest models available.

But GPT-4, ChatGPT o3, and o1-preview have since been succeeded as well, and the authors note that most studies predate reasoning models entirely. Nearly all the underlying data come from vignettes and simulated tasks rather than live encounters, where information exchange between model and user is a known failure point. Two of the four authors hold commercial positions in autonomous care, one as chief executive of an AI clinic company and one as an investor in that company and in OpenAI. 

The authors anticipate autonomous AI in some clinical workflows by 2030. But the infrastructure to support that remains largely unbuilt. Reimbursement has no category for care delivered without a clinician. Regulation and liability also assume a supervising human. Medical education still trains for tasks the authors expect will be automated. Physical procedures keep clinicians in the loop regardless, so the near-term question is which cognitive functions get carved out and who is responsible for them. 

"That in the near future AI-alone may provide better patient care than physicians or physician-controlled hybrids at 5 fundamental cognitive medical tasks is unsettling but seems probable," the authors conclude. 

Source: Emanuel EJ, et al. (2026 Aug 17) JAMA. Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?