A multi-agent AI system scored 83% on difficult case simulations, outperforming general physicians as Microsoft prepares for hospital trials.
An artificial intelligence diagnostic system developed by Microsoft correctly diagnosed complex simulated medical cases more than twice as often as generalist physicians, according to a study published Thursday in the journal Science.
The system, named the Diagnostic Orchestrator or MAI-DxO, coordinates multiple AI agents to mimic a consultation panel of doctors, TechCrunch reported. The software reviews patient records, asks questions, orders diagnostic tests, and debates possible conditions before reaching a final medical conclusion.
How the tool performed against human doctors
In evaluations using hundreds of challenging case reports from the New England Journal of Medicine, Microsoft’s system reached the correct diagnosis about 83 percent of the time. By comparison, licensed primary care physicians and general internists working individually with medical references scored about 36 percent, while doctors working without reference materials scored roughly 22 percent, according to The Verge.
The AI tool also diagnosed emergency department scenarios assembled from patient admission records at Beth Israel Deaconess Medical Center in Boston. On those emergency cases, the system achieved 80 percent accuracy, compared with an average of 47 percent for human doctors. Microsoft has packaged these testing scenarios into a benchmark called SDBench, which it plans to make open-source.
The study authors noted that real-world cases as complicated as those in the New England Journal of Medicine are typically handled by teams of specialists rather than lone general practitioners. Specialist teams were not part of the study’s comparison group.
Model capabilities and operational costs
Microsoft tested the Diagnostic Orchestrator framework using models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The orchestration framework works by organizing AI models into specialized agent roles, but researchers found that the raw capability of the underlying model accounted for most of the system’s performance.
Running OpenAI’s GPT-5 on its own with a basic prompt yielded 78.9 percent accuracy on complex cases at an estimated computing cost of about $8,400 per case. Adding the Diagnostic Orchestrator system reduced that cost by around $3,000 while raising accuracy to roughly 83 percent. When tested on newer GPT-5.5 models released in April 2026, the orchestration raised diagnostic accuracy from 72 percent to nearly 87 percent on complex cases, though it produced no significant accuracy or cost advantages on routine emergency department cases.
The peer-reviewed paper did not evaluate other recent models, such as GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon, or Anthropic’s Opus 5.5.
Hospital trials and next steps
Microsoft AI CEO Mustafa Suleyman described the study as progress toward building medical superintelligence, which the company defines as AI that exceeds the reasoning and coordination of human healthcare systems across care.
“It needs clinical validation,” Suleyman told The Verge. “That is a leap, but this is a major milestone along that journey.”
Microsoft plans to run real-world clinical trials at Beth Israel Deaconess Medical Center in Boston. In that project, consenting patients will receive standard care from human clinicians while the AI system evaluates their cases in parallel. Hospital staff and researchers will then compare the software’s recommendations against the clinical decisions made by doctors. Dominic King, Microsoft AI’s health lead, said the trials will likely get underway within the next couple of months.
Beyond diagnostic testing, Microsoft published outlines in BMJ Digital Health & AI detailing other clinical areas it aims to address, including treatment plans and clinical care coordination. The company is also working with the Mayo Clinic to build a dedicated healthcare foundation model scheduled for release next year.
