AI Panel Beats Physicians on Complex Diagnoses, Signaling a New Era in Medical Tech

A new diagnostic system developed by Microsoft has shown the ability to outperform human doctors in handling complex medical cases, according to a study published in Science this week. The system, known as MAI-DxO or simply DxO, achieved an 83% accuracy rate in simulations, more than double the 36% accuracy recorded by primary care physicians and general internists working individually. Mustafa Suleyman, CEO of Microsoft AI, described this milestone as a significant step toward what he calls 'medical superintelligence,' a state where AI surpasses current human-led health systems in reasoning and coordination.

The core innovation of DxO lies in its architecture, which functions like a virtual panel of specialists rather than a single algorithm. Multiple AI agents collaborate to question patients, order necessary tests, and debate potential diagnoses before arriving at a conclusion. This multi-agent approach was tested using complex cases from the New England Journal of Medicine. While the study notes that such difficult cases are typically managed by specialists or teams rather than generalists, the significant gap in accuracy highlights the potential of coordinated AI in navigating medical complexity that often stumps individual practitioners.

Cost and model dependency remain critical factors in the system's performance. Microsoft tested DxO with models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The most robust results were achieved using OpenAI’s GPT-5 and GPT-5.5 models. For instance, on complex cases, GPT-5.5’s accuracy jumped from 72% to nearly 87% when orchestrated by DxO, while slightly reducing costs. However, the benefits were less pronounced for emergency department cases, where the system offered little improvement over standalone models. Furthermore, the orchestration layer provided a larger boost to weaker models, such as Gemini 2.5 Flash, which saw its accuracy rise from 46.4% to nearly 69%, though it still lagged behind the top-tier models.

Looking ahead, Microsoft is positioning this technology as part of a broader infrastructure for healthcare AI. Suleyman revealed that the company is partnering with the Mayo Clinic to train a dedicated healthcare foundation model, with a release expected next year. He anticipates a 'coding style breakthrough' in 2027 that will allow for highly accurate predictions of how medical records might unfold, offering clinicians valuable foresight. Additionally, Microsoft is launching a real-world evaluation with Beth Israel Deaconess Medical Center in Boston. This study will compare DxO’s recommendations against human decisions in emergency and urgent care settings, aiming to provide compelling evidence for the system's scalability.

This development places Microsoft in direct competition with peers like Google and OpenAI, which have recently claimed similar capabilities. Google, for example, recently published a study on its AMIE diagnostic AI in The Lancet. Suleyman emphasizes that Microsoft’s focus on integrating AI into the full arc of care, rather than just answering exam questions, could be a key differentiator. With existing sales of dictation and radiology tools, the company views DxO as an expansion of its clinical decision support portfolio. As the technology moves from simulation to real-world testing, the industry watches closely to see if AI can truly bridge the growing gap between medical expertise supply and patient demand.