Standard medical licensing exams test static knowledge with single answers upfront, whereas clinical reality requires iterative sequential reasoning that traditional tools fail to capture. To address this structural mismatch, the MAI Health Team developed an AI diagnostic system named MAI-DxO, introducing the Sequential Diagnosis Benchmark to evaluate step-by-step diagnostic reasoning across complex clinical cases. As Microsoft AI outlines, the benchmark assesses performance across 304 complex cases taken from The New England Journal of Medicine case proceedings and a set of emergency-department presentations from MIMIC-IV-ED.
Rather than relying on one model for an immediate answer, the system operates like a panel of virtual physicians that coordinate an exchange over several rounds of debate, updating assessments as evidence arrives while weighing each test's likely value against its cost. MAI-DxO achieved 83 percent diagnostic accuracy on complex NEJM cases, whereas physicians achieved 22 percent without external reference resources and 36 percent with them. On more typical emergency-department cases from MIMIC-IV-ED, MAI-DxO achieved 80 percent accuracy versus 47 percent for physicians. When optimised more heavily for cost, one configuration achieved 81 percent accuracy on NEJM cases at less than half the cost of comparative systems, demonstrating that accuracy and cost-efficiency can scale together in clinical environments.
By systematically evaluating each test's likely diagnostic value against its financial cost during multi-agent deliberation, the system directly targets the primary inefficiency in hospital operations: redundant and low-value diagnostic workups. In fee-for-service revenue models, unnecessary testing has traditionally functioned as a volume-driven margin buffer, but as value-based reimbursement and fixed-payment bundles expand, clinics face strict financial penalties for diagnostic sprawl. By cutting the cost of comparative systems by more than half while maintaining an 81 percent accuracy rate, multi-agent diagnostic architectures alter the underlying unit economics of clinical care, converting what was once an unpredictable operational expense into a controllable, optimized variable.
MAI-DxO achieves diagnostic accuracy on complex NEJM cases that substantially exceeds human baselines while managing to operate at a fraction of the cost through optimized workups.
This architecture bypasses the traditional bottlenecks of clinical workflows by orchestrating multiple AI agents to select useful questions and order tests step by step. Across both datasets, no other human or AI approach was both more accurate and less costly than MAI-DxO, signaling a profound shift in how clinical validation can be automated. These gains persisted and improved when the evaluation was replaced or repeated with newer models, including GPT-5.5, without changing the orchestration method, suggesting the approach is not tied to one model generation.
The MAI Health Team is open-sourcing SDBench for select NEJM cases, including the benchmark, Gatekeeper, Judge, and cost-estimation code, to allow researchers to reproduce the evaluation and test other models. Furthermore, the organization aims to combine frontier AI with clinical practice to evaluate clinical reasoning capabilities across different hospital settings. As the MAI Health Team notes, the work is published in Science, alongside a broader vision for medical superintelligence published in BMJ Digital Health and AI.
How healthcare systems will absorb the capital expenditure required to deploy autonomous diagnostic orchestrators at scale while restructuring hospital operations remains unaddressed.
Editorial analysis
### How the story developed
In July 2026, clinical simulations in hematology-oncology saw the best of 32 tested AI setups reach only 68% accuracy, right before Google rolled out its AMIE diagnostic system in August. Microsoft’s MAI Health Team has now introduced MAI-DxO alongside the Sequential Diagnosis Benchmark, evaluating multi-step diagnostic reasoning across 304 complex cases from The New England Journal of Medicine and emergency records from MIMIC-IV-ED. Diagnostic AI is shifting from static exam recall to sequential reasoning, testing whether models can handle actual clinical decision chains.
Sources: our stories of 17.07.2026, 12.08.2026