Microsoft’s MAI-DxO Diagnosed Hard Cases Better Than Doctors — Here’s What the Science Study Actually Proves

Key takeaway: Microsoft’s MAI-DxO, an AI system that diagnoses patients step by step like a doctor, correctly diagnosed about 83% of 304 complex cases from the New England Journal of Medicine in a peer-reviewed study published in Science on October 22, 2026 — versus 36% for generalist physicians working with textbooks and the internet. The catch: the cases were simulated, not real patients.

What Microsoft actually built

MAI-DxO stands for Microsoft AI Diagnostic Orchestrator. It is not a single AI model trained to answer medical questions. Instead, it is an orchestration layer that turns a language model into a virtual panel of clinicians: several AI agents propose diagnoses, ask follow-up questions, order tests, debate one another’s reasoning, and update their conclusions as new information arrives.

That sequential process is the point. Most medical AI benchmarks hand a model the patient’s full history, lab results, and imaging at once and ask for a diagnosis. Real doctors never work that way. As Dr. Dominic King, corporate vice president of Microsoft AI’s health unit, put it: “No patient ever came to see me with a nice summary.” MAI-DxO starts with only a brief description of the problem, then decides what to ask, what to examine, and which tests to order — weighing the information a test would provide against its cost.

Microsoft tested the orchestrator on top of models from OpenAI, Google, Anthropic, Meta, and DeepSeek. Its best results came with OpenAI’s GPT-5 and GPT-5.5, and Microsoft says it plans to open-source the evaluation benchmark, called SDBench.

The headline numbers

On the 304 complex NEJM cases, converted into stepwise encounters where each requested test carried a virtual cost, MAI-DxO reached about 83% diagnostic accuracy. Generalist physicians working individually scored about 36% with textbooks and internet access, and about 22% without reference materials. On more typical emergency-department cases drawn from Beth Israel Deaconess Medical Center’s admissions data, the system scored 80% against 47% for physicians.

Microsoft also reports lower estimated diagnostic costs: in some configurations, the system reached its diagnosis at a fraction of the estimated cost of the human approach, by ordering fewer unnecessary tests.

What is confirmed, what is not

Confirmed: the results are peer-reviewed and published in Science, a significant step up from a company blog post. The sequential-evaluation design is genuinely different from the multiple-choice-style benchmarks that dominate medical AI testing, where, as King noted, AI systems now routinely score near 100%.

Not confirmed: whether this works on real patients. The study used historical, text-based cases — no live patients, no physical examinations, no medical images, and no pathology specimens. The human comparison group was individual general practitioners and general internists, not teams of specialists, which is how the hardest real cases are actually handled. The system has no regulatory approval and is not intended for clinical use.

Analysis: some of the gain also appears to come from the underlying models themselves rather than the orchestration. With GPT-5, the orchestrator added only about four percentage points over the model alone; with GPT-5.5, the gain was larger, from 72% to nearly 87%. On the emergency-department cases, GPT-5.5 alone scored about the same as with the orchestrator. That suggests the orchestration layer matters most on the hardest, most ambiguous cases — exactly where doctors themselves struggle.

Why this matters beyond medicine

The result is a template for a broader shift in AI evaluation: from “can the model answer correctly when given everything?” to “can it figure out what it needs to know, and at what cost?” That sequential, cost-aware reasoning applies far beyond diagnosis — to troubleshooting, engineering, customer support, and any job where gathering information is itself expensive.

It also lands in the middle of a fierce debate about AI in healthcare. Microsoft AI CEO Mustafa Suleyman has called the result a step toward “medical superintelligence.” Critics counter that beating solo generalists on simulated puzzles is not the same as beating real clinical teams on real patients. Both things can be true at once — and the honest reading is that this is a strong research result, not a product launch.

What to watch next

Microsoft says it has begun a study to test whether the system can be used in real patient care, starting with a large Boston hospital. Watch for three things: whether SDBench is actually released open-source so independent researchers can reproduce the results; whether real-world trials show the same accuracy and cost advantages; and how regulators respond when a system like this moves from research toward clinical use.

For now, the practical takeaway is narrow but real: AI that reasons step by step, asks its own questions, and weighs the cost of information is measurably better at hard diagnostic puzzles than AI that answers all at once. The gap between that and a doctor you can trust with your health is still wide — and closing it will take real-patient evidence, not just benchmark scores.

Sources: Reuters’ report on the study, and The Verge’s coverage of the Science paper and Microsoft’s announcement, linked below.

Sources: Reuters: Microsoft develops AI tool that can diagnose ‘sequentially’ like a doctor | The Hindu: Microsoft develops AI tool that can diagnose ‘sequentially’ like a doctor

Frequently asked questions

What is Microsoft’s MAI-DxO?

MAI-DxO (Microsoft AI Diagnostic Orchestrator) is an orchestration layer that coordinates several AI agents to act like a panel of doctors. The agents ask questions, order tests, debate diagnoses, and update their conclusions as new information arrives, rather than giving a single immediate answer.

How accurate was MAI-DxO compared with doctors?

On 304 complex cases from the New England Journal of Medicine, MAI-DxO reached about 83% accuracy, versus about 36% for generalist physicians using textbooks and the internet and 22% without references. On emergency-department cases, it scored 80% versus 47% for physicians.

Does this mean AI can replace doctors?

No. The study used simulated, text-based historical cases with no live patients, physical exams, or medical images, and the doctors tested were individual generalists, not specialist teams. The system has no regulatory approval and is not intended for clinical use.

What is SDBench?

SDBench is Microsoft’s sequential diagnostic benchmark. It converts 304 complex NEJM cases into stepwise encounters in which the AI starts with only a brief description and must request each question or test itself, with each test carrying a virtual cost. Microsoft says it plans to open-source the benchmark.

When will patients actually see this technology?

Not soon. Microsoft has begun a study on real patient care with a large Boston hospital, but any clinical use would require real-world trials and regulatory approval. For now, MAI-DxO is a research result, not a medical product.