Microsoft AI diagnostic system outperforms generalist doctors in study

The MAI-DxO system achieved 83% accuracy on complex medical cases compared to 36% for physicians using reference materials.

Microsoft AI diagnostic system outperforms generalist doctors in study
Photo: TechCrunch

Microsoft's AI diagnostic system, MAI-DxO, correctly diagnosed complex medical cases in a simulation more than twice as often as generalist physicians, according to a study published Thursday in Science. In a simulation using complex cases from the New England Journal of Medicine, the system was correct 83.2% of the time. Primary care doctors and general internists working individually with reference materials were correct 36.1% of the time.

The system, also called the Diagnostic Orchestrator, coordinates several AI agents that act like a panel of doctors to reason through symptoms, medical histories, and diagnostic tests. While the system achieved high accuracy on complex cases, it showed little improvement in accuracy or cost on emergency department cases when compared to using GPT-5 on its own.

Mustafa Suleyman, CEO of Microsoft AI, called the result a step toward medical superintelligence. He said the company is working with the Mayo Clinic to train a healthcare foundation model, with a release planned for next year. Suleyman expects healthcare models to get an order of magnitude better in capability within the next 24 to 36 months.

The study's data suggests much of the improvement comes from more powerful underlying AI models, with Microsoft's orchestration system adding gains on top. Microsoft tested DxO with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, but its best results came with OpenAI's GPT-5 and GPT-5.5.

Microsoft will evaluate the clinical reasoning capabilities of DxO across different hospital settings through a new partnership with Beth Israel Deaconess Medical Center in Boston. The company and its partners will compare DxO's recommendations against those of human doctors.

Key facts

  • MAI-DxO achieved 83.2% accuracy on complex New England Journal of Medicine cases.
  • Generalist physicians using reference materials were correct 36.1% of the time.
  • Microsoft is working with the Mayo Clinic to train a healthcare foundation model for release next year.

Three perspectives

Neutral

The system's accuracy gains appear to depend on the capability of the underlying AI models and the complexity of the medical cases.

Positive

Mustafa Suleyman described the results as a step toward medical superintelligence.

Negative

Cases of this complexity are usually handled by specialists or physician teams, who were not part of the comparison.

Your week in news, every Wednesday

The biggest stories of the week for your edition, in your inbox every Wednesday morning, your time. Free, and you can unsubscribe at any time.

Choose your news coverage

Please choose and confirm your coverage.

Series of major U.S. tech outages fuels cyberattack fears
Next storySeries of major U.S. tech outages fuels cyberattack fears

All sections

Edition