Why Did Microsoft Teach AI to Ask Questions Instead of Just Finding Answers?

Doctors do not begin a consultation already knowing what disease a patient has.

Most patients arrive with limited information about their symptoms. A fever, pain, swelling, or difficulty breathing rarely provides enough information to establish the underlying cause.

Doctors begin by asking questions.

When did the symptoms start? Does the patient have other medical conditions? What medications are they taking?

If necessary, physicians order blood tests, imaging studies, and other examinations. Once the results become available, they reconsider their initial diagnosis and determine whether further testing is needed.

Clinical diagnosis is therefore a process of gathering information progressively and repeatedly updating judgments.

Yet much of the evaluation of medical AI has historically been far removed from this process.

AI models have often been given a patient's symptoms, medical history, and test results all at once, then asked to identify the correct diagnosis.

It resembles a medical examination.

Real physicians, however, must also decide which questions to ask, which tests are necessary, and when to stop gathering information and reach a diagnosis.

Microsoft researchers focused on this distinction.

On October 22, 2026, the journal Science published a paper titled Sequential Diagnosis with Language Models, evaluating whether artificial intelligence could gather information and make decisions sequentially, in a way that more closely resembles a physician's diagnostic process.

The researchers developed a system that begins with limited patient information, asks relevant questions, selects diagnostic tests, revises its hypotheses, and eventually produces a final diagnosis.

They called the system the Diagnostic Orchestrator, or DxO.

The results were striking.

Across complex clinical cases from the New England Journal of Medicine (NEJM), DxO achieved diagnostic accuracy of 83.2%.

The average accuracy of participating physicians who were allowed to consult reference materials was 36.1%.

In a separate evaluation constructed from actual emergency department records, DxO achieved 80.3% accuracy, compared with an average of 46.8% among participating physicians.

But reducing the study to the conclusion that AI diagnoses patients better than doctors would be misleading.

The research was conducted in a controlled, text-based evaluation environment.

It did not compare AI and physicians across the full spectrum of clinical care, including face-to-face patient encounters, specialist consultations, treatment decisions, and patient safety.

Nevertheless, the study points toward an important direction in medical AI.

Medical AI competitiveness is expanding beyond how much medical knowledge a system can recall to how effectively it decides what information to collect, when to collect it, and how to revise its judgments.

Traditional Medical AI Evaluation Was More Like an Exam with All the Information Provided

One of the areas where medical AI has demonstrated strong performance is medical knowledge testing.

Models have been evaluated by answering questions similar to medical licensing examinations or by reading complex clinical cases and proposing diagnoses.

Recent language models have achieved high scores on such benchmarks.

But there is a fundamental problem.

The questions may already contain all the necessary information.

If a model is given the patient's symptoms, medical history, physical examination findings, blood test results, and imaging results, its primary task is to analyze the available information and identify the most likely disease.

In a real hospital, however, doctors do not possess all this information at the beginning of an encounter.

They must listen to the patient's symptoms and determine what additional information is needed.

They must decide which questions to ask and whether the information gained from a diagnostic test justifies its costs and burdens.

They must also determine whether to continue testing or make a diagnosis based on the information already available.

Microsoft researchers described this process as sequential diagnosis.

The SDBench framework developed for the study was designed to evaluate precisely this capability.

Rather than presenting a complete clinical case from the outset, SDBench initially provides only a brief patient description.

Additional information is then supplied progressively as the AI or physician asks questions or requests tests.

The evaluation measures both diagnostic accuracy and accumulated testing costs.

It therefore assesses not only whether a diagnosis is correct, but also how efficiently the diagnostic system obtains the information needed to reach that conclusion.

Turning 629 Clinical Cases into Sequential Diagnostic Problems

The researchers used two different medical datasets.

The first consisted of 304 clinicopathological conference cases published in the New England Journal of Medicine.

These cases often involve complex or rare diseases and are commonly used for medical education and clinical reasoning.

The second dataset included 325 emergency department admission cases extracted from MIMIC-IV-ED.

Because these cases were based on real emergency department electronic medical records, they reflected characteristics closer to everyday clinical practice than carefully prepared academic case reports.

Together, the researchers converted 629 cases into sequential diagnostic tasks.

Neither AI systems nor participating physicians could access all the test results from the beginning.

They received brief symptom descriptions and had to request additional information through questions or tests.

The researchers designed three primary roles within the evaluation framework.

The first was the Diagnostic Agent.

This role was performed by either a physician or an AI system. It asked questions, requested tests, and submitted a final diagnosis.

The second was the Gatekeeper Agent.

The Gatekeeper managed the original medical record and disclosed only the information that had been requested.

If the diagnostician asked about medical history, for example, it retrieved the relevant information from the source record.

If a particular test was requested, it provided the corresponding result.

When the requested information did not exist in the original record, the Gatekeeper generated a neutral synthetic response designed not to point toward a particular diagnosis.

This was intended to prevent the absence of information itself from becoming a diagnostic clue.

The third role was the Judge Agent.

The Judge compared the final diagnosis submitted by the AI or physician with the reference diagnosis.

The evaluation used a five-level physician-developed rubric and accepted diagnoses as correct when the degree of agreement was sufficient that the treatment approach would not materially change.

This design reproduced parts of clinical reasoning more concretely than conventional medical knowledge benchmarks.

However, the Gatekeeper's ability to generate neutral synthetic results also represents an important limitation.

Real medical records are often incomplete, patients' descriptions can be ambiguous, and actual test results contain measurement errors and interpretive uncertainty.

SDBench does not fully reproduce these complexities.

Why Microsoft Built a Virtual Medical Team Instead of Relying on a Single AI

The central technology developed in the study was DxO.

It does not operate by having one language model immediately produce a diagnosis.

Instead, it simulates multiple clinical expert roles, allowing diagnostic hypotheses to be examined from different perspectives.

When initial patient information becomes available, the system generates a set of possible diagnoses.

It then determines which questions should be asked next and updates its hypotheses as new information arrives.

It also examines alternatives that contradict leading diagnostic hypotheses, reducing the risk of reaching a premature conclusion.

When requesting tests, DxO considers both their expected diagnostic value and their cost.

The important point is that the system is not designed simply to order as many tests as possible.

Ordering every available test might increase diagnostic information, but it would also generate substantial costs.

Conversely, minimizing testing expenses too aggressively could cause important diseases to be missed.

DxO was designed to balance these competing considerations.

The researchers described the approach as model-agnostic orchestration.

In other words, it is not merely a diagnostic prompt tailored to one specific language model.

It is a structured framework that can organize sequential diagnosis using different models.

The research was initially developed using GPT-4.1, while major performance evaluations were subsequently conducted using GPT-5.

Before publication, the researchers also repeated the full evaluation using GPT-5.5.

According to the researchers, the advantages of structured diagnosis persisted across these model changes.

This suggests that the performance of medical AI depends on more than the underlying model.

The procedures used to collect information, the criteria for selecting tests, and the methods used to validate judgments can also be important determinants of performance.

AI Achieved 83.2% Accuracy in Complex Clinical Cases, Compared with 36.1% for Physicians

The most widely discussed results came from the evaluation involving NEJM clinical cases.

Using GPT-5, DxO achieved diagnostic accuracy of 83.2% when no explicit budget restriction was imposed on diagnostic testing.

Its average estimated testing cost was $5,431 per case.

Participating physicians who were allowed to consult reference materials achieved an average accuracy of 36.1%.

Their average estimated testing cost was $4,725.

One important distinction is that DxO achieved higher accuracy under these conditions but also incurred higher testing costs.

It would therefore be inaccurate to suggest that the 83.2% result demonstrated both superior accuracy and lower costs.

However, DxO maintained relatively high accuracy when testing budgets were constrained.

Under a configuration with an average estimated testing cost of $2,241, DxO achieved 80.9% accuracy.

At an average cost of $1,303, it achieved 79.6% accuracy.

These findings become particularly notable when compared with both the accuracy and estimated costs recorded by participating physicians.

They suggest that AI may be able to maintain high diagnostic accuracy while operating under tighter testing budgets.

The researchers analyzed these results through the concept of a cost-accuracy Pareto frontier.

A Pareto frontier represents combinations of performance and cost that cannot be simultaneously improved upon by another available approach.

Across multiple configurations, DxO demonstrated favorable combinations of diagnostic accuracy and testing costs compared with conventional language-model approaches and participating physicians.

However, these findings apply to the physicians, models, and cases included in the study.

They do not establish that AI performs better than all physicians or medical institutions in real clinical practice.

High Accuracy Also Appeared in Emergency Department Cases

If the system had performed well only on complex academic case reports, the broader significance of the findings would have been limited.

The researchers therefore conducted a separate evaluation using records from actual emergency department encounters.

The dataset consisted of 325 emergency department admission cases extracted from MIMIC-IV-ED.

In this evaluation, DxO achieved 80.3% diagnostic accuracy without a testing budget constraint.

Its average estimated testing cost was $4,979.

Participating physicians achieved an average accuracy of 46.8%, with average estimated testing costs of $3,400.

Once again, the AI system was more accurate under the unrestricted setting, but its testing costs were also higher.

When cost constraints were introduced, however, the results changed.

DxO achieved 73.2% accuracy at an average testing cost of $2,634.

At an average cost of $2,037, its diagnostic accuracy was 69.5%.

Both results represented higher diagnostic accuracy at lower estimated testing costs than the averages recorded by participating physicians.

The researchers considered the emergency department findings meaningful because the cases were relatively closer to real clinical workflows.

Nevertheless, these evaluations did not involve AI directly treating patients in emergency departments.

They were text-based diagnostic exercises constructed from existing medical records.

They did not fully reproduce changes in patients' conditions, physical examinations, collaboration among clinicians, delays in obtaining test results, or actual treatment decisions.

Why Did the AI Ask More Questions Before Ordering Tests?

One of the most interesting differences in the study concerned the diagnostic process itself.

Participating physicians tended to request tests relatively early.

DxO, by contrast, asked substantially more questions before ordering tests.

In the NEJM cases, physicians asked an average of 5.6 questions before requesting their first test.

DxO asked an average of 25.5 questions.

That is approximately 4.6 times as many.

This offers an important clue beyond the simple explanation that AI performed better because it possessed more medical knowledge.

Rather than immediately requesting extensive testing, DxO systematically gathered information about patient history and symptoms.

The researchers interpreted this as one potential reason the system could maintain accuracy while reducing testing costs.

Consider a situation involving a suspected infectious disease.

Questions about recent international travel, environmental exposure, or contact with particular animals can substantially change the range of likely diagnoses.

Ordering a broad series of tests without obtaining this information may increase costs unnecessarily.

By contrast, a thorough patient interview can narrow the list of possibilities and help identify which tests are actually needed.

The researchers identified structured history-taking as one area where DxO demonstrated advantages over conventional language-model approaches.

These advantages were especially relevant in cases where details such as unusual exposures or travel-related infections were diagnostically important.

This suggests a different role for medical AI.

Rather than improving accuracy by ordering more tests than physicians, AI may help make diagnosis more efficient by organizing questions systematically.

What Kinds of Errors Did the AI Make?

The researchers examined not only the proportion of correct diagnoses but also the types of errors made.

When DxO produced an incorrect diagnosis, it often selected a condition that was clinically related to the reference diagnosis or chose a diagnosis with an inappropriate degree of specificity.

Participating physicians, by comparison, showed a tendency to select diagnoses that were more distant from the reference diagnosis when they were incorrect.

This difference may reflect language models' ability to compare a broad range of medical possibilities.

The researchers found that language models demonstrated particular strengths in cases involving rare diseases or atypical presentations.

Physicians make decisions based on their areas of expertise and clinical experience.

But no individual physician can have firsthand experience with every rare disease or unusual presentation.

Large language models, having been trained on extensive medical literature spanning multiple specialties, may be capable of considering a broader range of conditions simultaneously.

Still, these results should not be interpreted as evidence that AI always makes clinically safer errors.

Even closely related diseases can require substantially different treatments and carry different prognoses.

Moreover, the study measured how closely a submitted diagnosis matched a reference diagnosis.

It did not evaluate the safety of treatments actually administered to patients.

An accurate diagnostic label and safe medical care are not the same thing.

The Conditions Hidden Behind Claims That AI Outperformed Doctors

At first glance, the study's figures make it appear that AI overwhelmingly surpassed human physicians.

A closer examination of the evaluation conditions, however, requires a more cautious interpretation.

First, the physicians did not examine real patients in person.

They performed diagnostic tasks within time limits using information provided through text.

For NEJM cases, physicians spent an average of 18.1 minutes when access to reference materials was permitted.

When reference searches were restricted, the average was 11.9 minutes.

These conditions differ considerably from a hospital setting in which physicians can observe patients, review extensive records, and consult specialists when necessary.

The NEJM cases also included complex conditions selected for educational significance.

They do not represent the distribution of diseases ordinarily encountered in outpatient clinics.

The evaluation did not include healthy patients or mild cases requiring little or no treatment.

A physician's responsibilities extend far beyond identifying the correct diagnosis.

Doctors must explain why tests are needed, evaluate their burdens and risks, and discuss treatment options with patients.

They must also consider patients' emotional states, social circumstances, and financial conditions.

These dimensions were not adequately captured by the benchmark.

The researchers themselves cautioned against generalizing the findings as proof that AI performs better than physicians in real clinical practice.

They also acknowledged that the performance gap might narrow when AI is compared with relevant specialists or multidisciplinary clinical teams.

The appropriate interpretation is therefore that, under specific controlled evaluation conditions, the AI diagnostic system achieved higher accuracy than the participating general-practice physicians.

It is not evidence that AI is ready to replace physicians across clinical practice.

Medical AI Competition Is Shifting from Models to Diagnostic Systems

Despite these limitations, the study offers important implications for the broader AI industry.

Applying a powerful language model to medicine does not automatically produce an effective diagnostic system.

In the research, DxO performed better than approaches that used existing language models through straightforward prompting.

Across the evaluated language models, the researchers reported that DxO's orchestration improved diagnostic accuracy by an average of 7.9 percentage points on NEJM cases and 4.4 percentage points on emergency department cases compared with more conventional model-use approaches.

This suggests that the process through which a model operates matters alongside the medical knowledge it possesses.

Rather than asking AI to immediately submit a diagnosis, the researchers organized a process of identifying possible conditions, selecting additional questions, and considering testing costs.

The system also compared competing hypotheses and incorporated mechanisms for reviewing diagnostic reasoning.

This architecture points toward medical AI evolving from a single model into a coordinated system involving multiple roles.

The principle resembles enterprise AI agents.

For AI to perform meaningful work inside an organization, answering questions is not enough.

It must gather necessary information, divide work into stages, consider costs and risks, and verify results.

Medical AI faces a similar challenge.

Building a highly accurate model is important.

But designing which information it should request and which actions it should be permitted to take may be equally consequential.

Can AI That Reduces Testing Costs Change Healthcare Economics?

Another important aspect of the study concerns healthcare costs.

Medical AI performance has traditionally been evaluated primarily through accuracy.

In clinical practice, however, costs can be nearly as important.

Diagnostic tests are essential for understanding a patient's condition, but not every test provides equal value.

Unnecessary testing increases expenses and may lead to additional examinations or procedures.

On the other hand, reducing diagnostic testing too aggressively may cause important illnesses to be missed.

For medical AI to create practical value, it must therefore consider both accuracy and costs.

DxO incorporated this trade-off directly into its diagnostic process.

When selecting tests, the system was designed to compare their expected diagnostic value with their estimated cost and determine whether additional testing was justified.

The findings showed that relatively high accuracy could be maintained under certain cost constraints.

This approach may be relevant to hospitals, insurers, and healthcare policymakers.

If AI can help clinicians ask more useful questions and select appropriate tests, it may improve efficiency by reducing unnecessary examinations.

But the costs used in the study were not actual hospital bills or amounts reimbursed by insurers.

They were estimates derived from standard pricing data from American healthcare institutions.

The benchmark also did not fully incorporate test turnaround times, pain or invasiveness associated with procedures, or whether tests would actually be available in a particular clinical setting.

The results should therefore not be interpreted as evidence that real hospital expenses would fall by the same proportions.

Actual cost savings would need to be validated in clinical practice.

What Is Microsoft Trying to Achieve in Medical AI?

The authors of the paper included Microsoft researchers.

Participants included Mustafa Suleyman of Microsoft AI, Eric Horvitz, and other researchers.

The work was supported by Microsoft without external research funding.

The authors were Microsoft employees and equity holders, and some were named on U.S. patent applications related to the sequential diagnosis evaluation framework.

These interests should be considered when evaluating the research.

At the same time, the study illustrates the direction Microsoft is pursuing in medical AI.

Competition across the AI industry is expanding beyond improving the performance of large language models toward applying those models to real industry workflows.

Healthcare is among the most complex of these environments.

It requires specialized knowledge, often involves incomplete data, and carries direct consequences for patient safety when decisions are incorrect.

Developing medical AI therefore requires more than model performance.

Information-gathering procedures, validation mechanisms, costs, and accountability structures must also be designed.

Microsoft's DxO focuses on structuring the diagnostic process and optimizing the relationship between accuracy and testing costs.

It suggests that medical AI may evolve beyond recommending possible diagnoses toward assisting with portions of the actual diagnostic workflow.

However, the study does not establish that Microsoft has validated a commercial product capable of making autonomous diagnoses in real clinical settings.

It presents a research system and benchmark findings.

Clinical trials involving patients and validation of real-world care outcomes remain necessary.

Will AI Replace Doctors or Strengthen Their Judgment?

The most important question raised by this study may not be whether AI can replace physicians.

It may instead be how AI can support the diagnostic process performed by physicians.

DxO's strengths included structured information gathering and repeated reassessment of diagnostic hypotheses.

Human physicians perform these tasks as well.

But real clinical environments impose constraints on time and workload.

Doctors must process substantial amounts of information while treating patients.

It can be difficult to continuously consider every diagnostic possibility or calculate the expected diagnostic benefit of every test relative to its cost.

AI may be able to support these processes.

It can suggest questions that might otherwise be overlooked, organize possible diagnoses, and explain why additional tests could be useful.

It may also identify alternatives that challenge an existing hypothesis, helping reduce premature diagnostic conclusions.

But final clinical judgments still require physicians.

Evaluating a patient's actual condition, explaining the risks of examinations and treatments, and making decisions appropriate to the individual's circumstances involve responsibilities that cannot be captured by diagnostic accuracy alone.

The researchers also proposed that AI should complement human clinical expertise and contextual judgment.

This aligns with a practical development path for medical AI.

Rather than independently performing every aspect of medical care, AI may help physicians make more accurate and efficient decisions.

Diagnosing Well Is Not the Same as Providing Good Medical Care

The study demonstrates progress in medical AI while also highlighting important boundaries.

AI selected questions, requested tests, and produced diagnoses with high accuracy based on initially limited information.

But actual medical care involves much more.

Patients may describe symptoms inaccurately.

Test results can be ambiguous.

Physicians monitor changes in patients' conditions over time and adjust treatment when necessary.

Even among patients with the same disease, appropriate tests and treatments may differ depending on age, underlying conditions, living environment, and financial circumstances.

The ability to predict a correct diagnosis does not establish the ability to perform all these responsibilities.

Patient safety, in particular, cannot be evaluated through a single accuracy metric.

Recognizing rare diseases matters.

But so do avoiding unnecessary tests, preventing harmful interventions, and communicating effectively with patients.

The study evaluated only part of this broader process: the collection and interpretation of information needed for diagnosis.

The 83.2% result is therefore evidence of a promising research capability, not an announcement that medical AI is ready to replace physicians.

The deeper value of the study lies in how it changes the evaluation of medical AI.

We should examine not only whether AI arrives at the right answer, but also which questions it asks, which tests it selects, how it considers costs, and when it decides that enough information has been gathered.

The Next Competition in Medical AI Is About Better Diagnostic Processes, Not Just Better Answers

The study published in Science illustrates a shift in the development of medical AI.

For years, AI attracted attention for achieving high scores on medical knowledge tests.

But the capability needed in clinical practice extends beyond producing correct answers.

It involves collecting information under uncertainty, comparing multiple possibilities, considering costs and risks, and deciding what to do next.

Microsoft organized these activities into a structured AI diagnostic system.

The result was stronger performance than approaches relying on more direct use of existing language models.

In comparisons with participating physicians, DxO also demonstrated higher diagnostic accuracy within the controlled evaluation environment and favorable combinations of accuracy and estimated testing costs under certain budget constraints.

But the most important question is not who won the benchmark.

It is which approach can actually improve the quality of medical care.

It is not enough for AI to remember more diseases and provide answers faster.

It must ask appropriate questions at appropriate times, select necessary tests, and revise earlier judgments when new evidence emerges.

Human physicians must also be able to verify those judgments and use them responsibly.

If these conditions can be met, medical AI could play a much larger role than it does today.

It could develop into a diagnostic support system that identifies overlooked questions, evaluates the possibility of rare diseases, and helps reduce unnecessary testing.

But reaching that point will require clinical validation involving actual patients and healthcare institutions, beyond controlled benchmarks.

The study demonstrates the potential for that future.

It does not provide the final answer.

Nevertheless, one important change is already visible.

Medical AI evaluation is moving beyond asking how much medical knowledge a model possesses toward examining how it uses that knowledge to conduct a diagnostic process.

Microsoft's central achievement was not teaching AI to identify a disease faster. It was developing a systematic way for AI to decide what to ask, which tests to request, and when to revise its judgment before reaching a diagnosis.

The future of medical AI may depend less on its ability to provide a single correct answer than on the accuracy and safety of the process through which that answer is reached.