MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM

arXiv:2509.20067 · cs.AI · Submitted 2026-08-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM".

Jane: The paper was written by the authors from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Summary: Tom: So, we’ve introduced the concept of MACD; now let’s look at what the paper actually *does* in this framework. The core idea is this multi-agent pipeline, which is really designed to mimic that professional growth we just talked about.

Jane: Think of it like a team of specialized agents—a Summarizer, a Refiner, and a Diagnostician—working together on four thousand three hundred ninety real patient cases from seven different diseases. They aren't just running one prompt; they are learning over time.

Lu: The process is quite sophisticated because the knowledge isn't static; it’s distilled through that Summarizer agent into an evolving set of concepts that the Refiner agent then cleans up, which makes sense for building a robust mental model.

Meng: I see this as highly scalable. The Summarizer and Refiner agents are taking raw data and turning it into structured, actionable knowledge, which is exactly how you want to manage massive datasets in a practical system.

Lalam: The purpose of the Diagnostician agent is to leverage that curated Self-Learned Knowledge when they're running new cases, making sure the AI isn't guessing based on generalized internet training alone.

Improvements: Tom: We’ve seen how the system works; but how much better is it performing compared to existing methods? The data shows some really impressive results.

Jane: The authors are claiming that their Self-Learned Knowledge outperforms established professional knowledge and even the Mayo Clinic knowledge, giving us a massive boost in accuracy—up to twenty-two point four percent better than using human-curated resources.

Lu: I find that interesting because it suggests the *way* the we distill the information is more compatible with how an LLM processes it than simply feeding it a pre-written guideline structure.

Meng: And when you look at the open-source models, they’re beating GPT-five by up to fifteen point eight percent, which is huge for me because these are models that can run on local hardware, making this performance achievable.

Lalam: I believe that this isn't just about being smarter; it' about the idea showing a consistent, predictable scaling effect across different LLMs, which suggests a much more universal and trustworthy form of medical logic.

Improvements: Tom: The paper also suggests that the MACD framework provides some great improvements in how we use this technology, not just in accuracy.

Jane: It’s not only about the core agents working together; they are also presenting a collaborative workflow where multiple diagnostician agents consult and debate before the human steps in if they can't agree.

Lu: This collaboration is crucial because it mimics how doctors actually talk through a difficult case, adding a layer of iterative refinement that goes far beyond what single-inference models achieve.

Meng: From an implementation standpoint, having this "MacD-human" workflow gives us a clear path for human oversight—the AI acts as decision support but not as the final authority, which is vital for clinical adoption.

Lalam: I think the implication here is that we are creating a truly symbiotic relationship between AI and the doctor, making sure that even when humans aren't available, we have a high-confidence diagnostic baseline to assist with.

Conclusion: Tom: We've covered so much ground today, from how these agents learn to the actual performance gains in "MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM."

Jane: I think the main thing we should remember is that we' are not just feeding prompts; we're building a system where the knowledge itself is adapting to evolving clinical reality.

Lu: This whole concept of "Self-Preference" in the learned knowledge suggests that AI is finding its own optimal way to understand medical concepts, which I think opens up some incredible new avenues for research.

Meng: The fact that this works even with smaller, local models makes it a very practical solution for deploying AI in underserved areas where we can’t rely on massive cloud resources.

Lalam: I'm excited about the idea of creating a "digital specialist" that serves as a reliable auxiliary tool, providing high-quality diagnostic support regardless of geographic location.

Tom: Well, that sounds like the perfect way to wrap up this discussion. It’s clear that "MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM" offers a very practical and powerful path forward for AI in medicine.

Jane: I agree, it moves us closer to a reliable, interpretable system that is truly designed to help us all achieve better healthcare outcomes.

Lu: It's an elegant solution that has successfully simulated the core process of clinical knowledge accumulation.

Meng: And from an engineering standpoint, we've found a way to make expert-level diagnostics accessible and deployable on-premise without the massive overhead of fine-tuning everything.

Lalam: We have hope for a more equitable global healthcare ecosystem thanks to this kind of technology.

cs.AI

Submitted: 2026-08-24

Updated: 2026-08-25

Code: https://github.com/qjdzj/MACD

Importance score: 86/100

The gist: The system utilizes a Multi-Agent Clinical Diagnosis (MACD) framework incorporating self-learned knowledge for Large Language Models (LLM).

Key concepts

MACD Multi-Agent Pipeline
This framework uses a team of specialized agents—a Summarizer, a Refiner, and a Diagnostician—to work together. This process mimics professional growth by allowing the AI system to learn from raw data and build an evolving understanding of complex clinical cases.
Self-Learned Knowledge
Instead of relying on static guidelines, this knowledge is distilled through the Summarizer and Refiner agents. It represents a curated set of concepts that adapts as the system processes data, creating a robust mental model for diagnosis.
MacD-human Workflow
This collaborative approach involves multiple diagnostician agents debating a difficult case. This iterative refinement provides high-confidence diagnostic support, ensuring that the AI acts as decision support and not as the final authority.

Terminology

Summary

The system utilizes a Multi-Agent Clinical Diagnosis (MACD) framework incorporating self-learned knowledge for Large Language Models (LLM). The architecture involves several specialized AI agents designed for comprehensive diagnostic support:

1. Diagnostic Agent Prompt Template:

This agent is designed to directly diagnose patients based on provided information. Its goal is to correctly diagnose the patient by providing the diagnostic criteria of the most confident pathology. It must provide a single diagnosis along with the most relevant diagnostic criteria, without writing any further information.

2. Knowledge Summarizer Agent Prompt Template:

This agent reviews a given report and summarizes the diagnostic evidence for an identified disease into two structured categories:

  • General Criteria: The 5 most relevant common clinical manifestations and diagnostic findings typically associated with the disease.

  • Rare Criteria: The 5 most relevant specific or unique clinical manifestations and diagnostic findings observed in a subset of patients with the disease.

The output must strictly adhere to a defined format, ensuring each category contains exactly 5 summarized criteria, focusing solely on diagnostic evidence directly extracted from the input report.

3. MACD-Human Collaboration Workflow Prompt Template:

This workflow positions the AI as a senior medical assistant whose primary duty is to assist a doctor by verifying previous diagnostic opinions and forming an independent diagnosis based on the patient’s complete situation. The process requires an impartial evaluation, acknowledging that prior opinions may be inconsistent, and providing the diagnostic criteria of the most confident pathology.

4. Judge Agent Prompt Template:

This agent functions as an expert Clinical Data Structurer and Senior Diagnostician tasked with constructing a Clinical Difference Table between two specific diseases for the Multi-Agent Diagnostic System. The analysis must strictly align with four dimensions: PATIENT HISTORY, PHYSICAL EXAMINATION, LABORATORY RESULTS, and IMAGING RESULT. The content must focus on high-value discriminatory features, prioritizing decisive differences that would appear in a real clinical report, while excluding non-specific, low-value generalities.

5. Comparative Analysis of Semantic Similarity Differences:

This process analyzes the clinical differences between multiple diagnostic candidates for the same patient case. It requires generating a detailed analysis that:

  • Identifies key features that best distinguish these conditions based on standard medical knowledge.

  • Strictly categorizes the differences into four dimensions: PATIENT HISTORY, PHYSICAL EXAMINATION, LABORATORY RESULTS, and IMAGING RESULT.

  • Must remain objective and not favor one diagnosis over the other.

  • The analysis must provide a structured differentiation analysis, with a count limit of a maximum of 6 differentiation points distributed across the four dimensions.

Diagnostic Definitions:

The system also maintains structured definitions for various conditions, including:

  • Pulmonary embolism: Defined by its location (pulmonary) and associated modifiers (embolus, thrombus).

  • Pericarditis: Defined by its location (pericardial) and associated modifiers (inflammation, effusion).

Improvements for AI systems

As a fastidious and diligent researcher, I have analyzed the MACD framework. While it represents a significant paradigm shift toward self-learning clinical AI, its current implementation faces inherent limitations in scalability and real-world generalization.

The following improvements are designed to mitigate these weaknesses, transforming the current framework into a robust, clinically deployable system that can handle complexity far beyond the scope of the original study's findings.

Improvement: Integrate a parallel pipeline for image-based knowledge extraction alongside text processing. The existing SLK generation process must be extended to incorporate visual features from radiology reports, such as specific patterns of consolidation, or quantifiable measurements of organ size/density.

Mechanism: Implement a specialized vision encoder (e.g., Vision Transformer) trained on medical images (CT/MRI). This encoder extracts salient features from the image and then feeds these visual embeddings into the Knowledge Summarizer Agent (M sum) for SLK generation.

What the Improved System Can Do: The system will no longer be limited to textual descriptions of findings. It can autonomously learn and incorporate visual heuristics (e.g, patchy ill-defined opacity in the right lung base) directly into its knowledge base, leading to superior diagnostic accuracy for diseases where imaging is primary evidence (e.g., Pneumonia, Pulmonary Embolism).

Improvement: Replace the iterative LLM-based refinement process with a structured, graph-based knowledge distillation mechanism. The current reliance on the Refiner Agent (M refine) is computationally expensive and prone to LLM hallucination in complex reasoning.

Mechanism: After the Summarizer Agent (M sum) extracts initial concepts, convert the raw output into a Knowledge Graph (KG). Nodes represent clinical findings (e.g., Elevated D-dimer), and edges represent causal or correlational relationships derived from the training cases. The Refiner Agent then becomes an optimization algorithm that removes redundant nodes/edges and identifies critical paths based on Acc scores, generating a structured, compressed knowledge representation (a subgraph) instead of a long text prompt.

What the Improved System Can Do: This drastically reduces computational overhead and token consumption during inference. The system achieves superior semantic density—the knowledge is not just listed, it is structurally related—allowing the Diagnostician Agent to reason over complex relationships instantly, accelerating diagnosis time by orders of magnitude while maintaining high accuracy.

Improvement: Refine the Judge Agent's decision-making process beyond simple semantic similarity and difference reporting. The current system relies on a neutral consensus mechanism.

Mechanism: Implement a Clinical Evidence Weighting function within the Judge Agent. This function assigns intrinsic weight to different types of evidence (e.g., Lab results Physical Exam History). When the three diagnostician agents disagree, the Judge Agent uses this weighting system to prioritize which agent's reasoning is more robust against external clinical standards (Professional Knowledge) and then generates a Differential Consensus Report based on weighted probability, not just semantic overlap.

What the Improved System Can Do: The system will provide a statistically grounded confidence score for its diagnosis. Instead of merely stating that consensus was reached, it can state: Diagnosis X is preferred with 85% certainty due to the confluence of high-weight laboratory findings (D-dimer) and consistent imaging features. This allows the AI to function as a reliable triage tool, not just a collaborative guess.

Improvement: Address the limitation that SLK is static once learned from the initial MIMIC-MACD dataset. Medical knowledge evolves (e.g., new diagnostic criteria, updated treatment protocols).

Mechanism: Implement a federated learning loop where the system periodically receives anonymized updates from clinical partners (or reads updated guidelines). The Knowledge Refiner Agent is adapted to ingest these new high-quality external data points, allowing the SLK to evolve into a Dynamic Self-Learned Knowledge. This requires a mechanism to identify and prune outdated or superseded concepts within the existing knowledge graph.

What the Improved System Can Do: The system remains clinically current. It prevents knowledge decay, ensuring that its recommendations align with the latest medical consensus, preventing its performance from degrading over time—a critical requirement for deployment in high-stakes clinical environments.

Sources

Related papers