From Documents to Spans: Span-Centric Learning for LLM-based Explainable Medical Coding

summary

Video file (mp4)

The gist

The gist The authors propose Span-Centric Learning (SCL), a training framework that strengthens Large Language Models’ coding ability at the span level and transfers this capability to full

In short

The authors propose Span-Centric Learning (SCL), a method to improve Large Language Models' ability to perform medical ICD coding from long clinical documents. SCL learns coding knowledge from short text segments (spans) and then applies this knowledge to full documents. This approach requires less training data and provides explicit supporting evidence for every predicted code, making the process more accurate and auditable.

Key concepts

ICD Coding
This is the process of assigning standardized diagnosis codes to clinical documents. It is vital for healthcare billing, tracking disease patterns, and analyzing patient health information. Accurate coding must be backed by clear textual evidence found within the medical record.
Span-Centric Learning (SCL)
SCL is a training framework that separates learning coding knowledge from applying it to entire documents. It first teaches the model how to recognize codes from short text segments (spans) and then uses this learned skill to code longer clinical documents, improving performance efficiently.
Code-Centric Data Expansion
This strategy creates a large dataset of span-level knowledge by generating different types of pairs: gold pairs from human knowledge bases, silver pairs derived from public data using LLMs, and synthetic pairs for codes not seen in the original documents. This enriches the model's understanding of specific medical codes.
Explicit Supporting Evidence
Instead of just predicting a code, SCL forces the model to generate or identify specific text spans within the document that justify its prediction. This provides clear, traceable evidence for each assigned code, which is essential for human review and clinical auditing.

Terminology used across episodes

This episode discusses

The paper

From Documents to Spans: Span-Centric Learning for LLM-based Explainable Medical Coding · Read on arXiv

Xu Zhang, Wenxin Ma, Chenxu Wu, Rongsheng Wang, Zhiyang He, Xiaodong Tao, *Kun Zhang*, *S. Kevin Zhou

School of Biomedical Engineering, USTC · MIRACLE Center, Suzhou Institute for Advance Research, USTC · Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology · State Key Laboratory of Precision and Intelligent Chemistry, USTC

Medical coding, which assigns standardized ICD codes to clinical documents, is a fundamental yet challenging task in modern healthcare. It requires extensive knowledge of a large code space and the ability to identify and interpret sparse supporting evidence in long, noisy clinical documents. Most existing methods rely on document-level supervision, which provides limited guidance on supporting evidence and insufficient training signals for rare codes. To address these limitations, we propose Span-Centric Learning (SCL), which separates coding knowledge acquisition from its application in full documents. SCL first learns coding knowledge from short spans, then learns to use this knowledge in full documents from a small set of annotated data. Experiments show that, our approach enables an 8B LLM to surpass state-of-the-art methods using only 248 complete documents for training. It also provides explicit supporting evidence for each predicted code, with strong alignment to human annotations. Further analysis shows that the model can learn knowledge of codes absent from documents through spans and apply that knowledge to full documents.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "From Documents to Spans".

Jane: The gist The authors propose Span-Centric Learning (SCL),

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper today, "From Documents to Spans: Span-Centric Learning for LLM-based Explainable Medical Coding," and the authors are Xu Zhang, Wenxin Ma, Chenxu Wu, Rongsheng Wang, Zhiyang He, Xiaodong Tao, Kun Zhang and S. Kevin Zhou. The title is pretty descriptive about what they're doing here >

Jane: It sounds a bit technical with "Span-Centric Learning," but at its core it’s about taking something that’s usually just labels on whole documents and breaking it down to look at smaller pieces, or spans, of text >

Lu: Yeah, the authors are addressing the problem that existing public datasets for ICD coding only give you the final code label without showing you which part of the document actually supported that code >

Meng: So if I'm hearing this right, they're trying to make sure these models don't just guess a code randomly based on the whole chart, but they need to point to specific words in the record >

Lalam: Exactly. The core idea is moving from document-level learning to span-level learning for coding accuracy and explainability >

The paper's summary: Tom: Okay, so the paper proposes a training framework called Span-Centric Learning, or SCL, and it uses a mixed training strategy. They use a small set of documents that *do* have annotated evidence spans to supervise the aggregation of evidence and code assignment over the whole document >

Jane: That means they’re not just feeding the model one big document at a time; they are teaching it how to look for those specific pieces of text, those spans, which is what makes it more grounded >

Lu: And they reinforce that span-level reasoning using a large collection of lightweight evidence spans, which helps the model learn robust associations at that smaller scale >

Meng: So the goal here is to get small LLMs to perform well without needing massive proprietary datasets or huge training costs, because they claim an eight point two-point improvement in macro-F1 with only twenty percent of standard SFT cost > <ref:2603.15270#pg3>

Lalam: That efficiency is a big deal because it means we can get better performance while keeping the model smaller and more interpretable for clinical use >

The paper's improvements: Tom: One major thing they point out is that this approach allows the model to learn previously unseen codes using only spans, which they call finding unseen codes from spans alone, which means it transfers that knowledge over to the full document coding task >

Jane: That’s interesting because usually, if a model hasn't seen a specific code before, it just fails on that code entirely. This method suggests the span-level supervision helps it generalize those codes >

Lu: They also found that adding span-level data actually improves evidence extraction beyond just the final code prediction; they saw recall go up when they added 150k evidence spans >

Meng: So it’s not just about getting the right diagnosis code; it helps the model find more complete and human-consistent supporting evidence, which is crucial for a production environment where we need to trust what the model is showing us >

Lalam: And they also show that this method results in explicit supporting evidence for every predicted code, which opens up a whole new workflow for auditing and revising decisions by humans >

Conclusion: Tom: So wrapping it up, the Span-Centric Learning framework is basically showing that you don't need dense document-level annotation to learn evidence-based coding if you focus on learning those code-specific patterns from spans >

Jane: It gives us a way to train smaller models effectively while still making sure they provide clear, traceable evidence for every decision they make, which is really helpful for clinical settings >

Lu: The span-centric perspective seems like a general strategy for training in clinical NLP tasks where document-level supervision is too costly but the span knowledge is more accessible >

Meng: From an engineering standpoint, it’s efficient because of the training cost, and it solves the deployment issue by making models smaller and more interpretable >

Lalam: The Span-Centric Learning for LLM-based Explainable Medical Coding paper really shows how focusing on local knowledge can transfer to global document tasks while keeping the interpretability needed for human review >

More episodes

← Home