Understanding the Limits of Agentic ICD Coding

arXiv:2609.13806 · cs.CL, cs.AI, cs.LG · Submitted 2026-09-12 · Read on arXiv

cs.CL, cs.AI, cs.LG

Submitted: 2026-09-12

Updated: 2026-09-12

Comments: Accepted to EMNLP 2026 Main Conference

License: http://creativecommons.org/licenses/by/4.0/

The gist: ICD-10-CM codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting.

Terminology

Abstract

ICD-10-CM codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting. Standard ICD-10-CM benchmarks report aggregate metrics that obscure performance on complex coding scenarios. We evaluate neural, workflow, and agentic systems on a rarity-stratified set of MIMIC-IV discharge summaries and identify two orthogonal failure modes. Neural classifiers exhibit a 0.43 micro-F1 gap between rare and common codes. Workflow systems handle rare codes well but score near zero on injury and external cause codes that require multi-step guideline following. A tool-augmented agentic configuration with structured access to official ICD-10-CM reference materials recovers up to 0.34 micro-F1 on this subset. No single system dominates across all conditions.

Related papers