Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights

arXiv:2610.00087 · cs.CL, cs.AI · Submitted 2026-09-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Legal text classification in Korean sexual offense cases".

Jane: Legal text classification in Korean sexual offense cases:

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, to recap where we are, we're looking at "Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights," and the main thesis is that classifying complex legal documents requires a careful comparison of different AI model types. The paper argues that traditional machine learning methods, medium language models, and state-of-the-art large language models all have their place, but the key finding is that fine-tuning smaller language models on legal data yields superior performance compared to those larger general-purpose models and even older traditional techniques.

Jane: That's a very clear claim; the authors are essentially saying that for this specific legal task, the quality of adaptation through fine-tuning is more impactful than just giving a model more raw computational power. The study claims this superiority across ten legal categories of sexual offense precedents.

Lu: It matters because it suggests that the complexity of legal language means that a model trained broadly on general text won't capture the specific nuances needed for these highly regulated areas unless it's specifically taught how to handle that domain first.

Meng: From a practical perspective, this means we don't need to immediately deploy the largest available models if we can achieve near-peak performance using a smaller, better-tuned model on the actual legal dataset. That impacts resource allocation and deployment strategy significantly.

Lalam: It’s important because it validates the idea that specialized AI is more effective than generalized AI when accuracy in sensitive areas like legal classification is at stake, which builds confidence in deploying targeted solutions.

Tom: And they emphasize that this isn't just about hitting a high number; they also focus on how to understand *why* the models get things wrong by using Explainable AI techniques to uncover systematic weaknesses in the classification process.

Jane: Exactly, Tom; it’s not just about getting the right answer once, but understanding the failure modes so we can actually fix those failures in our AI systems. They investigate whether errors come from ambiguous terminology or something else entirely.

Lu: The investigation into misclassification patterns using XAI is really where the creative potential lies; if we can map out exactly what linguistic features drive a wrong decision, we can build targeted interventions for model improvement.

Meng: I'm interested in how that analysis translates into actionable engineering steps; knowing which linguistic cues lead to errors helps us design better feature engineering or fine-tuning strategies later on.

Lalam: For the culture of using this technology, having transparent models that can explain their reasoning is essential for adoption in legal settings where accountability is paramount; it moves AI from a black box to a more trustworthy partner.

Tom: So, they’ve laid out the landscape: different model sizes perform differently, and understanding those differences requires looking at both performance metrics and the underlying reasons for misclassifications. This sets up a great discussion about what this means for real-world legal tech implementation.

Jane: It really highlights that in complex domains like legal text classification, simply scaling up the technology isn't always the best path; targeted refinement is often what delivers the actual high accuracy we need.

Conclusion: Tom: Wrapping up this discussion on "Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights," the authors, Jeongmin Lee and colleagues, conclude that the path forward involves prioritizing domain-specific pretraining and task-specific fine-tuning above model size for achieving high accuracy. They stress that while they found high performance levels, the primary limitation they identified is that current models often rely too much on explicit information instead of leveraging implicit contextual reasoning to make solid inferences about things like victim characteristics.

Jane: That reliance on explicit data versus implicit context is a critical point for the future; it means AI tools need to evolve beyond just reading words and start figuring out the subtle situational relationships between them. The implication is that we need more sophisticated ways for AI to grasp that unspoken understanding in legal texts.

Lu: This points directly toward the next wave of research, which should focus on integrating reinforcement learning techniques like Direct Preference Optimization and improving contextual embedding methods so these models can better capture those subtle nuances you mentioned.

Meng: From an engineering viewpoint, if we can successfully implement those contextual embedding refinements, it means our systems will become far more adaptable and less brittle when they encounter new or slightly varied legal phrasing in the future.

Lalam: For us, this means developing AI that can handle complex social and legal situations with a bit more intuition, which could make these tools much more effective for real-world application in sensitive areas.

Tom: So, to summarize the conclusion, this paper shows that success in legal text classification isn't just about having the biggest model; it’s about smart training strategies combined with explainable analysis to move AI toward a better understanding of legal context and implicit meaning.

Jane: And the broader implication is that for any high-stakes application involving complex documents, we need a robust framework that doesn't just promise accuracy but also demonstrates how the AI arrives at its decisions through interpretable reasoning.

Lu: It’s about building accountability into the AI itself, ensuring that when it makes a classification, we can trace back to the linguistic features and contextual understanding that led to that outcome.

Meng: So, the future work suggests a very practical goal: making these models inherently better at reasoning contextually rather than just being big containers for text.

Lalam: It sounds like we’re moving toward AI partners that are not only accurate but also deeply aware of the context, which is a huge step toward building truly reliable and accountable systems in legal tech.

Jeongmin Lee

University of Science and Technology (UST) · Electronics and Telecommunications Research Institute (ETRI)

cs.CL, cs.AI

Submitted: 2026-09-07

Updated: 2026-09-07

Comments: 22 pages, 3 figures. Published in Artificial Intelligence and Law

Journal ref: J. Lee, Artificial Intelligence and Law (2025)

DOI: 10.1007/s10506-025-09454-w

Code: https://github.com/KPFBERT/KPFbert

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 77/100

The gist: Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights investigates the evolution of AI models for classifying complex

Key concepts

Fine-tuning Small Language Models (Small LMs)
This involves taking a smaller language model, like KLUE-BERT, and training it specifically on legal data. The research showed this method achieved the highest accuracy, proving that tailoring the model to a specific legal domain is more effective than using massive general models.
Explainable AI (XAI)
XAI techniques were used to look inside how the AI made its decisions. By examining linguistic features, researchers found that models often rely on explicit details like victim's age rather than understanding subtle situational clues, highlighting a need for better contextual learning.
Domain Adaptation
This is the process of adapting a pre-trained model to perform well on a specific subject area, in this case, Korean sexual offense laws. The results confirmed that adapting the model through fine-tuning on legal precedents significantly improved its performance compared to using large, general-purpose models.
Contextual Misinterpretation
This refers to when the AI incorrectly understands the meaning of a word or phrase based on its surrounding context. The study found models struggled with location descriptions and semantic overlaps, indicating that they lack robust learning for understanding nuanced situational details.

Terminology

Summary

Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights investigates the evolution of AI models for classifying complex legal documents, demonstrating that fine-tuning small language models yields superior performance compared to large general-purpose models, while explainable AI (XAI) is crucial for understanding and ensuring transparency in these high-stakes applications.

Model Comparison and Performance

The study conducted a comprehensive evaluation of various legal text classification models across ten legal categories of sexual offense precedents, ranging from traditional machine learning techniques to state-of-the-art large language models (LLMs). The experimental results demonstrated that fine-tuning small-scale models (Small LMs) such as KLUE-BERT on legal data yields superior performance compared to large generalpurpose models such as GPT-3.5 and GPT-4.0, as well as traditional machine learning models. Specifically, KLUE-BERT achieved the highest accuracy of 99.3%, indicating that domain adaptation and fine-tuning play a more crucial role in legal document classification than model size alone. In contrast, medium and large language models (1B–100B parameters) showed performance ranging from 63.27 to 79.28% in accuracy, which was outperformed by the fine-tuned small models.

Data Construction and Preprocessing

The research leveraged two main datasets: the Crime Case Precedent Data and the KICS Data. The Crime Case Precedent Data was constructed from LBox, LegalSearch, and LJP criminal datasets, with a total of 4051 deduplicated cases. To improve class balance, we collected additional precedent data from LegalSearch to supplement underrepresented categories. Furthermore, the crime facts of precedents often contain a mixture of information on concurrent crimes, which was addressed by applying rules such that in cases where multiple charges are involved, the crime facts are usually written with sequences, and excluding cases not separated by charge. For the KICS data, since its direct use is restricted, the study utilized de-identified, synthetic crime descriptions resembling KICS data to preserve linguistic and structural characteristics while maintaining consistency.

Misclassification Analysis

To assess model performance beyond raw accuracy, the study analyzed misclassification patterns of the best-performing model, KLUE-BERT. The confusion matrix revealed three key challenges: contextual misinterpretation of location descriptions, ambiguity between similar offense categories, and limitations in processing legal terminology. A notable example was the failure to recognize that a public bus qualified as a crowded public place for an offense, suggesting that KLUE-BERT lacks robust contextual learning for location-based classifications. Additionally, the model struggled with semantic overlap, such as classifying Rape (label 0) as Indecent Act by Compulsion (label 1).

Explainable AI (XAI) Insights

The study employed XAI techniques to analyze model predictions and conduct an in-depth review of misclassification cases. By applying the transformers-interpret framework, which utilizes Layer Integrated Gradients (LIG), the researchers examined key linguistic features that drive the model’s decision-making process. For correct classifications, visualizations highlighted key legal indicators; for misclassified cases, word importance visualizations revealed that in one instance involving a minor offense, the case description did not explicitly mention the victim’s age, and the model relied on indirect linguistic cues like “mister,” “luring,” and “kid” to infer victim characteristics. This analysis underscores the limitation that models heavily rely on explicit information rather than leveraging subtle contextual cues to infer victim characteristics.

Conclusion and Future Directions

The findings suggest that AI-driven legal text classification models can serve as decision-support tools, emphasizing domain-specific pretraining and task-specific fine-tuning over model size. The study concludes that while high accuracy is achieved, the primary limitation lies in the model’s reliance on explicit information rather than implicit contextual reasoning. Future work is suggested to explore methods to enhance contextual reasoning by integrating reinforcement learning techniques like Direct Preference Optimization (DPO) and refining contextual embedding techniques to better capture relationships between legal concepts and situational nuances. The overall implication is that AI-assisted tools require human-AI collaboration and expert feedback loops to ensure they remain reliable, accountable, and aligned with real-world legal practices.

How it works

  1. The research systematically compares models ranging from traditional machine learning (SVM, Random Forest) to Small LMs (KLUE-BERT), Medium LMs (Llama-3.1-8B), Large LMs (polyglot-ko-12.8b), and Very Large LMs (GPT-4.0).

  2. Performance was measured using accuracy, precision, recall, and F1 score across the Crime Case Precedent Data for training and evaluation.

Improvements for AI systems

Based on the provided scientific paper, here are specific improvements that can be made to AI systems for legal text classification, and what those improved systems could do:

  1. Utilize a hybrid model architecture combining fine-tuned Small Language Models (SLMs) like KLUE-BERT with domain-specific knowledge graphs or semantic annotation layers.

  2. Implement an advanced Word Sense Disambiguation (WSD) module, specifically contrastive learning techniques, during the fine-tuning phase to explicitly emphasize the distinction between semantically similar legal terms (e.g., Quasi-Rape vs. Indecent Act by Compulsion).

  3. Integrate a Contextual Embedding layer trained on Korean legal corpora to capture nuanced contextual cues that are not captured by explicit keyword matching, specifically targeting implicit victim characteristics (like age inference from indirect linguistic signals).

  4. Develop an Explainable AI (XAI) pipeline using the proposed transformers-interpret framework (Layer Integrated Gradients) to generate visual attribution heatmaps for every classification decision on real-world data (like KICS data), allowing legal professionals to verify if the model is relying on legally sound indicators or spurious correlations.

  5. Employ a dynamic dataset balancing strategy during training that prioritizes the accurate representation of rare but critical offense categories (e.g., specific minor offenses) to mitigate class imbalance bias, ensuring high performance across all legal categories rather than just the most frequent ones.

  6. Integrate Reinforcement Learning techniques, such as Direct Preference Optimization (DPO), to fine-tune models based on expert preference data, enabling the system to learn subtle legal distinctions and improve its ability to differentiate between closely related offense categories with high precision.

These improved AI systems can:

  1. Perform highly accurate classification of legal texts in Korean sexual offense cases with state-of-the-art performance (aiming for >99% accuracy).

  2. Provide transparent, legally justifiable decisions by generating visual explanations (heatmaps) showing exactly which words or phrases drove a specific classification, thereby enhancing user trust and accountability.

  3. Accurately infer implicit contextual information in ambiguous legal texts, such as determining if a victim is a minor based on subtle linguistic cues, overcoming the limitation of models relying only on explicit keywords.

  4. Robustly handle the variability and unstructured nature of real-world legal records (KICS data) by generalizing beyond structured precedent data.

  5. Provide actionable insights for legal professionals by flagging potential misclassifications rooted in contextual misunderstanding or semantic overlap, guiding expert review loops for continuous model refinement.

Abstract

The advancement of natural language processing (NLP) has expanded AI-based text classification in the legal domain. However, accurately classifying legal documents remains challenging due to the complexity of legal texts and subtle differences between legal categories. This study evaluates legal text classification models ranging from traditional machine learning techniques to large language models (LLMs) using ten categories of Korean sexual offense precedents. The results show that fine-tuning small-scale models such as KLUE-BERT on legal data outperforms general-purpose models such as GPT-3.5 and GPT-4.0, as well as traditional machine learning models. KLUE-BERT achieved the highest accuracy of 99.3%, indicating that domain adaptation and fine-tuning can be more important than model size for legal document classification. We further employ explainable AI (XAI) techniques to analyze model predictions and misclassification cases. XAI analysis identifies linguistic features influencing model decisions and limitations in capturing subtle textual cues. Using KICS data, which closely resembles real-world legal case records, we further evaluate the model's generalization capabilities and find that it struggles to interpret implicit contextual cues. These findings highlight the importance of both performance and interpretability in legal AI and demonstrate how XAI can improve transparency in legal text classification. AI-assisted tools can support legal professionals in tasks including document classification, legal information retrieval, and case assessment.

Sources

Related papers