Text Chunking for Document Classification for Urban System Management using Large Language Models

arXiv:2504.00274 · cs.CL, cs.HC · Submitted 2025-03-31 · Read on arXiv

cs.CL, cs.HC

Submitted: 2025-03-31

Updated: 2025-03-31

Comments: 16 pages, 6 figures, 4 tables, 2 algorithms; Replication data and code can be found https://github.com/josh-rodriguez-csu/ChunkingforLLMs

DOI: 10.1049/smc2.70023

Code: https://github.com/josh-rodriguez-csu/ChunkingforLLMs

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Digital Twin — 21× in this paper · explained in 5 episodes
• Accuracy — 17× in this paper · explained in 1 episode
• Consistency — 8× in this paper · explained in 3 episodes
• Aggregation — 4× in this paper · explained in 1 episode
• Artificial Intelligence — 4× in this paper
• Benchmark — 3× in this paper · explained in 2 episodes
• Fidelity — 3× in this paper · explained in 2 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Explainability — 2× in this paper · explained in 3 episodes
• Hallucination — 2× in this paper · explained in 3 episodes
• Prompt Engineering — 2× in this paper · explained in 4 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Predictive Maintenance — 1× in this paper · explained in 2 episodes
• Resource Constraints — 1× in this paper · explained in 3 episodes
• Structured Reasoning — 1× in this paper · explained in 3 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding
• LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding
• On The Planning Abilities of OpenAI's o1 Models: Feasibility, Optimality, and Generalizability

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering