Can We Improve Educational Diagram Generation with In-Context Examples? Not if a Hallucination Spoils the Bunch
Evanfiya Logacheva, Arto Hellas, Tsvetomila Mihaylova, Juha Sorva, Ava Heinonen, Juho Leinonen
cs.CL
Submitted: 2026-08-21
Updated: 2026-08-24
License: http://creativecommons.org/licenses/by/4.0/
The gist: GenAI is widespread in educational applications; however, it is known to generate content with intrinsic and extrinsic hallucination.
Terminology
Abstract
GenAI is widespread in educational applications; however, it is known to generate content with intrinsic and extrinsic hallucination. We introduce a novel method for ICL diagram generation based on Rhetorical Structure Theory, which improves diagram faithfulness to its source text context. We find that ICL performance depends on task distribution and models' reasoning ability, with higher reasoning allowing better quality and performance for an out-of-distribution task. We perform an expert evaluation of 150 generated diagrams and analyze our findings using Bayesian GLMMs. Additionally, we use our evaluation rubric and samples from the data set for automated diagram evaluation, achieving statistically significant agreement with human evaluation.
Sources
- Why Language Models Hallucinate
- Large Language Models: A Survey
- Competitive Programming with Large Reasoning Models
- Epistemological Fault Lines Between Human and Artificial Intelligence
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering