Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

arXiv:2605.11960 · cs.CV · Submitted 2026-05-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters".

Jane: Chronicles-OCR introduces a comprehensive benchmark designed to evaluate the cross-temporal visual perception capabilities of Vision Large Language Models (VLLMs) across the entire evolutionary trajectory of Chinese characters,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: Hey everyone, I’m really pumped about this paper titled "Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters." Basically, it tackles how Vision Large Language Models handle a huge range of Chinese script evolution, from super old Oracle Bone Script all the way to Cursive Script. It claims that existing datasets only look at isolated periods and miss how characters change visually over thousands of years, which exposes some real bottlenecks in current VLLMs when they see things that are completely unconstrained or wildly different.

Jane: That sounds really interesting, Tom. So what's the main thrust here? What is this paper actually trying to prove about these models?

Lu: The paper sets out to evaluate the cross-temporal visual perception of Chinese characters across their entire lifecycle, which they map out from Oracle Bone Script in one thousand three hundred BCE up through Cursive Script. It claims that current methods fail because they don't account for the systematic shifts in visual distribution that happen over millennia when you look at such drastically varying morphologies.

Meng: So, it’s not just about seeing modern text well; it’s about handling historical chaos, right? That makes sense from a practical standpoint—if a model can't read ancient inscriptions, that limits its real-world utility in digital humanities or even historical preservation projects.

Lalam: I think what this paper is really highlighting is the severe perceptual gap when models are faced with unconstrained layouts and drastically varying character forms, which they call perceptual bottlenecks.

Tom: Exactly, Lalam! And they’ve designed a way to test this using two thousand eight hundred strictly balanced images spanning various media like tortoise shells and paper calligraphy to show this evolution. They even developed a novel Stage-Adaptive Annotation Paradigm for different historical stages.

Jane: The annotation method sounds complex, but I want to make sure I get it simple for our listeners. How does that adaptation work? Does it change how they label the characters depending on whether they are looking at an ancient script or a more modern one?

Lu: For archaic scripts like Oracle Bone, the researchers use fine-grained character-level annotations, which means bounding boxes for single characters and also provide modern character mappings to bridge the gap between what's there and what we know today. In contrast, for mature pre-modern scripts like Clerical or Running script, they switch to line- and paragraph-level transcriptions following the original reading order.

Meng: That distinction between character level versus paragraph level sounds important for setting up the evaluation tasks. So they aren't just asking the model to read a whole page at once, but testing different levels of understanding?

Tom: That’s right, Meng! They set up four rigorous quantitative tasks to isolate pure visual perception from any semantic reasoning, which is key for this study. They have Cross-period Character Spotting which is an end-to-end task requiring output of bounding box coordinates and modern character mappings, measured by the H-mean metric.

Paper summary: Jane: And then they have this fine-grained archaic character recognition, which seems designed specifically to isolate the pure morphological mapping accuracy of those pictographic glyphs. This is measured by Exact Match Accuracy, using a visual referring mechanism where the model has to recognize a specific highlighted archaic character and generate its modern counterpart.

Lu: They also have Ancient Text Parsing which tests comprehension of historical spatial layouts and reading sequences using a paragraph-level Normalized Edit Distance score, which strictly penalizes any sequence-ordering mismatches between the transcriptions. On top of that, they include Script Classification to probe the model’s macro-level understanding of morphological evolution across all seven scripts, measured by overall Accuracy.

Tom: Those results are pretty stark, and they show exactly where the current models struggle when faced with this kind of temporal variation. The findings on perceptual bottlenecks are really telling about what we need to work on next.

Jane: I agree, Tom; the paper points out a severe, twofold bottleneck in fine-grained grounding and morphological decipherment when models encounter archaic texts. They show that leading commercial models like GPT-five and Gemini two point five Pro register spotting H-mean scores near zero, which indicates a failure in end-to-end tasks due to a lack of robust grounding mechanisms for those unconstrained symbols.

Meng: That near zero score is pretty sobering for practical applications; it means if we deploy these models on anything historical, the initial visual recognition stage would likely fail catastrophically. How does that relate to the idea of semantic gap mentioned earlier?

Lalam: It confirms a massive, independent semantic gap beyond just spatial layout confusion when it comes to archaic scripts. Even when we give the model explicit visual guidance through bounding boxes, they consistently fail to map those pictographic glyphs to their modern counterparts.

Tom: And it gets worse when you look at the parsing results, because performance drops substantially as you move from mature scripts down to the archaic ones. For instance, Kimi K2 point 5 achieved a Normalized Edit Distance of zero point seven eight on Regular Script but plummets to merely zero point zero five on Oracle Bone Script because of both morphological deviation and the fundamentally different layout structures.

Jane: That drop in parsing accuracy really shows how sensitive these models are to structural consistency, which is a key finding from the Chronicles-OCR benchmark. It highlights that mature scripts stick to standardized conventions while ancient inscriptions have highly unconstrained, non-linear reading sequences.

Lu: But there’s this interesting contradiction in the Script Classification results that really needs attention. Models actually achieve exceptionally high classification accuracy on Archaic Scripts, like Seed2 point 0 Pro at ninety-six point six percent, by exploiting macro-level textural priors.

Meng: That contrast is what I find most telling from an engineering standpoint; it suggests a fundamental decoupling between stylistic recognition and fine-grained perception when we look at these models. They seem to rely heavily on macroscopic shape recognition instead of perceiving the delicate stroke dynamics needed to distinguish mature script categories.

Paper summary: Lalam: That decoupling really suggests that for culture, this means we might be relying too much on broad stylistic recognition while missing the subtle visual cues that define true character evolution.

Tom: So, what does this mean for digital humanities and how can we actually use these findings to improve things? This paper lays out some very clear optimization trajectories for where we need to focus our efforts.

Jane: The implication is that simply scaling up the model parameters doesn't fix everything; reasoning-enhanced variants often degrade because they introduce redundant or erroneous processes. We need targeted improvements in how these models ground themselves visually before they try to reason about the meaning.

Lu: From a creative perspective, this opens up possibilities for entirely new ways of modeling historical visual data that aren't just relying on standard modern text training sets. We could develop specialized pre-training regimes focused solely on morphological evolution trajectories instead of general text comprehension.

Meng: For me, from an engineering side, the focus needs to be on building more robust visual referring mechanisms that can handle the extreme noise and lack of standardization found in archaic physical media. We need models that are less susceptible to failure when input data doesn't conform to clean digital standards.

Lalam: I think the impact here is huge for how we preserve and interpret cultural heritage; if these AI systems can better handle the visual chaos of ancient scripts, they can unlock much deeper levels of understanding about the historical progression of Chinese culture.

Tom: So, to wrap up this discussion on Chronicles-OCR: it’s a detailed evaluation that shows current VLLMs have a significant struggle with the visual complexity of historical script evolution, especially when we look at unconstrained forms. The paper highlights specific failures in end-to-end spotting and fine-grained recognition when dealing with archaic symbols.

Jane: And the authors clearly lay out a path forward, indicating that scaling up isn't the only answer, but rather developing better mechanisms for grounding and handling morphological shifts is what’s needed. It’s a call for more nuanced visual understanding, not just bigger models.

Lu: I'm excited to see how researchers take this Stage-Adaptive Annotation Paradigm and apply it to other complex visual domains, because the way they adapted their labeling for Oracle Bone versus Cursive Script is a really creative way to isolate those issues.

Meng: From my side, I’m looking at how we can integrate these findings into new training loops that specifically penalize reliance on only macro-level textural priors when dealing with fine visual detail. That seems like a practical way to fix the classification inversion we saw.

Paper summary: Lalam: I think the most important part for us is how this advances our ability to engage with culture; if AI can accurately map those ancient symbols, it means we can access and understand historical narratives in ways that were previously locked away by visual barriers.

Tom: That’s a fantastic perspective, Lalam. So, the summary of Chronicles-OCR is that it provides the first comprehensive benchmark covering the full lifecycle of Chinese script evolution, showing exactly where current VLLMs hit a wall when dealing with unconstrained layouts and massive morphological shifts.

Jane: Precisely, Tom; it’s about measuring how these models perceive the systematic visual distribution shifts across thousands of years, which is crucial because existing datasets only look at isolated periods. This paper establishes a rigorous way to test those perceptual capabilities across the entire trajectory of Chinese characters.

Lu: It really pushes the boundary on what we expect from VLLMs in terms of handling historical, non-standardized visual information, which is a very rich area for future AI development. We can think about how this framework could be applied beyond Chinese script to other complex visual systems.

Meng: I just want to confirm the practical impact: if we adopt these benchmarks, we get a much clearer roadmap for improving model performance on historically sensitive tasks rather than just hoping they get better generally. That specificity is what matters for real-world deployment.

Lalam: I feel this work has implications for how we approach digital humanities projects; it suggests we need to build systems that are not only powerful but also deeply sensitive to the visual nuances of historical artifacts. This is a big step for cultural interpretation.

Tom: So, in conclusion, Chronicles-OCR gives us the first comprehensive evaluation benchmark covering the full lifecycle of Chinese script evolution from Oracle Bone Script to Cursive Script. It powerfully demonstrates that current VLLMs have severe perceptual bottlenecks when faced with unconstrained symbols and drastically varying morphologies.

Jane: And it shows the authors are pointing toward specific optimization trajectories, suggesting that reasoning-enhanced variants often degrade due to redundant processes. The paper’s main contribution is providing this detailed view of the perceptual challenges across the whole timeline.

Lu: I think we should be looking closely at how they handled the different annotation paradigms because that’s where the core insight into script-specific perception lies. Their approach to adapting annotations based on whether it’s an archaic or mature script is really clever.

Meng: I agree, the way they isolate the visual tasks—like pure detection versus end-to-end spotting—is important because it tells us exactly which part of the visual pipeline is failing most severely for these models.

Lalam: Ultimately, this research gives us a better lens through which to view the capabilities and limitations of AI when interacting with the deep history embedded in visual text. It shows where we need to focus our development efforts for cultural understanding.

Conclusion: Tom: So, we've just gone through all those technical details about how this benchmark works, and now it’s time to wrap up what all this means for us today on Chronicles-OCR.

Jane: Yeah, it feels like we’ve covered a lot of ground explaining the methodology behind testing AI's visual perception across thousands of years of Chinese characters.

Lu: From my side, I think the real takeaway is how this paper sets a new standard for evaluating models when dealing with historical data that has no standardized layout.

Meng: I just want to make sure we nail the core message for our listeners, so can you give us a simple explanation of what Chronicles-OCR actually is in plain English?

Lalam: Essentially, this paper introduces the first big test designed to see if AI models can truly "see" the whole life story of Chinese writing, from ancient symbols to modern script.

Tom: Exactly. The title itself, "Chronicles-OCR," hints at this deep dive into history through Optical Character Recognition capabilities.

Jane: It’s a comprehensive benchmark that covers the entire evolution of Chinese characters across seven distinct script stages in a single evaluation framework.

Lu: What’s striking is how they structured the data to force models to deal with both unconstrained archaic symbols and highly standardized modern layouts simultaneously.

Meng: I think it really shows us where the current limitations of AI are when it comes to handling extreme morphological variation and layout shifts over such vast timescales.

Lalam: Because this evaluation proves that current models struggle significantly when they have to map ancient, unstandardized symbols to their modern equivalents without clear visual cues.

Tom: That's the crux of it—showing that a model can be good at one script period but completely fails when presented with another due to visual chaos.

Jane: And the authors are pointing toward a clear path forward for improving these models by focusing on better visual grounding mechanisms rather than just adding more parameters.

Lu: I think this opens up really interesting avenues for future AI research, especially in developing specialized pre-training regimes focused specifically on morphological evolution trajectories rather than just general text comprehension.

Meng: That’s a good point about specialization; it suggests we should stop looking at one-size-fits-all solutions and start thinking about tailored visual understanding pipelines.

Lalam: And for culture, this means we can start building AI systems that aren't just reading text, but truly interpreting the deep historical context embedded in those ancient visual forms.

Tom: So, Chronicles-OCR isn't just another dataset; it’s a critical diagnostic tool showing us precisely what our vision models still lack when faced with the visual complexity of history.

Jane: It really frames the challenge for AI development as one of perception and structural understanding across different historical eras.

Lu: That framework is incredibly useful because it gives us a concrete way to measure those perceptual bottlenecks we've been talking about earlier in the show.

Meng: I think this paper will be really important for anyone building applications that need to interface with cultural heritage, as it shows us exactly where the visual recognition fails.

Lalam: I’m really hopeful that by using this benchmark, we can unlock a much deeper level of understanding regarding the historical progression of Chinese culture through AI technology.

Gengluo Li, Shangpin Peng, Xingyu Wan, Chengquan Zhang, Hao Feng, Xin Xu

Institute of Information Engineering, Chinese Academy of Sciences

cs.CV

Submitted: 2026-05-12

Updated: 2026-09-28

Code: https://github.com/VirtualLUOUCAS/Chronicles-OCR

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 82/100

The gist: Chronicles-OCR introduces a comprehensive benchmark designed to evaluate the cross-temporal visual perception capabilities of Vision Large Language Models (VLLMs) across the entire evolutionary

Key concepts

Stage-Adaptive Annotation Paradigm
This is a new method for labeling images that changes based on the script's age. For ancient scripts, it uses fine-grained character boxes and modern mappings. For mature scripts, it uses line and paragraph transcriptions following the original reading order to match how those texts are actually read.
Cross-period Character Spotting
This is an end-to-end task where the VLLM must simultaneously locate every archaic symbol in an image and provide its corresponding modern character mapping. It measures success using the H-mean metric, testing the model's ability to handle unconstrained symbols from different historical periods.
Fine-grained Archaic Character Recognition
This task isolates pure morphological accuracy by asking the model to recognize a specific archaic character highlighted in an image and map it to its modern equivalent. It uses a visual referring mechanism and is measured by Exact Match Accuracy to see if models can bridge the semantic gap.
Script Classification
This task tests the model's ability to categorize an image into one of the seven Chinese script types. While models are very good at this for ancient scripts by using broad textural patterns, they perform poorly on mature scripts, showing a split between macro and fine-grained perception.

Terminology

Summary

Chronicles-OCR introduces a comprehensive benchmark designed to evaluate the cross-temporal visual perception capabilities of Vision Large Language Models (VLLMs) across the entire evolutionary trajectory of Chinese characters, spanning from Oracle Bone Script to Cursive Script. This benchmark is crucial because existing datasets focus on isolated historical periods, failing to capture the systematic visual distribution shifts that occur over thousands of years, thereby exposing critical perceptual bottlenecks in current VLLMs when confronted with unconstrained layouts and drastically varying morphologies.

The gist

Chronicles-OCR provides the first comprehensive evaluation benchmark covering the full lifecycle of Chinese script evolution, achieving the first full-timespan coverage from unstandardized archaic symbols to mature pre-modern scripts.

Dataset and Annotation Paradigm

The dataset comprises 2,800 strictly balanced images encompassing highly diverse physical media, ranging from tortoise shells to paper-based calligraphy. To accommodate the drastic morphological and topological variations across different historical stages, the authors propose a novel Stage-Adaptive Annotation Paradigm. This paradigm is tailored to the distinct evolutionary characteristics of these scripts:

  1. For archaic scripts (Oracle Bone, Bronze, Seal), annotations consist of fine-grained character-level annotations including single-character bounding boxes to localize unconstrained symbols, alongside modern character mappings to bridge the profound semantic gap. Undeciphered characters are annotated with a special [UNK] token.

  2. Conversely, for mature pre-modern scripts (Clerical, Regular, Running, Cursive), the paradigm adopts line- and paragraph-level transcriptions following original reading order.

Stage-Adaptive Evaluation Tasks

The benchmark formulates four rigorous quantitative tasks to isolate visual perception from semantic reasoning:

  1. Cross-period Character Spotting (Oracle Bone, Bronze, Seal): This end-to-end task requires the VLLM to simultaneously output the bounding box coordinates and the corresponding modern Chinese character mapping for all archaic symbols present in an image, evaluated using the H-mean metric.

  2. Fine-grained Archaic Character Recognition: To isolate pure morphological mapping accuracy, this task introduces a visual referring mechanism where the model is prompted (e.g., “Recognize the archaic character highlighted by the red box in the image”) to generate the corresponding modern character, measured by Exact Match Accuracy.

  3. Ancient Text Parsing (All Seven Scripts): This task assesses comprehension of historical spatial layouts and reading sequences, utilizing a paragraph-level Normalized Edit Distance (NED) score, which strictly penalizes any sequence-ordering mismatches.

  4. Script Classification (All Seven Scripts): This task probes the model’s macro-level understanding of morphological evolution, requiring the model to classify an image into one of the “Seven Chinese Scripts,” evaluated using overall Accuracy (Acc).

Key Findings on Perceptual Bottlenecks

Evaluation results reveal substantial capability gaps in fine-grained visual perception. The paper demonstrates that current VLLMs exhibit a severe, twofold bottleneck in fine-grained grounding and morphological decipherment when confronted with archaic, unstandardized texts. Specifically:

- Cross-period Character Spotting:

Leading commercial models like GPT-5 and Gemini 2.5 Pro register Spotting H-mean scores near zero, indicating a catastrophic failure in end-to-end tasks due to a lack of robust grounding mechanisms for unconstrained symbols embedded in noisy physical media.

- Fine-grained Archaic Character Recognition:

Even when provided with explicit visual guidance via bounding boxes, models consistently fail to map these pictographic glyphs to their modern counterparts, confirming a massive, independent semantic gap beyond mere spatial layout confusion.

- Ancient Text Parsing:

Performance on parsing drops substantially from mature scripts (e.g., Kimi K2.5 achieving a NED of 0.78 on Regular Script) to archaic scripts, plummeting to merely 0.05 on Oracle Bone Script, due to both morphological deviation and the fundamental difference in layout structures—mature scripts adhere to standardized conventions while archaic inscriptions feature highly unconstrained, non-linear reading sequences.

Paradox in Classification Performance

An intriguing contradiction emerges in the Script Classification results: models achieve exceptionally high classification accuracy on Archaic Scripts (e.g., Seed2.0 Pro at 96.6%) by exploiting macro-level textural priors, yet this success sharply contrasts with their performance on Mature Scripts, where classification accuracy drops significantly. This inversion highlights a fundamental decoupling between stylistic recognition and fine-grained perception, suggesting that current models rely heavily on macroscopic shape recognition rather than perceiving the delicate stroke dynamics required to differentiate mature script categories.

Implications for Digital Humanities

The findings establish clear optimization trajectories: scaling up model parameters yields better overall performance, but reasoning-enhanced variants often degrade due to redundant or erroneous processes.

Improvements for AI systems

Here are the specific improvements and capabilities that can be derived from the Chronicles-OCR benchmark:

) 1. Development of an Evolution-Aware, Cross-Temporal Perception VLLM Foundation Model:

Improvement: Instead of training models solely on standardized modern text, researchers should fine-tune or pre-train Vision Large Language Models (VLLMs) using the Chronicles-OCR dataset across all Seven Chinese Scripts (Oracle Bone through Cursive). This requires developing a new pre-training paradigm that explicitly incorporates morphological evolution as a core feature space.

Capability: The resulting model can perform robust, zero-shot or few-shot perception on historical texts that were not included in its primary training data, overcoming the distribution shift barrier inherent in current models.

) 2. Creation of a Stage-Adaptive Annotation Pipeline:

Improvement: Implement the novel Stage-Adaptive Annotation Paradigm for data labeling and ground truth generation. This involves using fine-grained character bounding boxes and modern character mappings for archaic scripts, while employing sequence-level transcriptions for mature scripts (Clerical to Cursive).

Capability: This provides high-fidelity, multi-modal training data tailored to the specific visual characteristics of each historical era, allowing models to learn script-specific representations rather than a single modern representation.

) 3. Creation of Specialized Evaluation Metrics for Digital Humanities:

Improvement: Adopt and standardize the four rigorous evaluation tasks (Cross-period Character Spotting, Fine-grained Archaic Recognition, Ancient Text Parsing, and Script Classification). This includes using H-mean for spotting, Exact Match Accuracy for recognition (via visual referring), Normalized Edit Distance (NED) for parsing, and overall Accuracy for classification.

Capability: This allows researchers to systematically diagnose the exact perceptual bottlenecks of current VLLMs across different historical challenges (e.g., identifying whether a model fails due to spatial confusion, semantic gap, or stroke dynamics).

) 4. Targeted Model Architecture Design for Paleographic Features:

Improvement: Based on the analysis showing failure in fine-grained recognition (due to lack of specialized paleographic representations), future VLLM architectures should be augmented with specialized modules capable of mining micro-level, stroke-level features, rather than relying only on high-level structural priors.

Capability: The system can move beyond macroscopic shape recognition to perceive delicate stroke dynamics and contextual textures necessary for differentiating closely related mature script categories (e.g., distinguishing Cursive from Running script).

) 5. Mitigation of Reasoning Hallucination through Perceptual Grounding:

Improvement: Integrate a mechanism where explicit reasoning steps are only permitted or heavily constrained when the underlying visual perception task has achieved a high confidence score (as suggested by the degradation observed in think models).

Capability: This prevents reasoning-enhanced models from amplifying their own perceptual errors. The system will prioritize accurate visual grounding before attempting complex semantic interpretation, leading to more reliable outputs for historical decipherment tasks.

) 6. Automated Historical Document Cataloging and Sorting:

Improvement: Leverage the high classification accuracy observed on Archaic Scripts to build a robust automated cataloging system that can ingest diverse physical media (shells, bronzes, steles) and instantly classify them into their correct historical script stage.

Capability: This enables rapid, automated archival sorting and initial categorization of vast amounts of unstandardized historical artifacts without human intervention.

Sources

Related papers