What Users See Is Not What Models Read: Split-View PDFs in Document-to-LLM Supply Chains

arXiv:2606.15020 · cs.CR · Submitted 2026-06-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: Today's paper: "What Users See Is Not What Models Read".

Elias: The core finding of this research is that document-to-LLM pipelines suffer from semantic integrity failures because PDF renderers and extractors operate independently,

Nadia: First, who's behind it and why it matters.

Paper summary: Nadia: So, we're looking at this paper, "What Users See Is Not What Models Read: Split-View PDFs in Document-to-LLM Supply Chains," and it seems the core idea is that there's a serious problem with how document-to-LLM pipelines handle PDFs. Essentially, the authors show that these pipelines have semantic integrity failures because the tools responsible for rendering a PDF page and extracting its text operate independently, which lets attackers sneak in or extract text that looks fine to a human but carries different meaning for the AI model <ref:2606.15020#pg0>.

Elias: That's what caught my attention too, Nadia; it suggests a fundamental split-view PDF where the rendered page presents one set of semantics while the extracted text carries something else entirely, creating a supply chain issue upstream before the model even gets its input <ref:2606.15020#pg0>. It makes you wonder what kind of text an AI might actually be reasoning over when it's fed this divergent information.

Priya: From my side, I’m interested in what this means for the data itself; if the extracted text is semantically different from what a user sees, then any subsequent analysis or summary generated by an AI could be based on misinformation that was hidden from view <ref:2606.15020#pg1>. It raises questions about the trustworthiness of LLM outputs derived from these documents.

Nadia: Exactly, Priya; the paper claims they've identified twenty-five distinct extraction gaps across four families—semantic overrides, hidden semantic injection, reading-order splits, and font-decoding splits—and that these gaps are often not seen in previous work <ref:2606.15020#pg1>. It really highlights how much the PDF specification itself allows for these representation gaps between rendering and extraction <ref:2606.15020#pg0>.

Elias: I agree, Nadia; focusing on those specific mechanism-level mismatches, like the font-level /ToUnicode CMap override or span-level /ActualText marked-content attribute overrides mentioned in the abstract, shows that this isn't just a bug in one tool but a tolerated gap in how PDFs are processed generally <ref:2606.15020#pg0>. It points to a deeper issue with the PDF model itself.

Priya: And when you look at those four families, particularly reading-order splits and font-decoding splits, it suggests that the divergence isn't always malicious injection but can stem from how different systems interpret the structure of the document versus how a human reads it <ref:2606.15020#pg0>. That distinction between spatial reading order and content stream order is something we need to consider for privacy measurement.

Paper summary: Nadia: That's a good point, Priya; and Elias, you mentioned the four families; are there any particular gaps that seem more exploitable or easier to trigger in practice based on what you’ve seen? We're trying to figure out if this is just theoretical or something we can actually see happening on the ground <ref:2606.15020#pg1>.

Elias: The paper does mention that fourteen of those twenty-five extraction gaps don't have an exact path or mechanism-level match in prior studies, which is significant because it means we’re looking at novel ways to exploit these PDF structures <ref:2606.15020#pg1>. The authors also built a two-tier benchmark to systematically test whether these gaps actually propagate into the final LLM output through their summary and QA attacks <ref:2606.15020#pg1>.

Priya: That two-tier benchmark sounds like it’s very rigorous for separating the parser layer issues from the downstream impact on the model, which is important for understanding what the data actually shows <ref:2606.15020#pg1>. It helps us see if a hidden text insertion in a document translates into a flawed answer from an AI service.

Nadia: Right, so it’s not just finding the gap; it’s proving that the gap actually causes divergence when you test it against sixteen different processing stacks and seven commercial LLM services <ref:2606.15020#pg0>. That scale of testing shows how widely this problem is exposed, showing coverage ranging from twelve to twenty-one out of the gaps for each service <ref:2606.15020#pg1>.

Elias: It confirms the idea that exposure isn't tied to one specific model identity but rather to the ingestion stack—the APIs, cloud backends, and local runtimes—which is a crucial detail for understanding where we need to apply our scrutiny <ref:2606.15020#pg1>. The paper also suggests that these canaries serve as a way to fingerprint which specific PDF parser or loader path an LLM application is likely using <ref:2606.15020#pg1>.

Priya: Fingerprinting the ingestion stack sounds like a strong diagnostic tool; if we can identify the parser being used, it tells us exactly which part of the supply chain to focus our efforts on securing for privacy and integrity <ref:2606.15020#pg1>. It moves the discussion from just "is it broken" to "which component is causing the breakage."

Paper summary: Nadia: And that leads directly into what these gaps imply for security; if we can fingerprint the path, we can potentially target defenses more effectively <ref:2606.15020#pg1>. But what about defense? The authors tested a static screening scanner that flags all twenty-five benchmark gaps in their self-test, looking for things like "extractor-side replacement text" or "non-painted text operators" <ref:2606.15020#pg1>.

Elias: That scanner is interesting because it’s designed to catch the structural conditions that lead to these issues, but the paper notes that existing safety filters, like those in OpenDataLoader, often fail because they are based on fixed rendering-mismatch heuristics rather than checking for actual semantic consistency <ref:2606.15020#pg2>. It sounds like they’re not a fix for the underlying representation issue.

Priya: So, it suggests that relying solely on existing filters isn't sufficient because those filters don't understand the semantic divergence between what is visually present and what is extracted <ref:2606.15020#pg2>. This reinforces the need for solutions that check for consistency across modalities rather than just blocking specific text patterns.

Nadia: And when we look at defenses, the paper points to vision-based processing as the strongest defense against these text-layer split-view attacks, provided it's applied consistently across document sizes <ref:2606.15020#pg2>. But the authors also have a caution regarding OCR, stating that it’s only effective when the platform commits to visual processing across all document sizes because long documents can trigger fallback to cheaper text extraction routes <ref:2606.15020#pg2>.

Elias: That caveat about OCR highlights the performance trade-off; if you want perfect protection, you need consistent visual processing throughout the entire pipeline, which adds complexity to the system design <ref:2606.15020#pg2>. It also reminds us that these PDF Mirage attacks manipulate font and glyph interpretation to mask content for online services <ref:2606.15020#pg2>.

Priya: So, the implication is that document security in this context requires a layered approach: robust visual processing as a primary defense, combined with mechanisms to monitor ingestion stacks so we can identify where these representation gaps are most likely to occur <ref:2606.15020#pg1>. It’s about securing the entire pipeline upstream of the model reasoning process.

Nadia: That leads us perfectly into the conclusion of this work, "What Users See Is Not What Models Read: Split-View PDFs in Document-to-LLM Supply Chains," and its authors, Side Liu and Jiang Ming <ref:2606.15020#pg0>. They systematically identified twenty-five extraction gaps across four families, proving that document processing layers create divergent semantic views before the model even sees the data <ref:2606.15020#pg1>.

Paper summary: Elias: I think what this paper really contributes is mapping out these specific representation gaps and building that two-tier benchmark to test if those gaps actually cause real downstream issues in commercial LLM services <ref:2606.15020#pg1>. It moves the problem from a vague concern about AI input to a catalog of specific, measurable vulnerabilities in the PDF supply chain <ref:2606.15020#pg1>.

Priya: For privacy research, the implication is that document provenance and integrity checks need to be applied not just at the final model stage but throughout every extraction and rendering step, because hidden text can carry claims or data that are completely invisible to the user <ref:2606.15020#pg1>. It means what we collect from documents isn't just what’s visible on the screen.

Nadia: Exactly, Priya; it really puts the pressure on developers to address these gaps at the parsing and normalization layers rather than just treating them as post-processing issues <ref:2606.15020#pg2>. The practical reality is that if an attacker can control or influence that hidden layer, they can manipulate the AI’s perception of a document without ever changing what a user sees <ref:2606.15020#pg1>.

Elias: And from a cryptographic standpoint, Elias, the assumption here is that the text extracted by the model is not purely what was rendered visually, which means any cryptographic proof relying on visual fidelity of that text might be undermined if it relies on an unverified extraction layer <ref:2606.15020#pg0>. The parameters that break this are those related to font encoding and reading order, as the paper details <ref:2606.15020#pg1>.

Priya: So, when we think about the impact on the world, it’s not just about specific documents being compromised but about a fundamental erosion of trust in how we use AI to interpret complex visual information from documents <ref:2606.15020#pg1>. If the input itself is split into two different realities, the resulting analysis is inherently suspect <ref:2606.15020#pg1>.

Nadia: That’s what we’re seeing; the paper shows that this isn't a niche technical problem, but a widespread exposure across many stacks and services because of these tolerated gaps in the PDF standard itself <ref:2606.15020#pg0>. The challenge moving forward seems to be developing consistency checkers that go beyond simple text size filters to actually verify semantic alignment between the visual output and the machine-consumed text <ref:2606.15020#pg1>.

Conclusion: Nadia: So, to wrap up this discussion on "What Users See Is Not What Models Read: Split-View PDFs in Document-to-LLM Supply Chains," we've seen how document pipelines have these hidden semantic inconsistencies between what a person sees and what the AI actually processes.

Elias: I think that title really gets to the heart of it, focusing on that divergence between rendering and extraction—it suggests a fundamental mismatch in how data flows through this supply chain.

Priya: From my viewpoint, it points toward a significant gap in privacy measurement because if the extracted text differs from the visual text, we can't trust any downstream analysis based on that input.

Nadia: Exactly, Priya; the authors are showing us that this isn't just a rendering hiccup but a systemic failure where attacker-controlled or extractor-dependent text slips through unnoticed.

Elias: That systemic nature is key because it means the vulnerabilities aren't isolated bugs in one piece of software; they’re structural weaknesses permitted by how PDFs are represented.

Priya: And when you consider the scope, this has major implications for document provenance; if a hidden payload can be injected and remain invisible, tracking the true origin of information becomes incredibly difficult.

Nadia: Right, and that brings us to the real-world impact; we're looking at a situation where we might be feeding an AI documents that look clean but have malicious or misleading text embedded in ways only the extractor can see.

Elias: The authors’ work on mapping those twenty-five specific extraction gaps provides a concrete blueprint for understanding exactly which PDF features are most susceptible to this semantic divergence.

Priya: The implications for researchers and developers are that defenses can't just be about blocking visible text; they need checks that verify the consistency between the visual representation and the machine-read data.

Nadia: Precisely, so we’re moving toward building tools that check for semantic alignment across modalities rather than relying on simple heuristics.

Elias: And I'm curious about how those specific font-decoding and reading-order splits you mentioned might affect cryptographic proofs that rely on the assumption of visual fidelity.

Priya: That’s a big question, Elias; if the underlying text representation is mutable based on the extraction method, then any security layer built on top of that text gets fundamentally shaky.

Nadia: It makes us realize that securing this pipeline needs to happen far upstream at the parsing and normalization layers, long before the data ever reaches the reasoning model.

Elias: That structural focus is what makes this paper so valuable; it moves us away from treating these issues as isolated incident reports toward a comprehensive understanding of PDF processing flaws.

Tulane University

cs.CR

Submitted: 2026-06-12

Updated: 2026-10-05

Comments: Accepted by EEE Symposium on Security and Privacy (S&P 2027)

Code: https://github.com/py-pdf/pypdf

Project page: https://mozilla.github.io/pdf.js

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: The core finding of this research is that document-to-LLM pipelines suffer from semantic integrity failures because PDF renderers and extractors operate independently, allowing attacker-controlled or

Key concepts

Split-View PDF: A PDF where what a human sees (the rendered page) is different from what an extractor pulls out as text. This happens because the rendering process and the text extraction process operate independently, allowing malicious or unexpected data to exist only in the extracted stream, not on the visual page.
Extraction Gaps (EG): Specific technical flaws within PDF specifications that allow for inconsistent text extraction. These gaps range from font-level overrides to reading-order splits. They are the root causes of split-view PDFs, enabling text to be hidden or reordered only for the downstream LLM.
Two-Tier Benchmark: A testing method used by researchers that separates testing into two stages. Corpus A tests if an extractor detects a specific gap using inert tokens, while Corpus B tests how these gaps affect actual LLM tasks like summarization or question answering.

Terminology

Benchmark — 25× in this paper · explained in 2 episodes
Consistency — 20× in this paper · explained in 3 episodes
Elias — 16× in this paper · explained in 2 episodes
Nadia — 16× in this paper · explained in 2 episodes
Fingerprinting — 11× in this paper · explained in 2 episodes
RAG — 8× in this paper · explained in 3 episodes
Core — 4× in this paper · explained in 2 episodes
Robustness — 4× in this paper · explained in 5 episodes
Alignment — 3× in this paper · explained in 1 episode
Decode — 2× in this paper · explained in 2 episodes
Fidelity — 2× in this paper · explained in 2 episodes
Provenance — 2× in this paper · explained in 4 episodes
Semantic Alignment — 2× in this paper · explained in 3 episodes
Trace — 2× in this paper · explained in 2 episodes
Adversarial Robustness — 1× in this paper · explained in 3 episodes
Benchmarking — 1× in this paper · explained in 4 episodes
Calibration — 1× in this paper · explained in 7 episodes
Modality Gap — 1× in this paper · explained in 4 episodes
Ontology — 1× in this paper · explained in 2 episodes
Portable Document Format — 1× in this paper
Verification — 1× in this paper · explained in 2 episodes

Summary

The core finding of this research is that document-to-LLM pipelines suffer from semantic integrity failures because PDF renderers and extractors operate independently, allowing attacker-controlled or extractor-dependent text to be consumed by models while remaining invisible to the human user. This vulnerability stems from a split-view PDF where the rendered page presents benign content, but the extracted text carries divergent semantics, creating a critical supply chain failure upstream of model reasoning.

The gist

Document-to-LLM pipelines suffer from semantic integrity failures because PDF renderers and extractors operate independently, allowing attacker-controlled or extractor-dependent text to be consumed by models while remaining invisible to the human user.

Root Causes of Split-View PDFs

The split-view PDF arises not from implementation bugs in any single tool, but from specification-permitted representation or implementation-tolerated gaps in PDF’s dual rendering and extraction model. The paper identifies 25 confirmed extraction gaps (EG) across four root-cause families:

Semantic overrides: This includes the font-level /ToUnicode CMap override (EG01) and the span-level /ActualText marked-content attribute override (EG02). For example, EG04 sets Tr=3, a mode where the renderer generates glyph outlines but neither fills nor strokes text, leaving a payload in the content stream.
Hidden semantic injection: This family creates text that is visible to extractors but invisible in the rendered page.
Reading-order splits: This occurs when the PDF content stream order does not correspond to spatial reading order, causing extractors to return text in a different sequence than what a human reader perceives.
Font-decoding splits: These exploit ambiguities in Type 3 and CID-keyed fonts, where Unicode recovery depends on font metadata and fallback behavior, leading extractors to return different Unicode sequences than the rendered glyphs.

The Two-Tier Benchmark

To systematically evaluate these gaps, the authors developed a two-tier benchmark designed to separate mechanism validation from downstream impact:

Corpus A: Mechanism Canary Corpus:

This corpus consists of 25 minimal one-page PDFs, one for each confirmed extraction gap. Each sample isolates a single mechanism and embeds a semantically inert canary token to score whether the extractor returns the canary, visible text, or dropout. This tests upstream parser behavior (RQ1) and allows for fingerprinting of hidden ingestion behavior.

Corpus B: End-to-End LLM Corpus:

This corpus provides semantically rich documents that embed each confirmed gap into a downstream task carrier. It uses two modalities:

Summary Attack: Tests passage-level contamination, where the extractor view contains an attacker-side claim absent from the visible document. Success requires the model to repeat this claim during summarization.
QA Attack: Tests answer-span corruption, where a mechanism is woven into a short field or sentence, causing the extracted answer to be mutated or reordered compared to the visible text.

Evaluation Results and Deployment Exposure

The evaluation across 16 PDF processing stacks and 7 commercial LLM services showed that every gap produces render-extract divergence on at least one stack. Exposure is primarily shaped by the ingestion stack—APIs, cloud backends, web frontends, and local runtimes—not model identity alone. For instance, GPT-5.4 shows broad coverage (21/25), spanning all semantic override and hidden semantic injection gaps. Furthermore, deployment paths matter: Cloud deployment is a PDF security boundary, as routing through a hosted backend can change the parser or normalization layer, leading to different exposure sets (e.g., Azure dropping from 21/25 to 17/25).

Diagnostic Value and Defense Analysis

The extraction gaps serve dual roles: they are attack vectors and diagnostic probes. The authors developed a static screening scanner that triggers on all 25 benchmark gaps in their self-test, flagging structural conditions like extractor-side replacement text or non-painted text operators. While existing safety filters, such as those in OpenDataLoader, can suppress certain mechanisms (e.g., near-zero text size), they fail because they are fixed rendering-mismatch heuristics, not as a semantic consistency checker. Vision-based processing is the strongest defense against text-layer splitview attacks when applied consistently across document sizes. However, OCR is an effective defense only when the platform commits to visual processing across document sizes, as long documents can trigger a fallback to cheaper text extraction routes.

Ingestion-Stack Fingerprinting

The authors demonstrated that extraction gaps are diagnostic probes capable of fingerprinting the ingestion stack.

Improvements for AI systems

Based on the scientific paper, here are specific improvements for AI systems derived from these findings:

)Improved System Capabilities: Semantic Integrity Enforcement Layer (SIEL)

The core improvement is the implementation of a mandatory, auditable semantic integrity layer positioned precisely at the PDF-to-Text boundary. Instead of assuming that text extracted by a parser reflects visual reality, the system must enforce a contract between the human-visible claim set and the machine-consumed claim set.

Refinement of Ingestion Stacks:
Deployment-Aware Risk Assessment:
Ingestion Fingerprinting for Supply Chain Auditing:
Dynamic Defense Switching (Vision vs. Text):

)Specific Improvements and System Capabilities

A dedicated Split-View Guard module must be integrated into all PDF ingestion pipelines. This module continuously monitors the document stream for the four root-cause families identified (semantic overrides, hidden injection, reading-order splits, and font-decoding splits).
Enforcement Mechanism: For every detected gap (EG), the SIEL must execute a Consistency Check. If a divergence is found—where rendered output differs from extracted text for the same input—the system must halt inference and flag the document as having an unverified semantic boundary violation, preventing downstream reasoning based on potentially malicious or misleading text.
Dynamic Defense Switching: The system should implement a tiered defense strategy:
Contextual Triggering of Vision Processing: If the ingestion stack is identified (via fingerprinting) as one prone to visual-to-text divergence (e.g., certain cloud backends), the system should automatically trigger high-fidelity, page-by-page rendering and OCR for that document before text extraction proceeds.
Ingestion Fingerprinting: Implement a lightweight Canary Scanner that runs pre-ingestion to generate a unique signature based on the output tuple of key PDF objects (e.g., font metadata, marked-content attributes, and specific text recovery heuristics). This signature allows the system to rapidly identify which parser or ingestion stack is feeding the model context.
Supply Chain Auditing: Use the generated fingerprint to map every incoming document to its upstream processing path (e.g., This document came through PyPDF on Azure). This enables auditors to pinpoint exactly where a semantic integrity breach occurred in the supply chain, rather than just blaming the final LLM model.
Targeted Defense Tuning: Based on the identified gap family, the system can apply targeted mitigations (e.g., if a font-decoding split is detected, enhance Unicode recovery heuristics for that specific file type).
Mitigation of Geometric Hiding: Implement checks specifically targeting geometric vulnerabilities like off-page positioning (EG06) and zero-height text matrices (EG12), which are shown to be vulnerable under certain cloud deployment configurations.

)What the Improved AI System Can Do

The improved system moves beyond simple prompt filtering to become a proactive security and integrity monitor for document ingestion:

Predictive Integrity Vetting: Before any document is used for summarization or Q&A, the system performs a Split-View Check. It verifies that the text presented to the LLM matches what a human would see on the rendered page, actively blocking inputs where this fundamental contract is broken.
Automated Supply Chain Tracing: The system can instantly answer Where did this document come from? by analyzing its unique ingestion signature, allowing security teams to audit specific parser versions or cloud routing decisions responsible for the risk profile of a document batch.
Adaptive Risk Scoring: The system assigns a dynamic risk score to documents based on the ingestion path and the severity of detected extraction gaps (EG). A document routed through an API known to have high font-decoding vulnerabilities will receive a higher initial scrutiny than one routed through a more robust pipeline, allowing for risk-based prioritization.
Resilient Reasoning: By ensuring that the model only receives text that is consistent with its visual representation, the system drastically reduces the likelihood of semantic override or hidden semantic injection attacks from corrupting the model's reasoning process.