From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction

arXiv:2610.11818 · cs.CV, cs.AI · Submitted 2026-10-08 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-10-08

Updated: 2026-10-08

Code: https://github.com/uddipan77/Analysis-of-Lightweight-Vision-Language-Models-for-Document-OCR-and-Str

Terminology

• Fine-Tuning — 40× in this paper · explained in 5 episodes
• Adaptation — 9× in this paper · explained in 1 episode
• Benchmark — 8× in this paper · explained in 2 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Instruction Tuning — 3× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• LoRA Adapters — 2× in this paper · explained in 3 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Alignment — 1× in this paper · explained in 1 episode
• Approximate Normalized Levenshtein Similarity — 1× in this paper
• Benchmarking — 1× in this paper · explained in 3 episodes
• Character Error Rate — 1× in this paper
• Data Sovereignty — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Graphics Processing Unit — 1× in this paper
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Quantization — 1× in this paper · explained in 12 episodes
• Resource Constraints — 1× in this paper · explained in 3 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Pixtral 12B
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• Qwen2.5-VL Technical Report
• PaliGemma: A versatile 3B VLM for transfer
• PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model
• Gemma 3 Technical Report
• SmolVLM: Redefining small and efficient multimodal models
• ANLS* -- A Universal Document Processing Metric for Generative Large Language Models
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model
• DeepSeek-OCR: Contexts Optical Compression
• MiniCPM-V: A GPT-4V Level MLLM on Your Phone
• mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models