From Priors to Perception: Grounding Video-LLMs in Physical Reality

arXiv:2605.04515 · cs.CV · Submitted 2026-05-06 · Read on arXiv

cs.CV

Submitted: 2026-05-06

Updated: 2026-09-19

Code: https://github.com/LiamZhao326/From-Priors-to-Perception

Terminology

• Accuracy — 20× in this paper · explained in 1 episode
• Hallucination — 14× in this paper · explained in 3 episodes
• Core — 12× in this paper · explained in 2 episodes
• Fine-Tuning — 12× in this paper · explained in 5 episodes
• Grounding — 10× in this paper · explained in 3 episodes
• Computer Vision — 8× in this paper
• Visual-Anchored Reasoning Chain — 8× in this paper
• Alignment — 7× in this paper · explained in 1 episode
• Consistency — 7× in this paper · explained in 3 episodes
• Fidelity — 7× in this paper · explained in 2 episodes
• Visual Grounding — 7× in this paper · explained in 2 episodes
• Benchmark — 6× in this paper · explained in 2 episodes
• Verification — 6× in this paper · explained in 2 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Decoupling — 5× in this paper · explained in 2 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Benchmarking — 3× in this paper · explained in 3 episodes
• LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
• Pair-wise Consistency Accuracy — 3× in this paper
• Reasoning Alignment Score — 3× in this paper
• World Model — 3× in this paper · explained in 7 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Chain-of-Thought (CoT) — 2× in this paper · explained in 4 episodes
• Computational overhead — 2× in this paper · explained in 2 episodes
• Embodied AI — 2× in this paper · explained in 2 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• Human-in-the-Loop — 2× in this paper · explained in 3 episodes
• LoRA Fine-Tuning — 2× in this paper · explained in 3 episodes
• Quantization — 2× in this paper · explained in 12 episodes
• Supervised Fine-Tuning — 2× in this paper
• Transformer — 2× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• AdamW Optimizer — 1× in this paper · explained in 1 episode
• Adversarial scenarios — 1× in this paper · explained in 1 episode
• Causal Reasoning — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• GRPO — 1× in this paper · explained in 6 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Semantic Understanding — 1× in this paper · explained in 2 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Test-time Scaling — 1× in this paper · explained in 2 episodes

Sources

• Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
• VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
• PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
• Video-R1: Reinforcing Video Reasoning in MLLMs
• NOAH: Benchmarking Narrative Prior driven Hallucination and Omission in Video Large Language Models
• WorldModelBench: Judging Video Generation Models As World Models
• VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
• VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding
• TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility
• SAM 2: Segment Anything in Images and Videos
• IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Kling-Omni Technical Report
• InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
• VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models
• SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
• MiniCPM-V: A GPT-4V Level MLLM on Your Phone

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models