Video-Index: A Curated Meta-Benchmark for Video Understanding

arXiv:2610.00960 · cs.CV · Submitted 2026-10-01 · Read on arXiv

cs.CV

Submitted: 2026-10-01

Updated: 2026-10-01

Terminology

• Benchmark — 478× in this paper · explained in 2 episodes
• Accuracy — 224× in this paper · explained in 1 episode
• Hallucination — 25× in this paper · explained in 3 episodes
• Benchmarking — 20× in this paper · explained in 3 episodes
• Conference on Computer Vision and Pattern Recognition — 13× in this paper
• Embeddings — 12× in this paper · explained in 2 episodes
• Cosine Similarity — 8× in this paper · explained in 3 episodes
• Verification — 8× in this paper · explained in 2 episodes
• Multi-hop Reasoning — 6× in this paper · explained in 3 episodes
• Grounding — 5× in this paper · explained in 3 episodes
• Decode — 4× in this paper · explained in 2 episodes
• RAG — 4× in this paper · explained in 3 episodes
• Temporal Reasoning — 4× in this paper · explained in 2 episodes
• Foundation Model — 3× in this paper · explained in 3 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Instruction Tuning — 3× in this paper · explained in 2 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Agentic Evaluation — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Core — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Admissibility — 1× in this paper · explained in 2 episodes
• Agentic Search — 1× in this paper · explained in 1 episode
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Clustering — 1× in this paper · explained in 2 episodes
• Commentary — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Multi-Agent System — 1× in this paper · explained in 6 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Positional Bias — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Semantic Understanding — 1× in this paper · explained in 2 episodes
• Sycophancy — 1× in this paper · explained in 4 episodes
• Verifier — 1× in this paper · explained in 2 episodes
• World Model — 1× in this paper · explained in 7 episodes

Sources

• Lost in Time: A New Temporal Benchmark for VideoLLMs
• LVBench: An Extreme Long Video Understanding Benchmark
• VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models
• StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
• TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
• Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
• BenchAgents: Multi-Agent Systems for Structured Benchmark Creation
• Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
• Generalization through Memorization: Nearest Neighbor Language Models
• From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models
• MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
• Qwen3-VL Technical Report
• Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models
• Qwen3 Technical Report
• The Vendi Score: A Diversity Evaluation Metric for Machine Learning
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
• InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
• VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models