MoTE: Mixture of Task Experts for Multi-Task Video Understanding

arXiv:2608.24763 · cs.CV, cs.LG · Submitted 2026-08-25 · Read on arXiv

cs.CV, cs.LG

Submitted: 2026-08-25

Updated: 2026-08-25

Project page: https://vicuna.lmsys

Terminology

• Mixture of Task Experts — 39× in this paper
• Accuracy — 25× in this paper · explained in 1 episode
• Benchmark — 19× in this paper · explained in 2 episodes
• Transformer — 16× in this paper · explained in 2 episodes
• Adaptation — 12× in this paper · explained in 1 episode
• Instruction Tuning — 12× in this paper · explained in 2 episodes
• Conference on Computer Vision and Pattern Recognition — 6× in this paper
• Vision-language models — 4× in this paper · explained in 2 episodes
• CVF International Conference on Computer Vision — 3× in this paper
• Embeddings — 3× in this paper · explained in 2 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• Key Information Extraction — 3× in this paper
• Sparsity — 3× in this paper · explained in 4 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Catastrophic Forgetting — 2× in this paper · explained in 5 episodes
• Commentary — 2× in this paper · explained in 2 episodes
• Interpretability — 2× in this paper · explained in 15 episodes
• Mixture of Experts — 2× in this paper · explained in 2 episodes
• Modularity — 2× in this paper · explained in 9 episodes
• Nadia — 2× in this paper · explained in 2 episodes
• Optical Character Recognition — 2× in this paper
• Sparse Mixture-of-Experts — 2× in this paper
• AI for Creative Video Editing and Understanding — 1× in this paper
• Ablation — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• International Conference on Document Analysis and Recognition — 1× in this paper
• LoRA Adapters — 1× in this paper · explained in 3 episodes
• Mixture-of-Experts (MoE) — 1× in this paper · explained in 7 episodes
• Multi-objective Optimization — 1× in this paper · explained in 4 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Semantic Understanding — 1× in this paper · explained in 2 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• GLM-OCR Technical Report
• Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
• Event-VStream: Event-Driven Real-Time Understanding for Long Video Streams
• Improved Baselines with Visual Instruction Tuning
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• AURA: Always-On Understanding and Real-Time Assistance via Video Streams
• GPT-4 Technical Report
• An Overview of Multi-Task Learning in Deep Neural Networks
• StreamingEval: A Unified Evaluation Protocol towards Realistic Streaming Video Understanding
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
• DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming
• LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models