SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
cs.CV
Submitted: 2026-09-28
Updated: 2026-09-28
Project page: https://armenjeddi.github.io/scopd
Terminology
Sources
- Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- Evaluating Large Language Models Trained on Code
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Distilling the Knowledge in a Neural Network
- Similarity-Aware Token Pruning: Your VLM but Faster
- AVIS: Adaptive Test-Time Scaling for Vision-Language Models
- Trajectory-Refined Distillation
- VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information
- Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
- Visual-Advantage On-Policy Distillation for Vision-Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Self-Distillation Enables Continual Learning
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
- Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models