Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
cs.CV, cs.CL, cs.LG
Submitted: 2026-09-26
Updated: 2026-10-01
Code: https://github.com/Yrxxxxxxxx1007/LT-OPD
Terminology
Sources
- LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training
- Qwen3-VL Technical Report
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
- GPT-4o System Card
- G squared TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Kimi-VL Technical Report
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
- Group Sequence Policy Optimization
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models