DiD It in 87 Minutes: A Label-Free Softmax-to-Linear Adaptation of Vision Transformers for Object Detection
cs.CV, cs.LG
Submitted: 2026-08-23
Updated: 2026-08-23
Code: https://github.com/lightly-ai/lightly-train
Terminology
Sources
- Scaling Pre-training to One Hundred Billion Data for Vision Language Models
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- SAM 3: Segment Anything with Concepts
- Vision Transformer Adapter for Dense Predictions
- Focal Self-attention for Local-Global Interactions in Vision Transformers
- DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
- DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding
- The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
- The Linear Attention Resurrection in Vision Transformer
- Reformer: The Efficient Transformer
- Rethinking Attention with Performers
- cosFormer: Rethinking Softmax in Attention
- Contrastive Representation Distillation
- Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review
- ViT-AdaLA: Adapting Vision Transformers with Linear Attention
- Linearizing Vision Transformer with Test-Time Training
- Distilling the Knowledge in a Neural Network
- RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
- Attention Transfer Is Not Universally Effective for Vision Transformers
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models