One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts

arXiv:2610.12448 · cs.CV, cs.LG · Submitted 2026-10-08 · Read on arXiv

cs.CV, cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

Terminology

• Transformer — 28× in this paper · explained in 2 episodes
• Distillation — 23× in this paper · explained in 2 episodes
• Ablation — 15× in this paper · explained in 1 episode
• Accuracy — 14× in this paper · explained in 1 episode
• Adaptation — 10× in this paper · explained in 1 episode
• Alignment — 8× in this paper · explained in 1 episode
• Hypernetwork — 4× in this paper · explained in 2 episodes
• Regularization — 4× in this paper · explained in 3 episodes
• Adam — 2× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Sparsity — 2× in this paper · explained in 4 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Linear probe — 1× in this paper · explained in 2 episodes
• Mixture of Experts — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 3 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• What do Vision Transformers Learn? A Visual Exploration
• Adaptive Computation Time for Recurrent Neural Networks
• DeepSeek-V3 Technical Report
• Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
• Qwen3 Technical Report
• ST-MoE: Designing Stable and Transferable Sparse Expert Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models