Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models

arXiv:2605.23656 · cs.CV · Submitted 2026-05-22 · Read on arXiv

cs.CV

Submitted: 2026-05-22

Updated: 2026-09-18

Code: https://github.com/rwightman/pytorchimage-models

Terminology

• Accuracy — 19× in this paper · explained in 1 episode
• Transformer — 14× in this paper · explained in 2 episodes
• Distillation — 7× in this paper · explained in 2 episodes
• Fine-Tuning — 7× in this paper · explained in 5 episodes
• Downstream Tasks — 6× in this paper · explained in 2 episodes
• Recursive Block-Diagonal Coupling — 6× in this paper
• Instance Segmentation — 5× in this paper · explained in 3 episodes
• Transfer Learning — 5× in this paper · explained in 10 episodes
• alpha — 5× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Parameter-Efficient Fine-Tuning — 3× in this paper · explained in 2 episodes
• Pruning — 3× in this paper · explained in 4 episodes
• Quantization — 3× in this paper · explained in 12 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Knowledge Distillation — 2× in this paper · explained in 11 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Core — 1× in this paper · explained in 2 episodes
• Parameter-Efficient Fine-Tuning (PEFT) — 1× in this paper · explained in 4 episodes
• Regularization — 1× in this paper · explained in 3 episodes

Sources

• Net2Net: Accelerating Learning via Knowledge Transfer
• LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation
• FP8 Formats for Deep Learning
• On Efficient Training of Large-Scale Deep Learning Models: A Literature Review
• The Computational Limits of Deep Learning
• Wide Residual Networks

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models