Relational Representation Distillation

arXiv:2407.12073 · cs.CV, cs.AI · Submitted 2024-07-16 · Read on arXiv

cs.CV, cs.AI

Submitted: 2024-07-16

Updated: 2026-09-01

Code: https://github.com/giakoumoglou/distillers

Terminology

Distillation — 79× in this paper · explained in 2 episodes
Knowledge Distillation — 39× in this paper · explained in 11 episodes
Alignment — 12× in this paper · explained in 1 episode
Embeddings — 9× in this paper · explained in 2 episodes
Relational Representation Distillation — 9× in this paper
Accuracy — 6× in this paper · explained in 1 episode
Ablation — 5× in this paper · explained in 1 episode
Consistency — 5× in this paper · explained in 3 episodes
Benchmark — 4× in this paper · explained in 2 episodes
Representation Learning — 4× in this paper · explained in 2 episodes
Ablation study — 3× in this paper · explained in 1 episode
Contrastive Learning — 3× in this paper · explained in 5 episodes
Transferability — 3× in this paper · explained in 4 episodes
Transformer — 3× in this paper · explained in 2 episodes
Adam — 2× in this paper · explained in 2 episodes
Generalization — 2× in this paper · explained in 13 episodes
Kullback-Leibler (KL) Divergence — 2× in this paper · explained in 3 episodes
Quantization — 2× in this paper · explained in 12 episodes
Transfer Learning — 2× in this paper · explained in 10 episodes
Ablation Studies — 1× in this paper · explained in 1 episode
Benchmarking — 1× in this paper · explained in 3 episodes
Calibration — 1× in this paper · explained in 7 episodes
Decoupling — 1× in this paper · explained in 2 episodes
European Conference on Computer Vision — 1× in this paper
Objective Function — 1× in this paper · explained in 2 episodes
Predictive Coding — 1× in this paper · explained in 2 episodes
Redundancy — 1× in this paper · explained in 2 episodes
Regularization — 1× in this paper · explained in 3 episodes

Terminology

Related papers