One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models

arXiv:2609.36101 · cs.CV, cs.AI, cs.LG · Submitted 2026-09-28 · Read on arXiv

cs.CV, cs.AI, cs.LG

Submitted: 2026-09-28

Updated: 2026-09-28

Terminology

Modality Gap — 15× in this paper · explained in 4 episodes
Embeddings — 13× in this paper · explained in 2 episodes
Alignment — 10× in this paper · explained in 1 episode
Calibration — 9× in this paper · explained in 7 episodes
Vision-language models — 5× in this paper · explained in 2 episodes
International Conference on Learning Representations — 3× in this paper
AUC — 2× in this paper · explained in 2 episodes
Anisotropy — 2× in this paper · explained in 3 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
Foundation Models — 2× in this paper · explained in 7 episodes
Representation Learning — 2× in this paper · explained in 2 episodes
Accuracy — 1× in this paper · explained in 1 episode
Core — 1× in this paper · explained in 2 episodes
Cosine Similarity — 1× in this paper · explained in 3 episodes
Downstream Tasks — 1× in this paper · explained in 2 episodes
Generative Model — 1× in this paper · explained in 2 episodes
Generative Models — 1× in this paper · explained in 3 episodes
Multimodal Learning — 1× in this paper · explained in 2 episodes
Parsimony — 1× in this paper · explained in 2 episodes
Robustness — 1× in this paper · explained in 5 episodes

Sources

It's Not a Modality Gap: Characterizing and Addressing the Contrastive Gap

Related papers