What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?

arXiv:2610.09700 · cs.CV, cs.AI · Submitted 2026-10-07 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/rom1504/img2dataset

Terminology

• Accuracy — 23× in this paper · explained in 1 episode
• Embeddings — 18× in this paper · explained in 2 episodes
• Failure Modes — 10× in this paper · explained in 3 episodes
• Ablation — 9× in this paper · explained in 1 episode
• Transformer — 7× in this paper · explained in 2 episodes
• Benchmark — 6× in this paper · explained in 2 episodes
• Contrastive Learning — 6× in this paper · explained in 5 episodes
• Representation Learning — 6× in this paper · explained in 2 episodes
• Modality Gap — 5× in this paper · explained in 4 episodes
• Computational overhead — 4× in this paper · explained in 2 episodes
• Foundation Model — 4× in this paper · explained in 3 episodes
• Foundation Models — 4× in this paper · explained in 7 episodes
• Linear probe — 4× in this paper · explained in 2 episodes
• Cosine Similarity — 3× in this paper · explained in 3 episodes
• Generative Model — 3× in this paper · explained in 2 episodes
• Generative Models — 3× in this paper · explained in 2 episodes
• Zero-Shot Transfer — 3× in this paper · explained in 4 episodes
• alpha — 3× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• Computer Vision and Pattern Recognition — 2× in this paper
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• Model-Agnostic — 2× in this paper · explained in 2 episodes
• Self-Supervised Learning — 2× in this paper · explained in 8 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Clustering — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Gradient Flow — 1× in this paper · explained in 2 episodes
• In-Context Learning — 1× in this paper · explained in 4 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Predictive Coding — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Self-Distillation — 1× in this paper · explained in 7 episodes
• Semantic Understanding — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes

Sources

• SynCo: Synthetic Hard Negatives for Contrastive Visual Representation Learning
• ViTAMINS: An Empirical Study of Training Self-Supervised Vision Transformers with Synthetic Hard Negatives
• Explaining and Harnessing Adversarial Examples
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
• Do Better ImageNet Models Transfer Better?
• Scaling Language-Image Pre-training via Masking
• Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
• Microsoft COCO: Common Objects in Context
• CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions
• MLLMs-Augmented Visual-Language Representation Learning
• Fine-Grained Visual Classification of Aircraft
• SILC: Improving Vision Language Pretraining with Self-Distillation
• Representation Learning with Contrastive Predictive Coding
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• Perceptual Grouping in Contrastive Vision-Language Models
• Contrastive Learning with Hard Negative Samples
• EVA-CLIP: Improved Training Techniques for CLIP at Scale
• EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• Manifold Mixup: Better Representations by Interpolating Hidden States

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models