Multimodal Language Models as Text-to-Image Model Evaluators

arXiv:2505.00759 · cs.CV, cs.AI, cs.CL · Submitted 2025-05-01 · Read on arXiv

cs.CV, cs.AI, cs.CL

Submitted: 2025-05-01

Updated: 2026-09-01

Code: https://github.com/LAION-AI/aesthetic-predictor

Terminology

• Benchmark — 75× in this paper · explained in 2 episodes
• Consistency — 43× in this paper · explained in 3 episodes
• Alignment — 19× in this paper · explained in 1 episode
• Accuracy — 13× in this paper · explained in 1 episode
• Diffusion Model — 8× in this paper · explained in 8 episodes
• Diffusion Models — 6× in this paper · explained in 6 episodes
• Perplexity — 6× in this paper · explained in 3 episodes
• Generative Model — 5× in this paper · explained in 2 episodes
• Generative Models — 5× in this paper · explained in 2 episodes
• Transformer — 4× in this paper · explained in 2 episodes
• Alignment Scores — 3× in this paper · explained in 1 episode
• Latent Diffusion Model — 3× in this paper · explained in 2 episodes
• Latent Diffusion Models — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• Benchmarking — 2× in this paper · explained in 3 episodes
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Edge AI — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Rectified Flow — 1× in this paper · explained in 3 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Sources

• Consistency-diversity-realism Pareto fronts of conditional image generative models
• InstructPix2Pix: Learning to Follow Image Editing Instructions
• Muse: Text-To-Image Generation via Masked Generative Transformers
• All You May Need for VQA are Image Captions
• PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis
• Visual Programming for Text-to-Image Generation and Evaluation
• Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation
• A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication
• Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• Visual Programming: Compositional visual reasoning without training
• Improving Model Evaluation using SMART Filtering of Benchmark Datasets
• CLIPScore: A Reference-free Evaluation Metric for Image Captioning
• Imagen Video: High Definition Video Generation with Diffusion Models
• GenAI Arena: An Open Evaluation Platform for Generative Models
• Text encoders bottleneck compositionality in contrastive vision-language models
• Scaling up GANs for Text-to-Image Synthesis
• Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
• ImagenHub: Standardizing the evaluation of conditional image generation models
• VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models