Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation
cs.CV, cs.LG
Submitted: 2026-10-01
Updated: 2026-10-01
Terminology
Sources
- Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
- ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning
- A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation
- Emerging Properties in Unified Multimodal Pretraining
- Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences
- GEditBench v2: A Human-Aligned Benchmark for General Image Editing
- MLLM-as-a-Judge Exhibits Model Preference Bias
- Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
- Step1X-Edit: A Practical Framework for General Image Editing
- Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias
- Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling
- Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics
- Verbosity Bias in Preference Labeling by Large Language Models
- Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
- Fine-Grained Human Pose Editing Assessment via Layer-Selective MLLMs
- Unified Reward Model for Multimodal Understanding and Generation
- On the Adversarial Robustness of Multimodal LLM Judges
- MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing
- SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models