Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing

arXiv:2609.35708 · cs.CV · Submitted 2026-09-28 · Read on arXiv

cs.CV

Submitted: 2026-09-28

Updated: 2026-09-28

Project page: https://yanan-wang-cs.github.io/RefGAP/ABSTRACT

Terminology

• Fidelity — 24× in this paper · explained in 2 episodes
• Ablation — 7× in this paper · explained in 1 episode
• Calibration — 7× in this paper · explained in 7 episodes
• Consistency — 7× in this paper · explained in 3 episodes
• Computer Vision and Pattern Recognition — 5× in this paper
• Benchmark — 2× in this paper · explained in 2 episodes
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• KV cache — 2× in this paper · explained in 3 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Allocator — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 4 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Multimodal Inputs — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• SAM 3: Segment Anything with Concepts
• RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidance
• Prompt-to-Prompt Image Editing with Cross Attention Control
• Dual-Channel Attention Guidance for Training-Free Image Editing Control in Diffusion Transformers
• MediaPipe: A Framework for Building Perception Pipelines
• DINOv2: Learning Robust Visual Features without Supervision
• DirectSwap: Mask-Free Cross-Identity Training and Benchmarking for Expression-Consistent Video Head Swapping
• Head Similarity: Modeling Structured Whole-Head Appearance Beyond Face Recognition
• Qwen-Image Technical Report
• OmniGen2: Towards Instruction-Aligned Multimodal Generation
• LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing
• JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion
• IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models