PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset

summary

The gist

The gist The PRIMEdit framework enables faithful and disentangled edits in single- and multi-instance levels, as well as an applicability to more fine-grained partial instance level without the need

In short

PRIMEdit is a zero-shot framework for editing multiple objects in videos accurately and without unintended changes elsewhere. It uses Instance-centric Probability Redistribution (IPR) to ensure precise localization and Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. The method introduces the MIVE dataset and the Cross-Instance Accuracy score, proving it significantly outperforms existing methods in faithfulness and accuracy.

Key concepts

Instance-centric Probability Redistribution (IPR): This module modifies cross-attention layers by splitting the attention map into inside and outside an object's mask. It redistributes probabilities to ensure that tokens outside the instance mask have zero probability, preventing edits in unintended areas, while boosting probabilities within the instance to enhance editing faithfulness.
Disentangled Multi-instance Sampling (DMS): DMS minimizes attention leakage by using two sampling strategies: Series Noise Sampling (SNS) and Parallel Noise Sampling (PNS). It first independently edits each object and then harmonizes their resulting latents through PNS after latent fusion to produce a final, disentangled latent representation.
Cross-Instance Accuracy (CIA) Score: This is a new metric designed to quantify how much an edit leaks into other objects during multi-instance video editing. It's calculated by creating a similarity matrix where each row sets the highest score to 1, indicating perfect alignment between an instance and its caption, providing a strong correlation with human judgment.
MIVE Dataset: This is a newly created benchmark dataset for multi-instance video editing. It contains 200 videos featuring various numbers and sizes of objects, complete with corresponding instance-level captions and masks. This dataset allows researchers to conduct controlled analyses across different levels of scene complexity.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 18 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Quantum Fisher Information (QFI) — explained in 15 episodes

This episode discusses

The paper

PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset · Read on arXiv

Samuel Teodoro, Agus Gunawan, Soo Ye Kim, Jihyong Oh, Munchurl Kim

KAIST · Adobe Research · Chung-Ang University

Recent Artificial Intelligence (AI)-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When editing multiple objects only within their localized regions, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose P robability R edistribution for I nstance-aware M ulti-object Video Edit ing (PRIMEdit). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present the MIVE dataset for multi-instance video editing, including its subsets categorized by instance counts and sizes for controlled analysis across different levels of scene complexity. We also introduce the Cross-Instance Accuracy (CIA) Score that is firstly proposed to evaluate the editing leakage in multi-instance video editing tasks, which exhibits strong correlation with human evaluation as measured by Spearman's ρ= 0.86. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.

DOI: 10.1109/TPAMI.2026.3722239

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset".

Jane: The gist The PRIMEdit framework enables faithful and disentangled edits in single- and multi-instance levels,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: We're looking at "PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset" now and summarizing what they claim about this new approach to video editing.

Jane: The paper focuses on moving beyond global edits or single-object edits to handle multiple objects in a scene with more precision. They introduce the PRIMEdit framework, which is zero-shot, meaning it doesn't need extra training data from scratch for these localized edits.

Lu: They achieve this using two key modules: Instance-centric Probability Redistribution, IPR, which they use to ensure precise localization and faithful editing, and Disentangled Multi-instance Sampling, DMS to prevent attention leakage.

Meng: If we're thinking about what this means for shipping software that handles video editing now, what’s the immediate practical benefit of having these specific modules?

Tom: The core claim is that PRIMEdit enables faithful and disentangled edits in single- and multi-instance levels, and it even has an applicability to more fine-grained partial instance level without needing additional training.

Jane: To test this, they built a new evaluation benchmark called the MIVE Dataset, which consists of two hundred videos with varying numbers and sizes of instances, complete with instance-level captions and masks.

Lu: They also introduced the Cross-Instance Accuracy score, CIA Score. This metric is designed to quantify attention leakage in multi-instance editing tasks; they found it exhibits a strong correlation with human evaluation at a Spearman’s rho of zero point eight six.

Meng: So how does this move us away from the problems they identified, like unfaithful editing and attention leakage?

Tom: Previous methods often used global edit captions, which led to issues where edits bled into other parts of the video or weren't faithful to the specific object being edited. PRIMEdit addresses that by modifying each instance based on its corresponding edit caption.

Jane: It means for creators, they can give a prompt for a scene with three people and expect the edit to only affect those three people, not the furniture in the background.

Lu: However, they did flag a limitation: the model struggles with reflection consistency because their masks only cover the instance itself, not its reflection. That’s where future work needs to focus.

Conclusion: Tom: So we're wrapping up this whole thing about PRIMEdit now, looking at "PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset."

Jane: That title is pretty long, isn't it? It tells us right away that they’re focusing on how to redistribute probability to make sure the edits are instance aware.

Lu: The authors are working on representation learning and model training, and they’ve built a framework that uses two main modules: IPR for localization and DMS for sampling.

Meng: From what I’m hearing, it's about taking these zero-shot editing ideas and making them reliable enough to actually ship in production without constant retraining.

Tom: Exactly. The main implication is that we can finally move past those broad edits where the AI just randomly changes stuff across the whole video.

Lu: They’ve created this MIVE Dataset, which is pretty important because it gives us a controlled environment to test this on different scene complexities.

Tom: And they have this CIA Score, that Cross-Instance Accuracy score at zero point eight six correlation with human evaluation, which is a solid number for measuring that leakage we talked about earlier.

Jane: It suggests that the core concept isn't just about making things look good, it’s about building a system where every object gets its own clear boundaries in the editing process.

More episodes

← Home