Dino Forcing Flow Models: Do not denoise what you can predict

summary

The gist

The gist The proposed DINO Forcing (DF) method simplifies flow matching by predicting pretrained representations directly and using that prediction as self-conditioning, which substantially improves

In short

The DINO Forcing (DF) method simplifies flow matching by directly predicting pretrained representations like DINO and using that prediction to condition itself during generation. This removes the need for a second denoising trajectory and complex schedules. The approach achieves significantly faster convergence and better image quality in both latent and pixel spaces, proving that predicting what you can infer is more effective than denoisng unknown components.

Key concepts

Co-denoising: A prior method where the model simultaneously denoises the image and a pretrained representation. This requires managing two separate denoising paths and often involves complex schedules for both, which slows down training.
DINO Forcing (DF): The proposed simpler alternative. Instead of co-denoising, DF predicts the necessary pretrained representation directly in one pass and uses that prediction as a guide or condition for the main flow matching model during generation. It removes the need for a second ODE.
Representation Alignment Loss: A loss function used to train the model to ensure its predicted representation aligns with features from an external pretrained feature space, like DINO's output. This guides the flow matching process toward meaningful, learned visual features.
Do not denoise what you can predict: The core principle derived from the results. It suggests that auxiliary representations should not always be generated through a separate denoising trajectory; instead, if a representation can be inferred or predicted from the evolving sample, using that prediction as conditioning is sufficient for high-quality generation.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 18 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Quantum Fisher Information (QFI) — explained in 15 episodes

This episode discusses

Dino Forcing Flow Models: Do not denoise what you can predict · Paper Radio

The paper

Dino Forcing Flow Models: Do not denoise what you can predict · Read on arXiv

Arijit Ghosh, Lucas Degeorge, Paul Couairon, Alexei A. Efros, Vicky Kalogeiton, David Picard

ENPC (Ecole Polytechnique) · IP Paris (Institut Pasteur) · UC Berkeley

Co-denoising pretrained representations such as DINO can substantially improve the training speed and quality of flow matching models, but it introduces a second denoising trajectory and requires carefully designed schedules. We propose a simpler alternative: predict the pretrained representation directly, then condition the model on its own prediction. This removes the need for a second ODE and any representation-specific denoising schedules, while retaining the benefits of representation guidance. Our approach converges substantially faster and achieves better generation quality as measured by FID score. On ImageNet, it outperforms the state of the art in latent space at 2x fewer epochs than prior methods; in pixel space, it improves FID over comparable prior methods by more than 20%. These results support a simple principle: do not denoise what you can predict. Our code is openly available at https://github.com/arijit-hub/dino forcing.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Dino Forcing Flow Models".

Tom: The gist The proposed DINO Forcing (DF) method simplifies flow matching by predicting pretrained representations directly and using that prediction as self-conditioning,

Jane: First, who's behind it and why it matters.

Title and authors: Tom: So, we're looking at the paper "Dino Forcing Flow Models: Do not denoise what you can predict." Essentially, they are tackling the problem that when you use pretrained representations like DINO in flow matching, you often end up with two separate denoising processes which adds complexity and slow down training.

Jane: Right. They point out that co-denoising the images and the DINO embeddings introduces these paradoxes into the training process, specifically requiring a second denoising trajectory and those careful schedules that you have to design manually, according to page two of this paper #pg2 <ref:2610.11751#pg2>.

Lu: They propose a simpler alternative called DINO Forcing, where instead of denosing the representation alongside the image, you just predict that pretrained representation directly and then use that prediction to condition the model during generation, which is a schedule-free self-conditioning mechanism <ref:2610.11751#pg3>

Meng: So what's the main practical takeaway here is that this approach leads to substantially faster convergence for both latent and pixel space flow matching frameworks, which is something they show on page two of this paper #pg2 <ref:2610.11751#pg2>.

Tom: That speedup is significant when you look at the numbers they report on page two of this paper #pg2 <ref:2610.11751#pg2>. They show results on SiT-XL/two with eight hundred training epochs, and DINO Forcing reaches an FID of two point four seven in only one hundred epochs for latent space <ref:2610.11751#pg2>

Jane: That is a big difference when you compare it to prior methods that might need hundreds of epochs just to get decent results on their own, so the convergence speedup is really something to consider here <ref:2610.11751#pg2>

Lu: They also show improvement in pixel space by more than twenty percent over comparable prior methods, reaching an FID of five point five three at two hundred epochs for the L model <ref:2610.11751#pg2>

Meng: It really shows the benefit across both latent and pixel spaces, which is something they emphasize on page one of this paper #pg1 <ref:2610.11751#pg3>

Tom: The principle they’re pushing here is straightforward: do not denoise what you can predict, meaning the pretrained representation isn't an independent thing that needs its own trajectory; it's a property of the image that the model can infer as it emerges <ref:2610.11751#pg3>

Jane: It’s about inferring that property as the image is evolving, which seems like a much more natural way for a generative model to learn things than forcing it through a rigid schedule <ref:2610.11751#pg3>

Lu: They also show capabilities for inference and inversion with this method. At inference time, they replicate training by rolling out that representation prediction across the denoising steps, providing a continuously updated guide <ref:2610.11751#pg3>

Meng: And they even suggest that the predicted DINOv2 from the representation-conditioning branch can act as a standalone condition to test its utility, or even reconstruct an image given features and a null class <ref:2610.11751#pg3>

Tom: So to wrap up on "Dino Forcing Flow Models: Do not denoise what you can predict," the main point is that predicting the representation directly and using it as self-conditioning removes that second ODE and those representation-specific schedules entirely <ref:2610.11751#pg1>

Jane: This results in substantially faster convergence and better generation quality measured by FID score, with results showing improvements over prior methods in both latent and pixel spaces <ref:2610.11751#pg3>

Lu: It really supports the idea that we can simplify the pipeline by letting the evolving sample guide itself through predictions instead of forcing it into a rigid schedule <ref:2610.11751#pg3>

Meng: For practical engineering, cutting out those extra trajectories means less bookkeeping and fewer dependencies to manage when you're trying to build something scalable <ref:2610.11751#pg3>

Lalam: From a cultural perspective, this suggests that models can become more intuitive learners when they don't have to be explicitly told every small detail about the representation during the entire generation process <ref:2610.11751#pg3>

Jane: And they showed that this works even when you don't have a ton of extra supervision for the alignment part, which is pretty neat since those auxiliary representations are usually tricky to get right <ref:2610.11751#pg3>

Lu: They also showed inference and inversion capabilities with this method. At inference time, they replicate training by rolling out that representation prediction across the denoising steps <ref:2610.11751#pg3>

Tom: So it’s not just about making things faster; it’s about making them fundamentally more robust because of how they learn to predict things on the fly <ref:2610.11751#pg3>

Jane: They give us a much clearer path forward for making flow matching models both efficient and high quality without needing those heavy, second-pass denoising steps <ref:2610.11751#pg3>

Lu: So we're looking at how much simpler we can make the training loop by leaning into what the model can already predict about itself in "Dino Forcing Flow Models: Do not denoise what you can predict" <ref:2610.11751#pg3>

Tom: Definitely. Next up, let’s see how other teams are tackling those multimodal large language models as image classifiers.

The paper's summary: Tom: So we’ve seen how DINO Forcing Flow Models tackle that tricky co-denoising problem by just predicting the representation they need and using that as their own guide during generation. Jane Right, and what this really means is they’re cutting out a whole second denoising process and those extra schedules you have to design yourself.

Lu: They’re saying the pretrained representation isn't some separate thing that needs its own trajectory; it’s just a property of the image that the model can learn as it builds. Meng So, instead of forcing the model through two different denoising paths, they let it use what it already knows about itself to guide itself along a single path.

Tom: That’s the core idea, right? It’s about not denosing something when you can predict what you need to denoise it with. Jane And they show that this works even without needing extra supervision for the alignment part, which is pretty neat since those auxiliary representations are usually tricky to get right.

Lu: They also showed that at inference time, you just keep rolling out those predictions step by step, so you always have a continuously updated guide guiding the process. Meng And they even suggest that the predicted DINOv2 from the representation part can be used on its own to test how useful it is, or even reconstruct an image if you give it some features and a null class.

Tom: It’s not just about making things faster; it’s about making the whole thing more robust because of how they learn to predict these representations on the fly during generation. Jane They give us a much clearer path forward for building flow matching models that are both efficient and high quality without needing those heavy, second-pass denoising steps.

Lu: So we're looking at how much simpler we can make the training loop by leaning into what the model can already predict about itself in DINO Forcing Flow Models: Do not denoise what you can predict.

The paper's improvements: Tom: So we’re looking at how DINO Forcing Flow Models suggest that auxiliary representations don't always need to be generated alongside the main image, but can just be inferred as the image emerges during generation. Jane That means the model doesn't have to worry about generating a whole new set of features separately if it can just predict what it needs right then and there.

Lu: They showed that this conditioning mechanism works well even without needing external supervision for the alignment part, which is really cool because those auxiliary representations are usually tough to get right. Meng That’s a practical win for training because it cuts down on the overhead of adding more complex loss terms and schedules you have to tune.

Tom: And they emphasize that you can get those benefits just from the content of the pretrained representation itself, not necessarily by generating a whole new one. Jane It’s about inferring it as it emerges, which seems like a much more natural way for a generative model to learn things.

Lu: Plus, they’ve shown inference and inversion capabilities too. At inference time, you just keep rolling out those prediction steps across the denoising process so you always have that updated guide. Meng And they even suggest using the predicted DINOv2 from that conditioning branch as a standalone condition to test its utility or even reconstruct an image given some features and a null class.

Tom: So it’s not just about making things faster; it’s about making the whole system more robust because of how they learn to predict these representations on the fly. Jane They give us a much clearer path forward for building flow matching models that are both efficient and high quality without needing those heavy, second-pass denoising steps.

Conclusion: Tom: So we just finished our deep dive into "Dino Forcing Flow Models: Do not denoise what you can predict." Basically, they figured out how to stop wasting time on denoising representations when the model can just use its own predictions as guidance during generation. Jane Right. They’re cutting out that second ODE and those separate schedules entirely, which should make training much smoother for flow matching models across both latent and pixel spaces.

Lu: It really supports the idea that we can simplify the pipeline by letting the evolving sample guide itself through predictions instead of forcing it into a rigid schedule. Meng For engineering purposes, cutting out those extra trajectories means less bookkeeping and fewer dependencies to manage when you try to build something scalable.

Tom: That’s what it is. It’s about the model being smarter by watching what it’s already creating, rather than relying on external guidance for every single step in the process. Jane And they showed that this works even when you don't have a ton of extra supervision for the alignment part, which is pretty neat since those auxiliary representations are usually tricky to get right.

Lu: They also showed inference and inversion capabilities. At inference time, you just replicate training by rolling out that representation prediction across the denoising steps. Meng And they even suggest using the predicted DINOv2 from that representation-conditioning branch as a standalone condition to test its utility or even reconstruct an image given some features and a null class.

Jane: They give us a much clearer path forward for making flow matching models both efficient and high quality without needing those heavy, second-pass denoising steps. Tom So we're looking at how much simpler we can make the training loop by leaning into what the model can already predict about itself in DINO Forcing Flow Models: Do not denoise what you can predict.

Lu: Definitely. Next up, let’s see how other teams are tackling those multimodal large language models as image classifiers.

More episodes

← Home