BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling

summary

The gist

The gist The proposed method formulates constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling, leveraging Feynman–Kac sampling to

In short

The method treats constrained trajectory generation for diffusion and flow-matching policies as Bayesian posterior sampling. It uses a learned demonstration distribution as a prior and incorporates inference-time costs to tilt the sampling toward feasible, optimal trajectories. By leveraging Feynman–Kac sampling, the authors create a unified, retraining-free sampler for both diffusion and flow policies.

Key concepts

Bayesian Posterior Sampling: This approach frames trajectory generation as finding the most likely path given prior knowledge. The learned expert behavior acts as the starting distribution (prior), and a cost function derived from the task constraints modifies this distribution to favor safe and optimal paths during inference.
Feynman–Kac Corrector Framework: Originally used for diffusion models, this framework is adapted here to sample from the posterior. It allows for sampling trajectories by solving a related stochastic differential equation (SDE), effectively using the cost function as a weighting mechanism to guide the trajectory towards desired outcomes.
Cost-Tilted Likelihood: This component modifies the probability distribution of potential trajectories. Instead of relying solely on what the model learned, this likelihood term incorporates a cost derived from task objectives. This tilts the sampling process away from infeasible or suboptimal paths and towards those that minimize the defined cost.
Unified Sampler: The core innovation is creating a single sampling mechanism applicable to both diffusion policies and flow-matching policies. This is achieved by recasting the deterministic flow ODE into an equivalent stochastic process, allowing one method to serve both policy types without requiring separate training or inference procedures.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Federated Learning — explained in 16 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Sample Complexity — explained in 15 episodes
Generalization — explained in 13 episodes

This episode discusses

The paper

BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling · Read on arXiv

Sreevardhan Sirigiri, Weiming Zhi, Fabio Ramos

School of Computer Science, The University of Sydney · Australian Centre for Robotics, The University of Sydney · NVIDIA

Robots must generate trajectories that remain faithful to learned expert behavior while satisfying safety constraints and task-specific objectives specified only at inference time. We formulate constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling, with the learned demonstration distribution as a prior and an inference-time, cost-derived likelihood tilting it toward feasible, optimal trajectories. To sample from this posterior without any retraining of the base policy, we leverage the Feynman--Kac corrector framework, originally formulated for diffusion models, and extend it to deterministic flow-matching policies. The result is a unified, inference-time, retraining-free sampler for diffusion and flow policies. We validate the approach on pretrained Diffusion Policy, GR00T-N1.6, and π 0.5 checkpoints across simulated and real-world manipulation tasks, including planning around non-convex obstacles introduced at inference time, and show improvements over the base π 0.5 on zero-shot tasks.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling".

Dev: The gist The proposed method formulates constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So, we've talked about how this BayesFP method uses Bayesian posterior sampling combined with Feynman-Kac sampling to create a unified sampler for both diffusion and flow policies. We saw that this approach leverages the learned demonstration distribution as a prior and tilts it using a cost function derived from our constraints.

Dev: It seems like the main implication is removing that dependency on external safety filters or planners when we need to inject task-specific requirements during robot operation. It’s about steering the model's output directly toward feasibility and optimality in real time.

Taro: For autonomy research, this means we can deploy these powerful generative models in scenarios with strict, dynamic constraints that weren't explicitly trained into the policy itself. That makes a big difference for deployment outside of perfectly controlled lab settings.

Rosa: The authors are pointing to the fact that this unified sampler handles non-convex obstacles uniformly and they provide a concentration guarantee showing that the samples land near the constrained optimum when we use sufficiently high penalty weights and inverse temperature.

Dev: So, for the engineers on our side, it suggests a way to get high-quality, constrained paths from these policies without adding significant computational overhead or requiring complex re-planning loops during operation.

Taro: It's not that the AI is suddenly perfect, but it gives us a much more principled way to constrain what we know when we deploy these models in the physical world.

Rosa: That's it for BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling. This framework offers a way to sample trajectories that are faithful to learned behavior while strictly adhering to inference-time constraints and objectives.

Conclusion: Rosa: So, we're wrapping up our look at BayesFP by talking about what this paper actually is and who came up with it, Dev?

Dev: It’s titled "BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling," and the authors are focusing on making constrained path generation work for diffusion models and flow matching.

Rosa: It seems like the core idea is taking those pre-trained policies—the ones we already have—and treating them like a prior distribution, then using cost functions to tilt that distribution toward what we need at inference time.

Dev: Exactly. The method uses Bayesian posterior sampling, where the learned demonstration data is your starting point, and the cost function handles the safety and task goals you’re imposing on it while you’re running it live.

Rosa: So, if I'm driving a robot in a warehouse and suddenly I need to avoid an unexpected obstacle, this method lets me ask for a path that respects that avoidance without having to re-train the entire policy first.

Dev: That’s the point. The Feynman-Kac framework helps them sample from that tilted distribution using a weighted process, which they adapted for these flow policies by just changing how they look at the mathematical structure of the flow equations.

Rosa: It sounds like a way to bridge the gap between what an AI learned in training and what it needs to actually *do* when it’s faced with new, real-time constraints.

Dev: It gives us a unified sampler that works for both diffusion and flow models, which is pretty neat because they’re different types of generative models.

Rosa: And the results they showed on things like GR00T-N1 point six suggest this isn't just theory; it actually holds up when you test it on real manipulation tasks, including those tricky non-convex obstacles <ref:2606.21014#pg1>.

Dev: The paper shows that with enough penalty weight and temperature, the samples really do concentrate around the feasible and optimal solutions they were aiming for.

Rosa: It’s a significant step because it provides a principled way to constrain these powerful policies without having to manually design complex safety layers on top of them later.

Dev: It moves the problem from something you try to fix after training, to something you bake into the sampling process itself.

Rosa: So this paper gives us a clear path for how we can deploy generative models in environments where strict, dynamic rules are necessary for physical interaction.

More episodes

← Home