What Type of Inference is Active Inference?

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "What Type of Inference is Active Inference?".

Jane: Here's the gist of the paper. The gist The VFE of an augmented model can be rewritten as the VFE of the predictive model plus explicit entropy-correction terms, making the EFE contribution transparent and yielding a full variational characterization of EFE-based planning How it works 1.

Tom: The paper reformulates Active Inference by showing that the VFE of an augmented model can be written as the VFE of the predictive model plus explicit entropy-correction terms, which makes the EFE contribution transparent This decomposition is achieved by combining two entropy corrections: a planning correction and epistemic corrections 2.

Jane: Proper EFE-based planning requires combining two entropy corrections: the planning correction of Lázaro-Gredilla et al.

2024: , which turns the expected-utility variational objective into policy optimization, and the epistemic corrections of Nuijten et al.

Title and authors: Tom: We’re now looking at the beginning of "What Type of Inference is Active Inference?" and the authors are Wouter W. L. Nuijten, Mykola Lukashchuk, and Thijs van de Laar with Bert de Vries as supervisor.

Jane: So, what's the big takeaway from just looking at those names and that title? What’s the actual core idea they are pushing?

Lu: The title suggests they are investigating the fundamental nature of active inference itself, asking what kind of inference we are really performing when we use expected free energy.

Tom: They immediately point out that recent work showed EFE minimization can be written as Variational Free Energy minimization on a generative model augmented with epistemic priors.

Jane: That sounds like they are trying to bridge the gap between the theoretical goal of active inference and the practical implementation using generative models and uncertainty about the model itself.

Meng: If you’re thinking about running this on production systems, I need to know if this framework is still tractable when we add all these explicit entropy correction terms.

Lalam: The paper establishes that the VFE of an augmented model can be rewritten as the VFE of the predictive model plus explicit entropy-correction terms, making the EFE contribution transparent.

Tom: That transparency is key because it lets you isolate exactly what part of the planning process is handling uncertainty versus just pure prediction.

Jane: So, they are essentially showing that we can separate how much of our objective is about predicting the next step versus how much is about reducing our ignorance about the model parameters.

Lu: They then prove that proper EFE-based planning requires combining these epistemic corrections with a planning correction that turns marginal inference into policy optimization, yielding a full variational characterization of EFE-based planning.

Tom: That's the core finding: you need both the entropy corrections and the specific planning correction to get proper EFE-based planning.

Jane: It clarifies exactly which corrections are needed for cross-entropy planning versus full expected free energy based planning.

The paper's summary: Tom: Now let's look at the actual summary of "What Type of Inference is Active Inference?" and what it summarizes about the process.

Jane: They summarize that this work shows how to write the VFE as a predictive model's VFE plus those entropy correction terms we just talked about.

Lu: Specifically, they show that proper EFE-based planning requires combining those epistemic corrections with a planning correction that turns marginal inference into policy optimization.

Meng: So, the summary boils down to this: to do EFE-based planning right, you have to combine two specific types of corrections into your objective function.

Lalam: They give us the full EFE-based planning objective expression, which is given by min q Fpˆq + X T t=one 2H

q(ytxt, θ): − H

q(xtxt−one ut): − H

q(ytxt): <ref:2606.04935#pg1>.

Tom: That formula is the meat of it. It’s a combination of the planning correction and the EFE correction derived from Theorem one <ref:2606.04935#pg1>.

Jane: It looks like they’ve put everything into one equation that defines the objective for what they call EFE-based planning.

Lu: They derive this objective by adding the planning correction of Lázaro-Gredilla et al. two thousand twenty-four to the marginal-EFE corrections from Theorem one <ref:2606.04935#pg2>.

Tom: So, it’s not just a theoretical idea; they actually derived this combined objective by combining known components into a new structure.

Jane: It’s about showing that this structure is what properly captures the goal-directed and information-seeking behaviors of active inference together.

The paper's improvements: Tom: Let’s talk about the specific improvements they suggest, which are really focused on how to make this work better than just using marginal VFE minimization.

Jane: They show how epistemic priors identify the corrections that transform marginal VFE minimization into EFE minimization through three different lemmas.

Lu: Lemma four shows that the state epistemic prior contributes minus H

q(xtxt−one ut): to the objective function.

Meng: That means they are explicitly saying that uncertainty about the current state transition is a cost we need to account for in our objective.

Lalam: And Lemma five shows that the action epistemic prior contributes plus H

q(ytxt, θ): minus H

q(ytxt): to the objective.

Tom: So, they are using these specific entropy corrections to produce EFE minimization inside a marginal variational objective.

Jane: They also have Lemma six showing that the observation epistemic prior contributes plus H

q(ytxt, θ): minus H

q(ytxt): to the objective as well.

Lu: These three lemmas collectively produce EFE minimization inside a marginal variational objective, which is a significant step because it shows how to move from simple marginal VFE minimization toward true EFE minimization.

Meng: If we can achieve this, it means we are using uncertainty about what the environment will do and what the observations will tell us in a way that actually drives goal-directed behavior.

Conclusion: Tom: So to wrap up, the conclusion summarizes how this entire framework works and what it implies for planning systems.

Jane: It clarifies that the full EFE-based planning objective is formulated as a standard Bethe free energy over the modified factor graph, solved using message passing with channel reparameterization.

Lu: This leads to a construction where the four normalized conditional distributions are introduced as channels: rux(u1xzero), rxxu(xone times zero u1), ryxθ(y1xone θ), and ryx(y1xt).

Meng: So the system is jointly optimized over beliefs and these channels, which is a very specific way of doing things.

Lalam: This approach allows the VBP scheme to generalize to arbitrary time horizons by introducing a time-local channel at each time step, avoiding the min-max structure found in AIF by having only a single channel that enters only the dynamics kernel.

Tom: And they note that this VBP scheme continuously interpolates between marginal inference, which is a uniform channel, and full cross-entropy planning, which is the converged channel.

Jane: This interpolation suggests a flexible system where it can switch between simple inference and more complex planning as needed depending on the task.

Lu: They also mention that the opposing signs of the entropy corrections induce a min-max structure in the joint optimization over beliefs and channels, which in practice requires damped channel updates for stable convergence.

Meng: I’m hearing that means we need to be careful with how we update those channels if we want this to actually run reliably without it oscillating wildly.

Lalam: That damping parameter lambda also currently requires manual per-environment adjustment, which is a limitation they admit and restrict their analysis to discrete state spaces where exact factor evaluations are available.

Tom: It’s important to remember that this work provides a principled message-passing formulation of EFE-based planning.

Jane: It gives us a concrete way to build these systems based on known mathematical structures, moving away from ad hoc methods toward something more principled.