Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure".

Jane: The gist The integrated diagnosis-and-search workflow characterizes the integrated diagnosis-and-search workflow; it does not isolate the causal contributions of difficulty classification, policy routing, and post-cleanup BO.

Tom: First, who's behind it and why it matters.

Title and authors: Jane: So this paper is called "Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure," and the authors are from Texas A andM University. Would someone say that out loud?

Tom: Yeah, it’s a long title, but the main idea is that they’re not trying to invent new test goals or assertions; they are sticking to an existing coverage plan and closing those gaps with targeted search.

Lu: They start by building a common initial coverage database from unmodified CRV regressions, then they extract the residual bins—the targets that are still unhit—and then they map each target to executable testbench controls using AST/COI analysis.

Meng: So they take the residual bins you already have and use that structural analysis to find exactly which knobs can influence them.

Lalam: And after that, they compare continued CRV, reduced-space random search, and reduced-space Bayesian optimization under the same twenty simulation budget.

Jane: That’s a very concrete comparison because it shows how this new workflow stacks up against existing techniques when you only have a fixed budget.

Tom: They also remove targets only if formal verification provides complete unreachability proofs, which acts as a powerful filter before the next stage of optimization begins.

Lu: After that, they classify the targets they keep into groups based on whether they are constraint-heavy or temporal-heavy, and then run a group-specific Bayesian optimization policy.

The paper's summary: Jane: So what are the specific improvements they suggest? It seems like their main enhancement is using formal verification as a targeted diagnostic oracle inside the closure loop.

Tom: That’s right, they use formal verification for two roles: first, reachability triage, where it asks if a target is provably unreachable under your current design and environment assumptions before you spend simulation budget on it.

Lu: And the second role is difficulty diagnosis, where for reachable hard cases, formal artifacts like witness structure or enabling-condition patterns give clues about whether the blocker is a narrow conjunction of value constraints or a long sequential dependency.

Meng: So instead of just guessing which part of your design is causing the issue, you get formal hints on whether it’s a simple constraint problem or something more complex that needs a lot of time.

Lalam: They then assign difficulty scores based on temporal and constraint knobs to label targets as TemporalHeavy or ConstraintHeavy.

Jane: This leads to their difficulty-conditioned Bayesian optimization acquisition function, where the term Bdiff (x) is set to zero for Constraint-heavy and inconclusive targets, but it’s active for TemporalHeavy ones.

Tom: And they use that term to condition the UCB acquisition function by adding a difficulty-conditioned term Bdiff (x) which is min(zero point three, 0.08nt (x)) only for TemporalHeavy targets and zero otherwise.

Lu: The objective function being optimized is J(x, b) equals one point zero H plus zero point three S plus zero point two F minus zero point zero naught one C minus Iinvalid where F is set based on formal analysis.

Meng: That objective function seems designed to heavily reward targets that can be hit by simulation or are formally reachable quickly, which makes sense for a production environment where you want high-quality results efficiently.

The paper's improvements: Jane: So to wrap up the paper "Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure," they show that this workflow improves last-mile closure on two OpenTitan IPs while preserving their mature UVM environments.

Tom: The results show that in those audited post-cleanup runs, the twenty newly covered bins comprised three direct and seventeen collateral hits for AES, and ninety-three Entropy Source bins comprised seventeen direct and seventy-six collateral hits for that IP.

Lu: And the final raw coverage figures are two hundred sixty-one/two hundred seventy-one forty-nine point three one percent for AES and two thousand two hundred ten/two thousand two hundred seventy-seven forty-nine point zero six percent for the Entropy Source.

Meng: And that translates to ninety-seven point zero three percent effective coverage, which is a real win when you're working with a fixed simulation budget.

Lalam: What this means for us is that this diagnosis-and-search workflow is a systematic way to improve closure on these specific chips without needing to run massive, unconstrained BO runs over the full unreduced space.

Jane: It really shows that integrating formal verification into the loop gives you better last-mile closure on these OpenTitan IPs while keeping those mature UVM environments intact.

Tom: Yeah, it's a structured approach that moves from broad mapping to focused optimization based on what's formally possible.

Lu: I think the key part is using that linkage map derived from AST parsing and cone of influence tracing instead of treating the whole knob space as a black box.

Meng: Because every simulation run costs time, reducing the number of knobs you search over makes a huge difference in terms of deployment cost for us.

Lalam: So they're not inventing new testbench control logic; they're just using existing ones to find the path to the target condition.

Conclusion: Tom: So we've just talked about how this workflow uses formal verification to guide Bayesian optimization to close coverage gaps on OpenTitan IPs without rewriting tests.

Jane: Exactly, and the results show a solid improvement in last-mile closure on those chips while keeping the existing UVM environments intact.

Lu: The final raw coverage figures are two hundred sixty-one/two hundred seventy-one forty-nine point three one percent for AES and two thousand two hundred ten/two thousand two hundred seventy-seven forty-nine point zero six percent for the Entropy Source.