VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models

summary

Video file (mp4)

The gist

The gist: Flow-based Vision-Language-Action (VLA) policies are inherently stochastic, and this paper introduces VeriVLA, an adaptive test-time verification framework that uses a training-free

In short

Flow-based Vision-Language-Action (VLA) policies are inherently random during testing, leading to potentially harmful suboptimal actions in fine manipulation tasks. VeriVLA introduces a training-free two-axis trigger to detect critical states based on motion variation and sample variance. When triggered, a JEPA latent world model ranks action chunks by predicting future latent states and task progress, offering accurate verification without pixel reconstruction.

Key concepts

Two-Axis Trigger
This mechanism detects critical moments in the action sequence using two signals: temporal variation within a chunk (how much actions change from one to the next) and cross-sample variance (how different different samples are). It activates when these signals exceed set thresholds, signaling a state where an action is likely important or risky.
JEPA-based Latent World Model
This model operates in a compressed latent space instead of pixels. It uses a JointEmbedding Predictive Architecture to predict future states based on the current action and task goal. This allows the verifier to rank candidate actions efficiently by predicting how much progress each action will make without needing a full image reconstruction.
Delta Progress Prediction
This is the core function of the verifier, which estimates exactly how much a specific sub-chunk of an action advances the overall task goal. It uses a learnable query token to condition this prediction, allowing it to assess action quality by estimating future task completion rather than just predicting where the agent will be physically.
Flow-Based VLA Policies
These are generative AI policies that map visual observations and language instructions directly to sequences of actions. Because they rely on noise during inference, they produce different results for the same input multiple times, making them stochastic and requiring a verification method to ensure safety.

Terminology used across episodes

This episode discusses

The paper

VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models · Read on arXiv

Xueli Sun, Yuxiang Lu, Yantong Liu, Xianzhe Fan, Xi Chen, Zhe Liu†, Hengshuang Zhao*

The University of Hong Kong

Flow-based Vision-Language-Action (VLA) policies are inherently stochastic: different noise seeds can produce action chunks of varying quality from the same observation. At test time, however, the policy typically commits to a single sample without an explicit estimate of its quality. In fine-grained manipulation, executing a suboptimal action can irreversibly disrupt task progress and lead to failure. To exploit policy stochasticity, we introduce a training-free two-axis trigger that uses two initial samples to measure within-chunk temporal variation and cross-sample variance, detecting the critical states. Once triggered, a JEPA-based latent world model ranks candidate action chunks by predicting action-conditioned future latent states and the corresponding changes in task progress. By operating in latent space, the verifier avoids pixel reconstruction and enables efficient and more accurate ranking. Across three real-world tasks, i.e., pen insertion, pipetting, and apple placement), our method raises the average completion score from 50.8% to 65.5%, while adding 326 ms for sampling and verification per triggered decision on a single RTX 4090. Consistent gains are also observed in simulation, where the average success rate rises from 66.4% to 73.6% in the clean setting of RoboTwin 2.0, from 62.8% to 69.6% in its randomized setting, and from 96.9% to 97.6% on LIBERO.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: Today's paper: "VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models".

Dev: The gist: Flow-based Vision-Language-Action (VLA) policies are inherently stochastic, and this paper introduces VeriVLA,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So what we just talked about is VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models. The core idea is using a training-free two-axis trigger to spot when a sample quality might be critical, and then using a JEPA based latent world model to rank those samples by predicting future states and task progress >

Dev: The thesis is that this method exploits the inherent stochasticity of flow policies by only performing the expensive verification step when it's actually likely to be useful, which saves computation time compared to checking every single sample >

Taro: It matters because in fine-grained manipulation, executing a bad action can irreversibly disrupt task progress, so having a way to verify that potential failure is important for real deployment >

Rosa: They introduce the two-axis trigger which uses within-chunk temporal variation and cross-sample variance to detect critical states where the policy's output is highly diverse >

Dev: Then, when triggered, the JEPA based latent world model ranks those action chunks by predicting action-conditioned future latent states and the corresponding changes in task progress, which is done in latent space instead of pixels >

Taro: This means they avoid pixel reconstruction and get a more accurate ranking because they are operating in that lower dimensional latent space >

Rosa: The results show consistent gains across real tasks like pen insertion, pipetting, and apple placement where completion scores improved from fifty point eight percent to sixty-five point five percent >

Dev: And in simulation, the success rates improved significantly from sixty-six point four percent to seventy-three point six percent in the clean RoboTwin two point zero setting > <ref:2610.21985#pg1>

Conclusion: Rosa: So looking at VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models, the authors are essentially proposing a way to adaptively verify samples from these stochastic policies without needing a goal image >

Dev: It’s about using that training-free two-axis trigger to decide when to spend the time and compute power on ranking those action chunks using the latent world model >

Taro: The implication is that you can improve frozen generative VLAs at deployment by making them smarter about when they need verification based on internal motion patterns >

Rosa: They’ve shown that this approach gives consistent gains across three real-world tasks and simulation environments, moving scores up significantly >

Dev: The paper confirms that using latent dynamics modeling and pretraining provides the benefits, especially because without that dense world modeling objective, the model can't learn the necessary task progress information >

More episodes

← Home