Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought

summary

The gist

The gist The Numerical Visual Chain-of-Thought (NV-CoT) framework enables Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete

In short

Numerical Visual Chain-of-Thought (NV-CoT) allows Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete tokens. This framework expands the model's action space to a Euclidean space, enabling direct generation of bounding box coordinates as actions. This significantly improves localization precision and final answer accuracy across both supervised fine-tuning and reinforcement learning settings.

Key concepts

Continuous Action Space: Instead of choosing from a fixed list of text tokens, NV-CoT treats the output as a continuous set of numerical values representing coordinates (like [x1, y1, x2, y2]). This allows the model to select any precise location within an image space. This contrasts with old methods that used discrete patches or textified coordinates.
Gaussian/Laplace Policy: For reinforcement learning, NV-CoT replaces categorical policies with a Gaussian policy over the continuous coordinate space. This policy models both the mean and standard deviation of the bounding box distribution. Stochasticity is introduced using reparameterized sampling to allow for varied, precise spatial predictions.
Regression Loss (SFT): During supervised fine-tuning, NV-CoT uses a regression loss instead of token cross-entropy. This loss supervises the model to predict the actual continuous bounding box coordinates by minimizing the difference between its predicted mean and the ground truth bounding box.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes

This episode discusses

The paper

Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought · Read on arXiv

Kesen Zhao, Beier Zhu, Junbao Zhou, Xingyu Zhu, Zhongqi Yue, Hanwang Zhang

Nanyang Technological University · Chalmers University of Technology and University of Gothenburg · University of Science and Technology of China

Recent MLLMs increasingly rely on visual chain-of-thought to perform region-grounded reasoning over images. However, existing approaches ground regions via either textified coordinates (causing modality mismatch and semantic fragmentation) or fixed-granularity patches that both limit precise region selection and often require non-trivial architectural changes. In this paper, we propose Numerical Visual Chain-of-Thought (NV-CoT), a framework that enables MLLMs to reason over images using continuous numerical coordinates. NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space, allowing models to directly generate bounding-box coordinates as actions with only minimal architectural modification. The framework supports both supervised fine-tuning and reinforcement learning. In particular, we replace categorical token policies with a Gaussian (or Laplace) policy over coordinates and introduce stochasticity via reparameterized sampling, making NV-CoT fully compatible with GRPO-style policy optimization. Extensive experiments on three benchmarks against 11 representative visual reasoning baselines demonstrate that NV-CoT significantly improves localization precision and final answer accuracy, while also accelerating training convergence, validating the effectiveness of continuous-action visual reasoning in MLLMs. Project page: https://kesenzhao.github.io/NV-CoT/

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Thinking with Images as Continuous Actions".

Jane: The gist The Numerical Visual Chain-of-Thought (NV-CoT) framework enables Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete tokens,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we've covered the high level idea of NV-CoT, focusing on how it moves from discrete tokens to continuous numerical coordinates for region grounding. The core claim is that this approach overcomes the limitations of textified coordinates, which cause semantic fragmentation, and fixed patches, which limit selection.

Jane: Exactly. The thesis is that by expanding the MLLM action space into a continuous Euclidean space and allowing the model to directly generate bounding-box coordinates as actions with minimal architectural change, we get more flexible and precise localization in the visual world >

Lu: What they claim is that this expansion allows models to reason over images using these continuous numerical coordinates, which is fundamentally different from previous methods like text-based approaches that use tokens like "four eight" or patch-based approaches constrained by fixed spatial granularity > <ref:2602.23959#pg2>

Meng: So, the key difference seems to be that NV-CoT allows for flexible and precise localization because it’s predicting continuous values instead of discrete text tokens, which is a big deal for accuracy.

Lalam: And they support this with two training paradigms: supervised fine-tuning and reinforcement learning, showing it works across different optimization settings >

Tom: They detail how they implement this in SFT by using a regression loss to supervise the continuous bounding-box coordinates against the ground truth, defining it as LSFT(theta) equals X t∈TV minus log pi theta(ot o<t, q, I0) plus lambda X t∈TA mu t minus b* t squared >

Jane: That regression loss is crucial because it directly supervises the continuous nature of the output, contrasting with standard cross-entropy losses that ignore geometric proximity >

Lu: For reinforcement learning, they adapt it by replacing categorical policies with a Gaussian policy over coordinates and introducing stochasticity via reparameterized sampling to model this continuous coordinate regression >

Meng: So in RL, they are using a Gaussian policy where the model predicts both the mean and the standard deviation of the bounding box distribution and then samples from that distribution for its actions >

Lalam: And they use analytic importance ratios based on Gaussian likelihoods for GRPO-style RL to compute these importance ratios, which makes continuous localization compatible with mainstream reinforcement learning algorithms >

Tom: They also introduce a Laplace policy variant, using the Laplace distribution and an L1 loss in supervised fine-tuning to encourage sharper and more robust localization predictions >

Jane: So it's not just one method; they are exploring different ways to model this continuous action space, which includes Gaussian policies, Laplace policies, and specific loss functions for both SFT and RL >

Lu: The results on three benchmarks against eight baseline models demonstrate that NV-CoT significantly improves localization accuracy, final answer accuracy, and convergence speed across the board >

Meng: That's a solid set of results. It shows that this continuous action approach is indeed better than the methods they compared against in terms of how accurate the localization is and how fast it learns >

Lalam: And qualitatively, visualizations show NV-CoT consistently generates tighter bounding boxes compared to the backbone model, proving that modeling coordinates as continuous actions captures spatial structure more effectively >

Conclusion: Tom: So looking at "Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought," it seems like the big move here is ditching the text tokens for continuous numerical coordinates for region grounding >

Jane: It really is. The authors are showing that this numerical approach, whether trained with supervised fine-tuning or reinforced with reinforcement learning, leads to better localization precision and final answer accuracy >

Lu: What this means in practice is that we can build MLLMs that are much better at tasks requiring precise spatial reasoning over images than the current state-of-the-art methods allow >

Meng: For deployment, it implies we can move towards systems where region grounding is inherently more robust because they aren't relying on brittle text representations or fixed grid structures >

Lalam: It’s a plug-and-play framework that mitigates the modality mismatch and semantic fragmentation issues inherent in text-based visual chain-of-thought methods while also avoiding the limitations of fixed patch designs >

Tom: So, it's not just a tweak; it's a fundamental change in how we teach these models to interact with images spatially >

Jane: And they’re showing that iterative refinement, like allowing multiple zoom-in tool calls, actually helps the model perform better when thinking with images >

Lu: So the implication is that for visual question answering or optical character recognition, we can expect much more accurate and efficient region grounding because the model learns to reason in this continuous numerical space >

Meng: It points towards a future where vision models can be trained to be inherently better at spatial tasks if we give them the right continuous action space to explore >

Lalam: This work is about advancing multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding >

More episodes

← Home