The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

arXiv:2605.09883 · cs.CV, cs.AI · Submitted 2026-05-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "The Cartesian Shortcut".

Tom: This research identifies and empirically validates a pervasive vulnerability in current Multimodal Large Language Models (MLLMs) known as the "Cartesian Shortcut." This shortcut occurs because models systematically exploit orthogonal grid-based…

Jane: First, who's behind it and why it matters.

Title and authors: Tom: We're now looking at the title and the folks who put this paper together; it’s "The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space," written by Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, and Zhicheng Wang. It's a lot of names from top labs at Google and Stanford.

Jane: It’s interesting to see such a broad collaboration on this topic; it suggests this isn't just one lab’s idea but something that has gained significant attention across the AI research community. The authors are clearly aiming to address a widespread issue rather than just proving a single capability.

Lu: Their inclusion of figures from various domains, like Sudoku and path-finding tasks in the Polaris-Bench setup, shows they've built a very comprehensive diagnostic tool to test this hypothesis across many different types of spatial reasoning.

Meng: I wonder if having so many top researchers on this project helps ensure the methodology is rigorous enough to catch these kinds of subtle shortcuts that might be missed by smaller teams. It adds a layer of validation we engineers need when building things that are supposed to be reliable.

Lalam: Having established figures from such respected institutions lends a lot of weight to their findings; it makes us trust the diagnosis that the Cartesian Shortcut is indeed pervasive across different model families, not just one specific architecture.

The paper's summary: Tom: So, what they’re saying in the summary of "The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space" is pretty straightforward: they found that models often exploit grid-based layouts by turning them into explicit text coordinates to help them solve visual problems. This means the model is using text deduction instead of actually seeing the spatial relationships.

Jane: That’s a really helpful way to put it; essentially, the shortcut allows these models to 'offload' their complex visual logic onto simple text instructions, which confounds how we measure their actual vision capabilities on those grid-based tests.

Lu: They go into detail that they analyzed over three thousand eight hundred synthetic questions across nine major benchmarks and found that in over fifty-six percent of those cases, frontier models explicitly write out coordinates like "starting at (twenty-three), moving to (thirty-five)" in their Chain-of-Thought reasoning.

Meng: That statistic is pretty compelling; if more than half the time they are doing this text conversion instead of thinking spatially, it tells us the reliance on that shortcut is quite strong in current state-of-the-art systems. It’s a clear indicator of where their actual understanding might be weakest.

Lalam: What this means for our culture is that we need to start questioning *how* these models solve things, not just *if* they get the right answer on a standard test; it forces us to look at the internal process itself.

The paper's improvements: Tom: Moving into what they suggest as improvements, the authors introduce Polaris-Bench, which is their controlled diagnostic testbed. It’s not just one test; they reformulate fifty-three visual reasoning tasks in Polar coordinate space and pair them with a Cartesian version to keep the logic consistent.

Jane: The core improvement here seems to be this deliberate disruption of the orthogonal structure that models usually exploit, which is what they call breaking that prior. They are forcing the model to reason in a way that doesn't rely on those text-based shortcuts anymore.

Lu: The methodology involves a four-stage pipeline, starting with task curation from existing benchmarks like MMMU Pro and then ensuring logical alignment between the Cartesian and Polar tasks before generating examples to make sure everything is solvable.

Meng: That level of careful construction—the cross-topology alignment—is where the engineering rigor really shines; they aren't just throwing random tasks at it; they are systematically designing a scenario where the shortcut should fail if it's present.

Lalam: I think this structured approach to creating the testbed is what makes their results so powerful because it’s not just a random comparison; it’s a controlled experiment designed specifically to expose this deficiency.

Conclusion: Tom: So, wrapping up on "The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space," the main conclusion is that current Multimodal Large Language Models fundamentally lack topology-invariant visual reasoning capability. They can't reason reliably across different spatial structures, and this isn't just a minor weakness.

Jane: Exactly; the paper shows that the performance drop is consistent across different model families and scales, even when the logical equivalence between Cartesian and Polar layouts is perfectly maintained, which really confirms that coordinate transformation alone suffices to disrupt their reasoning ability.

Lu: This finding points toward a necessary shift in developing vision systems where generalization across diverse topological structures becomes the main goal instead of just optimizing for standard grid performance.

Meng: From an engineering standpoint, this suggests we need to move beyond simply training on standard datasets and start designing evaluation suites like Polaris-Bench that actively probe for these kinds of structural dependencies.

Lalam: This work is incredibly important because it establishes a clear target for future development: achieving visual reasoning that is truly layout-agnostic, which is a much more robust form of intelligence.

Tom: It’s a lot to digest, but the message is clear: we need systems that see the world spatially without getting trapped by how we draw the grid on top of it. That's what this paper delivers.

Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang

Google DeepMind · Stanford University

cs.CV, cs.AI

Submitted: 2026-05-11

Updated: 2026-09-30

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: This research identifies and empirically validates a pervasive vulnerability in current Multimodal Large Language Models (MLLMs) known as the "Cartesian Shortcut." This shortcut occurs because models

Key concepts

Cartesian Shortcut: This is a vulnerability where large language models systematically convert grid-based images into explicit text coordinates, such as (2,3). This allows the model to bypass complex visual understanding by offloading the logic onto simpler text-based deduction.
Polaris-Bench: A controlled testbed designed to expose the shortcut. It reformulates 53 visual reasoning tasks into Polar coordinate space while keeping a Cartesian version as a reference. This systematically disrupts the orthogonal structure models rely on for their shortcuts.
Topology-Invariant Visual Reasoning: The ability of a model to reason visually regardless of how the input is structured (e.g., grid vs. polar). The study found that current MLLMs severely lack this capability, as their reasoning gains are dependent on the specific orthogonal structure they are given.
Cartesian-to-Polar Transformation: The process of mathematically converting visual problems from a standard Cartesian grid format into a Polar coordinate system. This transformation is used in Polaris-Bench to test if models can maintain logical equivalence when the underlying spatial structure changes.

Terminology

Accuracy — 83× in this paper · explained in 1 episode
Benchmark — 48× in this paper · explained in 2 episodes
Alignment — 24× in this paper · explained in 1 episode
Trace — 11× in this paper · explained in 2 episodes
Core — 9× in this paper · explained in 2 episodes
Calibration — 8× in this paper · explained in 7 episodes
Verification — 5× in this paper · explained in 2 episodes
In-Context Learning — 4× in this paper · explained in 4 episodes
LLM-as-a-Judge — 4× in this paper · explained in 3 episodes
Consistency — 3× in this paper · explained in 3 episodes
Chain-of-Thought (CoT) — 2× in this paper · explained in 4 episodes
Grounding — 2× in this paper · explained in 3 episodes
Hallucination — 2× in this paper · explained in 3 episodes
Multimodal Reasoning — 2× in this paper · explained in 5 episodes
Reasoning Traces — 2× in this paper · explained in 3 episodes
Robustness — 2× in this paper · explained in 5 episodes
Ablation — 1× in this paper · explained in 1 episode
Adaptation — 1× in this paper · explained in 1 episode
Distribution Shift — 1× in this paper · explained in 5 episodes
Fidelity — 1× in this paper · explained in 2 episodes
Foundation Model — 1× in this paper · explained in 3 episodes
Foundation Models — 1× in this paper · explained in 7 episodes
Generalization — 1× in this paper · explained in 13 episodes
Modality Gap — 1× in this paper · explained in 4 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Scaffolding — 1× in this paper · explained in 3 episodes
Visual Grounding — 1× in this paper · explained in 2 episodes

Summary

This research identifies and empirically validates a pervasive vulnerability in current Multimodal Large Language Models (MLLMs) known as the Cartesian Shortcut. This shortcut occurs because models systematically exploit orthogonal grid-based layouts common in visual reasoning benchmarks by converting them into explicit textual coordinates, thereby offloading visual reasoning onto text-based deduction. The paper introduces Polaris-Bench, a controlled diagnostic testbed that reformulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts to systematically disrupt this shortcut and expose a critical deficiency: the lack of topology-invariant visual reasoning capability in MLLMs.

The Cartesian Shortcut Identification

The study identifies the Cartesian Shortcut by analyzing over 3,800 synthetic-based questions across 9 prominent visual reasoning benchmarks. The analysis reveals that in over 56% of these examples, frontier models explicitly discretize the input image into Cartesian coordinates, generating text such as 'starting at (2,3), moving to (3,5)'. This behavior allows models to heavily offload visual logic onto text-based deductive reasoning, which significantly confounds the evaluation of visual reasoning on grid-based testbeds.

The Polaris-Bench Methodology

To systematically dismantle the shortcut, the authors introduce Polaris-Bench, comprising 53 procedurally generated visual reasoning tasks in Polar coordinate space, each accompanied by a Cartesian counterpart as a controlled reference under consistent logical constraints. This design leverages the Cartesian-to-Polar topological transformation to break the orthogonal structure that models exploit. The benchmark is constructed through a four-stage pipeline:

Task Curation from existing benchmarks (e.g., MMMU Pro, MathVista).
Task Design and Cross-Topology Alignment, ensuring logical alignment and visual calibration.
Example Generation with careful randomization of visual and logical axes to ensure solvability.
Multi-stage Quality Assurance involving author review, LLM-assisted code audits, iterative convergence, and a final human validation study to confirm data quality.

Empirical Results on Performance Collapse

Comprehensive evaluation across 14 state-of-the-art MLLMs reveals a striking and consistent pattern. Frontier models achieving 70–83% accuracy on Cartesian layouts collapse to 31–39% on Polar equivalents, with the largest degradation approaching 47 points. Crucially, this degradation persists even when the transformation preserves complete logical equivalence, confirming that the coordinate transformation alone suffices to disrupt model reasoning. Furthermore, while reasoning gains are substantial on Cartesian layouts, they are severely diminished on Polar equivalents, exposing a fundamental lack of topology-invariant visual reasoning capability.

Analysis of Reasoning Modes and Task Categories

The evaluation examines performance across different reasoning modes (high vs. non-reasoning) and task taxonomies (e.g., Visual Pattern Matching, Navigation and Routing). The results show that the collapse is non-uniform, with the Combinatorics & Probability category suffering the steepest degradation in accuracy. The analysis further demonstrates that while models can leverage textual discretization to assist visual reasoning, this shortcut is not a shallow heuristic that can be overridden by instruction; even when explicitly guided to convert Polar layouts back to Cartesian grids, models often fail to execute this re-mapping.

Conclusion and Implications

The findings establish that current MLLMs fundamentally lack topology-invariant visual reasoning capability. The work concludes that the Cartesian Shortcut is a systemic vulnerability rather than a model-specific limitation, as the performance collapse is consistent across diverse model families, scales, and reasoning configurations. The paper emphasizes that developing visual reasoning that generalizes reliably across diverse topological structures is essential for truly robust multimodal intelligence. Future research should focus on genuine layout-agnostic visual reasoning.

Key Findings Enumerated:

The Cartesian Shortcut is validated by observing explicit invocation of Cartesian coordinates in over 56% of intermediate Chain-of-Thought reasoning steps on grid-based tasks.
PolarisBench re-formulates 53 visual reasoning tasks in Polar coordinate space, paired with Cartesian counterparts, to create a controlled diagnostic testbed.
Frontier models show a performance collapse of up to 45 points when moving from Cartesian (70–83%) to Polar (31–39%) layouts.
The degradation persists even under conditions of complete logical equivalence, confirming the topological transformation is the primary disruptive factor.
Reasoning gains on Cartesian layouts are severely diminished on Polar equivalents, indicating that the amplification effect of reasoning is dependent on the existence of an orthogonal prior structure.
Human raters show a small accuracy reduction (e.g., 5.7 percentage points overall) from Cartesian to Polar layouts, providing a human reference point that contextualizes model evaluation results.

Improvements for AI systems

Here are specific, high-impact improvements for AI systems based on the findings of The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space:

) 1. Implementation of Topology-Invariant Reasoning Modules (TIRM)

Instead of relying solely on grid-based visual processing, integrate a module that explicitly handles coordinate system transformations (Cartesian to Polar and vice versa) as a first layer of visual processing.

The improved AI system will be able to solve spatial reasoning tasks across arbitrary topologies (grid, polar, octagonal, hexagonal) without needing explicit text-based discretization. This directly counters the Cartesian Shortcut vulnerability where models exploit orthogonal grid layouts for text-based deductions.

) 2. Cross-Topology Evaluation Framework (CTEF)

Develop a standardized evaluation suite that mandates paired testing: every visual reasoning task must be tested on both its Cartesian and Polar topological equivalents under identical logical constraints, as defined by Polaris-Bench.

The improved system will be able to provide robust performance metrics that isolate the effect of topology transformation from inherent model capacity. This allows researchers to definitively prove whether a model's high score on a standard grid is due to genuine visual understanding or shortcut learning based on coordinate discretization.

) 3. Reasoning Mode Sensitivity Tuning

Implement dynamic inference control that monitors the nature of the input layout and automatically adjusts reasoning depth based on topological complexity.

The improved system will be able to selectively apply high-reasoning Chain-of-Thought (CoT) only when the input strongly suggests a reliance on explicit coordinate mapping (e.g., Cartesian layouts), while defaulting to more perception-heavy, topology-invariant reasoning strategies for non-grid inputs. This mitigates the tendency of frontier models to offload visual logic onto text when a grid is present.

) 4. Enhanced Perceptual Grounding Calibration

Integrate a visual calibration sub-module that dynamically adjusts rendering parameters (e.g., font sizing, cell scaling near the pole) based on detected topological shifts, ensuring that performance gaps are due to reasoning difficulty rather than perceptual disadvantage caused by coordinate distortion.

The improved system will achieve higher accuracy in tasks involving complex spatial transformations and geometry (like Pipe Lengths or Area Balancing) across all topologies. It prevents the systematic degradation observed when models struggle to discretize Polar layouts into text, ensuring visual perception remains grounded regardless of coordinate system.

) 5. Automated Shortcut Detection and Mitigation

Develop a Shortcut Detector component that analyzes intermediate reasoning traces for explicit invocation of Cartesian coordinates (e.g., row 2, (x,y)). If detected in a grid context, the system triggers an alternative reasoning path that avoids textual discretization.

The improved system will possess self-correction capabilities against shortcut learning. When presented with a grid problem, it will be programmed to prioritize direct visual/geometric manipulation over generating intermediate text coordinates, thereby preserving topology-invariant reasoning capabilities and preventing performance collapse on Polar equivalents.

Abstract

As current Multimodal Large Language Models rapidly saturate canonical visual reasoning benchmarks, a key question emerges: do these strong scores genuinely reflect robust visual understanding? We identify a pervasive vulnerability, the Cartesian Shortcut: models frequently discretize the orthogonal grid-based layouts prevalent in visual reasoning benchmarks into explicit textual coordinates, offloading reasoning from visual perception to text-based deduction. To re-evaluate visual reasoning when this shortcut is unavailable, we introduce Polaris-Bench, which re-formulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts that preserve task semantics, disrupting the orthogonal structure that models exploit. Comprehensive evaluation across 14 state-of-the-art MLLMs reveals that frontier models achieving 69 -- 83% on Cartesian layouts collapse to 31 -- 39% on Polar equivalents. Moreover, thinking gains largely vanish on Polar layouts, prompting interventions fail to close the gap, and comparable drops arise on other non-orthogonal layouts. These findings reveal that current MLLMs' visual reasoning performance is strongly coupled to orthogonal grid structure, a fragility that is consistent across model families but far smaller in humans, who maintain 88.8% accuracy on Polar layouts. The benchmark, evaluation suite, and leaderboard are publicly available at https://google-deepmind.github.io/polaris-bench.