Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Grounding

arXiv:2608.07353 · cs.CL, cs.AI, cs.IR, cs.LG · Submitted 2026-08-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Grounding".

Jane: The paper was written by Karim Radouane, Jose G Moreno and Lynda Tamine from University of Toulouse and Institut de Recherche en Informatique de Toulouse.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: On today's show we're digging into a study that asks whether large language models actually understand concepts, or just pattern-match their way through questions. The authors use geography as their test case, looking at direction, distance, and topology, and they probe the models on three properties: abstraction, compositionality, and grounding.

Jane: What I find compelling is that they don't rely on existing question sets. They build their own benchmark, generating relational facts about UK wards, like "Prescot is west of Todmorden," and turning those facts into yes/no and multiple choice questions. That gives them precise control over what every question is testing.

Lu: The clever part is that each concept instance is a triplet: subject, relation, object. And the relations are computed from real geometry. Bearings produce direction, distances produce close or far, and GeoSPARQL produces topology, so the ground truth comes from coordinates, not from somebody's guess.

Meng: The headline results are genuinely mixed. Multiple choice accuracy looks respectable, with the best model above 70 percent, but consistency on yes/no questions often falls below 26 percent. That means a model can answer a question correctly and then give the opposite answer when the same fact is phrased negatively, which is not what a stable concept should look like.

Lalam: The stakes go well beyond geography. If eye systems are going to retrieve information and manage knowledge responsibly, we need to know whether their internal representations behave like concepts: generalizing to new cases, composing with each other, and connecting to real-world measurements. That's exactly what this paper tries to quantify.

Jane: And they're careful to separate task performance from concept understanding, because high accuracy could just come from surface patterns in the questions. So they layer internal probing on top, looking at the model's representations across layers, not just its final answers. That's a recurring theme in how they interpret every result.

Tom: The summary we've just given is the skeleton; the introduction fleshes out each finding with the research questions behind it. It also explains why the authors think previous probing work missed the core issue. That's the natural place for us to continue.

Page 1: Tom: So we've set the stage. Page one is the introduction, and the main contribution starts with the research gap they identify. They argue that prior work on concepts in language models fails on three counts: much of it uses multimodal models where text and images share representations, the text-only studies cover just one concept property at a time, and the probes are built around downstream tasks rather than around the concepts themselves.

Jane: That third point is the one that matters most. If you test a model with questions borrowed from some benchmark, you can't tell whether a wrong answer comes from a missing concept or from a failure of task skills like format following or reading comprehension. The authors want to isolate the concept, so they generate questions directly from the concept's instances and properties.

Lu: And they pick direction, distance, and topology because these are spatial commonsense concepts that rarely show up explicitly in text. A model probably never encounters these specific ward pairs in its training text, so it can't just memorize the fact. That makes the benchmark a genuine test of abstraction rather than recall.

Meng: They formulate four research questions to structure everything. RQ1 asks whether QA performance is even a valid proxy for concept understanding. RQ2 asks whether the models encode abstract representations that generalize across tokens and regions, RQ3 looks at compositionality, and RQ4 asks whether concepts can be grounded in real-world numerical knowledge.

Lalam: What I value here is how they turn philosophical properties into testable hypotheses. Abstraction becomes "does the same concept representation generalize to unseen instances." Compositionality becomes "does the representation of a composite concept relate to the representations of its parts." That's how you make progress in the debate about whether models have understanding.

Tom: The page closes with a preview of the results, and they're provocative. Models show moderate accuracy but shaky consistency, most encode concepts well except the Mistral family, compositionality tracks performance, and grounding fails even when all the numbers are provided. Page two then positions this work against prior probing and geography research, and it closes with a strong positioning claim.

Page 2: Jane: Page two reviews the prior work, and the authors take the definitional issues seriously. They note that "concept" means something different in cognitive science than in machine learning, but three properties keep appearing across disciplines: abstraction, compositionality, and grounding. Their entire benchmark is organized around those three.

Lu: The closest ancestors are in mechanistic interpretability. Gurnee and Tegmark showed that models store latitude and longitude in early layers and can decode a map of locations. Patel and Pavlick then showed that large models can map their linguistic representations onto a grounded conceptual space using a few examples. Both of those threads feed directly into this paper.

Meng: But the authors also point out where those studies fall short. Most rely on multimodal setups with images, so you can't tell whether grounding happens inside the language model itself. And the text-only studies tend to examine one property in isolation, so their comparison table makes the gap visible: no prior work tests all three properties in text-only models.

Lalam: There's also a useful distinction between knowledge and representation. A model can recall geographic facts, that's knowledge. But representation is about whether the internal geometry actually organizes concepts in a structured way, so this paper probes both, which is why they separate task performance from linear probing of hidden states.

Jane: On the geography side, they bring in GeoQA challenges like the vagueness of geographic concepts, the difficulty of identifying correct spatial relations, and trouble generalizing neighborhood relationships across scales. Those challenges justify their choice of UK wards and their controlled triplet generation, because they want to sidestep the messiness of real-world question corpora.

Tom: The section ends with a positioning statement that this is the first concept-centric benchmark to jointly test those three properties in text-only models. After that, the next page starts building the formal machinery to back that claim up.

Page 3: Tom: Page three pins down the terminology, and this matters because "concept" is a slippery word. The authors adopt a knowledge-representation view where a concept is an abstract category like direction, and the instances are concrete exemplars like east or west. Each instance is expressed as a relational triplet: subject, relation, object.

Jane: So "Prescot west of Todmorden" is a positive triplet, and "Prescot east of Todmorden" is its negation. The negation is essential for their consistency metric, because if a model believes the first fact, it should reject the second. Consistency measures whether the model handles both versions coherently.

Lu: They also define the three properties precisely. Abstraction means the instances of a concept form its semantic type, so a probe should classify unseen instances into the right type. Compositionality is specifically conjunctive here, logical AND, combining close and west into a composite fact. Grounding means mapping linguistic expressions to numerical meaning, like distances in kilometers and bearings in degrees.

Meng: The methodology then splits into two tracks. Task performance gives accuracy and consistency on the questions themselves. Probing performance trains linear classifiers on internal representations to see whether the concept is actually encoded, and that two-track design runs through all the experiments that follow.

Lalam: And the concept vocabulary is deliberately small: four directions, two distance values, two topology relations, plus their negations. With just those building blocks, they can construct atomic facts, pairwise compositions, and three-way compositions. Keeping the space simple is what makes the probe results interpretable.

Jane: The section closes with a table of the relations used throughout: north, south, east, west, close, far, within, borders, each paired with its negation. Those become the raw material for the benchmark, which brings us to page four and the dataset generation.

Page 4: Jane: Page four walks through the benchmark construction, and the scale is immediately impressive. They use 506 UK metropolitan district wards spread across 25 districts, and the wards form three discontinuous regions. The middle region generates the data, while the upper region is held out for out-of-distribution testing later.

Lu: The relations come from real geometry, which is the part I like most. Pairwise geodesic distances give close or far, bearings give cardinal directions, and GeoSPARQL predicates over actual ward geometries give within and borders. The direction function maps angles to compass points: east from 45 to 135 degrees, south from 135 to 225, west from 225 to 315, and north everywhere else.

Meng: The distance threshold is set to the mean of the pairwise distance distribution, which lands at 47 point 76 kilometers. Below that is close, above is far. And the threshold is not included in the questions, which lets the authors later estimate each model's own implicit notion of closeness and compare it to the dataset label.

Jane: Compositional triplets are built by combining atomic relations that share the same subject and object, so a pairwise composition might be "close and west," and the three-way version adds topology on top. Each triplet then becomes a yes/no question and a three-option multiple choice question, with negated triplets added to test consistency. In the multiple choice setting, distractors are sampled so they don't satisfy the relation, and the correct option's position is randomized.

Tom: The resulting dataset is enormous, 1 point 79 million binary questions and over 115,000 multiple choice questions. At that scale, the models can't have memorized the answers from pretraining, because these specific facts simply don't appear in their training text.

Lalam: And because the whole pipeline is algorithmic, from geometry to questions to ground truth, you could regenerate the benchmark for any region or any relation family. That reproducibility is what makes the probing methodology reusable, and the next page shows what happens when you actually run these questions through current models.

Page 5: Tom: Page five reports the raw question-answering results, and the first pattern jumps out immediately. Multiple choice accuracy beats yes/no accuracy everywhere, which is expected because the answer choices scaffold the task. The strongest model, Llama-3 point 1-8B, reaches 71 point 7 percent on multiple choice, while Qwen3-4B leads the yes/no task at 56 point 14 percent.

Jane: But accuracy on its own is misleading. Consistency on the yes/no task is usually below 26 percent, meaning when a fact is negated, the model often fails to reverse its answer. That's exactly the signature of pattern-matching rather than stable concept representation.

Lu: They also test explicit thinking prompts, chain-of-thought for the Llama and Mistral models and reasoning mode for the Qwen family, and it doesn't help. Llama's multiple choice accuracy actually falls from 71 point 7 percent to 50 point 6 percent, and consistency drops from 48 point 7 percent to 16 point 6 percent. Extra computation on top of an unstable representation just amplifies the noise.

Meng: The distance threshold analysis is the most revealing part of this section. The authors estimate each model's implicit "close" boundary by fitting density curves to the model's close and far predictions. In the yes/no task, most models treat closeness far more strictly than the dataset's 47 point 76 kilometers.

Jane: So the models think "close" means much shorter distances than the data suggests?

Meng: Exactly. Several models land around 20 kilometers, and one small Qwen sits at 7 point 5. In the multiple choice task, the revealed thresholds cluster near the dataset threshold, typically 43 to 45 kilometers, because the answer options act as anchors that pull the model toward the dataset scale. The paper flags that as a format-induced recalibration, not evidence of better spatial understanding.

Lalam: The continental scale test reinforces the point. Small Qwen models sit near chance at the US scale, with no real close/far discrimination at 1,637 kilometers, while larger models hold onto some ability. So the notion of "close" depends on scale, format, and capacity, which is not what a grounded concept should do.

Tom: The section concludes that raw QA performance is not a reliable proxy for concept understanding. That motivates the targeted probing that follows, starting on page six with the abstraction tests.

Page 6: Jane: Page six shifts from task performance to internal representations. The authors construct a dedicated probing dataset by subsampling a thousand binary and a thousand ternary compositional questions, then adding all atomic decompositions and their negations, for 14,000 question instances total. They also make sure that each composite question and its atomics share the same answer options, which is essential for clean decomposition.

Lu: The abstraction test works with a linear probe at every layer. You take the average token embedding of a question, push it through a classifier that predicts one of seven conceptual classes, like direction, distance, topology, or their combinations. If the representation actually carries the concept, a linear separator should find it.

Meng: On a random train/test split, most models are essentially perfect, 99 point 95 percent to 99 point 98 percent accuracy. The concepts are clearly present in the representations. But then the authors make it harder with out-of-distribution splits, withholding specific tokens like east and west, or entire regions, to see whether the concept generalizes beyond what the probe saw during training.

Jane: And here the Mistral family stands out, in the wrong direction. Where other models reach around 76 percent on the geographic split and 80 to 83 percent on token splits, the Mistral models collapse to 37 to 42 percent on the geographic split and 22 to 29 percent on the token splits.

Tom: So their earlier task accuracy didn't reflect a real concept at all.

Jane: Exactly. That's not a marginal deficit; it suggests those models never formed the abstract concept. And the per-layer curves reinforce it, since for Mistral the signal stays weak across depth.

Lu: For most models, the concept signal strengthens as you go deeper, which fits earlier results about concepts emerging in later layers.

Tom: And for Mistral, nothing sharpens as you go up the layers?

Lu: Right. The paper doesn't fully explain why, but the pattern is consistent enough to point at architectural or training differences.

Lalam: This reframes their earlier QA numbers. Mistral's moderate task accuracy was not backed by abstract representations. The model apparently found a way to answer the questions without forming the general concept, and that is exactly the kind of thing that task-only evaluation will always miss.

Tom: Abstraction, then, is well supported in most families and conspicuously absent in one. Which raises the question of whether compositionality behaves the same way, and page seven starts to answer that.

Page 7: Tom: Page seven takes on compositionality in stages. First they measure the compositionality gap: cases where a model answers all the atomic subquestions correctly but then fails the composite question. In multi-hop reasoning, that gap is notoriously large, but this paper finds something quite different.

Jane: For conjunctive compositions, combining "close and west" into a single question, models actually do better on the composite than on the atomics. That's the opposite of the multi-hop result. The conjunction seems to narrow down what's being asked, which makes the composed question easier rather than harder.

Lu: They define two gap metrics. CGA, compositional gap accuracy, counts composite questions that are answered wrong even though every atomic part was answered right. CGC, compositional gap consistency, adds the twist of using paired positive and negative questions, so a model that flips on negation still counts as failing even if it nails the positive wording.

Meng: The results are sobering. Smaller models show the largest gaps and the weakest consistency. Mistral-v0 point 3-7B shows the smallest binary gap at 13 percent.

Jane: Wait, that sounds like a good result for Mistral.

Meng: Except its absolute accuracy is low, meaning the atomics and composites are failing together. Qwen3-4B reaches a higher 59 point 7 percent accuracy while carrying a larger gap, which is a different kind of failure.

Jane: In the multiple choice task, Llama-3 point 1-8B posts the highest compositional accuracy at 54 point 5 percent, but it also shows a bigger gap than some small models. The authors interpret that as a trade-off: better absolute performance does not guarantee tighter alignment between atomics and composites.

Lalam: The gap metrics capture behavior at the output, but they can't tell us whether the internal geometry actually composes linearly. For that, we need to look at whether the representation of "close and west" sits close to the sum of "close" and "west" in embedding space. That's the focus of the next page.

Page 8: Jane: Page eight opens up the internal geometry. The authors compute cosine similarity between the embedding of a composite question and the sum of its atomic embeddings, layer by layer, across the whole network. If the model composes concepts in a roughly additive way, that similarity should be high and stable.

Lu: The Mistral models diverge again, showing the lowest and most variable cosine similarities, especially in later layers where other models settle into a coherent compositional structure. There's also a consistent ordering: two-concept compositions look more compositional than three-concept ones, which makes sense because more parts means more room for interference.

Meng: The prediction-side analysis is where the story gets sharp. They train logistic regression probes on frozen representations and compare three ways of combining atomics: summing logits, summing embeddings, and averaging probabilities. For Qwen and Llama models, the correlations are strong, with logits above 0 point 85 in most cases. For Mistral, logit correlations sit around 0 point 4.

Jane: And those correlation numbers track accuracy. The Qwen and Llama models reach roughly 80 percent on binary questions while Mistral hovers near 65 percent, and the multiple choice numbers separate the same way. The paper's argument is that compositionality is not a side effect; it's a central factor in whether concepts can support correct answers.

Lalam: This is the kind of evidence that moves the debate from "do models understand" to "under what conditions does the representational geometry support composition." The answer depends on model family, and identifying that dependency opens the door to studying which training choices produce compositional structure and which ones undermine it.

Tom: So compositionality holds in some families and fails in others, and the failures line up with performance. The final property, grounding, gets a very different kind of test: the paper hands the model all the numerical information and sees whether it uses it.

Page 9: Jane: Page nine is the grounding test, and the experiment is almost unfair in the model's favor. Every question comes with explicit context: coordinates, distance in kilometers, bearing in degrees, and even the threshold definition. The model is told that wards count as close at or below 47 point 76 kilometers, so if grounding happened, the questions would become trivial.

Lu: So the experiment gives the model everything it needs, in plain numbers.

Jane: Everything except the ability to use those numbers. In the yes/no setting, accuracy hovers around chance, roughly 50 percent, with low consistency. In the multiple choice setting, a few models clear the 33 percent random baseline, with Qwen3-8B performing best at 66 point 7 percent accuracy and 44 point 7 percent consistency on the distance concept, but the overall picture is weak.

Meng: The paper computes the change relative to the no-grounding condition, and the best average improvement is 1 point 93 percent in accuracy and 3 percent in consistency. That's practically nothing.

Lu: So providing exact numbers doesn't just fail to help, it might as well not be there.

Meng: Right. Supplying exact numerical facts barely moves performance, which suggests the models are not integrating the numbers with the linguistic concepts at all.

Jane: There's no systematic advantage for direction or distance over topology, even though direction and distance are precisely quantifiable. If a model understood what "west of" means numerically, the bearing information should resolve the question immediately. Instead, the behavior looks like in-context guessing over memorized patterns.

Lalam: The authors frame this as a reliance on memorized linguistic patterns rather than true numerical grounding, which aligns with earlier findings that text-only language models struggle to connect words to non-linguistic referents. The uncomfortable implication is that a model can use "close" fluently in prose while having no stable connection to physical proximity.

Tom: So grounding is the weakest of the three properties across every model family tested. That failure has concrete consequences for retrieval and knowledge management systems, which is exactly where the paper's final pages point.

Page 10: Tom: The closing pages tie the findings together and sketch the way forward. The core deliverable is the concept-centric benchmark itself, built from relational triplets generated from real geometry, with algorithmic question generation and ground truth. Because everything hangs off the triplet representation, the methodology extends to any concept expressible as relation families.

Lu: The authors explicitly name other targets, like the concept of truth, or patient gender in healthcare. If you can define the relation families and their negations, the pipeline applies unchanged. That extensibility follows directly from the clean formulation on page three.

Meng: They also list the limitations honestly. Only two geographic regions, UK and US. The chosen concepts may not capture the complexity of other real-world concepts. And the probes are restricted to linear classifiers, which can only detect linearly separable structure, so concepts encoded nonlinearly would go unseen.

Jane: On the information retrieval side, the finding that most models generalize well to out-of-distribution concept instances suggests exploring axiomatic approaches and mechanistic interpretability for ranking, where the concept of relevance itself becomes the object of study. That's a concrete research direction this paper enables.

Lalam: And the grounding failure is the strongest argument in the paper for externally grounded architectures. If language models can't anchor concepts to numbers on

Conclusion: Tom: So wrapping up this one: the paper turned spatial concepts into three testable properties, and the bottom line is that task accuracy hides more than it reveals.

Jane: Right, because the same model that scores 70 percent on multiple choice can flip its answer when you negate the fact, so you can't trust the number alone.

Tom: The most striking result for me was grounding. You hand the model exact coordinates, distances, bearings, even the threshold definition, and performance barely moves.

Jane: That's the part that should worry us. It means "close" and "west" live in the model as linguistic patterns, not as numerical relationships to the world.

Tom: And yet abstraction mostly held up. Most models generalized to unseen regions and tokens just fine, except the Mistral family, which collapsed on exactly those tests.

Jane: So the paper's real contribution is separating those three properties, because they don't move together. A model can abstract without grounding, and compositionality only explains predictions in some architectures.

Tom: That distinction matters for anyone building retrieval systems on top of LLMs. Knowing that relevance is compositional in one model but not another changes how you design the pipeline.

Jane: And their call for concept-centric benchmarks is the natural next step, because downstream tasks clearly aren't enough to tell us what a model understands.

Tom: We'll leave the paper there, but its question lingers: if models can't ground even simple spatial ideas, what does that mean for the knowledge they retrieve and rank?

Jane: And that's exactly where we're heading next, with a paper that takes a hard look at whether retrieval models actually understand relevance or just match surface patterns.

Tom: Stay with us.

Karim Radouane, Jose G Moreno, Lynda Tamine

University of Toulouse · Institut de Recherche en Informatique de Toulouse

cs.CL, cs.AI, cs.IR, cs.LG

Submitted: 2026-08-07

Updated: 2026-08-10

Comments: Preprint

Code: https://github.com/rd20karim/concept-probing

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 65/100

Key concepts

Abstraction: In this paper, abstraction means a model can generalize a concept like 'east' to unseen examples. The authors test this by probing whether the model's internal representations classify new instances correctly, rather than just recalling memorized facts.
Compositionality: Compositionality here refers to combining simple concepts, like 'close' and 'west', into a composite fact. The paper tests whether a model's representation of a combined concept relates to the representations of its parts, using logical AND operations.
Grounding: Grounding means linking linguistic expressions to real-world numerical values, such as distances in kilometers or bearings in degrees. The paper tests if models can map 'close' to an actual distance threshold, but finds they often fail, even when numbers are provided.
Consistency: Consistency measures whether a model gives opposite answers to a fact and its negation. For example, if a model says 'Prescot is west of Todmorden' is true, it should say 'Prescot is east of Todmorden' is false. Low consistency indicates unstable concept understanding.

Terminology

Grounding — 53× in this paper · explained in 3 episodes
Accuracy — 46× in this paper · explained in 1 episode
Abstraction — 37× in this paper · explained in 1 episode
Consistency — 34× in this paper · explained in 3 episodes
Core — 22× in this paper · explained in 2 episodes
Benchmark — 20× in this paper · explained in 2 episodes
Generalization — 16× in this paper · explained in 13 episodes
Embeddings — 11× in this paper · explained in 2 episodes
Cosine Similarity — 7× in this paper · explained in 3 episodes
Information Retrieval — 7× in this paper
Interpretability — 7× in this paper · explained in 15 episodes
Additivity — 5× in this paper · explained in 1 episode
Large Language Models (LLMs) — 5× in this paper · explained in 6 episodes
Mechanistic interpretability — 5× in this paper · explained in 2 episodes
Latent Representation — 4× in this paper · explained in 2 episodes
Latent Representations — 4× in this paper · explained in 3 episodes
Downstream Tasks — 3× in this paper · explained in 2 episodes
International Conference on Machine Learning — 3× in this paper
Adam — 2× in this paper · explained in 2 episodes
Alignment — 2× in this paper · explained in 1 episode
Calibration — 2× in this paper · explained in 7 episodes
Fine-Tuning — 2× in this paper · explained in 5 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
Foundation Models — 2× in this paper · explained in 7 episodes
Generalizability — 2× in this paper · explained in 6 episodes
Linear Probing — 2× in this paper · explained in 2 episodes
RAG — 2× in this paper · explained in 3 episodes
Standardized Metrics — 2× in this paper · explained in 3 episodes
World Model — 2× in this paper · explained in 7 episodes
Benchmarking — 1× in this paper · explained in 3 episodes
Compositional Gap Accuracy — 1× in this paper
Compositional Gap Consistency — 1× in this paper
Compositional Generalization — 1× in this paper · explained in 4 episodes
Decoupling — 1× in this paper · explained in 2 episodes
Latent Reasoning — 1× in this paper · explained in 2 episodes
Robustness — 1× in this paper · explained in 5 episodes
Transformer — 1× in this paper · explained in 2 episodes

Summary

The paper investigates whether Large Language Models (LLMs) can genuinely understand concepts, moving beyond surface-level task performance. The authors state: Understanding concepts is fundamental to generalization. Despite their impressive performance on a wide range of tasks, Large Language Models (LLMs) still struggle with genuine concept understanding. They argue that prior work often conflate[s] multiple skills or lack[s] precise control over the underlying concepts and their properties.

To address this, they design tests on their core properties: abstraction, compositionality, and groundness, building a concept-centric benchmark, targeting spatial concepts such as direction, distance, topology, and their compositions, and use question answering tasks serving as a proxy. The overarching goal is formalized as: How can we translate abstraction, compositionality, and grounding into testable hypotheses and develop the empirical probing tests allowing us to quantify the extent to which LLMs align with?

The paper situates itself in the debate about whether LLMs are statistical parrots or can organize symbolic, structured representations of concepts. They identify a research gap: prior work on concept probing (i) mostly considers multi-modal LLMs, (ii) covers only one core property of concepts in text-only LLMs, and (iii) designs probes guided by downstream tasks rather than concepts per se, thereby confounding conclusions about both LLMs' concept understanding and task skills.

(RQ1) Are LLMs' performances on a geo-spatial question-answering task sufficient to assess their actual ability to understand underlying concepts?
(RQ2) Do LLMs encode abstract geo-spatial concept representations? Are concept representations generalizable across linguistic tokens and geographic regions?
(RQ3) Are LLMs able to compose geo-spatial concepts? Can LLMs' predictions be explained by composing concept representations?
(RQ4) Can concept representations be grounded in real-world knowledge?

Following knowledge representation literature, the authors define a concept as an abstract category or class that defines a set of object entities sharing common properties and roles. A concept instance is a concrete exemplar of a concept, representing a specific real-world entity that belongs to the extension of that concept. Instances are represented as relational triplets ⟨x, r, y⟩ (e.g., ⟨Prescot, west of, Todmorden⟩). Three core properties are examined:

Abstraction: a concept C is assumed to be the abstraction of its instances I(C). Therefore, I(C) form the semantic type of C.
Compositionality: the composition of atomic concepts C1,..., Cn yields a new composite concept C1 ×... × Cn, focusing on conjunctive (logical AND) composition.
Grounding: the awareness of concepts, expressed using natural language constructs, with the physical entities in the world they are assumed to apply to.

The concepts investigated are: Direction (north, south, east, west), Distance (close, far), and Topology (within, borders), with corresponding negated relations. Triplets are generated from UK metropolitan district wards using geometric measurements from GraphDB and YAGO2GEO. Distances and bearings are mapped to relational instances via functions phi d,th and phi theta. The distance threshold is set to the mean of the pairwise distance distribution (th = 47.76 km). Both atomic and compositional triplets (pairwise and higher-order compositions) are generated via Algorithm 1. Questions are generated as Binary-QA (Yes/No) and MCQ (3-options), with distractor generation for MCQ. The dataset contains 1,799,390 Binary-QA instances and 115,294 MCQ instances. A dedicated probing dataset DPB is constructed by subsampling 1,000 binary and 1,000 ternary compositional questions with all atomic decompositions and negations, totaling 14,000 QA instances.

The authors evaluate popular text-only LLMs (Llama-8B, Mistral-7B/8B, Qwen-0.6B, 1.7B, 4B, 8B) using two evaluation proxies: task performance (accuracy and consistency, where consistency measures whether paired positive/negative questions are both answered correctly) and probing performance (linear classifiers on internal layer representations).

Results show that across all settings, MCQ accuracy consistently exceeds Binary-QA accuracy, with Llama-3.1-8B achieving 71.7% MCQ and Qwen3-4B 56.14% Binary-QA. However, "consistency remains low overall, especially for Binary-QA (often < 26%), indicating unstable factual recall despite moderate accuracy. Explicit thinking (CoT/reasoning) is largely ineffective: it yields marginal gains at best and often degrades performance. The distance threshold analysis reveals that Closeness perception is systematically stricter than the dataset label, with models' revealed thresholds T∗ ≪ T0, and that Task format re-calibrates the implicit threshold in MCQ. Additionally, Distance concept understanding degrades at continental scale for smaller models. The authors conclude: QA-based evaluation has inherent limitations as a proxy for concept understanding: surface-level accuracy does not reliably reflect robust understanding across tasks and regions, particularly for challenging concepts such as distance."

Using layer-wise linear probing with standard random splits and OOD settings (region-based and token-based), the authors find: "most evaluated LLMs achieve near-perfect performance on the standard random split (99.95−99.98% accuracy) and generalize well to OOD settings, reaching 75−83% on the Geo-Level split, 80−83% on the single Token-level split, and over 99% on the compositional Token-level split. However, the Mistral-family models exhibit substantially lower accuracy across all splits –random split 52.8−58.0%, Geo-Level split 37.8−42.2%, single Token-level split 22.5−29.4%, and compositional Token-level split 48.5−55.4%– indicating significantly weaker concept encoding and generalization. The authors summarize: Most models clearly recognize concepts via their latent representations... However, the Mistral family consistently underperforms across all splits and layers, suggesting limited concept abstraction capacity in its architecture."

Compositionality is assessed through three measures: the compositionality gap, additivity of latent representations (cosine similarity between compositional question embeddings and summed atomic embeddings), and prediction compositionality (logit additivity, embedding summation, and probability averaging correlations). Key findings:

Unlike [39], in conjunctive-based compositionality, LLMs perform better on compositional questions than on atomic ones. Smaller models show the largest compositional and consistency gaps.
Mistral models exhibit the lowest and most variable compositionality correlation in cosine similarity analyses; 2-Concept compositions consistently show higher compositionality than 3-Concept ones.
From prediction-based analysis: "Mistral achieves moderate Binary-QA accuracy (∼65%) with weak compositional correlations (logit correlations ∼0.4)... In contrast, Qwen and LLaMA attain higher Binary-QA accuracy (∼80%) and comparable MCQ performance (∼35–39%), while maintaining consistently strong correlations across settings (logits ≳0.85, probabilities ≳0.7), indicating a substantially stronger alignment between compositional structure and predictive behavior."

The authors conclude: models with compositional embeddings preserve prediction consistency and achieve more robust results, unlike Mistral-like architectures.

Grounding is tested by augmenting question contexts with numerical real-world information (coordinates, distances, bearings, and an explicit distance threshold). Results show: "In the MCQ setting, several models surpass the random baseline (33%), with Qwen3-8B achieving the strongest results (66.7% accuracy and 44.7% consistency on the distance concept). However, overall consistency remains limited, indicating only weak grounding ability. In Binary-QA, accuracy largely concentrates around chance level (∼50%) and is consistently paired with low consistency. The authors find: Concepts are not naturally grounded in the evaluated LLMs; instead, the results indicate a reliance on in-context concept memorization rather than true numerical meaning grounding, adding that despite enriching concept-based questions with real-world factual information, such as geometrical measurements, it does not improve related QA-task performance in LLMs even with an explicit distance threshold definition."

The paper concludes: while LLMs encode and partially compose concepts, factual consistency and real-world grounding remain critical bottlenecks. The methodology is designed to be extensible: our probing methodology [is] designed to generalize across spatial concepts and extend to other concepts, tasks, and domains. Limitations acknowledged include reliance on only two real-world geographic regions, the possibility that the studied concepts may not fully capture the complexity of other real-world concepts, and that probing experiments are limited to linear classifiers.

The authors identify implications for information and knowledge management: (i) Information access and retrieval — LLMs' abstraction ability suggests new paths moving from the current designs of LLMs for relevance ranking toward new approaches of axiomatic IR or mechanistic interpretability for IR; (ii) Mining multimodal content — limited grounding provides concrete evidence of why multi-modal... models must go beyond traditional two-tower models, arguing for externally grounded models that complement LLMs with parametric mechanisms for explicit grounding in the same line as parametric retrieval augmented generation; (iii) Evaluation — findings reveal a critical limited ability of downstream task evaluation to actually probe LLMs on their understanding of conceptual knowledge, calling for new benchmarks suited to concept probing, annotated with core properties, standardized metrics, and reference concepts, configurations with published baselines.

The code is available at https://github.com/rd20karim/concept-probing.

Improvements for AI systems

The paper shows moderate QA accuracy coexists with very low consistency (<26% for Binary-QA), meaning models answer paired positive/negative questions inconsistently.

Improvement: Add a consistency loss that trains on paired positive/negative questions (e.g., Is A west of B? / Is A east of B?) and require both to be correct. Use consistency as an evaluation metric in addition to accuracy.
Resulting capability: The AI system maintains high accuracy and high consistency (>80%), eliminating unstable factual recall and self-contradictory answers on negated/opposite queries.

The paper finds Qwen/LLaMA models show logit correlations ≳0.85 between compositional questions and summed atomic concept embeddings, while Mistral shows weak correlations (0.4).

Improvement: Add a training regularizer that pushes the representation of a compositional question (e.g., north AND far AND within) to be close to the vector sum of its atomic concept representations. For architectures with weak compositionality, intervene at the layer level by replacing the compositional representation with the summed atomic representation at inference time.
Resulting capability: The system can answer novel, unseen compositional queries by explicitly composing learned atomic concepts, with predictive behavior that is logically aligned and robust for both 2-concept and 3-concept combinations.

The paper shows a model family (Mistral) can achieve task accuracy yet fail abstraction probing (22–38% on OOD token/geo splits), while other families reach 75–83%.

Improvement: Build a standardized concept-probing suite (random split, region-based OOD split, token-based OOD split) and use its results as a model selection and early-stopping criterion, instead of relying only on downstream task accuracy.
Resulting capability: The AI system reliably generalizes to unseen geographic regions and unseen linguistic surface forms, rather than memorizing training patterns; architecturally weak models are detected and rejected before deployment.

The paper shows that even when numerical facts (coordinates, distances, bearings) are injected into context, performance stays at chance level (50% Binary-QA), indicating in-context memorization rather than true numerical understanding.

Improvement: Replace facts in prompt with a parametric retrieval mechanism (like RAG) that retrieves structured geometric data, feeding them into a dedicated differentiable module that computes distance/bearing/topology from coordinates rather than relying on memorized verbal labels. Train on continuous outputs (distance in km, bearing in degrees) and map to relations deterministically afterward.
Resulting capability: The system answers geo-spatial questions by actually computing geometry (e.g., Prescot is 23 km west of Todmorden, therefore close), works at continental scale where current small models degrade, and does not collapse when the numerical context is removed.

The paper finds the models' implicit close/far threshold is systematically stricter than the dataset threshold (T* ≪ T0 ≈ 47.76 km) and shifts depending on question format.

Improvement: Append a learned calibration head that estimates the model's implicit threshold and aligns it to the target threshold via a calibration loss. Expose this threshold as an interpretable output so the system can report its own decision boundary.
Resulting capability: The AI system's distance perception is controllable and consistent across tasks: a pair of points is classified as close vs. far with the same internal boundary regardless of whether the question is Binary-QA or MCQ, and the system can declare its threshold in kilometers.

The paper shows MCQ accuracy consistently exceeds Binary-QA accuracy, and the task format re-calibrates the implicit distance threshold, indicating answers are not stable across surface formats.

Improvement: Train on multiple QA formats (Binary-QA, MCQ, open-ended) jointly with a cross-format consistency loss that penalizes the model for changing its answer to the same underlying question when the format changes.
Resulting capability: The system gives format-invariant answers: the same underlying geo-spatial fact is answered correctly and consistently regardless of whether it is posed as Yes/No, multiple-choice, or natural-language generation.

The paper shows chain-of-thought prompting yields marginal or negative gains, and models still struggle with compositional grounding.

Improvement: Replace open-ended CoT with a neuro-symbolic reasoning layer that explicitly applies formal composition rules (AND-composition of atomic relations over ⟨x, r, y⟩ triplets) and checks logical consistency of the final composed relation before generating a response.
Resulting capability: The system provides logically guaranteed multi-hop spatial reasoning — if A is north of B and B is within region C, it can derive and assert A is north AND within C with traceable intermediate steps, closing the compositional gap and removing reliance on fragile free-text reasoning.

Distance understanding degrades at continental scale for smaller models, and the benchmark only used one geographic region (UK wards).

Improvement: Use a curriculum that trains on multiple geographic scales (local wards, regional districts, national borders) with region-mixing augmentations, so distance/bearing representations are scale-invariant.
Resulting capability: The AI system maintains consistent geo-spatial concept understanding across local and continental queries, and transfers to entirely new geographic regions without retraining.

Net effect: The improved AI system does not merely answer geo-spatial QA accurately; it demonstrates verified abstraction (generalizes to unseen tokens/regions), verifiable compositionality (its answers follow logically from atomic concept vectors), explicit numerical grounding (it computes relations from real-world geometry rather than memorizing), calibrated and format-invariant perception, and transparent reasoning traces — directly addressing the paper's identified bottlenecks of factual consistency and real-world grounding.