Skill Issue: Are Skills Language-Invariant in LLMs?

arXiv:2608.25832 · cs.CL, cs.AI, cs.GT, cs.LG · Submitted 2026-08-26 · Read on arXiv

cs.CL, cs.AI, cs.GT, cs.LG

Submitted: 2026-08-26

Updated: 2026-08-26

Code: https://github.com/TextArena/TextArena

Terminology

• Benchmark — 23× in this paper · explained in 2 episodes
• Fidelity — 10× in this paper · explained in 2 episodes
• Verification — 9× in this paper · explained in 2 episodes
• Accuracy — 6× in this paper · explained in 1 episode
• Adam — 5× in this paper · explained in 2 episodes
• alpha — 5× in this paper · explained in 1 episode
• Cooperation — 4× in this paper · explained in 2 episodes
• Core — 4× in this paper · explained in 2 episodes
• Verifier — 4× in this paper · explained in 2 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Consistency — 3× in this paper · explained in 3 episodes
• Cross-lingual Transfer — 3× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 3× in this paper
• Adaptation — 2× in this paper · explained in 1 episode
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• Low-Resource Languages — 2× in this paper · explained in 4 episodes
• Verifiability — 2× in this paper · explained in 2 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Alignment Methods — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Elias — 1× in this paper · explained in 2 episodes
• Explainability — 1× in this paper · explained in 3 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Zero-Shot Transfer — 1× in this paper · explained in 3 episodes

Sources

• OpenAI Gym
• XOR QA: Cross-lingual Open-Retrieval Question Answering
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models
• No Language Left Behind: Scaling Human-Centered Machine Translation
• $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
• GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
• Cross-Lingual Exploration for Parametric Knowledge
• Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environments
• MindAgent: Emergent Gaming Interaction
• The Llama 3 Herd of Models
• GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
• TextArena
• Toucan: Many-to-Many Translation for 150 African Language Pairs
• Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
• IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer
• Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs
• Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages
• Ministral 3

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering