Do Knowledge and Reasoning Oriented Benchmarks Induce Stable Representation Signatures in LLMs?
cs.LG
Submitted: 2026-10-16
Updated: 2026-10-16
Comments: NeurIPS 2026 Interpretability as a Science Workshop
License: http://creativecommons.org/licenses/by/4.0/
The gist: Internal signatures could support monitoring, adaptive computation, and mechanistic analysis, but benchmark prompts can be separable without any shared computational distinction.
Terminology
Abstract
Internal signatures could support monitoring, adaptive computation, and mechanistic analysis, but benchmark prompts can be separable without any shared computational distinction. We test whether a prespecified knowledge/reasoning label defines a reusable internal signature in five contemporary 7B to 31B models. We treat the task as the unit of inference probes hold out complete tasks, labels are permuted across tasks, and the best layer and stream are reselected for every permutation. Raw probes recover an eight-versus-eight-subject MMLU partition in all five models (macro-F1 0.78 -- 0.81), but prompt text predicts it nearly as well: under an identical probe configuration, the activation advantage is only 0.017 -- 0.070 macro-F1 and is not significant. At the prespecified 0.01 level, lexical adjustment removes significance while the adjusted states still identify individual subjects, and the selected site changes across task panels and selection rules. Additionally, two released base-training trajectories show no common direction. These pooled layerwise measures recover the task label but identify no transferable knowledge/reasoning signature at the tested sensitivity, which cannot exclude moderate effects. The resulting protocol gives future internal monitors and intervention studies a sequence of tests to pass. Project page:.
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks