What Must Survive? Exact Task-Information--State Frontiers for Resource-Sufficient Learning
cs.LG, cs.IT, math.IT
Submitted: 2026-09-18
Updated: 2026-09-18
Comments: 9 pages, 0 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: A system may be compressed before its downstream task is fully known.
Terminology
Abstract
A system may be compressed before its downstream task is fully known. We ask how much retained state is then necessary and how much can be saved by limited advance task information. For a finite family of linear tasks, a task message is revealed before state formation and the exact task only afterwards. For an advice alphabet of size K, the exact frontier is p*(K)= C in (T C), with the b-bit frontier obtained by setting K= (2 b,). Thus advance task information reduces state through partitions whose joint task operators have low rank. We also give an approximate singular-value frontier, a common-core lower bound and exact direct-sum law, and strong NP-hardness of finding an optimal advice partition. The hardness persists at every fixed positive approximation tolerance. Three examples illustrate the result. A well-conditioned softmax attention construction gives an exact 524, 288 to1, 024 coordinate frontier when nine bits resolve one of 512 continuations. A domain-decomposed digital twin yields an interface-plus-local-state law and a weighted partition problem for heterogeneous regions. A hierarchical multi-task model gives a two-stage frontier in which three bits reduce the required state from 3136 to 448 coordinates, with further task information approaching the irreducible 328-coordinate single-task floor.
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks