Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU k Networks on the Sphere
math.NA, cs.LG, cs.NA
Submitted: 2025-10-05
Updated: 2026-09-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: We establish two related but logically distinct results for shallow ReLU k neural networks on the unit sphere d.
Terminology
Abstract
We establish two related but logically distinct results for shallow ReLU k neural networks on the unit sphere d. First, for an arbitrary set of inner neural-network parameters, the best L 2(d) approximation of a fixed target function with smoothness r> 2 admits an asymptotic lower bound given by a constant multiple of n-1/2 k+1/2, where denotes the antipodal separation distance of the normalized inner-parameter set. This lower bound depends explicitly on the parameter configuration through and applies without additional assumptions on the parameters. Second, for antipodally quasi-uniform parameters, n-1/d, and the lower bound establishes the exact saturation order n-d+2k+1 over 2d for such parameter families: a target function with regularity greater than d+2k+1 over 2 and satisfying the required parity condition can be approximated at this rate, whereas approximation at any strictly faster rate forces the target function to be zero. Our results therefore place linearized neural-network approximation within the classical saturation framework and show that, although ReLU k network spaces can outperform finite elements of the same degree, this advantage is intrinsically limited.
Sources
- Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
- Integral Representations of Sobolev Spaces via ReLU$^k$ Activation Function and Optimal Error Estimates for Linearized Networks
- Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon Transform
- Tractability of approximation by general shallow networks
- Highly localized kernels on space of homogeneous type
Related papers
- Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
- A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions
- Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
- Windowed thinning and query complexity for the bouncy particle and Zigzag samplers
- Data-efficient Kernel Methods for Learning Hamiltonian Systems
- Adjoint Method versus Physics-Informed Neural Networks in PDE-Constrained Inverse Problems