Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox
cs.LG
Submitted: 2026-09-25
Updated: 2026-09-25
Code: https://github.com/Ahmadtr/depth-laws-precision
Terminology
Sources
- Why Does Post-Training Quantization Work?
- Training with Quantization Noise for Extreme Model Compression
- The Quantization Benefits of Residual-Free Transformers
- Scaling Laws for Neural Language Models
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
- A White Paper on Neural Network Quantization
- Three Quantization Regimes for ReLU Networks
- When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
- Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks