pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks
cs.LG, cs.NA, math.NA
Submitted: 2025-04-10
Updated: 2026-08-26
Code: https://github.com/inEXASCALE/pychop
License: http://creativecommons.org/licenses/by/4.0/
The gist: Motivated by the growing demand for reduced-precision arithmetic in computational science, we exploit lower-precision emulation in Python -- widely regarded as the dominant programming language for
Terminology
Abstract
Motivated by the growing demand for reduced-precision arithmetic in computational science, we exploit lower-precision emulation in Python -- widely regarded as the dominant programming language for numerical analysis and machine learning. Low-precision paradigms have revolutionized deep learning by enabling more efficient computation and reduced memory footprint while maintaining model fidelity. To better enable numerical experimentation with and exploration of reduced-precision computation, we developed pychop, which supports customizable floating-point formats and a comprehensive set of rounding modes in Python, allowing users to benefit from fast, reduced-precision emulation in numerous applications. pychop also provides flexible interfaces for array and tensor backends, enabling efficient reduced-precision emulation on both CPUs and GPUs for neural network deployment. In this paper, we offer a comprehensive exposition of the design and applications of pychop. Furthermore, we present empirical results on reduced-precision emulation for image classification and object detection using published datasets, illustrating the sensitivity to low precision and delivering valuable insights into its quantization-aware training and post-quantization impacts. Establishing itself as a foundational tool for advancing mixed-precision algorithms, pychop enables in-depth investigations into the effects of numerical precision in scientific computing and deep learning deployment, facilitating the development of novel hardware accelerators.
Sources
- Improved Regularization of Convolutional Neural Networks with Cutout
- FP8 Formats for Deep Learning
- 8-bit Numerical Formats for Deep Neural Networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks