A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models
cs.LG
Submitted: 2026-08-27
Updated: 2026-08-27
Terminology
Sources
- CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compression
- HAPM -- Hardware Aware Pruning Method for CNN hardware accelerators in resource constrained devices
- EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
- Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
- LIMINAL: Exploring The Frontiers of LLM Decode Performance
- CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs
- Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
- A White Paper on Neural Network Quantization
- HAWQV3: Dyadic Neural Network Quantization
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- GLU Variants Improve Transformer
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks