Predicting Quantization Price for Selecting PTQ Configurations Before Deployment
cs.CL, cs.LG
Submitted: 2026-09-23
Updated: 2026-09-23
Terminology
Sources
- QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
- Extreme Compression of Large Language Models via Additive Quantization
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
- SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
- GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
- SqueezeLLM: Dense-and-Sparse Quantization
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
- LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
- BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
- RUQuant: Towards Refining Uniform Quantization for Large Language Models
- VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering