LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression
Zhuowen Liu, Longkun Hao, Shiyu Feng, Xiaowen Chang, Ruiqun Li, Changqun Li
cs.LG, cs.AI
Submitted: 2026-08-15
Updated: 2026-08-18
Comments: 12 pages, 5 figures, 5 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- GPT-4 Technical Report
- The Llama 3 Herd of Models
- Distilling the Knowledge in a Neural Network
- Qwen Technical Report
- I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
- SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression
- Mistral 7B
- AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
- Qwen3 Technical Report
- ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
- A White Paper on Neural Network Quantization
- OPT: Open Pre-trained Transformer Language Models
- WinoGrande: An Adversarial Winograd Schema Challenge at Scale
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks