Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights

arXiv:2609.02652 · cs.LG · Submitted 2026-09-02 · Read on arXiv

cs.LG

Submitted: 2026-09-02

Updated: 2026-09-02

Code: https://github.com/ggml-org/llama.cpp

Terminology

• Decode — 29× in this paper · explained in 2 episodes
• Perplexity — 23× in this paper · explained in 3 episodes
• Quantization — 19× in this paper · explained in 12 episodes
• Calibration — 13× in this paper · explained in 7 episodes
• Elias — 4× in this paper · explained in 2 episodes
• In International Conference on Learning Representations — 4× in this paper
• Transformer — 4× in this paper · explained in 2 episodes
• Verification — 4× in this paper · explained in 2 episodes
• Benchmark — 3× in this paper · explained in 2 episodes
• Provenance — 3× in this paper · explained in 3 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Core — 1× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 1× in this paper
• Generative AI — 1× in this paper · explained in 2 episodes
• KV cache — 1× in this paper · explained in 3 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes

Sources

• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
• QuIP: 2-Bit Quantization of Large Language Models With Guarantees
• SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
• Extreme Compression of Large Language Models via Additive Quantization
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
• MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
• Fast Matrix Multiplications for Lookup Table-Quantized LLMs
• Measuring Massive Multitask Language Understanding
• SqueezeLLM: Dense-and-Sparse Quantization
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
• VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
• SpinQuant: LLM quantization with learned rotations
• Pointer Sentinel Mixture Models
• LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
• Qwen3 Technical Report
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
• QTIP: Quantization with Trellises and Incoherence Processing
• GPTVQ: The Blessing of Dimensionality for LLM Quantization

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks