SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems
cs.AI
Submitted: 2026-08-24
Updated: 2026-08-25
Code: https://github.com/WishArdently/SA-RSQ
License: http://creativecommons.org/licenses/by/4.0/
The gist: Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead.
Terminology
Abstract
Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead. Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget. We propose Sparse Activation-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact (Index, Probability) tuples. The stored tuples decouple per-item storage from codebook dimensionality; for a fixed selected support, gradients propagate through the routing weights and weighted reconstruction without relying on a straight-through estimator. Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item. A preliminary Next-Distribution Prediction study and a one-week online A/B test further demonstrate the practical potential of SA-RSQ, with relative lifts of +2.51% in CTR and +3.66% in CPM.
Sources
- Qwen3-VL Technical Report
- Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
- Deep Learning Recommendation Model for Personalization and Recommendation Systems
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization
- R3-VAE: Reference Vector-Guided Rating Residual Quantization VAE for Generative Recommendation
- Generative Recommendation: Towards Next-generation Recommender Paradigm
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
- FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasets
- OneRec-Think: In-Text Reasoning for Generative Recommendation
- SGDR: Stochastic Gradient Descent with Warm Restarts
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection