Cubit: Token Mixer with Kernel Ridge Regression
cs.LG, cs.CL
Submitted: 2026-05-07
Updated: 2026-08-26
Terminology
Sources
- Neural Machine Translation by Jointly Learning to Align and Translate
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- Efficiently Modeling Long Sequences with Structured State Spaces
- Mixtral of Experts
- Multiplicative LSTM for sequence modelling
- Virtual Width Networks
- DeepSeek-V3 Technical Report
- Optimal Rates and Saturation for Noiseless Kernel Ridge Regression
- Regularized least squares learning with heavy-tailed noise is minimax optimal
- RWKV-7 "Goose" with Expressive Dynamic State Evolution
- HGRN2: Gated Linear RNNs with State Expansion
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
- Theory, Analysis, and Best Practices for Sigmoid Self-Attention
- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
- Fast Transformer Decoding: One Write-Head is All You Need
- JetMoE: Reaching Llama2 Performance with 0.1M Dollars
- Simplified State Space Layers for Sequence Modeling
- Retentive Network: A Successor to Transformer for Large Language Models
- Kimi K2: Open Agentic Intelligence
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks