PairAlign: A Framework for Autoregressive Tokenization via Self-Alignment with Applications to Audio Tokenization
cs.LG, cs.CL, cs.SD, eess.AS
Submitted: 2026-05-07
Updated: 2026-08-29
Terminology
Sources
- vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Constitutional AI: Harmlessness from AI Feedback
- Revisiting Feature Prediction for Learning Visual Representations from Video
- AudioChat: Unified Audio Storytelling, Editing, and Understanding with Transfusion Forcing
- Monotonic Chunkwise Attention
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- High Fidelity Neural Audio Compression
- KTO: Model Alignment as Prospect Theoretic Optimization
- A-JEPA: Joint-Embedding Predictive Architecture Can Listen
- Sequence Transduction with Recurrent Neural Networks
- Efficiently Modeling Long Sequences with Structured State Spaces
- The Curious Case of Neural Text Degeneration
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
- Monotonic Multihead Attention
- Discrete Audio Tokens: More Than a Survey!
- Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment
- Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks