FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

arXiv:2601.06199 · eess.AS, cs.AI, cs.SD · Submitted 2026-01-08 · Read on arXiv

eess.AS, cs.AI, cs.SD

Submitted: 2026-01-08

Updated: 2026-08-31

Code: https://github.com/myshell-ai/MeloTTS

Terminology

• Adaptation — 17× in this paper · explained in 1 episode
• Benchmark — 17× in this paper · explained in 2 episodes
• Abstraction — 12× in this paper · explained in 1 episode
• Hierarchical Temporal Abstractor — 7× in this paper
• Instruction Tuning — 7× in this paper · explained in 2 episodes
• Temporal Abstraction — 7× in this paper · explained in 2 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Ablation — 4× in this paper · explained in 1 episode
• Ablation study — 3× in this paper · explained in 1 episode
• Alignment — 3× in this paper · explained in 1 episode
• Computational overhead — 3× in this paper · explained in 2 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• Downstream Tasks — 2× in this paper · explained in 2 episodes
• Granularity Mismatch — 2× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
• Redundancy — 2× in this paper · explained in 2 episodes
• Semantic Understanding — 2× in this paper · explained in 2 episodes
• alpha — 2× in this paper · explained in 1 episode
• Ablation Studies — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 2 episodes
• Automatic Speech Recognition (ASR) — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Feature Extraction — 1× in this paper · explained in 2 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
• GPT-4 Technical Report
• Common Voice: A Massively-Multilingual Speech Corpus
• Multilingual and Cross-Lingual Intent Detection from Spoken Data
• Qwen2-Audio Technical Report
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• Kimi-Audio Technical Report
• Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities
• Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
• The Llama 3 Herd of Models
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Gaussian Error Linear Units (GELUs)
• Long-Form Speech Generation with Spoken Language Models
• A Call for Clarity in Reporting BLEU Scores
• KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension
• Voxtral
• Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
• Mixed Precision Training

Related papers

• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization
• Towards Audio Token Compression in Large Audio Language Models
• WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection
• ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions