Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

arXiv:2603.17558 · cs.CL, cs.SD · Submitted 2026-03-18 · Read on arXiv

cs.CL, cs.SD

Submitted: 2026-03-18

Updated: 2026-09-07

Comments: Published in IEEE Transactions on Audio, Speech and Language Processing, 2026

Journal ref: IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 3956-3969, 2026

DOI: 10.1109/TASLPRO.2026.3719331

Code: https://github.com/YuCeong-May/Zipper-LoRA

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Adaptation — 59× in this paper · explained in 1 episode
• Fine-Tuning — 20× in this paper · explained in 5 episodes
• Decoupling — 17× in this paper · explained in 2 episodes
• Parameter-Efficient Fine-Tuning — 10× in this paper · explained in 2 episodes
• Embeddings — 8× in this paper · explained in 2 episodes
• Low-Resource Languages — 6× in this paper · explained in 4 episodes
• Alignment — 5× in this paper · explained in 1 episode
• Parameter-Efficient Fine-Tuning (PEFT) — 5× in this paper · explained in 4 episodes
• Mixture-of-Experts (MoE) — 4× in this paper · explained in 7 episodes
• In-Context Learning — 3× in this paper · explained in 4 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Two-stage training — 3× in this paper · explained in 2 episodes
• Automatic Speech Recognition — 2× in this paper
• Domain Adaptation — 2× in this paper · explained in 5 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• LoRA Adaptation — 2× in this paper · explained in 2 episodes
• Mixture of Experts — 2× in this paper · explained in 2 episodes
• Adaptability — 1× in this paper · explained in 1 episode
• Adaptive Weighting — 1× in this paper · explained in 1 episode
• Automatic Speech Recognition (ASR) — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Continual Learning — 1× in this paper · explained in 2 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Cross-lingual Transfer — 1× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 1× in this paper
• Feature Extraction — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• LLM Integration — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Meta-learning — 1× in this paper · explained in 2 episodes
• Parameter Efficiency — 1× in this paper · explained in 3 episodes
• Proceedings of International Conference on Machine Learning — 1× in this paper
• Proceedings the International Conference on Learning Representations — 1× in this paper
• Regularization — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Sparsity — 1× in this paper · explained in 4 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Two-Stage Training Strategy — 1× in this paper · explained in 3 episodes

Sources

• OpenAI GPT-5 System Card
• The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes
• Qwen3 Technical Report
• Fun-ASR Technical Report
• Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
• FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration
• MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
• Qwen2.5-Omni Technical Report
• Bridging the gap: A comparative exploration of Speech-LLM and end-to-end architecture for multilingual conversational ASR
• A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR
• MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
• SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
• AudioPaLM: A Large Language Model That Can Speak and Listen
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
• Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
• Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
• LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
• Adaptive Shared Experts with LoRA-Based Mixture of Experts for Multi-Task Learning
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering