New LoRA Skills Should Read but Never Write
cs.LG
Submitted: 2026-09-25
Updated: 2026-09-25
Terminology
Sources
- Git Re-Basin: Merging Models modulo Permutation Symmetries
- BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
- Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks
- Knowledge Fusion of Large Language Models Via Modular SkillPacks
- The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- Linear Mode Connectivity and the Lottery Ticket Hypothesis
- Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
- The Llama 3 Herd of Models
- Towards a Unified View of Parameter-Efficient Transfer Learning
- Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
- Parameter-Efficient Transfer Learning for NLP
- LoRA: Low-Rank Adaptation of Large Language Models
- LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
- EMR-Merging: Tuning-Free High-Performance Model Merging
- Editing Models with Task Arithmetic
- Model Stock: All we need is just a few fine-tuned models
- Dataless Knowledge Fusion by Merging Weights of Language Models
- Label-Free Cross-Task LoRA Merging with Null-Space Compression
- The Power of Scale for Parameter-Efficient Prompt Tuning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks