Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models
cs.AI
Submitted: 2025-10-24
Updated: 2026-09-21
Comments: Published at ICLR 2026. Code: https://github.com/ZoYujin/NuSA-CL
Code: https://github.com/ZoYujin/NuSA-CL
License: http://creativecommons.org/licenses/by/4.0/
The gist: Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in a wide range of real-world tasks without additional
Terminology
Abstract
Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in a wide range of real-world tasks without additional task-specific training. However, in real deployment scenarios with evolving environments or emerging classes, these models inevitably face distributional shifts and novel tasks. In such contexts, static zero-shot capabilities are insufficient, and there is a growing need for continual learning methods that allow models to adapt over time while avoiding catastrophic forgetting. We introduce NuSA-CL (Null Space Adaptation for Continual Learning), a lightweight memory-free continual learning framework designed to address this challenge. NuSA-CL employs low-rank adaptation and constrains task-specific weight updates to lie within an approximate null space of the model's current parameters. This strategy minimizes interference with previously acquired knowledge, effectively preserving the zero-shot capabilities of the original model. Unlike methods relying on replay buffers or costly distillation, NuSA-CL imposes minimal computational and memory overhead, making it practical for deployment in resource-constrained, real-world continual learning environments. Experiments show that our framework not only effectively preserves zero-shot transfer capabilities but also achieves highly competitive performance on continual learning benchmarks. These results position NuSA-CL as a practical and scalable solution for continually evolving zero-shot VLMs in real-world applications.
Sources
- On Tiny Episodic Memories in Continual Learning
- Don't Stop Learning: Towards Continual Learning for the CLIP Model
- Visual Instruction Tuning
- Improved Baselines with Visual Instruction Tuning
- Fine-Grained Visual Classification of Aircraft
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection