CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization
cs.DC, cs.AI
Submitted: 2026-09-14
Updated: 2026-09-14
License: http://creativecommons.org/licenses/by/4.0/
The gist: Amid the rapid advancement of physical-world intelligence, cloud-edge collaborative large language models (LLMs) have emerged as a promising roadmap for practical LLM deployment.
Terminology
Abstract
Amid the rapid advancement of physical-world intelligence, cloud-edge collaborative large language models (LLMs) have emerged as a promising roadmap for practical LLM deployment. However, existing cloud-edge paradigms struggle to balance global consensus with local personalization, which fails to satisfy the need for a unified knowledge foundation on the cloud and domain-specific adaptation at the edge. To address this, we introduce, for the first time, a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, reaching cloud-edge evolving in coordination. We then propose CIDERS, an efficient solver that decomposes the model into a learnable backbone and a messenger. While the cloud performs knowledge transfer to the learnable backbone, the key lies in embedding global trajectories into each local personalization step via consensus-variate correction to reconcile personalization with consensus. We provide a comprehensive theoretical analysis, including a geometric characterization of the local trajectory and a full convergence guarantee, revealing an explicit trade-off structure between personalization and global convergence. Extensive experiments demonstrate that CIDERS consistently outperforms competitive baselines on the compressed edge path, with 3.1x and 1.7x gains on mathematical reasoning and code generation, respectively, and a 10% relative gain on instruction metrics. Mechanism experiments attribute these gains to early consensus-corrected coordination and task-aware distillation. Overall, CIDERS offers a viable path toward consensus-guided continuous personalization in cloud-edge LLM systems.
Sources
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- Towards Edge General Intelligence via Large Language Models: Opportunities and Challenges
- SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization
- Model-Distributed Inference for Large Language Models at the Edge
- A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
- SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
- SplitCom: Communication-efficient Split Federated Fine-tuning of LLMs via Temporal Compression
- Federated Learning with Personalization Layers
- A Survey on Federated Fine-tuning of Large Language Models
- Towards Building the Federated GPT: Federated Instruction Tuning
- A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning
- FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning
- Selective Aggregation for Low-Rank Adaptation in Federated Learning
- FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA
- Distilling the Knowledge in a Neural Network
- Approximation Methods for Bilevel Programming
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Pushing CPU Speech Synthesis to the Wall: Extreme Inference Tuning under Serverless Architecture and Billing