Feedback Control for Multi-Objective Graph Self-Supervision
cs.LG
Submitted: 2026-02-04
Updated: 2026-09-05
Comments: ICML 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Can multi-task self-supervised learning on graphs be coordinated without the usual tug-of-war between objectives? Graph self-supervised learning (SSL) offers a growing toolbox of pretext objectives
Terminology
Abstract
Can multi-task self-supervised learning on graphs be coordinated without the usual tug-of-war between objectives? Graph self-supervised learning (SSL) offers a growing toolbox of pretext objectives like mutual information, reconstruction, and contrastive learning, yet combining them reliably remains challenging due to objective interference and training instability. Most multi-pretext pipelines use per-update mixing, forcing every parameter update to be a compromise and leading to three failure modes: Disagreement (conflict-induced negative transfer), Drift (nonstationary objective utility), and Drought (hidden starvation of underserved objectives). We argue that coordination is fundamentally a temporal allocation problem: deciding when each objective receives optimization budget, not merely how to weigh them. We introduce ControlG, a control-theoretic framework that recasts multi-objective graph SSL as feedback-controlled temporal allocation by estimating per-objective difficulty and pairwise antagonism, planning target budgets via a Pareto-aware log-hypervolume planner, and scheduling with a Proportional-Integral-Derivative (PID) controller. Across 9 datasets, ControlG consistently outperforms state-of-the-art baselines while producing an auditable schedule that reveals which objectives drove learning.
Sources
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- Automated Self-Supervised Learning for Graphs
- Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalization
- Towards Graph Contrastive Learning: A Survey and Beyond
- Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks
- Semi-Supervised Classification with Graph Convolutional Networks
- AdaTask: Adaptive Multitask Online Learning
- The Hypervolume Indicator: Problems and Algorithms
- Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
- Multi-Task Learning as a Bargaining Game
- Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
- Deep Graph Contrastive Representation Learning
- Geom-GCN: Geometric Graph Convolutional Networks
- Multi-scale Attributed Node Embedding
- Pitfalls of Graph Neural Network Evaluation
- Large-Scale Representation Learning on Graphs via Bootstrapping
- Graph Attention Networks
- Deep Graph Infomax
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks