One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models
cs.LG
Submitted: 2026-07-21
Updated: 2026-09-02
Comments: EMNLP 2026 Main Conference
Code: https://github.com/Jo-eyang/OMG-VLM
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Inductive Representation Learning on Large Graphs
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning
- UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs
- Tokenized Graph Transformer with Neighborhood Augmentation for Node Classification in Large Graphs
- Open Graph Benchmark: Datasets for Machine Learning on Graphs
- LLaGA: Large Language and Graph Assistant
- Semi-Supervised Classification with Graph Convolutional Networks
- Training Graph Neural Networks with 1000 Layers
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- Visual Instruction Tuning
- GraphGPT: Graph Instruction Tuning for Large Language Models
- Graph Attention Networks
- GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning
- Graph4MM: Weaving Multimodal Learning with Structural Information
- Learning Transferable Visual Models From Natural Language Supervision
- Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks