XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication

XB RIDGE: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication

Summary

This paper introduces XB RIDGE, a decode-free communication protocol designed to enable effective communication between heterogeneous large language models (LLMs) from different model families. The authors identify a core challenge they term the entity grounding problem, where continuous latent-space bridges that transfer representations across different architectures suffer from rare-token compression collapse, losing discrete entity identities (like names, numbers, or rare tokens) in the continuous bottleneck. In bridge-only experiments, the receiver recovers contextual information but loses exact entity identities, with bridge-only F1 scores around 30%.

To address this, XB RIDGE decomposes communication into two complementary mechanisms:

1. Lexical Anchor Mapping (LAM): A deterministic, training-free function that maps the sender's original context tokens to the receiver's vocabulary, providing discrete entity anchors without autoregressive decoding. This preserves entity identity by placing the sender's original tokens, including entity mentions, as receiver-native lexical inputs.

2. Latent Enrichment Bridge (LEB): A set of four gated cross-attention modules inserted into the receiver at layers 6, 13, 20, 27, allowing the receiver to query the sender's hidden states for contextual enrichment. This transfers continuous contextual information that can be bound to the lexical anchors.

The key insight is that LAM supplies which entities appear in the output, while LEB supplies how the receiver reasons about them. The entity anchors ground the bridge's contextual signals to specific entities through the receiver's own self-attention, as the receiver's queries already encode which entities are present and where they appear.

Key Results:

• Performance: Across three model families (Llama-3.1-8B, Qwen2.5-7B, Mistral-7B), seven benchmarks, and both communication directions, XB RIDGE outperforms text-based communication (NLComm) on all seven tasks for each model pair, with average gains of +21.4 pp (Llama→Qwen), +14.3 pp (Qwen→Llama), and +20.9 pp (Mistral→Qwen).

• Latency: XB RIDGE achieves 11× lower latency than text-based communication (0.15 s vs. 1.70 s per sample) by eliminating autoregressive decoding.

• Same-architecture setting: XB RIDGE exceeds a KV-sharing baseline (KVComm) on six of seven tasks, and surpasses even FullComm (where the receiver reads the full context directly) on six of seven tasks.

• Efficiency: LEB requires only 264M trainable parameters (3.8% of the receiver), is trained on a small balanced sample set of 587 samples in under 10 minutes, and adds negligible inference overhead.

• Ablation: Removing LAM drops performance to 30.3% F1 (close to NoComm), while removing LEB reaches 56.5% F1, confirming both mechanisms are necessary. The full XB RIDGE achieves 78.8% F1 on HotpotQA.

• Entity perturbation experiment: Swapping LAM input changes the output entity, while swapping LEB input alone does not change the output entity, confirming clear role separation between the two mechanisms.

• Composability: Two independently trained bridges (Llama and Mistral as senders) combine at inference without retraining, achieving 70.4% F1 on HotpotQA compared to 67.0% for single-sender full context and 56.8% for dual NLComm.

• Sender scaling: LAM is sender-size invariant; even a 1.5B sender achieves 59.5% average F1, approaching FullComm, while 7B and 14B senders surpass it.

The paper concludes that heterogeneous LLMs can communicate at the latent level while preserving entity fidelity, representing an early step toward cross-architecture multi-agent collaboration. Limitations include single-direction bridges and untested multi-turn latent dialogue or many-agent composition scenarios.

Improvements to AI Systems:

1. Entity-Faithful Cross-Model Communication: Implement XB RIDGE’s dual mechanism (Lexical Anchor Mapping + Latent Enrichment Bridge) in multi-agent LLM systems to enable heterogeneous models (e.g., Llama, Qwen, Mistral) to exchange task-relevant information without losing rare entities (names, numbers, IDs). The improved system can achieve 78.8% F1 on entity-dense QA (HotpotQA) versus 30% with bridge-only latent transfer, enabling reliable collaboration across model families in production.

2. Decode-Free Latent Coordination for Real-Time Agents: Replace text-based inter-agent messaging with XB RIDGE’s latent bridge to cut per-sample communication latency from 1.70s to 0.15s (11× faster). The improved system can support real-time multi-agent reasoning (e.g., live customer support, financial trading, or robotics planning) where sub-second coordination is critical, while preserving entity accuracy.

3. Training-Efficient Cross-Architecture Adaptation: Use XB RIDGE’s lightweight LEB (264M parameters, 3.8% of receiver) trained on only 587 samples in under 10 minutes to adapt a new receiver model to accept latent inputs from any sender. The improved system can rapidly onboard new LLMs into an existing multi-agent network without full fine-tuning or large datasets, reducing deployment cost and enabling dynamic model swapping.

4. Composable Multi-Sender Knowledge Aggregation: Leverage XB RIDGE’s composability to combine independently trained bridges from multiple senders (e.g., Llama and Mistral) at inference without retraining. The improved system can fuse complementary knowledge from diverse models—achieving 70.4% F1 on HotpotQA versus 56.8% for text-based dual communication—for tasks requiring multi-perspective reasoning (e.g., medical diagnosis, legal analysis, or code review).

5. Entity-Controllable Generation via Anchor Manipulation: Use LAM’s deterministic token mapping to directly control which entities appear in the receiver’s output by editing the anchor input (as shown in the perturbation experiment). The improved system can perform targeted entity substitution or correction in generated responses (e.g., fixing a wrong date or name in a summary) without retraining, enabling precise factual editing in content generation pipelines.

6. Sender-Size-Invariant Knowledge Transfer: Exploit XB RIDGE’s sender-size invariance to allow small, efficient models (e.g., 1.5B) to communicate with larger receivers (e.g., 8B) at near-FullComm performance (59.5% F1). The improved system can deploy lightweight edge models as senders in distributed settings, reducing computational load while maintaining high-quality collaborative outputs—ideal for resource-constrained environments like mobile or IoT multi-agent systems.

Heterogeneous multi-agent LLM systems, where agents are powered by different model families, can outperform homogeneous configurations by reducing redundant reasoning patterns. Yet existing communication protocols either operate through text, discarding the sender's internal representations, or require architectural homogeneity for latent-level transfer. We identify the entity grounding problem in cross-architecture communication: cross-attention bridges that transfer continuous representations across different LLM families suffer from rare-token compression collapse, where entity identity is lost in the continuous bottleneck (bridge-only F1 30%). We propose XBRIDGE, a decode-free communication protocol that addresses this through two mechanisms. Lexical Anchor Mapping (LAM) maps the sender's original context tokens to the receiver's vocabulary, providing discrete entity anchors. A Latent Enrichment Bridge (LEB) lets the receiver query the sender's hidden states for contextual enrichment. The entity anchors ground the bridge's contextual signals to specific entities through the receiver's own self-attention. Across three model families (Llama, Qwen, and Mistral), seven benchmarks, and both communication directions, XBRIDGE outperforms text-based communication on all seven tasks for each model pair while achieving 11x lower latency, and in a same-architecture setting it also exceeds a KV-sharing baseline on six of seven tasks. LEB requires only 264M trainable parameters (3.8% of the receiver), is trained on a small balanced sample set, and adds negligible inference overhead.