Harnessing the Synergy between LLM Agents and Knowledge Graphs for Urban Socioeconomic Prediction

The paper Harnessing the Synergy between LLM Agents and Knowledge Graphs for Urban Socioeconomic Prediction proposes a new framework called SLAK (Synergize LLM Agent and Knowledge Graph learning model) to improve the prediction of urban socioeconomic indicators, such as population and commercial activity. The authors note that while existing studies leverage Knowledge Graphs (KG) to model heterogeneous urban data, "these approaches heavily rely on heuristic ideas and expertise to extract task-relevant knowledge from diverse data, which may not be optimal for specific tasks. Additionally, they tend to overlook the inherent relationships between different indicators, limiting the prediction accuracy."

To address these limitations, the authors propose a framework that integrates the reasoning and representation learning on KG with LLM agents across three distinct levels:

1. Node Level: Semantic-enriched KG Embedding

The framework utilizes an embedding language model (LM) to capture semantic information. The authors construct an urban knowledge graph (UrbanKG) to model multi-sourced urban data and finetune an embedding language model (LM) to generate embeddings for KG entities with semantic information. Specifically, to align the model with the UrbanKG, they finetune the model on a KG completion task using the scoring function of a classic KG representation learning model TransE.

2. Path Level: Meta-path Extraction and Knowledge Fusion

The framework uses the reasoning capabilities of LLMs to automate the discovery of task-relevant knowledge. We construct an LLM agent to automatically discover meta-paths in the UrbanKG in order to extract knowledge relevant to the specific socioeconomic prediction task. The agent is provided with the schema of UrbanKG described in natural language to generate several potential meta-path schemes. Once meta-paths are identified, the model extracts meta-path-based sub-KGs which are processed through a KG representation learning model. To combine this knowledge, the authors design a semantic-guided attention module for knowledge fusion with meta-paths. This involves obtaining the natural language encoding of meta-paths by using the conjunction word THAT to construct a nested clause for each meta-path, which is then used to calculate the importance of meta-paths based on their semantic embeddings.

3. Task Level: Cross-task Communication Mechanism

To enhance performance through knowledge sharing, the authors introduce a cross-task communication mechanism to further enhance performance by enabling knowledge sharing across tasks at both LLM agent and KG levels.

• Agent-level communication: The LLM agents for different tasks collaborate to generate more diverse and comprehensive meta-paths. This involves agents recommending meta-paths to one another and updating their own selections based on knowledge from other tasks.

• KG-level communication: The embeddings from different tasks are adaptively merged for better socioeconomic prediction. The framework uses a semantic-guided knowledge fusion module to adaptively fuse the region embeddings from different tasks using a task description as a query.

Experimental Results and Efficiency

The effectiveness of the SLAK framework was evaluated using two city-scale datasets: Beijing and Shanghai. The results demonstrate that our model surpasses existing methods by 3.9-76.2% in terms of R squared on eight indicator prediction tasks. The authors also compared their method to traditional meta-path searching algorithms, such as random search and genetic algorithms, finding that our model achieves better performance and lower time cost, specifically reducing the time cost by over 10×. Furthermore, a case study revealed that the meta-paths discovered in the second round through multi-agent collaboration are generally longer and more diverse, capturing more complex relationships such as Economic Attraction or Regional Competitive Dynamics.

1. Temporal-Dynamic SLAK (T-SLAK)

• Improvement: Integrate temporal reasoning into the KG embeddings and the LLM agent’s meta-path discovery process to account for the time-varying nature of urban data.

• What the system can do: Predict not just current socioeconomic states, but the trajectory and velocity of change (e.g., forecasting the exact year a neighborhood will undergo gentrification or predicting the seasonal fluctuations of commercial activity in specific districts).

2. Counterfactual Urban Policy Simulator

• Improvement: Expand the LLM agents from knowledge discoverers to intervention simulators by allowing them to perform what-if reasoning on the UrbanKG.

• What the system can do: Simulate the socioeconomic impact of hypothetical urban planning decisions, such as If a new subway line is constructed between Point A and Point B, how will it redistribute population density and commercial revenue across the surrounding five districts?

3. Multi-Modal Urban Intelligence Engine

• Improvement: Augment the UrbanKG with non-structured, multi-modal data (e.g., satellite imagery, real-time traffic sensor streams, and social media sentiment) using vision-language models (VLMs) to create a Perceptual-Semantic KG.

• What the system can do: Detect real-time socioeconomic shifts that precede official statistical reporting, such as identifying a sudden increase in local economic vitality by correlating social media check-in spikes and satellite-detected changes in nighttime light intensity with existing KG entities.

4. Explainable Urban Governance Agent (X-SLAK)

• Improvement: Implement a formal Causal Traceability layer that maps the LLM-discovered meta-paths to causal inference models, ensuring the reasoning is not just associative but causative.

• What the system can do: Provide urban planners with human-readable, evidence-based justifications for predictions (e.g., "Commercial activity in District X is predicted to rise by 15% because the new transit hub creates a high-connectivity meta-path between residential zones and the central business district").

5. Hierarchical Multi-Agent Urban Orchestrator

• Improvement: Transition from task-level communication to a hierarchical multi-agent architecture where specialized agents (e.g., a Transport Agent, a Housing Agent, and an Environmental Agent) report to a Master Urban Planner Agent.

• What the system can do: Manage complex, cross-domain urban crises by identifying how a failure in one sector (e.g., a sudden flooding event affecting transport) will cascade through the KG to impact other indicators (e.g., declining commercial activity and shifting population movement).

Socioeconomic prediction aims to leverage various urban data to predict the socioeconomic indicators of regions such as population and commercial activity level, which plays an important role in understanding urban regions and supporting decision-making. Existing studies leverage knowledge graphs (KG) to model heterogeneous urban data, and further apply graph representation learning methods for socioeconomic prediction. However, these approaches heavily rely on heuristic ideas and expertise to extract task-relevant knowledge from diverse data, which may not be optimal for specific tasks. Additionally, they tend to overlook the inherent relationships between different indicators, limiting the prediction accuracy. Motivated by the remarkable abilities of large language models (LLMs), in this work, we propose a synergistic framework of LLM agents and KG, which integrates the reasoning and representation learning on KG with LLM agents. We first construct an urban knowledge graph (UrbanKG) to model multi-sourced urban data and finetune an embedding language model to generate embeddings for KG entities with semantic information. Then we leverage the reasoning power of LLM to identify relevant meta-paths in the UrbanKG for each type of socioeconomic prediction task, and design a semantic-guided attention module for knowledge fusion with meta-paths. Moreover, we introduce a cross-task communication mechanism to further enhance performance by enabling knowledge sharing across tasks at both LLM agent and KG levels. On the one hand, the LLM agents for different tasks collaborate to generate more diverse and comprehensive meta-paths. On the other hand, the embeddings from different tasks are adaptively merged. Experiments on two datasets demonstrate the effectiveness of the synergistic design between LLM and KG, providing insights for information sharing across socioeconomic prediction tasks.