RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation

Large Language Models (LLMs) are being adapted for recommendation systems, but current methods struggle to construct decision-relevant contexts from heterogeneous evidence and face severe context-efficiency bottlenecks. This paper proposes RRCM, a ranking-driven retrieval-and-reasoning framework over collaborative and metadata memories for LLM-based agentic recommendation, which learns adaptively when and what evidence to retrieve based on final recommendation quality.

The gist

RRCM learns a policy that decides whether to recommend directly, retrieve collaborative evidence, retrieve item metadata, or interleave both through reasoning by optimizing an outcome-only ranking reward using Group Relative Policy Optimization.

Problem Formulation: Ranking-Driven Context Construction

The fundamental task is to predict the next item or a list of preferred items based on a user’s chronological interaction history. The core intuition is that LLMs can infer preferences from lightweight signals like item titles for popular items, but effective recommendation may require evidence not contained in parametric memory. The model starts with a lightweight user-history context consisting primarily of item titles and learns a policy to determine if additional evidence is needed.

Dual-Memory Retrieval Corpus

RRCM constructs a unified retrieval corpus, denoted as M, that organizes heterogeneous recommendation evidence into a searchable textual format. This corpus contains two complementary memories:

1. Collaborative Memory: Documents representing interaction sequences from historical users (e.g., “User 123 History: [Matrix, Inception, Interstellar,]”). This provides behavioral evidence about how users with similar historical preferences continue their interactions.

2. Meta Memory: Documents describing rich existing item metadata (e.g., “Movie Name: Inception; Director: Nolan; Genre: Sci-Fi; …”). This memory provides grounded item-level evidence, such as categories, creators, genres, brands, and prices.

Interleaved Reasoning and Memory Retrieval

Inference is modeled as a multi-turn loop of reasoning and memory access guided by a learned policy πθ(ak sk). The trajectory consists of:

1. Reasoning and Assessment: The model analyzes the current state to infer preference and assess if available evidence is sufficient.

2. Action Decision: If additional evidence is required, the model generates natural-language queries to retrieve from M via the unified retrieval interface, denoted as Retrieve(q). The state is updated with the retrieved passage. This loop repeats until a termination action is taken when the context is deemed sufficient for a final recommendation string yˆ.

Ranking-Driven Policy Optimization

The policy is optimized using reinforcement learning based directly on final recommendation quality, treating retrieval as an on-demand context construction action rather than a default preprocessing step with fixed rules. The training reward R(τ) reflects both ranking accuracy and structural validity:

(3)

R(τ) = Σn∈N wn · InTop@n(Lcand, igt) + λ · Iparse

where InTop@n measures whether the ground-truth item igt appears in the top-n positions of the grounded candidate list Lcand, and Iparse ensures format validity. This objective is instantiated with token-level Group Relative Policy Optimization (GRPO), which encourages retrieving evidence only when it directly improves final recommendation quality.

Significant Improvement with Selective Evidence Acquisition

Experiments show that RRCM consistently improves recommendation performance over strong traditional and LLM-based baselines by learning to selectively retrieve collaborative/meta memories only when needed, reducing unnecessary context construction and improving efficiency. Ablation studies confirm the contribution of each component: removing collaborative filtering information (RRCM w/o CF), item metadata retrieval (RRCM w/o META), or reasoning (RRCM w/o RE) consistently reduces performance, validating the joint integration of both memories and reasoning. The policy behavior shifts over training, moving from exploring by retrieving more evidence and producing longer reasoning traces to a state where it becomes increasingly selective, retrieving collaborative histories or item metadata only when additional evidence is necessary.

Case Studies

The framework utilizes an instruction prompt template that explicitly delineates reasoning, querying, retrieval results, and the final answer. Case studies demonstrate this in practice: for a book recommendation, the model first retrieves collaborative histories to identify similar reading patterns, then retrieves item metadata for the strongest candidate to verify genre and series information, and finally synthesizes both sources of evidence to make a grounded recommendation. For long-tail items, RRCM's ability to generate queries and retrieve relevant metadata is shown to be especially beneficial when item information is sparse or difficult for the LLM to infer from titles alone.

Conclusion

RRCM outperforms all compared baselines across three public datasets by learning a unified RL policy that jointly decides whether retrieval is needed, what to retrieve, and how to leverage the retrieved memory into reasoning.

Here are specific improvements to AI systems based on the RRCM (Ranking-Driven Retrieval over Collaborative and Meta Memories) framework, detailing what these improved systems can achieve:

The core improvement is shifting LLM recommendation from a static prompt-filling or fixed retrieval pipeline approach to an adaptive, evidence-driven, agentic reasoning process. The improved system will be characterized by its ability to dynamically decide when and what external knowledge (collaborative behavior or item metadata) is necessary to make a final decision.

Here are the specific improvements and capabilities:

1. ---

2. The system will transition from generating recommendations based solely on surface-level user history (e.g., titles) to performing multi-step, evidence-gathering reasoning loops using an integrated retrieval engine.

3. It can perform a dynamic sequence of actions:

4. Query the unified retrieval corpus (Collaborative Memory or Meta Memory) using natural language queries generated by the LLM policy.

5. Receive specific, grounded evidence (e.g., Find user histories of people who liked Item X or Find the genre and director of Item Y).

6. Re-reason over this newly acquired context to synthesize a final recommendation, rather than relying on pre-programmed heuristics or static context injection rules.

7. The system will achieve superior performance by optimizing its retrieval strategy directly against the final ranking metric (Outcome-Driven Reinforcement Learning).

8. It learns an adaptive policy that dictates:

9. Whether retrieval is necessary at all (context sufficiency assessment).

10. Which memory source to query (Collaborative vs. Meta Memory) when evidence is needed.

11. How to formulate the most effective natural-language queries required to extract the most decision-relevant information for a specific instance, rather than using fixed, pre-defined retrieval rules or handcrafted CF injection methods.

12. The system will operate with significantly improved context efficiency and reduced latency by employing selective evidence acquisition.

13. It avoids the context-length bottleneck by only retrieving external information when the current lightweight context is insufficient to make a reliable ranking decision, leading to shorter, more concise reasoning traces and lower inference costs during high-volume recommendation tasks.

14. The system will exhibit enhanced robustness and accuracy for difficult recommendation scenarios (e.g., long-tail or niche items).

15. When faced with a niche item (where title-only context is insufficient), the system can proactively generate targeted queries to retrieve specific item metadata (like author, genre, or series ID) to ground its understanding of the item's attributes before making a recommendation.

16. The system will demonstrate superior generalization across different data modalities and domain characteristics through learned adaptability.

17. The policy is trained end-to-end with a ranking reward that balances accuracy against structural validity, allowing it to automatically adjust its reliance on collaborative signals versus item metadata based on the specific characteristics of the input dataset (e.g., prioritizing metadata retrieval for long-tail items in Goodreads, and collaborative history for pattern-driven music preferences).

18. The system will provide explainable and grounded recommendations.

19. Because every decision is preceded by an explicit reasoning step (block) that either concludes sufficiency or details the evidence retrieved (via and), the final recommendation is inherently traceable back to the specific pieces of collaborative history or metadata that drove it, significantly improving interpretability.

Large Language Models (LLMs) have emerged as a promising paradigm for next-generation recommender systems, offering strong semantic understanding and natural-language reasoning abilities. Despite recent progress, current LLM-based recommenders still face key challenges in constructing decision-relevant contexts from heterogeneous evidence. First, existing methods often rely on fixed context construction strategies: collaborative behavioral evidence and item-side metadata are typically incorporated through predefined prompts, static retrieval pipelines, or handcrafted injection mechanisms, making it difficult to determine what information is truly beneficial for each instance. Second, heterogeneous evidence introduces a severe context-efficiency bottleneck. Rich metadata and collaborative interaction records can quickly overwhelm the context window, while aggressive compression or heuristic filtering may discard fine-grained evidence critical for accurate recommendation. To address these challenges, we propose RRCM, a ranking-driven retrieval-and-reasoning framework over collaborative and metadata memories for LLM-based agentic recommendation. RRCM starts from a lightweight user-history context and learns whether to recommend directly, retrieve collaborative evidence, retrieve item metadata, or interleave both through reasoning. Both memories are represented in natural language and accessed through a unified retrieval interface, enabling flexible evidence acquisition without handcrafted CF injection or fixed retrieval rules. We optimize this memory-reading policy with an outcome-only ranking reward, instantiated using group relative policy optimization, so that retrieval decisions are directly driven by final top-k recommendation quality. Extensive experiments show that RRCM significantly outperforms traditional baselines and diverse LLM-based recommendation approaches.