MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models

arXiv:2605.28009 · cs.CL, cs.AI, cs.LG · Submitted 2026-05-27 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models".

Jane: Memory-augmented large language models extend reasoning beyond fixed context windows by maintaining long-term memory across interactions,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we've just been introduced to the paper titled "MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models," which is looking at how memory systems in these large language models can get messy over time. Basically, the core idea they are pushing is that current memory setups often mix up different types of information so that a user's specific facts and their general rules start blending together, which leads to bad outputs.

Jane: That sounds like a really important problem because if the AI mixes up what's a specific event from last week with its general knowledge about how things work, the answers it gives aren't going to be reliable at all. The abstract makes it clear that they are identifying this issue as heterogeneous memory contamination where functionally distinct memories end up being treated as interchangeable evidence.

Lu: From a creative standpoint, I see the potential here for truly nuanced reasoning because if we can keep those functional boundaries tight, the AI won't just regurgitate old facts but will actually be able to reason across different domains without losing its footing. This framework seems to offer a way to structure knowledge so it remains distinct while still being connected.

Meng: I'm curious about how this translates into something that actually runs well in a real application, because if the memory reorganization is too complex, it could just slow down the entire interaction process significantly for users. The summary mentions they are trying to preserve these functional boundaries across the whole memory lifecycle, from writing to retrieval and even how evidence is put together.

Lalam: I think what this means for our culture is that we can build AI assistants that remember our personal context without them starting to forget or act out of character because their long-term memory got cross-contaminated with unrelated stuff. If the AI can reliably separate its episodic memories from its procedural ones, it fosters a more trustworthy interaction environment for everyone.

Tom: Exactly, and what's really compelling about this is that they aren't just suggesting a fix; they are proposing a type-aware framework to handle those functional boundaries during both the creation and the retrieval stages of memory. They’re trying to stop that shared space from becoming one big, messy pool.

Jane: It sounds like the thesis is centered on introducing this specific mechanism to ensure that when an AI needs evidence, it pulls from a store that matches what it's actually asking about, rather than getting a mixed bag of things. This directly tackles the issue of how context-specific events can turn into overgeneralized claims.

Paper summary: Lu: The concept of assigning each memory exactly one functional role at write time seems like a very disciplined approach to knowledge management; it forces structure right from the start, which is something I find very promising for complex reasoning tasks. This single-type constraint is what keeps things isolated before they even get stored away.

Meng: From an engineering standpoint, decomposing raw dialogue into these type-specific atomic memories and then recording dependencies in a relational knowledge graph sounds like a lot of overhead to implement efficiently, especially when the memory grows very large during long interactions. I wonder how scalable this reorganization step is without introducing significant latency during the writing phase.

Lalam: If we can do that reorganization effectively, it means our AI can maintain a much richer and more organized internal state over longer conversations, which is essential for building deep, personalized user profiles that feel intuitive and accurate. It moves us away from just storing things linearly toward storing them intelligently based on their function.

Tom: Right, so we're moving from an unordered set of propositions to something structured where the connections between different types of memory are explicitly defined in a graph, which is a big step in how we model long-term context. This moves beyond just having a big memory dump and starts thinking about how those facts relate to each other across different categories.

Jane: The paper claims that this type-aware framework helps prevent contamination by keeping functionally distinct memories separate during the writing process, which is a crucial first step in maintaining integrity. They are focused on making sure semantic, episodic, and procedural memories stay in their designated silos initially.

Lu: That relational knowledge graph construction is where I see some of the deepest potential; by encoding dependencies between these typed atoms, you're not just storing isolated facts; you're storing a map of how those facts influence each other across memory types. That kind of structured dependency modeling opens up possibilities for much more sophisticated inference.

Meng: But the retrieval part also sounds tricky because they are doing query-adaptive routing, which means the system has to estimate which memory type is relevant before it even looks at the stores, and if that estimation is wrong, we lose that efficiency gain. How do they ensure that their confidence distribution over memory types isn't skewed by a poorly constructed initial query?

Paper summary: Lalam: If we can get the routing right, it means the AI doesn't waste time searching through irrelevant data just because it’s trying to be thorough; it intelligently targets the right type of knowledge first, which should make interactions feel much faster and more relevant for our users. It feels like a level of intelligent filtering that's really important for user experience.

Tom: So, we have this two-pronged approach: strong reorganization when writing to set up the types correctly, and then dynamic routing at retrieval time to use those types selectively based on the immediate query, which is what they call retrieval-time dynamic memory routing. That sounds like a very smart way to handle the complexity of long-term memory access.

Jane: The paper emphasizes that this dual mechanism is what allows them to preserve functional boundaries during both construction and retrieval, which directly addresses the failure modes of contamination they identified earlier in the summary. It's about applying constraints consistently throughout the entire memory lifecycle.

Lu: The experimental results are what really give us a sense of how effective this structure is in practice, especially when you look at the gains reported on benchmarks like HaluMem and LoCoMo where they show improvements like eighty-nine point five three percent antihallucination accuracy on one benchmark. Those kinds of concrete numbers really ground the theoretical framework in measurable performance improvements.

Meng: I’m looking closely at those results, especially the finding that uncertainty errors are mostly tied to write-time contamination, which suggests that fixing the initial structure is more impactful than trying to clean up retrieval errors later on. That points toward prioritizing memory construction quality.

Lalam: If we can focus our efforts on improving the initial memory organization—the write-time part—then we see a direct path to reducing those untrustworthy outputs that come from overgeneralized claims, which is exactly what users need to trust an AI with their long-term data.

Tom: And they also showed a reduction in retrieved tokens on LoCoMo by up to five point eight times compared to prior methods, which means this structure isn't just making the output better; it’s making the process more efficient too. That's a win for both quality and speed.

Jane: So, when we talk about the implications of "MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models," we're really talking about moving towards AI systems that manage their own knowledge base with more intentionality and less accidental blending of concepts. It’s about building a system where facts stay facts, events stay events, and rules stay rules.

Paper summary: Lu: I think the broader impact is that this level of internal organization could unlock AI capabilities that require sustained reasoning over very long sequences of interactions without losing track of the initial premises or the specific context that started the conversation. It’s about creating a persistent cognitive state that feels genuinely coherent.

Meng: For practical implementation, I see this as a way to make memory systems more modular; instead of one giant monolithic store, you have specialized stores for different types of information, which makes updating or verifying specific chunks much cleaner and less prone to accidental interference. That modularity is something engineers can actually work with.

Lalam: From my view, the most significant cultural shift this represents is moving from a reactive memory system where we constantly have to correct the AI's drift, to a proactive system where we design the AI's memory architecture upfront so it operates reliably within its defined constraints. That level of reliability builds public trust in these kinds of powerful tools.

Tom: So, to wrap up on this paper about "MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models," we've seen that the core idea is a type-aware framework that enforces functional boundaries during memory construction and retrieval. This tackles the contamination problem by treating functional boundaries as reliability constraints across the whole memory lifecycle, which leads to measurable improvements in accuracy and retrieval efficiency.

Jane: It really boils down to preventing functionally distinct memories from interfering with each other, ensuring semantic facts don't overwrite procedural rules or episodic events inappropriately during generation. The authors show that this selective use of heterogeneous memory is more effective than just scaling up the amount of memory you pull into a system indiscriminately.

Lu: I think the implication is that long-term reasoning becomes possible because the AI doesn't have to constantly re-learn how its stored knowledge fits together; it has a structured way to access and compose that knowledge based on its type. That structured composition is what allows for more sophisticated, sustained reasoning abilities.

Meng: For us in engineering, the practical implication is that we can design memory modules with clear roles, which simplifies debugging and scaling because if something goes wrong, we know exactly which memory silo it's coming from and where the contamination might have occurred.

Lalam: Ultimately, this work suggests that reliable long-term reasoning depends on principled organization and a selective use of heterogeneous memory; it’s about designing the system to respect what each piece of information is supposed to be, which fosters a much more dependable interaction environment overall.

Conclusion: Tom: So we've been diving deep into MemGuard, and now it's time to wrap up our chat about this paper titled "MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models."

Jane: It really boils down to how the authors tackle that messy problem of different types of memories getting mixed up inside these long-term AI systems.

Lu: Exactly, and the core idea is using a type-aware framework to make sure semantic facts don't bleed into procedural rules or episodic events inappropriately.

Meng: From an engineering standpoint, it seems like they've focused on making that distinction at the very moment memory is written, which makes sense for stability.

Lalam: This work suggests that reliable long-term reasoning really hinges on having a principled organization of what the AI remembers and how it uses those different kinds of memories.

Tom: And that selective use of memory is what really makes this paper interesting; it’s not just about storing more data, but about storing it intelligently.

Jane: That selective approach means we can build systems where the AI isn't just pulling a random pile of information, but rather retrieving exactly the right type of context needed for the task at hand.

Lu: It opens up possibilities for sustained reasoning because the AI doesn't have to constantly re-learn how all its stored knowledge fits together; it has a structured way to access and compose that knowledge based on its type.

Meng: I see why that structured composition matters, especially when you think about scaling these models; having clear silos makes debugging and verifying specific chunks much cleaner than trying to clean up a massive, undifferentiated memory blob.

Lalam: For us in the culture side, this means we can design AI assistants that remember our personal context with more intentionality and less accidental blending of concepts, which fosters a much more dependable interaction environment for everyone.

Tom: So we've seen how MemGuard uses write-time reorganization and retrieval-time routing to keep those functional boundaries intact.

Jane: It really shows that by respecting what each piece of information is supposed to be, we can move toward AI systems that manage their knowledge base with much more intentionality.

Lu: And the results they showed on benchmarks like HaluMem and LoCoMo give us concrete evidence that this structural approach leads to tangible improvements in accuracy and efficiency.

Meng: Those measurable improvements are what I look for most; it proves that focusing on the initial organization quality actually yields a lot of practical gains down the line.

Lalam: This paper really shows us a path toward building long-term reasoning capabilities because we are designing the system to respect its internal constraints from the very beginning.

Tom: And that brings us to where we need to go next; we've covered how MemGuard works and why it matters for reliability, so what does this mean for the future of memory-augmented AI?

University of Illinois Urbana-Champaign

cs.CL, cs.AI, cs.LG

Submitted: 2026-05-27

Updated: 2026-09-30

Importance score: 89/100

The gist: Memory-augmented large language models extend reasoning beyond fixed context windows by maintaining long-term memory across interactions, but existing systems often suffer from heterogeneous memory

Key concepts

Heterogeneous Memory Contamination
This occurs when different types of memory—like facts versus personal events—become mixed or interchangeable. This mixing confuses the model, leading to incorrect generations because knowledge from one functional domain leaks into another, causing errors.
Type-Aware Memory Reorganization
During writing, this process breaks down raw conversation into small 'atomic memories' and strictly assigns each one a single functional type (semantic, episodic, or procedural). This ensures that knowledge stays isolated in its correct silo from the start.
Relational Knowledge Graph
This is a structured map used to record how different memory atoms are related to each other. It tracks dependencies between memories while maintaining strict separation based on their assigned functional type, allowing for controlled cross-referencing later.
Query-Adaptive Type Routing
Instead of retrieving everything indiscriminately, this method uses the user's query to estimate which memory types are most relevant. It then limits retrieval to only the specific memory stores needed, ensuring that only the correct functional knowledge is accessed.

Terminology

Summary

Memory-augmented large language models extend reasoning beyond fixed context windows by maintaining long-term memory across interactions, but existing systems often suffer from heterogeneous memory contamination where functionally distinct memories become interchangeable and mislead generation. MEMGUARD introduces a type-aware framework to preserve these functional boundaries during construction and retrieval, improving reliability by up to 28.27% while reducing retrieved tokens.

How it works

MEMGUARD addresses the failure mode of heterogeneous memory contamination by treating functional boundaries as reliability constraints across the entire memory lifecycle—writing, retrieval, and evidence composition. At write time, the framework performs type-aware memory reorganization, decomposing raw dialogue into type-specific atomic memories and recording their dependencies in a relational knowledge graph. This ensures that each atom is assigned exactly one functional type (semantic, episodic, or procedural), enforcing a single-type constraint to prevent knowledge from being compressed into a shared representation.

Write-Time Memory Reorganization

The process involves several steps:

  1. Type-Aware Knowledge Decomposition: The LLM decomposes the conversation into non-overlapping memory atoms, each assigned an explicit functional type (e.g., semantic, episodic, procedural).

  2. Self-Verified Extraction: A verification step identifies missing atoms and recovers them to ensure every atom is atomic and tied to a single memory type.

  3. Relational Knowledge Graph Construction: The system builds a directed typed graph over the atom set, where edges encode typed dependencies among them, preserving cross-atom relationships while maintaining type isolation at the storage level.

  4. Type-Isolated Memory Writing: Each atom is routed to its corresponding typed store (e.g., Semantic Store), and operations are restricted to ADD, UPDATE (within the same type), or SKIP. This prevents functionally distinct memories from overwriting one another within their specific silos.

Retrieval-Time Dynamic Memory Routing

At retrieval time, MEMGUARD utilizes two stages: query-adaptive type routing and relational composition. First, a prompt-based soft router estimates the relevance of each memory type based on the query, outputting a confidence distribution over memory types. This allows for type-specific budget kτ proportional to w, where retrieval is performed only over the corresponding store Mτ, making retrieval query-adaptive.

Relational Knowledge Composition

To recover necessary cross-memory dependencies missed by routing, MEMGUARD expands primary results over the relational knowledge graph G. For each retrieved node, a Breadth-First Search (BFS) is performed up to a maximum hop depth (hmax) to collect reachable nodes. These reachable nodes are composed into relation-aware context entries using concatenation with explicit relation labels, scored by query relevance with hop decay, which restores cross-memory dependencies without compromising storage boundaries.

Key Results and Findings

Experiments on benchmarks like HaluMem and LoCoMo demonstrate that MEMGUARD substantially improves memory reliability. On HaluMem, it achieves 89.53% (+28.27%) antihallucination accuracy and 71.49% (+9.38%) memory update correctness. On LoCoMo, the framework retains competitive performance while retrieving up to 5.8× fewer memory tokens than prior methods. The analysis shows that unverifiability errors are predominantly associated with write-time contamination (97.7%), indicating that unsupported or overgeneralized knowledge is often introduced before retrieval occurs, while factuality errors are associated with retrieval-time contamination (63.8%).

Conclusion

The paper concludes that reliable long-term reasoning depends on principled organization and selective use of heterogeneous memory. MEMGUARD's design—preserving functional boundaries through write-time reorganization and dynamic routing at retrieval time—effectively mitigates cross-type interference, thereby preventing persistent hallucinations in memory-augmented LLMs. The framework shows that selectively retrieving the right memory types by preserving functional boundaries is more effective than scaling retrieval indiscriminately.

The gist: MEMGUARD introduces a type-aware framework to preserve functional boundaries during memory construction and retrieval, improving reliability by up to 28.27% while reducing retrieved tokens.

Write-Time Memory Reorganization

The process involves several steps:

Improvements for AI systems

Here are specific improvements for AI systems based on the MEMGUARD framework:

The core improvement is shifting from a monolithic, semantically similar memory store to a structured, type-aware memory governance system that preserves functional boundaries across the entire lifecycle (writing, retrieval, composition).

Here is what an improved AI system can do:

Use of explicit functional roles during memory construction:

The system will decompose incoming conversational data into distinct memory atoms, each assigned a specific functional type (Semantic, Episodic, Procedural) at the point of writing. This prevents episodic events from being overgeneralized into stable facts or procedural rules from being treated as simple data points.

Type-Isolated Storage:

Instead of storing all memories in one shared vector space, the system will maintain separate memory stores for each type (e.g., a dedicated store for Asthma Constraints vs. a store for Headache Success Cases). This structural isolation prevents functional incompatibility from causing contamination during storage.

Query-Adaptive Routing:

During retrieval, the system will use a soft router to estimate the utility of each memory type based on the user's query. It will then allocate a specific budget of tokens to only retrieve memories from relevant stores (e.g., if the query is about medication safety, it prioritizes querying Semantic and Procedural stores while minimizing retrieval from Episodic stores that might be irrelevant). This reduces noise significantly compared to uniform similarity search.

Relational Knowledge Graph Composition:

When composing an answer, the system will not rely solely on top-K retrieved snippets. Instead, it will leverage a typed relational knowledge graph constructed at write time. It can perform graph-guided composition, allowing it to trace necessary cross-type dependencies (e.g., linking a procedural recommendation from one memory type with a semantic constraint from another) without ever merging the underlying memory entries themselves.

Increased Reliability and Reduced Hallucination:

The system will exhibit up to 28% improvement in anti-hallucination accuracy (as shown on HaluMem). By enforcing principled organization and selective use of evidence, the model is far less likely to generate unsupported claims or make unverifiable statements, leading to higher reliability in long-horizon conversations.

Optimized Efficiency:

MEMGUARD can retrieve up to 5.8× fewer memory tokens than prior methods while maintaining high accuracy, suggesting that this principled approach is more efficient than scaling retrieval indiscriminately across the entire dataset.

In summary, the improved AI system moves from a reactive, similarity-based retrieval mechanism to a proactive, governed memory architecture that treats knowledge as structured evidence with defined roles.

Sources

Related papers