**How Agentic Memory Research Unifies Long-term and Short-term Memory for LLM Agents**
Imagine designing a language model (LLM) agent that can effortlessly store, retrieve, and manage both long-term and short-term memory without relying on hand-tuned heuristics or additional controllers. Sounds like a tall order, right? Well, researchers from Alibaba Group and Wuhan University have proposed a framework called AgeMem, which makes this possible. In this article, we’ll dive into how AgeMem achieves this unification and what it means for LLM agent design.
**The Struggle with Memory in LLMs**
Current LLM agent frameworks treat memory as two separate entities:
1. **Long-term memory**: stores user profiles, job information, and previous interactions across sessions.
2. **Short-term memory**: the current context window, which holds the active dialogue and retrieved documents.
These two components are designed in isolation, leading to several issues:
* Long-term and short-term memory are optimized independently, resulting in limited interaction and poor generalization.
* Heuristics determine when to store, update, or discard memory, which can be brittle and miss rare but important events.
* Additional controllers or trained models increase complexity and costs.
**AgeMem: A Unified Memory Framework**
AgeMem removes the external controller and folds memory operations into the agent’s policy itself. This approach allows the agent to make more informed decisions about what to store, retrieve, summarize, and discard. Memory operations are exposed as tools, which the agent can use at each step. These tools include:
* **Long-term memory tools**: `ADD`, `UPDATE`, and `DELETE` to manage long-term memory.
* **Short-term memory tools**: `RETRIEVE`, `SUMMARY`, and `FILTER` to manage short-term context.
The agent interacts with these tools to build, maintain, and retrieve memory. For instance, when the agent decides to “ADD” a new memory item, it can also choose to “UPDATE” an existing entry or “DELETE” an outdated object.
**Three-Stage Reinforcement Learning**
AgeMem is trained using reinforcement learning in three stages:
1. **Stage 1: Long-term memory development** – The agent interacts with the environment, observing information that may become relevant later. It uses `ADD`, `UPDATE`, and `DELETE` to build and maintain long-term memory.
2. **Stage 2: Short-term memory control under distractors** – The short-term context is reset, and long-term memory persists. The agent receives distractor content and must handle short-term memory using `SUMMARY` and `FILTER` to maintain helpful content.
3. **Stage 3: Integrated reasoning** – The final question arrives, and the agent retrieves from long-term memory using `RETRIEVE`, controls short-term context, and produces a response.
**Reward Design and Step-Wise GRPO**
AgeMem uses a variant of Group Relative Coverage Optimization (GRPO) with a step-wise reward function. The total reward has three components:
* **Job reward**: scores response quality between 0 and 1 using an LLM evaluator.
* **Context reward**: measures the quality of short-term memory operations, including compression, summarization, and preservation of question-related content.
* **Memory reward**: measures long-term memory quality, including the fraction of high-quality stored objects and the relevance of retrieved objects to the question.
**Experimental Results**
The researchers fine-tuned AgeMem on the HotpotQA training split and evaluated it on five benchmarks:
* ALFWorld for text-based embodied tasks.
* SciWorld for science-themed environments.
* BabyAI for instruction following.
* PDDL tasks for planning.
* HotpotQA for multi-hop question answering.
AgeMem outperformed the best baseline, Mem0, on all five benchmarks, with a median score of 41.96 on Qwen2.5-7B-Instruct and 54.31 on Qwen3-4B-Instruct.
**Key Takeaways**
* AgeMem turns memory operations into specific tools, allowing the agent to decide when to store, update, or discard memory.
* Long-term and short-term memory are trained together using a three-stage RL setup.
* The reward function combines job accuracy, context management quality, and long-term memory quality with uniform weights and penalties for context overflow and extreme dialogue size.
* AgeMem consistently outperforms memory baselines on average scores and memory quality metrics.
**Conclusion**
AgeMem presents a design pattern for future agentic systems. By integrating memory operations into the agent’s policy, AgeMem demonstrates the potential for more efficient and effective memory management. This approach can lead to better performance and improved decision-making in LLM agents.
