Procedural Memory in Agents: A Comprehensive Guide

Introduction

Procedural memory is the cognitive system that stores knowledge of how to perform tasks—the skills and procedures that become automatic with practice. For AI agents, procedural memory represents the ability to internalize and automate repetitive task patterns, decision-making processes, and interaction sequences, leading to more efficient and context-aware responses over time [citation:1]. Unlike semantic memory (facts and concepts) or episodic memory (personal experiences), procedural memory captures the "how-to" knowledge that enables fluent, unconscious execution of complex behaviors [citation:5].

This guide explores the foundations of procedural memory in AI agents, examining the core concepts, architectural frameworks, and practical implementation strategies that enable agents to learn from experience and perform tasks with increasing efficiency.

What Is Procedural Memory?

In cognitive science, procedural memory is a type of long-term memory that involves the retention of procedures and skills—such as typing, riding a bike, or playing a musical instrument—which are performed automatically without conscious thought [citation:1]. For AI agents, procedural memory enables the internalization of reusable patterns of reasoning, tool sequences, and recovery tactics [citation:1].

The agent utilizes procedural memory to automate repetitive tasks, decision-making processes, and interaction patterns, leading to more efficient and context-aware responses over time [citation:1]. This stands in contrast to working memory (which holds current context) and semantic memory (which stores factual knowledge).

Key Properties of Procedural Memory

Procedural memory in AI agents exhibits several distinguishing characteristics:

Implicit operation: Like human procedural memory, agentic procedural memory operates below the level of conscious deliberation. The agent executes learned procedures without needing to reason through each step [citation:5].
Skill automation: Repeated tasks become automated, reducing cognitive load and execution time [citation:2].
Experience-based refinement: Procedural knowledge improves with practice, becoming more efficient and reliable [citation:1].
Transferable expertise: Procedural skills can be extracted, packaged, and transferred between agents [citation:2][citation:3].
Deterministic execution: Unlike probabilistic reasoning, well-learned procedures execute with consistency and reliability [citation:2].

Procedural Memory Frameworks for AI Agents

The CoALA Framework

The CoALA (Cognitive Architectures for Language Agents) framework organizes agent memory into working, episodic, semantic, and procedural categories [citation:2]. While working memory (context) and semantic memory (RAG) are well-solved, procedural memory remains a bottleneck. CoALA suggests three possible approaches for capturing procedural memory in AI Agents [citation:2]:

Rewriting the code of the Agent: Directly modifying the agent's executable logic
Editing the weights of the LLMs: Updating model parameters through fine-tuning
Editing the instructions of the AI Agent: Modifying prompts and system instructions

In practice, it is fairly uncommon for agents to modify their model weights or rewrite their code. However, it is more common for agents to modify their own prompts [citation:2].

LIDA's Procedural Memory Module

The Learning Intelligent Decision Agent (LIDA) cognitive architecture implements procedural memory through a modified schema mechanism inspired by Piaget's constructivism [citation:5]. In this framework, procedural memory stores an internal model of the agent's actions and their situation-dependent outcomes, generating predictions about what might occur if the agent were to execute an action in a given context [citation:5].

Action Selection uses these predictions to choose the next behavior based on the agent's motivations and goals. These modules are also instrumental in supporting mental simulation and planning through the execution of internal behaviors [citation:5].

Starting with no a priori knowledge of its environment, an agent based on this implementation can learn an accurate internal model of its interactions with an environment and use that knowledge to select goal-directed and exploratory behaviors [citation:5].

The Memp^p Framework

Memp^p is a task-agnostic framework that treats procedural memory as a first-class optimization object [citation:1]. The framework explores how different strategies for memory construction, retrieval, and updating affect overall performance. Core explorations include [citation:1]:

Construction: Distilling past agent trajectories into both fine-grained, step-by-step instructions and higher-level, script-like abstractions
Retrieval: Experimenting with various key-building strategies such as query-vector matching and keyword-vector matching
Update: Implementing diverse strategies including ordinary addition, validation filtering, reflection, and dynamic discarding

Evaluation on TravelPlanner and ALFWorld shows that as the memory repository is refined, agents achieve steadily higher success rates and greater efficiency on analogous tasks. Moreover, procedural memory built from a stronger model retains its value—migrating the procedural memory to a weaker model yields substantial performance gains [citation:1].

CodeMem: Deterministic Procedural Memory

CodeMem reframes the LLM as an architect of executable workflows [citation:2]. Instead of standard chat-based tool calling, the agent utilizes a sandbox to write, validate, and save successful logic into a persistent procedural memory bank. This approach solves the reproducibility crisis inherent in probabilistic models by shifting complex logic from volatile context windows into deterministic code [citation:2].

Key design principles include [citation:2]:

Context efficiency: Intermediate data transformations occur in the sandbox's memory, not the LLM's context window, allowing infinite-depth processing
Logic reliability: Python code executes deterministic logic rather than relying on probabilistic LLM predictions
Reproducibility: Versioned scripts ensure consistent execution across sessions

Skill Acquisition Pathways

Research has identified three primary pathways for acquiring procedural memory in AI agents [citation:3][citation:6]:

1. Human-Authored Skills

Domain experts manually author skill packages (SKILL.md files) that encapsulate procedural knowledge. This approach provides reliability guarantees but suffers from scalability constraints [citation:3]. The Anthropic Skills repository, released as an open standard in December 2025, accumulated over 62,000 GitHub stars within four months [citation:3].

2. Autonomous Skill Discovery

Agents discover and learn skills through exploration and iterative feedback. The EXIF framework employs an exploration-first strategy where an exploration agent interacts with the environment to retrospectively generate feasible, environment-grounded skill datasets [citation:9].

An iterative feedback loop evaluates performance to identify areas for improvement, guiding subsequent exploration in a closed-loop data generation process [citation:9]. Experiments on Webshop and Crafter demonstrate EXIF's ability to effectively discover meaningful skills without human intervention [citation:9].

3. Automated Skill Extraction

Procedural knowledge is systematically extracted from existing open-source software repositories. The extraction pipeline involves repository structural analysis, semantic skill identification through dense retrieval, and standardized translation to the SKILL.md format [citation:6].

This approach demonstrates that systematic extraction from agentic repositories enables scalable acquisition of procedural knowledge that augments LLM capabilities without requiring model retraining [citation:6].

Procedural Memory vs. Other Memory Types

Dimension	Procedural Memory	Semantic Memory	Episodic Memory
Content	Skills, procedures, "how-to" [citation:1]	Facts, concepts, general knowledge [citation:3]	Personal experiences, events [citation:3]
Operation	Implicit, automatic [citation:5]	Explicit, declarative	Explicit, contextual
Example	Tool sequence pattern [citation:1]	"API authentication uses OAuth" [citation:3]	"Last Tuesday, approach X failed" [citation:3]
Representation	Code, workflows, scripts [citation:2]	Knowledge graphs, vectors [citation:3]	Events, trajectories [citation:1]

Neural Procedural Memory: A New Frontier

Recent research has introduced Neural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit instructions [citation:4]. By distilling procedural skills from historical contrastive experiences into steering vectors in the activation space, NPM directly activates the task-relevant neural mechanisms to guide task execution [citation:4].

Key findings include [citation:4]:

NPM performs comparably to baselines using explicit textual instructions
Combining implicit steering with explicit workflows provides complementary advantages
Steering vectors encode consistent task logic, forming organized structures within the activation space

This suggests that implicit activation steering provides a promising approach for managing agent memory, potentially overcoming the text-action disconnect that often limits explicit instruction-based methods [citation:4].

Key Design Considerations

Context Cost vs. Execution Efficiency

Standard tool calling incurs context cost of approximately ∑(Sprompt + Shistory + Stool_output_i) across each step. CodeMem reduces this to approximately Sprompt + Scode_block + Sfinal_result, significantly reducing token consumption while enabling infinite-depth processing [citation:2].

Probabilistic Instability vs. Deterministic Reliability

Because LLMs are non-deterministic, relying on procedural prompts or instructions does not guarantee consistent adherence. CodeMem treats procedural memory as frozen code, ensuring deterministic execution without depending on the model's instruction-following capability [citation:2].

Skill Portability

Procedural memory built from a stronger model retains its value when migrated to a weaker model, yielding substantial performance gains. This suggests that skills are separable from the model that discovered them [citation:1].

Related Concepts

AI Agent Memory Architectures — The broader landscape of memory systems for AI agents
Episodic Memory for AI Agents — Personal experiences and event-based recall
Semantic Memory Systems — Facts, concepts, and general knowledge
Agent Skills — Modular packages of procedural knowledge
Model Context Protocol — Standardized tool connectivity
AI Agent Architecture — Foundation Agent, Core Components, Agent Systems

Conclusion

Procedural memory is essential for AI agents that must perform repetitive tasks efficiently, learn from experience, and operate reliably in dynamic environments. The field has evolved from theoretical cognitive frameworks to practical implementations like CodeMem and Memp^p that treat procedural memory as a first-class optimization object [citation:1][citation:2].

As one researcher notes, "by turning earlier trajectories into reusable templates like patterns of reasoning, tool sequences, and recovery tactics, it can progress step by step, learning from every failure and success, until even the most convoluted missions become routine" [citation:1]. This capability to distill, chronicle, and re-apply lessons from experience is the bedrock of human learning and the pivotal gateway through which an agent ascends toward self-directed refinement [citation:1].

For developers building production AI agents, procedural memory is not an optional enhancement—it is a foundational capability that determines whether agents can internalize expertise, perform consistently across sessions, and evolve toward mastery over time.

References

Fang, Runnan, et al. Memp^p: Exploring Agent Procedural Memory. arXiv. 2025.
Gaurav, Nishant, et al. CodeMem: Architecting Reproducible Agents via Dynamic MCP and Procedural Memory. arXiv. 2025.
Xu, Renjun, et al. Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward. arXiv. 2026.
Zhao, Chengfeng, et al. Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering. arXiv. 2026.
Kugele, Sean. Constructivist procedural learning for grounded cognitive agents. Cognitive Systems Research. 2025.
Bi, Shuzhen, et al. Automating Skill Acquisition through Large-Scale Mining of Open-Source Agentic Repositories. arXiv. 2026.
Zhang, Zeyu, et al. Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information. arXiv. 2025.
Wheeler, Schaun, et al. Procedural Memory Is Not All You Need: Bridging Cognitive Gaps in LLM-Based Agents. arXiv. 2025.
Yang, Yongjin, et al. Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback. arXiv. 2025.

Search This Blog

Agentic Mind