GoClaw features a multi-tiered memory architecture designed to provide agents with both short-term session recall and long-term persistent context. This system is optimized to operate efficiently on local servers or NAS devices while ensuring multi-user security, high performance, and continuous learning.
This page is the overview: scopes and who may write them, what goes into the prompt, search settings, the memory lifecycle and administration. The engine's internals (tables, ranking formulas, chunk IDs, the session summarizer and memory flush, the background job queue, where the code lives) are in Memory Engine Architecture.
GoClaw divides memory storage into two main layers: an embedded database for dynamic, queryable records, and Markdown files for structured archives and core identity.
By default the dynamic records live in an embedded SQLite database, gc_state.sqlite in the state directory (~/.goclaw/, or GOCLAW_STATE_DIR), running in WAL (Write-Ahead Logging) mode. database.driver can put the same tables in PostgreSQL, MySQL or ClickHouse instead. The memory tables are:
memories): the content, its scope, a classification domain, and a JSON metadata column (provenance, superseded_by, …).memory_chunks): text segments, with their embeddings when an embedding provider is configured, that search runs on.memory_tags): one row per tag on a memory (memory_id, tag), such as critical, superseded or unverified.memory_entities, memory_relations). A copy for graph queries is kept in a second database: kg.sqlite in the state directory, or PostgreSQL with memory.graph.backend: postgres.For bootstrap state and historical logging, GoClaw uses standard text files in each agent's workspace:
Memory.md: Long-term curated knowledge base. Injected into the prompt on every turn (see Session Context Injection); it changes only when someone edits it, runs memory_consolidate, or rebuilds it with goclaw memory replay.IDENTITY.md: The agent's core character definition, instructions, and behavior limits.memory/archive/*.md: Archived historical chat sessions, kept for audit and memory replay.The org wiki holds curated pages of organization knowledge (data dictionaries, runbooks, how systems connect), edited by people on the Web UI's Wiki page and by the agent with wiki_write (human admins and SOP/cron runs only; wiki_read is open to all).
org_wiki_revisions; the newest version of a slug is the page, and older versions can be restored.wiki, so memory_search (and SOP and cron runs) find it. Wiki pages are not injected into the prompt and are not subject to the agent-notes budget.<state dir>/media, listed in wiki_attachments) and served at /media/<id>.POST /api/memory/wiki/distill) asks the default agent's model to turn the learned facts in agent_notes into pages under learned/<domain>, saved as agent edits for a person to review. It runs only when triggered./api/memory/wiki (with /history, /restore, /search, /attachments and /distill).tools.wikijs): read-only copies under the slugs wikijs/<site>/<locale>/<path>, searchable as memory scope wikijs. See Wiki.js as Org Knowledge.To maintain security and prevent context contamination, memory is divided into distinct scopes with strict access boundaries:
| Scope | Purpose | Storage | Writer | Reader |
|---|---|---|---|---|
personal |
User preferences, custom facts, and private configurations. | Database | That user (memory_write), fact extraction from their own messages, and the session summary of a conversation that was theirs alone |
Only that specific user |
agent_notes |
Organization knowledge: learned facts and preferences, host details, schema notes. | Database | Administrators (memory_write, @notes:); roles or users listed under memory.write_access (memory_write only); SOP, cron and webhook runs; fact extraction from an administrator's messages |
All users & agents |
global |
Shared team guidelines, instructions, and standard values. | Database | The roles and users listed under memory.write_access while it is enabled (the default); administrators when it is disabled |
All users & agents |
agent_config |
Core settings and parameters modifying agent behavior. | Database | Administrators and the roles/users listed under memory.write_access (memory_write); administrators (@config:) |
System prompt injector |
memory_write refuses any other scope. Three more scopes are written by GoClaw itself: wiki (the searchable copy of org wiki pages), wikijs (copies of Wiki.js pages) and db_cache (schema notes from db_analyze / db_enrich). Memory.md is not a scope: it is a file in the agent's workspace, injected on every turn.
personal scope, and never read a user's personal memory. This prevents automated tasks from corrupting or polluting a human user's private context.agent_config or agent_notes needs an administrator or a role or user listed under memory.write_access; global, while write_access is enabled, needs a listed role or user. A regular user is refused and pointed to personal. Autonomous runs (SOP, cron and webhook) may write agent_notes (they carry out admin-authored procedures) and never global or agent_config: they work on outside input, which must not become a standing instruction.On every turn, GoClaw assembles the context of the run (SessionContexts in agent.go) in two parts. The session's part holds for the whole conversation and follows the static system prompt. The message's part is what was found for this message; it follows the person's message, after the history:
Session's part (system prompt, after its static part)
Memory group
1. Critical constraints (agent notes tagged `critical`, always kept)
2. Memory.md (a file over 20,000 characters keeps its first 14,000 and last 6,000)
3. Agent configuration (`agent_config` records, verbatim)
4. A hint with the agent-note count
5. Agent notes that name the current user
6. RECENT MEMORY (global records of the last 7 days, and the user's personal records)
7. PREVIOUS CONTEXT (the summary of trimmed earlier conversation)
System group
8. Configured datasources, GitLab and SonarQube instances, observability instances, MCP servers
9. The newest memory files (up to 15)
User group
10. USER.md (only when the request has no identified user)
11. The current user's identity block
Message's part (after the person's message, as <message_context>)
12. The current time
13. Agent notes: up to 5 notes matching the message
14. LEARNED KNOWLEDGE (learned facts and preferences ranked by the message, not shown above)
15. PERSONAL MEMORY (the user's 5 personal records most relevant to the message)
16. Context fetched by message hooks
17. The queries `db_query` ran without error earlier in the conversation
The split is for the providers' prompt caches, which read a prompt from its start: with what was found for each message in the system prompt, ahead of the history, every turn's first call wrote the whole conversation to the cache again. On Claude models that take one (Sonnet 5.5, Opus 5.5, Haiku 5.5, Opus 5, Opus 4.8, Fable and Mythos) the message's part is a system message after the person's message, which a person's text cannot imitate; elsewhere it is a paragraph or part of the message after the person's words. The person's text itself is sent as the session keeps it, with no timestamp in front: the time is in the message's part.
SOP, cron and webhook runs get no personal records (in 6 and 15) and no PREVIOUS CONTEXT. Tool errors are not stored at all: a failing call gets its hint in the same turn (see Auto-Learn).
To keep the context window efficient, each part has its own cap: RECENT MEMORY takes at most 6,000 characters of global records and 6,000 of personal records, walking from newest to oldest so the most recent context survives. The memory groups are then held to memory.context_budget_chars (default 32,000 characters; 0 means no limit). The session's sections get at most three quarters of it, whatever the message found, so that the session's part does not change with the message; the message's sections get what is left. Over its share, the lowest-priority sections are dropped first: in the session's part the note-count hint, then RECENT MEMORY and PREVIOUS CONTEXT, with critical constraints, agent configuration and Memory.md dropped last; in the message's part the matching notes, then PERSONAL MEMORY, then LEARNED KNOWLEDGE.
In multi-user environments (such as shared Discord servers or MS Teams channels), GoClaw guarantees strict privacy boundaries:
To maintain instant response times during chat, GoClaw processes all memory-heavy tasks—such as chunking, embedding generation, and indexing—asynchronously in the background.
flowchart LR
IN["Incoming Write"] --> DB["Write to SQLite"]
DB --> Q["Add to Background Queue"]
Q --> W["Worker Processes Chunking & Vector Generation"]
GoClaw utilizes a Hybrid Reciprocal Rank Fusion (RRF) search algorithm to find relevant context. Rather than relying solely on semantic vector search, it combines:
memory.search.provider and memory.search.model) to capture conceptual meaning. Without a provider, search is keyword-only.memory.search.min_score (default 0.35). memory.search.max_results (default 6) caps the results.Ranking always puts something first, however little matches. So a result must also show evidence that it answers the query: a cosine similarity to the query of at least the floor, or at least half of the query's terms in its text. The floor is memory.search.min_similarity when set; otherwise 0.62 for gemini-embedding-001, and no floor for other embedding models (their similarities sit on another scale). A negative min_similarity turns the check off.
0.4, ensuring highly relevant but old facts are still discoverable.Documents are parsed and split into target sizes of 400 tokens (1,600 characters) with an overlap of 80 tokens (320 characters). The chunker splits text at paragraph, newline, and sentence boundaries to preserve the logical flow of information.
Memory undergoes a continuous lifecycle of summarization, archiving, and consolidation:
flowchart LR
CHAT["Active Chat"] --> TRIM["Auto-Trimming & Archiving"]
TRIM --> FACT["Fact Extraction"]
FACT --> SCOPE["agent_notes / personal"]
SCOPE --> CONS["Nightly Consolidation"]
When a session's message count or token usage exceeds configured thresholds, GoClaw triggers an automatic cleanup:
memory/archive/*.md).personal memory scope.The system cron job sys-memory-consolidate runs every day at 03:00 and queues a memory_consolidate job on the memory worker. It:
agent_notes facts and preferences that say the same thing into one record. Each merged source is retired as superseded and recorded in memory_revisions (via consolidation), so a merge can be undone like any other revision. Unverified, superseded and critical records are left out.It does not touch Memory.md, which stays curated by hand.
Memory.md grows only on request, through the memory_consolidate tool (admins and SOP/cron runs). It takes the global records and the daily memory files (memory/YYYY-MM-DD.md) older than memory.consolidate_after_days (default 30) or the call's lookback_days, has the model extract the key facts, appends them to Memory.md under a dated heading, deletes those global records, moves the daily files to memory/archived/, and re-indexes. dry_run: true lists what it would take.
The agent tends to re-learn a topic in slightly different words, and each version used to stay live and be injected. When a learned fact or preference (agent_notes) is written, older ones on the same subject are retired, never deleted:
writeLearnedRecord catch the same IP, a <service> <field> is … value, and a few single-valued preferences (response language, report format, timezone, …).belief_revision job on the memory worker finds up to three older live records of the same kind that share at least 40% of the new record's words. A small call to the background model (the one fact extraction uses: Gemini where ai.gemini has a key, otherwise the default agent's own model) then decides, for each, whether the new record fully replaces it. When unsure, it keeps the old record.A retired record is tagged superseded. Its metadata records superseded_by, superseded_at and superseded_reason, and each decision is a row in memory_revisions (goclaw memory revisions lists them). Undo one with goclaw memory revisions undo <memory-id>, everything one run retired with goclaw memory revisions undo --run <run-id>, or through POST /api/memory/revisions/undo. The record returns to the prompt, and the undone decision stands as a veto: the same record never replaces it again. Removing the superseded tag by hand records no veto. Superseded records leave the prompt and plain memory_search; memory_search with include_history: true returns them, marked. Records from an unconfirmed session (unverified) never take part: they can neither retire a record nor be retired.
goclaw memory revise applies the same judge to records stored before this existed, oldest first, so a topic learned in several versions collapses onto its newest. Run it with --dry-run first: that only reports what would be retired.
If the main Memory.md file is corrupted or deleted, administrators can trigger a Memory Replay. This utility parses archived chat logs, keeps each turn with its speaker, and regenerates the consolidated core memory base using an LLM. It builds the memory from what people said, decided and confirmed; the agent's own replies are context and can be wrong, so a claim only the agent made is left out or kept under an "Unverified" section.
When a tool call fails, GoClaw checks the error against known patterns for that tool (auto_learn.go). No model is involved.
db_query, db_analyze, uptrace_query, mcp, gitlab, zabbix_query, http_request, message / notify.created_at does not exist in table users"). An infrastructure or credential error (network failure, timeout, 401, a login page) is only logged: the agent cannot fix it by changing its approach.agent_notes row is written and no tool description changes, so a later turn does not remember the mistake. What the agent should do every time is written down by an admin, as a saved query, a tool preset or a wiki page.Administrators can inspect and maintain memory from the terminal. These commands go through the running gateway's API, so start the gateway first. goclaw memory check and goclaw memory migrate read the database directly, and so does the backup the CLI takes before clear, clear-user, forget and migrate.
# View memory database statistics
goclaw memory stats
# Export the knowledge catalog (agent_notes and agent_config only) to a JSON file
goclaw memory export --file catalog.json
# Import a previously exported knowledge catalog
goclaw memory import --file catalog.json
# Rebuild the semantic vector index from scratch
goclaw memory reindex
# Remove near-duplicate chunks and agent notes
goclaw memory dedupe --ratio 0.85
The catalog export is not a backup: for a portable snapshot of the whole database (and, with flags, sessions, workspace files and wiki attachments) use goclaw db export (see Backup & Maintenance).
With the default SQLite driver, dynamic memory lives in two database files in the state directory, and a disk failure takes it with them. Schedule periodic backups of:
gc_state.sqlite: memories, sessions, SOPs, cron jobs and the rest of the state (or the file database.path names);kg.sqlite: the knowledge graph (or the file memory.graph.path names);workspace/ and agents/<id>/workspace/): Memory.md, IDENTITY.md, daily memory files and session archives;credentials/: the secrets vault (secrets.enc) and the pairing files, and config.yaml. Without them a restore loses every ${secret:...} credential and every pairing.mkdir -p ~/goclaw-backups
# Run a safe online backup of the SQLite databases while the gateway runs
sqlite3 ~/.goclaw/gc_state.sqlite ".backup $HOME/goclaw-backups/gc_state-$(date +%Y%m%d).sqlite"
sqlite3 ~/.goclaw/kg.sqlite ".backup $HOME/goclaw-backups/kg-$(date +%Y%m%d).sqlite"
# Copy the workspaces (add "agents" when you run more than one agent)
tar czf ~/goclaw-backups/workspace-$(date +%Y%m%d).tar.gz -C ~/.goclaw workspace
With database.driver set to PostgreSQL or MySQL, back up that database with pg_dump or mysqldump (and the graph's PostgreSQL database, if memory.graph.backend is postgres).
For a portable snapshot that goclaw db import can load on another machine, use goclaw db export --output backup.gclaw (add --sessions, --files, --attachments and the other flags for more; see goclaw db export --help).
The destructive memory commands (goclaw memory clear, clear-user, forget, …) first save a copy of gc_state.sqlite to <state dir>/backups/gc_state_<timestamp>.sqlite. They skip this for other drivers.