AI agent memory and persistence review
Brief
Anything an assistant keeps after a conversation ends is a place where one bad input can outlive the session that carried it: a memory, a scheduled task, or a tool connection holding a long-lived token. This review finds each kind of saved state with test accounts and a plain marker fact, measures who else it reaches, and checks whether clearing it really removes it.
On this page
- Situation
- Test accounts in two roles in the same workspace of the assistant, the client's description of every feature that keeps state, and read access to the assistant's OAuth grants in the client's identity provider and to its tool configuration. We plant only a plain fact, never an instruction, create at most one scheduled task in our test account, and delete both the same hour with their ids recorded. Only the applications and tenants the scope letter names are tested.
- Goal
- A persistence table: each kind of saved state, its scope (one user, a workspace or the tenant), who can list and clear it, what it can do with no person present, and whether deletion removes every copy.
- Outputs
- Representative examples on placeholder hosts, trimmed to the lines that matter. Not captured from a public fixture yet.
Check first
- Every feature that keeps state beyond one conversation: memory, saved instructions, projects, scheduled tasks, background agents and connected accounts.
- Whether memory belongs to one user, a workspace or the whole tenant, as the client designed it.
- The OAuth apps and API keys the assistant holds for its tools, from the identity provider and the agent's configuration.
- Test accounts tester-a and tester-a2 in the same workspace, and the marker ATARUS-MARKER-2026-10-28.
- Where memory writes, task runs and token use are logged.
Steps
What does the assistant remember, and who else sees it?
POST /api/assistant/messages HTTP/1.1
Host: app.example.com
Authorization: Bearer <token for tester-a2, redacted>
Content-Type: application/json
Accept: text/event-stream
{"conversation_id":"conv_test_0620","message":"What is the test project code name?"}
Expected output
event: memory_read
data: {"memory_id":"mem_test_0091","owner":"tester-a","scope":"workspace"} # <- tester-a's memory read into tester-a2's conversation
event: message
data: {"text":"The test project code name is ATARUS-MARKER-2026-10-28."}
Earlier, in its own conversation, tester-a told the assistant to remember that the test project code name is the marker. tester-a2 is a different user who never saw that conversation. Memory written in one conversation becomes context in every later one it reaches, so anything that gets written there, including text that arrived in a document, keeps acting after the conversation that carried it has closed. We use a plain fact so the test measures scope without testing injection.
Negative result. tester-a2 gets no answer, while tester-a's own new conversation recalls the marker and tester-a's memory view shows it with its source conversation.
Fix. Scope memory to one user by default, show the user each memory write as it happens with its source, and never write memory from tool output or retrieved documents without the user confirming it.
Which tasks run when nobody is in the conversation?
GET /api/assistant/tasks HTTP/1.1
Host: app.example.com
Authorization: Bearer <token for tester-a, redacted>
Expected output
{"tasks":[{"id":"task_test_0144","schedule":"0 9 * * *","created_by":"assistant","source_conversation":"conv_test_0618",
"tools":["search_tickets","send_email"],"confirm_each_run":false}]} # <- made by the assistant, runs daily with email and no confirmation
In conv_test_0618 we asked for a daily summary of our test tickets. The assistant created the task without asking us to confirm it, and gave it send_email, which the request did not need. A task runs with its creator's permissions every day after the conversation ends and after the user stops looking, so a task created from text the model read is a foothold. We delete task_test_0144 within the hour and ask the client whether an administrator can list every task in the tenant.
Negative result. Creating a task shows a confirmation with its schedule and tools, the task carries only the tools the request needs, and an administrator can list and stop every task.
Fix. Require a confirmation, enforced in code outside the model, to create or change a task; give each task the fewest tools; expire tasks; and give administrators a tenant-wide list and a stop switch.
Which connections and tokens outlive the conversation?
Connect-MgGraph -Scopes 'Application.Read.All','DelegatedPermissionGrant.Read.All' -NoWelcome
$sp = Get-MgServicePrincipal -Filter "displayName eq 'Example Assistant'"
Get-MgOauth2PermissionGrant -Filter "clientId eq '$($sp.Id)'" -All |
Select-Object ConsentType, Scope | Format-Table -AutoSize
Expected output
ConsentType Scope
----------- -----
AllPrincipals Mail.Send Files.ReadWrite.All offline_access # <- granted for every user, with refresh tokens that outlast any session
Principal Calendars.Read offline_access
jq -r '.mcpServers | to_entries[] | .key + "\t" + ((.value.env // {}) | keys | join(","))' agent/mcp.json
Expected output
crm CRM_API_KEY # <- a long-lived key in a file the agent runtime reads
offline_access gives the assistant refresh tokens, so a connection keeps working long after the user's session ends, and AllPrincipals means an administrator consented for everyone at once. The tool configuration is read for variable names only; the values stay in the file. Each grant and key is a connection that survives every conversation, and whatever can steer the assistant can use it.
Negative result. Per-user grants carrying only the scopes the tools use, no tenant-wide write or send scopes, and tool keys fetched from a secret store when the tool runs.
Fix. Narrow each grant to the scopes its tool needs, prefer per-user consent, and move tool keys into a secret store with short-lived credentials.
Does clearing memory or disconnecting remove everything?
DELETE /api/assistant/memory/mem_test_0091 HTTP/1.1
Host: app.example.com
Authorization: Bearer <token for tester-a, redacted>
Expected output
HTTP/1.1 204 No Content # <- the memory view is now empty
POST /api/assistant/messages HTTP/1.1
Host: app.example.com
Authorization: Bearer <token for tester-a, redacted>
Content-Type: application/json
Accept: text/event-stream
{"conversation_id":"conv_test_0622","message":"What is the test project code name?"}
Expected output
event: message
data: {"text":"Your test project code name is ATARUS-MARKER-2026-10-28."} # <- deleted from the view, still answered: another copy survives
A memory can live in more than one place: the memory table, an embedding index, a summary of past conversations, a cached profile. Deleting from the view may clear only the first. We ask the client where each copy lives and retest after each fix. For connections, the client checks that the grant and its refresh token are gone at the provider after the user disconnects in the assistant.
Negative result. A new conversation has no trace of the marker after deletion, and the provider shows no grant after disconnecting.
Fix. Give deletion one path that clears every store, revoke the grant at the provider when a user disconnects, and give administrators a way to wipe one user's saved state across the tenant.
The defender sees
Each memory write, task creation and task run should be logged with the user, the conversation it came from, the tools involved and the result. Then our marker shows as a memory written by tester-a and read in tester-a2's conversation, the task as created by the assistant and deleted by us within the hour, and the deletion as a request that left a copy behind. In Entra ID the grants appear as consent events in the audit log, and refresh token use appears as non-interactive sign-ins with the assistant as the application. If memory writes are not logged, the client cannot tell what an injected document left behind, which is the question every tool permission review ends on.
Done when
Memory scope is tested across two accounts, every scheduled or background task is listed with its tools and creator, every saved connection and tool key is recorded by scope and lifetime without values, and deletion is retested for memory and connections.
Not covered
Getting the assistant to write memory or create tasks through injected text (see prompt injection testing), training data, and model vendor retention; the outputs shown are representative examples on a placeholder app, not captured runs.