I Built an AI Memory That Knows When to Forget
A consent aware memory layer that remembers with receipts and forgets on demand.
LIVE SIGNAL · MEMORY GOVERNANCE · V0.1 · CONCEPT
0103
In plain English
Most AI memory demos optimize for remembering more. This concept explores the harder question: what should an agent be allowed to remember, for how long, and how can a person inspect or erase it? The result is a blueprint for useful personalization without turning memory into an invisible archive.
Business value
- Less manual work
- Clearer data flow
- Better operational visibility
- Easier maintenance
Memory Control Plane
A concept dashboard showing why each memory exists, where it came from, when it expires, and how a user can correct or remove it.
- Role
- Product strategy, system design, governance model, UX prototype, evaluation plan
- Design · Build · Ship
- Timeframe
- 2026
- Domain
- AI agents, privacy, data governance
- Category
- AI Systems
- Designed a consent aware memory layer for AI agents as an independent concept study.
- Stored structured, attributable facts instead of retaining full conversations by default.
- Bound every memory to a purpose, source, consent state, sensitivity level, and expiry rule.
- Put policy checks before semantic retrieval and made selective forgetting a core workflow.
- Designed a consent aware memory layer for AI agents as an independent concept study.
- Stored structured, attributable facts instead of retaining full conversations by default.
- Bound every memory to a purpose, source, consent state, sensitivity level, and expiry rule.
- Put policy checks before semantic retrieval and made selective forgetting a core workflow.
The Useful Kind of Forgetful
Persistent memory can make an agent feel dramatically more capable. It can carry preferences, project constraints, and important decisions from one session into the next. That continuity can save time and make the experience feel personal. But it also creates a second system behind the conversation, one the user may never see or understand.
For this independent research concept, I treated memory as governed data instead of a convenience feature. Every remembered fact needed a reason to exist, a source, a clear owner, and a visible path to deletion. The design question was not simply how to help an agent remember more. It was how to make memory useful without asking people to surrender control.
Memory Without Consent Becomes Surveillance
Most agent memory demos reward recall. The more an agent remembers, the smarter it appears. That incentive is incomplete. A useful detail can become risky when it survives longer than its purpose, appears in the wrong context, or cannot be traced back to a source.
The system had to preserve continuity without keeping complete conversations forever. It also had to answer practical questions in plain language. Who approved this memory? Which source supports it? Why is it being recalled now? When does it expire? Can one sensitive fact be removed without destroying everything else?
The core challenge was to make appropriate recall the default. Memory should be selective, explainable, and reversible from the moment it is created.
Ownership
Everything I designed, built, and was accountable for.
Product & UX
- Redaction and forgetting workflows
- Control UX and evaluation plan
Additional scope
- Product framing and threat modeling
- Memory contract and data model
- Retrieval architecture with policy first
Key decisions
The calls I made, what I rejected, and why: these are the tradeoffs that shaped the system.
Store structured facts, not full transcripts
Automatically embedding and retaining complete conversations.
Structured memories are easier to explain, correct, expire, and delete while reducing unrelated data exposure.
The smartest memory system is not the one that remembers everything. It is the one that can explain why it remembers and forget cleanly.
Govern Before You Retrieve
I designed each saved fact as a structured memory envelope. The envelope records the subject, purpose, source, consent state, sensitivity, retention period, creation time, and ownership boundary. Raw conversation content stays temporary unless a deliberate save rule applies. This reduces unnecessary storage and makes every retained fact easier to inspect.
Retrieval begins with policy, not similarity. The system checks ownership, allowed purpose, active consent, expiry, and sensitivity before any semantic search occurs. Only permitted memories enter the candidate set. The agent then ranks candidates by relevance, source strength, and available context budget. A highly similar fact never overrides an expired consent state or an incompatible purpose.
The user interface exposes what the agent remembers and why. Each item shows its source, intended use, and retention window. People can correct a fact, revoke consent, or request deletion without clearing unrelated context.
A forget request starts a coordinated removal flow. It blocks future access, removes the vector and structured record, invalidates cached and derived copies, and returns a deletion receipt that proves completion without repeating the private content. Observability records the decision path and operational result while excluding the memory payload itself. The design makes forgetting a core capability instead of an afterthought.
0104
consent · retention · forgetting loop
Frontend
Backend
Database
Infrastructure
Also used
Memory must be earned. A useful detail is not automatically permission to retain it.
Useful by Design, Forgetful on Purpose
- Consent aware memory model
- Retrieval with policy first
- Verifiable selective forgetting
- Provenance visible to users
The result is a testable reference architecture, memory control interface, deletion workflow, and acceptance plan. It is not presented as a client deployment or production benchmark. Its value is in making the hard parts visible before implementation.
The concept defines four clear tests. There must be no silent storage. Every recall must show useful provenance. Expired or revoked memories must be excluded automatically. A forget request must remove the selected fact and its derived copies in a verifiable way.
Together, these tests turn responsible memory from a policy statement into behavior that product, engineering, security, and users can inspect. The architecture gives a team a practical starting point for building personalization that earns trust instead of assuming it.
0 silent memories · design target
Concept criterion
A safer personalization pattern in which stored context can be explained, audited, expired, corrected, and revoked before deployment. This is a reference architecture and design target, not a measured production result.
“Rusty understands the difference between adding features and making software actually usable. He looks at how people work, finds the friction, and improves the system in a way that makes daily operations feel smoother.”
Operations Stakeholder
Internal Platform Team — name under NDA
What I Would Validate Next
The hardest part was not vector search. It was deciding when memory should exist at all. Once expiry and deletion were designed alongside capture, the rest of the architecture changed. Provenance became a product feature instead of background metadata. Consent became an active state that retrieval must respect every time.
I also learned that deletion needs the same level of design attention as creation. Removing one database row is not enough when vectors, caches, summaries, and derived artifacts may still influence future responses. A trustworthy system has to know where a memory traveled and prove that each copy is no longer usable.
The next validation step would be a moderated usability study. I would test whether people understand what the agent remembers, can predict why it will use a detail, and feel confident correcting or removing it. A strong result would be quiet confidence. Users should not need to study the architecture to feel in control.
The smartest memory system is not the one that remembers everything. It is the one that can explain why it remembers and forget cleanly.
Better AI memory is relevant, attributable, limited by time, and reversible.
10+
3
5+
10+
Building AI that needs memory without the baggage?
I design agent systems that preserve useful context while keeping consent, provenance, retention, and deletion visible from day one.
Design a Safer Agent