Sanjay Mukhyala

Project Case Study · AI Infrastructure · Local-First Systems

Cortex

Every AI tool I use — ChatGPT, Claude, Poke, Claude Code — starts from zero. I re-explain the same preferences, projects, and context in every window, and none of it carries across tools. Cortex is a local-first memory layer that fixes this: it reads my conversation exports and local memory files, extracts the durable facts about me, checks them against what it already knows, and propagates the approved set back into every assistant in the format each one expects.

Next.js 16 · Prisma 7 · SQLite · TypeScript · Zod · Anthropic SDK · MCP

4 pipeline stages
10 memory categories
5+ connected tools
3 storage layers

1. The idea

Cortex treats personal context as a first-class, portable asset. Instead of each assistant maintaining its own siloed, lossy memory, Cortex builds one curated knowledge base and keeps it synchronized across tools. It ingests exports and local memory files, uses Claude with structured output to extract atomic durable facts, deduplicates and conflict-checks them against the existing store, and — once approved — writes the profile back out as ChatGPT custom instructions, a Claude CLAUDE.md, or a Poke API push. The whole system runs locally against a SQLite database: no SaaS, no lock-in, the data lives in my home directory.

2. The pipeline

Cortex is the sync layer between the tools I already use: it reads context out of each one, runs it through a synchronous 4-agent pipeline, and writes the curated result back to all of them. Every stage operates on the typed output of the previous one — each input and output is a Zod schema, so the contract is validated at runtime and the shape never drifts. Ingest normalizes raw exports into conversations; Extract turns conversations into categorized facts; Deduplicate reconciles them against the existing store; Commit persists and routes each memory to review or auto-approval before it propagates.

Reads from ChatGPT Claude Poke Claude Code
01
Ingestnormalize exports, hash-dedup conversations
02
ExtractClaude structured output → typed facts
03
Deduplicatereconcile & flag conflicts
04
Commitstore, then review or auto-approve
Syncs back to ChatGPT Claude Poke Claude Code

3. Ingest

The Ingest agent accepts the messy reality of AI exports and produces one normalized shape. It unzips ChatGPT archives and flattens their branching message trees, parses Claude.ai JSON, and reads Claude Code CLAUDE.md / MEMORY.md files directly. Each conversation is fingerprinted with a content hash, so re-importing an export never re-processes conversations that have already been seen — the pipeline is idempotent by construction.

ChatGPT ZIP/JSON, Claude.ai JSON, CLAUDE.md / MEMORY.md
normalized conversations content hash source metadata dedup skip-set

4. Extract & classify

The Extract agent is where Claude does the heavy lifting. A Zod schema is converted to a JSON schema and handed to the model as a tool, forcing structured output: the model returns a list of atomic, durable facts rather than prose. Quality filters run in the prompt — implementation details and one-off debugging context are discarded unless they are framed as a lasting preference. Every surviving fact is tagged with a category, a temporality (durable / current / expired), and a sensitivity flag that governs whether it can leave the machine.

identity Name, age, location, devices, accounts
education_career School, courses, roles, labs, founding work
projects Active builds, repos, products, experiments
preferences Tools, styles, tastes, naming choices
relationships Collaborators, contacts, team, pets
goals · research · writing_voice Ambitions, interests, and tone signals
workflows · temporary Setup, tooling, and time-boxed context

5. Deduplicate & conflict detection

A new fact is rarely brand new — it usually relates to something already stored. The Deduplicate agent indexes existing memories by category for fast lookup, then uses the LLM to compare each incoming fact against its neighbors. This is not string matching: it reasons about meaning, classifies the relationship into one of four conflict types, and attaches a suggested action. Non-sensitive facts from trusted sources can resolve automatically; anything ambiguous or sensitive is escalated to review.

contradictionKeep oneNew fact disagrees with an existing memory. Surface both and pick the current truth.
duplicateDropSame fact, already stored. Skip it and keep the store clean.
refinementMergeNew fact adds detail to an existing one. Combine into a richer memory.
supersedeReplaceNew fact makes an old one stale. Retire the old, promote the new.

6. Commit & review

The Commit agent writes survivors to SQLite and decides how each one reaches me. Two modes govern that decision. Review Queue is the default for new or sensitive memories — nothing propagates until I approve it in the dashboard. Auto Update applies to trusted sources like Claude Code and Claude Desktop: those memories flow through automatically unless they trip a conflict, in which case they always fall back to review. Conflicts never resolve silently.

Review Queue Auto Update
Default for new & sensitive memories Trusted sources: Claude Code, Desktop, Poke
Nothing propagates without approval Auto-propagates unless a conflict fires
You resolve conflicts and edit content Conflicts always fall back to review

7. Three storage layers

Cortex models memory in three layers because each serves a different consumer. The structured database is the source of truth. The markdown export is what humans and Claude read. The workspace is what stays hot — a small, privileged set of the memories that matter right now, kept close so the assistant reaches them first.

Layer 1SQLiteTyped records for memories, conflicts, sources, review items, and sync runs. The source of truth.
Layer 2Markdown CLAUDE.mdHuman-readable export wrapped in cortex markers, so re-syncing never clobbers hand-written sections.
Layer 3WorkspaceA small hot set of active memories, served with priority over a much larger background store.

The workspace is inspired by Verbalizable Representations Form a Global Workspace in Language Models (Anthropic, 2026). The paper shows a language model keeps a small, privileged set of representations — the J-space, surfaced by the Jacobian lens (J-lens) — that it is “poised to verbalize,” sitting atop a much larger volume of automatic processing. Only a handful of these vectors (the paper finds roughly 25 meaningfully active at a time) form this foreground, and the model can deliberately pull a concept in when instructed to hold it in mind. It is a computational echo of conscious attention: a narrow, reportable foreground over a wide automatic background.

Cortex borrows the shape, not the internals. Its workspace is a small foreground of memories held in mind — the cortex_hold_in_mind tool maps directly to the paper's directed-attention move — served ahead of the background store on every MCP query. What decides which memories stay in that foreground is Cortex's own engineering: a bounded set of active slots, a relevance score blending keyword overlap, category match, recency, and co-occurrence, re-run in response to activity signals (MCP queries, file changes, manual pins), plus a time decay so untouched memories fall back to the background. The paper describes no such decay — that half-life is a product choice for keeping the foreground fresh, an approximation of the same narrow-foreground idea rather than anything the paper claims.

8. The MCP server

The whole point is that memory has to reach the assistants where I actually work. Cortex exposes a Model Context Protocol server that runs over stdio (native to Claude Desktop) or Streamable HTTP (for Poke and external tools). The tools are stateless, which is what makes cross-tool sync possible: any client can pull the same context bundle, and workspace queries feed back into the scoring loop that decides what stays hot.

tool cortex_get_context — returns the workspace plus related background memories as one bundle.
tool cortex_search_memories — keyword and category search across the full store.
tool cortex_hold_in_mind — pins a memory into the workspace as a manual signal.
tool cortex_suppress / cortex_release — temporarily hide or restore a memory.

Why it matters

The fragmentation of context across AI tools is a real tax: every new window is a cold start. Cortex removes that tax by making personal memory a portable, user-owned layer rather than a per-app silo. Facts are extracted once, curated once, and follow me everywhere — and because everything lives locally with an approval gate, I keep control over what is remembered and what is allowed to leave the machine.

The broader pattern — normalize noisy sources, extract typed atomic facts, reconcile against a store with explicit conflict semantics, and decay toward what is actually used — generalizes past AI memory. Any system that has to merge streams of overlapping information into one trusted, evolving picture can use the same architecture.

Browser Demo GitHub Back to projects