System overview

Memory Crystal is a layered memory system, not a single table with embeddings.

Core layers

Idempotent completed-turn capture from agent events
Grounded, bounded distillation
Storage in Convex
Embedding and indexing
Recall and reranking

Convex as operational backend

Convex handles structured memory records, per-key namespaces, and query APIs with predictable latency.

Each memory record includes:

content summary
memory type
source and scope metadata
salience score
retention policy

Embedding strategy

The self-hosted MCP server supports Gemini, OpenAI, or Ollama embeddings through EMBEDDING_PROVIDER. Hosted durable memories use the account owner's configured provider path, with no shared fallback for model-funded distillation. Text is chunked conservatively to preserve coherence. Over-chunking hurts retrieval quality.

STM vs LTM

Short-term memory

Recent message text used for exact-turn continuity through scoped lexical search and bounded chronological fallback. Raw messages do not carry vectors.

Long-term memory

Durable facts/events optimized for cross-session recall.

A grounded distillation process promotes high-signal STM windows into LTM at explicit session boundaries or a bounded due-work fallback.

Deterministic ranking

Vector recall finds similar meaning while lexical recall preserves exact names and fresh facts. Static ranking weights add recency, strength, confidence, provenance, supersession, and scope without another model call.

Recall pipeline

const [semantic, lexical, recent] = await Promise.all([
  vectorSearch(queryEmbedding, 30),
  lexicalMemorySearch(query, 20),
  recentTranscriptSearch(scope, 20),
]);
const ranked = rerank([...semantic, ...lexical, ...recent], intent, scope);
return ranked.slice(0, 8);

Multi-tenant safety

Every hosted request is scoped by API key and user account. Channel-scoped memory isolation adds another boundary when you want per-agent or per-client separation inside the same account.

Observability

Useful metrics:

capture success rate
recall latency p95
precision@k on probe set
stale-memory incidence

Failure modes and mitigations

noisy capture -> stricter write filters
stale context -> TTL + freshness boosts
empty recall -> fallback modes + diagnostics

Architecture principle

Fast, scoped, and explainable recall beats “retrieve everything and hope the model figures it out.”

That principle is why Memory Crystal remains stable as memory volume grows.