memory-graph-plugin

Persistent knowledge graph memory for the Fractal Synapse agent framework. Stores every conversation turn as an episode, asynchronously extracts entities and relationships from them, and injects relevant context before each agent turn — automatically, without the agent deciding when to query memory.

How it works

Memory retrieval is automatic middleware, not an agent tool. Before every user turn the plugin:

1. Classifies the message (conversational, correction, task continuation, new topic)
2. Skips retrieval for conversational messages ("ok", "thanks", "got it")
3. Embeds the message and runs parallel KNN vector searches over entities and episodes
4. Formats the results into a <memory_context> block and appends it to the user message

Extraction runs asynchronously after each assistant or tool turn — it never blocks a response. A setInterval heartbeat retries failed extractions, expires stale entities, and applies confidence decay to old facts.

Features

• Automatic context injection — memory is retrieved every turn via before-turn hook; the agent never decides when to call it
• Embedding-based message classifier — four categories (conversational, correction, task continuation, new topic) using centroid vectors, not keyword lists
• Knowledge graph schema — entities, typed relationships, episodes stored in SQLite with 5 migrations
• Vector search — KNN over entity and episode embeddings via sqlite-vec, cosine distance
• Async extraction pipeline — OpenAI Structured Outputs (gpt-4o-mini) extracts entities and relationships from each episode, confidence thresholds applied before storage
• Entity deduplication — case-insensitive name + type matching, weighted confidence average, longer description wins
• Relationship supersede — facts are never deleted; valid_until marks them expired and superseded_by links to the replacement
• Tool + episode memory — user turns, assistant turns, and tool results are all stored and extracted
• Startup retry — pending episodes from a previous crash are scheduled on initializeAgent
• Mechanical heartbeat — setInterval tasks for extraction retry (15 min), entity expiry (1 hour), confidence decay (24 hours)
• Provider-agnostic embeddings — works with any EmbeddingsInterface (nomic-embeddings-plugin, openai-embeddings-plugin, etc.)
• memory_graph agent tool — 8 actions for direct agent access to the graph when needed

Pre-requirements

This plugin depends on three other plugins being configured and available:

• sqlite-database-plugin — provides DatabaseInterface with SQLite + sqlite-vec
• An embeddings plugin — any plugin implementing EmbeddingsInterface (e.g. nomic-embeddings-plugin)
• agent-core — before-turn hook, EmbeddingsInterface, DatabaseInterface

Installation

npm install @fractal-synapse/memory-graph-plugin

Quick Start

import { SqliteDatabasePlugin } from '@fractal-synapse/sqlite-database-plugin';
import { NomicEmbeddingsPlugin } from '@fractal-synapse/nomic-embeddings-plugin';
import { MemoryGraphPlugin } from '@fractal-synapse/memory-graph-plugin';
import { Agent } from '@fractal-synapse/agent-core';

// 1. Set up the database
const db = new SqliteDatabasePlugin({ dbPath: '/path/to/data/agent.db' });

// 2. Set up embeddings (any provider works)
const embeddings = new NomicEmbeddingsPlugin();

// 3. Create the memory plugin (migrations are registered automatically)
const memory = new MemoryGraphPlugin({
  db,
  embeddings,
  openAIApiKey: process.env.OPENAI_API_KEY,
});

// 4. Open the database (runs all migrations including memory-graph tables)
db.initialize();

// 5. Register the memory_graph tool with your tool registry
for (const def of memory.getToolDefinitions()) {
  toolRegistry.registerTool(def.name, def);
}

// 6. Add the plugin to the agent
const agent = new Agent({ plugins: [db, embeddings, memory] });

The plugin starts working immediately. Every user and assistant turn is stored; context is injected automatically before each turn.

Configuration

interface MemoryGraphPluginConfig {
  // Required — SQLite database with sqlite-vec extension
  db: DatabaseInterface;

  // Required — any embeddings provider implementing EmbeddingsInterface
  embeddings: EmbeddingsInterface;

  // OpenAI API key for entity/relationship extraction via gpt-4o-mini
  // If omitted, extraction is disabled (episodes are stored but not extracted)
  openAIApiKey?: string;

  // OpenAI model for extraction. Default: "gpt-4o-mini"
  extractionModel?: string;

  // Maximum token budget for injected memory context per turn. Default: 500
  // Truncation order: episodes → relations → entities
  tokenBudget?: number;

  // Minimum confidence to store an extraction. Default: 0.4
  // Extractions below this value are discarded entirely
  confidenceMinimum?: number;

  // Strong-confidence threshold. Default: 0.7
  // Extractions at or above this are stored with full confidence
  // Extractions between confidenceMinimum and this are stored with the raw confidence value
  confidenceThreshold?: number;

  // Optional logger compatible with LoggingInterface
  logger?: LoggingInterface;
}

Database Schema

The plugin creates 5 tables via the migration system:

Table	Description
mg_entities	Named entities with type, description, confidence, and validity window
mg_relationships	Typed edges between entities — never deleted, superseded instead
mg_episodes	Raw conversation turns (user, assistant, tool) and their extraction status
mg_entity_embeddings	vec0 virtual table — one vector per entity, used for KNN search
mg_episode_embeddings	vec0 virtual table — one vector per episode, used for KNN search

The vector table dimension is set at construction time from config.embeddings.dimensions. Changing embedding providers after data has been stored requires dropping and recreating both vec0 tables and re-embedding all content.

Entity types

person · project · tool · concept · decision · organization · location · event

Relationship types

works_on · uses · decided_to_use · assigned_to · part_of · created_by · depends_on · knows · related_to · mentions

Context Injection

Before every non-conversational turn the plugin injects a <memory_context> block appended to the user message:

<memory_context>
Entities:
- fractal-synapse (project): Agent framework in development
- memory-graph-plugin (tool): Persistent memory plugin being built

Relations:
- memory-graph-plugin → part_of → fractal-synapse

Recent episodes:
- [2026-03-18] [assistant] Decided to use sqlite-vec for vector search
</memory_context>

Token budget is applied before injection. Content is truncated in this order: episodes first, then relations, then entities — the most precise facts are always kept.

Message Classifier

The classifier decides whether to run vector retrieval for a given turn. Embedding retrieval is skipped for conversational messages to avoid wasting an embedding call and injecting irrelevant context.

Category	Examples
conversational	"ok", "thanks", "got it", "sounds good", "perfect"
correction	"actually that's wrong", "hold on, you got that backwards", "let me clarify"
task_continuation	"continue with that", "next step", "also can you", "building on that"
new_topic	"different subject", "let's switch topics", "separate thing"

The classifier uses embedding-based centroid matching — one embed(message, 'classification') call is compared against four pre-computed category centroids. This classifies natural phrasings that keyword rules would miss.

Category embeddings are computed once in initializeAgent() and reused for the lifetime of the session.

Extraction Pipeline

After each assistant turn and tool result, extraction runs asynchronously via setImmediate:

1. Fetch the episode from mg_episodes
2. Call OpenAI gpt-4o-mini with a JSON Schema response format — returns entities and relationships with 0.0–1.0 confidence scores
3. Apply confidence thresholds: discard below confidenceMinimum (default 0.4), store the rest
4. Upsert entities — deduplicate by lower(name) + type, merge confidence with weighted average
5. Embed each entity name with 'document' mode and store in mg_entity_embeddings
6. Upsert relationships — deduplicate on head + relation + tail WHERE valid_until IS NULL
7. Mark extracted = 1 on the episode

If extraction fails, the episode stays at extracted = 0 and the heartbeat retries it every 15 minutes.

Confidence Thresholds

confidence >= 0.7    → stored with raw confidence
confidence 0.4–0.7   → stored with raw confidence (lower weight in practice)
confidence < 0.4     → discarded

The memory_graph Tool

The plugin exposes a single memory_graph tool for direct agent access to the graph. Use getToolDefinitions() to register it:

for (const def of memory.getToolDefinitions()) {
  toolRegistry.registerTool(def.name, def);
}

Action	Description
retrieve	Vector search over entities + episodes for a natural language query
retrieve_linked	Fetch entity by ID + all valid 1-hop connections
create_entity	Insert entity with name, type, description → embed and store
create_relation	Resolve head/tail names → insert typed relationship
update_entity	Update description, confidence, valid_until, last_seen
supersede	Set valid_until = now on a relationship, link to replacement
store_episode	Manually insert an episode — useful for storing important context
search_graph	Vector search + 1-hop entity expansion + related episodes

Heartbeat

Three setInterval tasks run in the background during an active agent session:

Task	Interval	Operation
Extraction retry	15 minutes	Query mg_episodes WHERE extracted = 0 → schedule extraction
Expiry check	1 hour	Set confidence = 0 on entities past their valid_until date
Confidence decay	24 hours	Multiply confidence by 0.9 for entities not updated in 7+ days

Intervals are started in initializeAgent and cleared in cleanupAgent.

Relationship Supersede

Relationships are never deleted. When a fact changes, mark the old relationship as superseded and record the replacement:

// Via the memory_graph tool
await agent.tool('memory_graph', {
  action: 'supersede',
  relationship_id: 'old-rel-id',
  new_relationship_id: 'new-rel-id',
});

All queries filter WHERE valid_until IS NULL to see only currently valid facts. The full history is always preserved for auditing.

Testing

# Run all tests (unit + integration)
npm run test:run

# Watch mode during development
npm run test:watch

101 tests across 10 files — 5 unit test suites and 5 integration test suites using real SQLite :memory: instances.

Notes

• Kernel context is deferred to memory-graph-kernel-task.md. Dynamic retrieval is the only injection path for now.
• GLiNER + GLiREL (Python extraction service) is deferred to extraction-service-task.md. OpenAI is the only extractor for now.
• Two embedding calls per non-conversational turn — one with 'classification' mode for the classifier, one with 'query' mode for vector retrieval. These cannot be shared; different modes produce vectors optimized for different tasks.
• Switching embedding providers after data has been stored requires dropping and recreating the mg_entity_embeddings and mg_episode_embeddings vec0 tables and re-embedding all content. Treat the embedding provider as a deployment-time decision.