Agent Core

Core AI agent functionality for the Fractal Synapse project. This package provides a comprehensive agent framework with registry-based architecture for building sophisticated AI agents with tool integration, model management, and sub-agent capabilities.

Features

• Registry Architecture: Centralized tool and model management through ToolRegistry and ModelRegistry
• Agent Definitions: Configurable agent behavior and capabilities through AgentDefinition class
• Sub-Agent System: Isolated task execution using worker threads for complex delegation
• Tool Integration: Flexible tool definition and registration system with cleanup support
• Context Folding: Automatic compression of older tool calls in long sessions — summaries replace verbose history in model calls while the full data remains available in the UI
• State Management: Built-in state store for context sharing between agents and tools
• Model Flexibility: Support for multiple AI providers with dynamic runtime configuration and BYOK support
• Worker Thread Isolation: Safe sub-agent execution in separate threads for complex tasks
• File Attachments: Multi-modal support for images, text files, and binary documents
• AI SDK Integration: Built on top of Vercel's AI SDK with extensible model support

Installation

This package is part of the Fractal Synapse monorepo and is used internally by other packages.

npm install agent-core

Usage

Complete Agent Setup

import {
  Agent,
  ToolRegistry,
  modelRegistry,
  AgentDefinition,
  ToolDefinition,
  useSubagentToolDefinition
} from 'agent-core'
import { tool } from 'ai'
import { z } from 'zod'

// 1. Configure providers on the shared modelRegistry
// initFromEnv() is called automatically at module load for CLI usage.
// For desktop / BYOK, configure providers explicitly:
modelRegistry.configureProvider({
  provider: 'openai',
  config: { type: 'api-key', key: 'sk-...' }
})

// 2. Create custom tools
const timeToolDefinition = new ToolDefinition(
  'get-current-time',
  tool({
    description: 'Get the current date and time',
    parameters: z.object({
      timezone: z.string().optional().describe('Timezone (default: UTC)')
    }),
    execute: async ({ timezone = 'UTC' }) => {
      return new Date().toLocaleString('en-US', { timeZone: timezone })
    }
  }),
  'Returns the current date and time in the specified timezone',
  'Use this tool when users ask about the current time or date. Default timezone is UTC unless specified.'
)

// 3. Set up Tool Registry
const toolRegistry = new ToolRegistry()
toolRegistry.registerTool('get-current-time', timeToolDefinition)
toolRegistry.registerTool('use-subagent', useSubagentToolDefinition)

// 4. Create custom Agent Definition
const customAgentDefinition = new AgentDefinition(
  'TimeAssistant',
  'A helpful assistant that can provide time information and delegate complex tasks',
  `You are a helpful AI assistant specialized in time-related queries and task delegation.

## Your Capabilities:
- Provide current time and date information for any timezone
- Handle scheduling and time-sensitive questions
- Delegate complex tasks to sub-agents when beneficial

## Guidelines:
- Always be helpful and accurate with time information
- Use sub-agents for complex tasks that require specialized focus
- Provide clear, concise responses`,
  ['get-current-time', 'use-subagent'],
  'openai-gpt-4o'
)

// 5. Create and use the agent
const agent = new Agent({
  agentDefinition: customAgentDefinition,
  toolRegistry: toolRegistry,
  modelRegistry
})

// Stream response with full capabilities
const stream = agent.prompt("What time is it in Tokyo? Also, can you help me plan a meeting schedule?")

// Get the full text response
const text = await agent.execute("What time is it in Tokyo? Also, can you help me plan a meeting schedule?")

File Attachments

The agent supports sending files alongside text messages through an optional attachments parameter. Files must be pre-read and base64-encoded by the caller -- agent-core never reads from disk.

AttachedFile Interface

interface AttachedFile {
  /** Original filename including extension, e.g. "diagram.png" or "notes.md" */
  filename: string
  /** IANA media type, e.g. "image/png", "text/plain", "application/pdf" */
  mediaType: string
  /** Base64-encoded file content. Never a file path. */
  data: string
}

Usage

Pass an array of AttachedFile objects as the second argument to prompt() or execute():

import { Agent, AttachedFile } from 'agent-core'

const agent = new Agent()

// Stream a response with an image attachment
const stream = agent.prompt('Analyze this chart', [
  {
    filename: 'chart.png',
    mediaType: 'image/png',
    data: base64String  // pre-encoded by caller
  }
])

// Execute with multiple attachments
const text = await agent.execute('Summarize these documents', [
  {
    filename: 'notes.md',
    mediaType: 'text/markdown',
    data: base64String
  },
  {
    filename: 'report.pdf',
    mediaType: 'application/pdf',
    data: base64String
  }
])

Image Support and Graceful Degradation

Image and binary file support is determined per-model via a supportsImages flag in the model registry. Query support with modelRegistry.modelSupportsImages(modelName).

When a model does not support images or binary files, the agent gracefully degrades:

• Text files (text/*, application/json, etc.) are always injected inline as decoded text with a [File: filename] header.
• Images are converted to a text note: [Image attached: filename -- this model does not support image input].
• Binary files (PDFs, etc.) are converted to a text note: [File attached: filename (media-type) -- this model does not support file input].

This ensures the agent is always aware that a file was attached, even when it cannot process the content directly.

Extracting Attachments from History

Use extractUserMessageParts() to parse multi-modal content from stored conversation history for UI rendering:

import { extractUserMessageParts, UserMessageDisplayPart } from 'agent-core'

const parts: UserMessageDisplayPart[] = extractUserMessageParts(userMessage)
// parts is an array of { type: 'text' | 'image' | 'file', ... }

---

API Reference

Agent Class

Constructor

new Agent(options?: AgentOptions)

interface AgentOptions {
  agentDefinition?: AgentDefinition;
  toolRegistry?: ToolRegistry;
  modelRegistry?: ModelRegistry;
}

Properties

• messages: ModelMessage[] - Conversation history
• system: string - System prompt for the agent (read-only when using AgentDefinition)

Methods

• prompt(text: string, attachments?: AttachedFile[]): ReadableStream<any> - Send a message and get streamed response
• execute(text: string, attachments?: AttachedFile[]): Promise<string> - Execute a prompt and return the complete response
• appendToHistory(message: ModelMessage): Promise<void> - Append a message to conversation history without triggering agent execution
• cleanup(): Promise<void> - Clean up all registered tools
• getSystemPrompt(): string - Get the complete system prompt including tool instructions

appendToHistory(message: ModelMessage)

Appends a message directly to the agent's conversation history without triggering a model execution turn. The message is pushed to this.messages and a message:user event is emitted, which triggers the history plugin to persist the message to disk via its debounced save handler.

This method is designed for cases where an external system needs to inject content into a conversation for the user to see, but the agent should not process or respond to it. The primary use case is the message gateway delivering background job results in silent/append-only mode.

import { Agent } from 'agent-core'

const agent = new Agent()

// Silently append a background job result — agent does not react
await agent.appendToHistory({
  role: 'user',
  content: 'Daily digest: 3 tasks completed, 1 pending.',
  _source: 'background-job:job-123',  // UI renders this as a special bubble
})

Key behaviors:

• No agent execution: Unlike receiveMessage(), this does not call prompt() and the model never sees or processes the message.
• Persistence: The message:user event triggers the AgentHistoryPlugin save handler, so the message is persisted to disk.
• UI rendering: Messages with _source metadata are rendered by the UI as special injected-message bubbles (e.g., "Scheduled task result") rather than plain user messages.

ToolRegistry Class

Methods

• registerTool(name: string, toolDefinition: ToolDefinition): void - Register a tool
• getTool(name: string): ToolDefinition | undefined - Get a tool by name
• checkTool(name: string): boolean - Check if a tool exists
• getToolsForAgent(toolNames: string[]): ToolDefinition[] - Get multiple tools for an agent
• getAllToolNames(): string[] - Get all registered tool names

ModelRegistry Class

Manages AI model instances with dynamic provider configuration. A shared singleton modelRegistry is exported from the package and auto-initialized from environment variables on import.

Provider API

• configureProvider(entry: ProviderConfig): void - Configure a provider and register all its models
• unconfigureProvider(provider: ProviderId): void - Remove a provider and all its models
• initFromEnv(): void - Auto-configure from standard environment variables (called automatically)
• getConfiguredProviders(): ProviderId[] - List configured provider IDs
• getProviderApiKey(provider: ProviderId): string | undefined - Get raw API key (internal use only)

Model Access

• getModel(name: string): any - Get a model instance by registry name (e.g. 'openai-gpt-4o')
• checkModel(name: string): boolean - Check if a model is registered
• getDefaultModel(): any - Get the best available model instance
• getDefaultModelName(): string | undefined - Get the name of the best available model
• getAllModelNames(): string[] - Get all registered model names
• getReasoningModelNames(): string[] - Get names of reasoning-capable models

See Model Registry for the full provider reference, model name catalogue, and BYOK usage.

AgentDefinition Class

Constructor

new AgentDefinition(
  name: string,
  description: string,
  instructions: string,
  toolNames?: string[],
  modelName: string
)

Methods

• addTool(toolName: string): AgentDefinition - Add a tool to the agent (chainable)
• addTools(toolNames: string[]): AgentDefinition - Add multiple tools (chainable)
• removeTool(name: string): AgentDefinition - Remove a tool by name (chainable)

ToolDefinition Class

Constructor

new ToolDefinition(
  name: string,
  tool: Tool,
  description: string,
  instructions: string,
  cleanup?: () => Promise<void>,
  deferred?: boolean,
  label?: string,
  summarizeCall?: (args: any, result: any) => ToolCallSummary,
  foldable?: boolean    // default: true
)

summarizeCall

Optional method for generating compact one-line summaries of a tool call's inputs and outputs. Used by the context folding system to compress older tool calls in long sessions while preserving the full data in conversation history and the UI.

summarizeCall: (args, result) => ({
  args: `path: ${args.path}`,
  result: `Read ${args.path} (${result.lineCount} lines)`
})

foldable

Set to false to opt this tool out of context folding entirely. The tool's calls will never be compressed, even in very long sessions. Defaults to true.

Architecture

The agent-core package follows a registry-based architecture that promotes modularity and extensibility:

Registry Pattern

• ToolRegistry: Centrally manages all available tools, allowing agents to discover and use tools by name
• ModelRegistry: Manages AI models, enabling easy switching between different language models
• AgentDefinition: Declarative configuration that defines agent behavior, capabilities, and dependencies

Benefits

• Separation of Concerns: Tools, models, and agent logic are managed independently
• Reusability: Tools and models can be shared across multiple agents
• Testability: Each component can be tested in isolation with mock registries
• Extensibility: New tools and models can be added without modifying existing agents

State Management

The package includes a built-in state management system for sharing context between agents and tools:

import { createAgentStateStore } from 'agent-core'

// State is automatically created for each agent instance
// Tools can access and modify shared state through the experimental_context

Features

• Shared Context: Tools can access and modify state that persists across tool calls
• Agent Isolation: Each agent instance has its own state store
• Tool Communication: Tools can communicate with each other through shared state
• Built on Zustand: Lightweight and performant state management

For detailed state management documentation, see Agent State Management.

Context Folding

In long sessions with many tool calls, agent-core automatically compresses older tool calls to reduce token consumption. This is called "folding."

How it works

• Each tool call generates a one-line summary immediately after it executes (via summarizeCall on the tool definition)
• The full args and result are preserved in this.messages — used by the UI and conversation history
• Before each model call, a projected copy of the messages is built where older tool calls are replaced with their summaries
• The model sees compact summaries; the user always sees full data

Fold triggers

The ContextFolding class applies three rules automatically:

Rule	When
Accumulation	After each step: if older tool calls exceed 5,000 estimated tokens and the last 15 calls are protected
Cache-lost idle	At turn start: if more than 5 minutes elapsed since the last turn
Turn boundary	At turn start: turn 0 records are folded when entering turn 2+

Opting out per tool

Set foldable: false on a ToolDefinition to prevent that tool's calls from ever being folded:

new ToolDefinition(
  'write-file',
  writeTool,
  'Writes content to a file',
  'Use to write or overwrite files.',
  undefined,   // cleanup
  undefined,   // deferred
  undefined,   // label
  undefined,   // summarizeCall
  false        // foldable — never compress write operations
)

Implementing summarizeCall

new ToolDefinition(
  'read-file',
  readTool,
  'Reads a file',
  'Use to read file contents.',
  undefined,
  undefined,
  'Read File',                    // label
  (args, result) => ({
    args: `path: ${args.path}`,
    result: `Read ${args.path} (${result.lines} lines)`
  })
)

If summarizeCall is not provided, a generic fallback is used automatically.

Sub-Agents

The package supports sub-agent execution through worker thread isolation:

Worker Thread Isolation

• Process Isolation: Sub-agents run in separate worker threads
• Memory Safety: Sub-agents cannot access main agent state or tools
• Timeout Protection: Sub-agent execution is limited by configurable timeouts
• Error Containment: Sub-agent failures don't crash the main agent

Use Cases

• Complex Task Delegation: Delegate specialized tasks to focused sub-agents
• Experimental Operations: Test risky operations in isolation
• Parallel Processing: Execute independent subtasks concurrently
• Resource Management: Prevent memory leaks from long-running subtasks

Example

// Sub-agents are used through the use-subagent tool
const result = await agent.execute("Use a sub-agent to analyze this complex data set")

Testing

This package uses Vitest for testing.

Run Tests

# Run tests once
npm run test:run

# Run tests in watch mode
npm run test:watch

# Run tests interactively
npm test

Test Structure

Tests are located in the tests/ directory with comprehensive utilities and organized test files:

tests/
├── agent.test.ts                    # Core Agent class functionality tests
├── agent-definition.test.ts         # AgentDefinition class and configuration tests  
├── agents/
│   └── main-agent.test.ts          # Tests for the built-in main agent definition
├── tools/
│   └── use-subagent.test.ts        # Sub-agent tool functionality and worker thread tests
└── utilities/                       # Shared testing utilities and infrastructure
    ├── index.ts                    # Central exports for all test utilities
    ├── assertions/                 # Custom test assertions and matchers
    │   └── index.ts               # Agent-specific assertion helpers
    ├── mocks/                      # Mock implementations for testing
    │   ├── ai-sdk.ts              # Mocked AI SDK responses and models
    │   ├── language-models.ts     # Mock language model implementations
    │   └── workers.ts             # Mock worker thread implementations
    └── test-helpers/               # Helper functions for test setup
        ├── agents.ts              # Agent creation and configuration helpers
        ├── registries.ts          # Registry setup and management helpers
        └── tools.ts               # Tool definition and testing helpers

Example Test

import { describe, it, expect } from 'vitest'
import { Agent } from '../src/agent'

describe('Agent', () => {
  it('should create an agent instance', () => {
    const agent = new Agent()
    expect(agent).toBeInstanceOf(Agent)
  })
})

Development

Build

npm run build

Watch Mode

npm run dev

Dependencies

Core Dependencies

• ai: Vercel AI SDK for LLM interactions and tool definitions
• @ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google: Provider integrations
• zhipu-ai-provider: Z.ai / Zhipu GLM models
• @openrouter/ai-sdk-provider: OpenRouter multi-model gateway
• ollama-ai-provider: Local Ollama models
• zustand: Lightweight state management for agent context
• zod: Schema validation for tool parameters and type safety

Built-in Node.js Features

• worker_threads: Used for sub-agent isolation and parallel execution
• path: File path utilities for worker script resolution

Related Documentation

• Model Registry - Provider configuration, model catalogue, and BYOK usage
• Agent State Management - Detailed state store documentation
• Project Architecture - Overall project structure
• Tool Development Guide - Creating new tools