dev-docs/architecture/plugin-points.md
Rama main · réplica de lectura del repositorio original.
# Pluggable Architecture Guide
CopilotKit is built around extension points. Almost everything is optional and replaceable. This guide catalogs **every** pluggable part, where it's configured, and what happens when you don't provide it.
---
## Overview: All Extension Points
```mermaid
graph TB
subgraph "Frontend (React / Angular / Vanilla)"
FT["Frontend Tools<br/><i>Functions agents can call</i>"]
CTX["Agent Context<br/><i>Data agents can read</i>"]
RTC["Tool Call Renderers<br/><i>Custom UI for tool calls</i>"]
HIL["Human-in-the-Loop<br/><i>Approval before execution</i>"]
RAM["Activity Renderers<br/><i>Custom activity messages</i>"]
RCM["Custom Message Renderers<br/><i>Inject UI before/after messages</i>"]
SUG["Suggestions Config<br/><i>AI or static suggestions</i>"]
SUBS["Event Subscribers<br/><i>React to lifecycle events</i>"]
end
subgraph "Backend (Runtime)"
BM["Before Middleware<br/><i>Auth, logging, transforms</i>"]
AM["After Middleware<br/><i>Post-processing</i>"]
RUNNER["Agent Runner<br/><i>How agents execute</i>"]
TS["Transcription Service<br/><i>Audio → text</i>"]
end
subgraph "Agent Level"
MW["AG-UI Middleware<br/><i>Intercept agent pipeline</i>"]
end
```
---
## Frontend Extension Points
### 1. Frontend Tools
**What:** Functions in your app that agents can call during a conversation.
**Where configured:**
- React: `useFrontendTool()` hook or `frontendTools` provider prop
- Angular: `copilotKit.addTool()` or `tools` in config
- Vanilla: `copilotKit.addTool()`
**Default when not provided:** No tools — agent can only send text messages.
```typescript
// Type signature
type FrontendTool<T> = {
name: string;
description?: string;
parameters?: z.ZodType<T>;
handler?: (args: T, context: FrontendToolHandlerContext) => Promise<unknown>;
followUp?: boolean; // Re-run agent after tool completes
agentId?: string; // Scope to specific agent
};
```
```mermaid
sequenceDiagram
participant Agent
participant Core as CopilotKitCore
participant Tool as Your Tool Handler
Agent->>Core: TOOL_CALL_START { name: "myTool" }
Agent->>Core: TOOL_CALL_ARGS { ... }
Core->>Tool: handler(args)
Tool-->>Core: result
Core->>Agent: TOOL_CALL_RESULT
opt followUp = true
Core->>Agent: Re-run agent with result
end
```
---
### 2. Agent Context
**What:** JSON data that gets sent to agents as context (like "the user is on the settings page").
**Where configured:**
- React: `useAgentContext()` hook
- Angular / Vanilla: `copilotKit.addContext()` / `removeContext()`
**Default when not provided:** No extra context — agent only sees messages and tool definitions.
```typescript
type AgentContextInput = {
description: string; // Human-readable label
value: JsonSerializable; // Any JSON value
};
```
---
### 3. Tool Call Renderers
**What:** Custom React components that render while a tool is being called — showing progress, args, and results.
**Where configured:**
- React: `useRenderToolCall()` hook or `renderToolCalls` provider prop
- Angular: `renderToolCalls` in config
**Default when not provided:** Generic built-in rendering.
```typescript
type ReactToolCallRenderer<T> = {
name: string; // Tool name to render
args: z.ZodSchema<T>; // Schema for type-safe args
agentId?: string; // Scope to specific agent
render: React.ComponentType<
| { status: "in-progress"; args: Partial<T>; result: undefined }
| { status: "executing"; args: T; result: undefined }
| { status: "complete"; args: T; result: string }
>;
};
```
```mermaid
graph LR
IP["in-progress<br/><i>Args streaming in<br/>Partial<T> available</i>"]
EX["executing<br/><i>Handler running<br/>Full args available</i>"]
CO["complete<br/><i>Result available</i>"]
IP --> EX --> CO
```
---
### 4. Human-in-the-Loop
**What:** Tools that pause and wait for user input before continuing. The user sees a custom UI with approve/deny buttons.
**Where configured:**
- React: `useHumanInTheLoop()` hook or `humanInTheLoop` provider prop
- Angular: `humanInTheLoop` in config
**Default when not provided:** No approval required — tools execute immediately.
```typescript
type ReactHumanInTheLoop<T> = Omit<FrontendTool<T>, "handler"> & {
render: React.ComponentType<{
args: T;
status: "in-progress" | "executing" | "complete";
respond: (result: unknown) => Promise<void>; // Call this to approve/deny
}>;
};
```
```mermaid
sequenceDiagram
participant Agent
participant Core as CopilotKitCore
participant UI as Your Approval UI
participant User
Agent->>Core: TOOL_CALL { name: "deleteUser" }
Core->>UI: Render with status: "executing"
UI->>User: "Delete user X?"
User->>UI: Clicks "Approve"
UI->>Core: respond("approved")
Core->>Agent: TOOL_CALL_RESULT
Agent->>Agent: Continues
```
---
### 5. Activity Message Renderers
**What:** Custom UI for structured activity messages (non-chat messages like progress indicators or MCP app outputs).
**Where configured:**
- React: `useRenderActivityMessage()` hook or `renderActivityMessages` provider prop
**Default when not provided:** Built-in MCP Apps renderer is included. Other activity types show generic display.
```typescript
type ReactActivityMessageRenderer<T> = {
activityType: string; // Use "*" for wildcard
agentId?: string;
content: z.ZodSchema<T>;
render: React.ComponentType<{
activityType: string;
content: T;
message: ActivityMessage;
agent: AbstractAgent | undefined;
}>;
};
```
---
### 6. Custom Message Renderers
**What:** Inject custom UI before or after specific messages (e.g., add a "copy" button, show state snapshots).
**Where configured:**
- React: `useRenderCustomMessages()` hook or `renderCustomMessages` provider prop
**Default when not provided:** No custom rendering — standard message display.
```typescript
type ReactCustomMessageRenderer = {
agentId?: string;
render: React.ComponentType<{
message: Message;
position: "before" | "after";
runId: string;
messageIndex: number;
agentId: string;
stateSnapshot: any;
}> | null;
};
```
---
### 7. Suggestions Configuration
**What:** Configure AI-generated or static prompt suggestions shown to users.
**Where configured:**
- React: `useConfigureSuggestions()` hook
- Core: `suggestionsConfig` in config
**Default when not provided:** No suggestions.
```typescript
// AI-generated suggestions
type DynamicSuggestionsConfig = {
instructions: string; // What to suggest
minSuggestions?: number; // Default: 1
maxSuggestions?: number; // Default: 3
available?: SuggestionAvailability; // When to show
providerAgentId?: string; // Which agent generates them
consumerAgentId?: string; // Which agent receives them ("*" = all)
};
// Static suggestions
type StaticSuggestionsConfig = {
suggestions: Array<{ title: string; message: string }>;
available?: SuggestionAvailability;
consumerAgentId?: string;
};
type SuggestionAvailability =
| "before-first-message" // Default for static
| "after-first-message" // Default for dynamic
| "always"
| "disabled";
```
```mermaid
graph TB
subgraph "Suggestion Types"
DYN["Dynamic<br/><i>AI generates suggestions<br/>from instructions</i>"]
STA["Static<br/><i>You provide fixed<br/>suggestion list</i>"]
end
subgraph "Availability"
BFM["before-first-message"]
AFM["after-first-message"]
ALW["always"]
DIS["disabled"]
end
DYN -.->|default| AFM
STA -.->|default| BFM
```
---
### 8. Event Subscribers
**What:** Listen to lifecycle events — connection status, tool execution, agent changes, errors.
**Where configured:**
- Any: `copilotKit.subscribe(subscriber)`
- Returns: `{ unsubscribe() }` for cleanup
**Default when not provided:** No listeners — events still fire internally.
```typescript
type CopilotKitCoreSubscriber = {
onRuntimeConnectionStatusChanged?: (event) => void;
onToolExecutionStart?: (event) => void;
onToolExecutionEnd?: (event) => void;
onAgentsChanged?: (event) => void;
onContextChanged?: (event) => void;
onSuggestionsChanged?: (event) => void;
onSuggestionsStartedLoading?: (event) => void;
onSuggestionsFinishedLoading?: (event) => void;
onPropertiesChanged?: (event) => void;
onHeadersChanged?: (event) => void;
onError?: (event) => void;
};
```
---
## Backend Extension Points
### 9. Before Request Middleware
**What:** Intercept HTTP requests before they reach the handler. Use for auth, logging, request transformation.
**Where configured:** `CopilotRuntime` constructor — `beforeRequestMiddleware`
**Default when not provided:** Requests pass through unchanged.
```typescript
type BeforeRequestMiddleware = (params: {
runtime: CopilotRuntime;
request: Request;
path: string;
}) => MaybePromise<Request | void>;
// Return modified Request, or void to pass through
// Return a Response to short-circuit (e.g., 401)
```
```mermaid
graph LR
REQ["Incoming Request"]
BM["beforeRequestMiddleware"]
HANDLER["Route Handler"]
REJECT["401 / Error Response"]
REQ --> BM
BM -->|pass through| HANDLER
BM -->|reject| REJECT
```
---
### 10. After Request Middleware
**What:** Run code after the response is prepared. Use for logging, metrics, cleanup.
**Where configured:** `CopilotRuntime` constructor — `afterRequestMiddleware`
**Default when not provided:** No post-processing.
```typescript
type AfterRequestMiddleware = (params: {
runtime: CopilotRuntime;
response: Response;
path: string;
}) => MaybePromise<void>;
```
---
### 11. Agent Runner
**What:** Controls how agents are executed and how thread state is managed.
**Where configured:** `CopilotRuntime` constructor — `runner`
**Default when not provided:** `InMemoryAgentRunner` — in-process, ephemeral (threads lost on restart).
```typescript
abstract class AgentRunner {
abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
}
```
| Implementation | Storage | Persistence | Use case |
| --------------------- | ----------- | ----------- | -------------------------------- |
| `InMemoryAgentRunner` | RAM | No | Development, stateless apps |
| `SQLiteAgentRunner` | Disk | Yes | Production, long-running threads |
| Custom | Your choice | Your choice | Redis, PostgreSQL, etc. |
```mermaid
graph TB
RT["CopilotRuntime"]
RUNNER["runner (AgentRunner)"]
RT --> RUNNER
subgraph Implementations
IM["InMemoryAgentRunner<br/><i>Default — in-process</i>"]
SQ["SQLiteAgentRunner<br/><i>Persistent on disk</i>"]
CU["YourCustomRunner<br/><i>Redis, Postgres, etc.</i>"]
end
RUNNER -.-> IM
RUNNER -.-> SQ
RUNNER -.-> CU
```
---
### 12. Transcription Service
**What:** Convert audio files to text. Enables the `/transcribe` endpoint.
**Where configured:** `CopilotRuntime` constructor — `transcriptionService`
**Default when not provided:** `/transcribe` endpoint returns 404.
```typescript
abstract class TranscriptionService {
abstract transcribeFile(options: {
audioFile: File;
mimeType?: string;
size?: number;
}): Promise<string>;
}
```
---
## Agent-Level Extension Points
### 13. AG-UI Middleware
**What:** Intercept and transform the agent execution pipeline. Cross-cutting concerns like logging, filtering, and backward compatibility.
**Where configured:** At the agent level (outside CopilotKit core).
**Default when not provided:** Direct agent execution.
```typescript
abstract class Middleware {
abstract run(
input: RunAgentInput,
next: AbstractAgent,
): Observable<BaseEvent>;
}
// Built-in implementations:
// - FunctionMiddleware — wrap a function as middleware
// - FilterToolCallsMiddleware — filter which tools are sent
```
```mermaid
graph LR
INPUT["RunAgentInput"]
MW1["Middleware 1<br/><i>e.g., logging</i>"]
MW2["Middleware 2<br/><i>e.g., tool filtering</i>"]
AGENT["Agent.run()"]
INPUT --> MW1 --> MW2 --> AGENT
```
---
## Complete Map: Where Each Extension Plugs In
```mermaid
graph TB
subgraph "Provider / Config"
P["CopilotKitProvider<br/>or provideCopilotKit()"]
P --> FT_P["frontendTools"]
P --> RTC_P["renderToolCalls"]
P --> RAM_P["renderActivityMessages"]
P --> RCM_P["renderCustomMessages"]
P --> HIL_P["humanInTheLoop"]
P --> HDR["headers"]
P --> CRD["credentials"]
P --> PRP["properties"]
P --> DC["showDevConsole"]
end
subgraph "Hooks / Service Methods"
UFT["useFrontendTool()"]
UAC["useAgentContext()"]
URT["useRenderToolCall()"]
UHL["useHumanInTheLoop()"]
UCS["useConfigureSuggestions()"]
URA["useRenderActivityMessage()"]
URC["useRenderCustomMessages()"]
end
subgraph "CopilotRuntime"
RT["new CopilotRuntime()"]
RT --> AGENTS["agents (required)"]
RT --> RUNNER["runner"]
RT --> BM["beforeRequestMiddleware"]
RT --> AM["afterRequestMiddleware"]
RT --> TS["transcriptionService"]
end
subgraph "Core API"
SUB["copilotKit.subscribe()"]
AT["copilotKit.addTool()"]
AC["copilotKit.addContext()"]
end
```
---
## Summary Table
| Extension Point | Location | Config Method | Default | Optional |
| ---------------------------- | -------- | ----------------------------- | ------------------- | -------- |
| **Frontend Tools** | Frontend | Hook / Provider / `addTool()` | None | Yes |
| **Agent Context** | Frontend | Hook / `addContext()` | None | Yes |
| **Tool Call Renderers** | Frontend | Hook / Provider | Generic rendering | Yes |
| **Human-in-the-Loop** | Frontend | Hook / Provider | Immediate execution | Yes |
| **Activity Renderers** | Frontend | Hook / Provider | MCP Apps included | Yes |
| **Custom Message Renderers** | Frontend | Hook / Provider | None | Yes |
| **Suggestions Config** | Frontend | Hook / Config | None | Yes |
| **Event Subscribers** | Frontend | `subscribe()` | None | Yes |
| **Before Middleware** | Backend | Runtime constructor | Pass-through | Yes |
| **After Middleware** | Backend | Runtime constructor | None | Yes |
| **Agent Runner** | Backend | Runtime constructor | InMemoryAgentRunner | Yes |
| **Transcription Service** | Backend | Runtime constructor | None (404) | Yes |
| **AG-UI Middleware** | Agent | Agent-level config | Direct execution | Yes |