agent-server/README.md
Rama main · réplica de lectura del repositorio original.
# Eval-Server
HTTP API wrapper for Browser Operator - provides WebSocket server with Chrome DevTools Protocol (CDP) integration for browser automation.
## Overview
The eval-server exposes browser automation capabilities via HTTP API endpoints. It manages WebSocket connections to browser agents and provides REST APIs for:
- Sending tasks to agents (`/v1/responses`)
- Capturing screenshots via CDP
- Retrieving page content
- Managing browser tabs
**Note:** Evaluation orchestration and LLM-as-a-judge logic lives in the separate `evals/` Python project, which calls these APIs.
## Architecture
```
eval-server/
└── nodejs/
├── src/
│ ├── api-server.js # HTTP REST API endpoints
│ ├── client-manager.js # WebSocket client management
│ ├── rpc-client.js # JSON-RPC 2.0 communication
│ ├── config.js # Configuration management
│ ├── logger.js # Winston logging
│ └── lib/
│ ├── EvalServer.js # Core server + CDP integration
│ └── HTTPWrapper.js # HTTP API wrapper
└── package.json
```
## Quick Start
### Installation
```bash
cd eval-server/nodejs
npm install
```
### Configuration
Copy `.env.example` to `.env` and configure:
```bash
# WebSocket server port
PORT=8080
# HTTP API server port
API_PORT=8081
# Authentication
AUTH_KEY=your-secret-key
# Chrome DevTools Protocol endpoint
CDP_HOST=localhost
CDP_PORT=9223
```
### Start Server
```bash
npm start
```
The server will start:
- WebSocket server on `ws://localhost:8080`
- HTTP API server on `http://localhost:8081`
## HTTP API Endpoints
### Core Endpoint
#### `POST /v1/responses`
Send a task to a connected browser agent and get response.
**Request (OpenAI):**
```json
{
"input": "Click the submit button",
"url": "https://example.com",
"wait_timeout": 5000,
"model": {
"main_model": {
"provider": "openai",
"model": "gpt-5-mini",
"api_key": "sk-..."
},
"mini_model": {
"provider": "openai",
"model": "gpt-5-nano",
"api_key": "sk-..."
},
"nano_model": {
"provider": "openai",
"model": "gpt-5-nano",
"api_key": "sk-..."
}
}
}
```
**Request (LiteLLM):**
```json
{
"input": "Navigate to google.com",
"url": "about:blank",
"wait_timeout": 5000,
"model": {
"main_model": {
"provider": "litellm",
"model": "qwen3:14b",
"endpoint": "http://localhost:4000",
"api_key": "sk-litellm-key"
},
"mini_model": {
"provider": "litellm",
"model": "qwen3:14b",
"endpoint": "http://localhost:4000"
},
"nano_model": {
"provider": "litellm",
"model": "qwen3:14b",
"endpoint": "http://localhost:4000"
}
}
}
```
**Endpoint Configuration:**
- `endpoint` can be specified per model tier (e.g., `main_model.endpoint`)
- Or at top-level `model.endpoint` to apply to all tiers
- Falls back to `LITELLM_ENDPOINT` environment variable if not provided
- Required for LiteLLM provider unless set via environment variable
**Response:**
```json
[
{
"id": "msg_abc123",
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Done - clicked submit button",
"annotations": []
}
],
"metadata": {
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429"
}
}
]
```
The `metadata` field contains `clientId` and `tabId` which can be used for screenshot capture and other CDP operations.
### CDP Endpoints
#### `POST /page/screenshot`
Capture screenshot of a specific browser tab.
**Request:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"fullPage": false
}
```
**Response:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"imageData": "data:image/png;base64,iVBORw0KG...",
"format": "png",
"fullPage": false,
"timestamp": 1234567890
}
```
#### `POST /page/content`
Get HTML or text content of a page.
**Request:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"format": "html"
}
```
**Response:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"content": "<html>...</html>",
"format": "html",
"length": 12345,
"timestamp": 1234567890
}
```
#### `POST /page/execute`
Execute JavaScript code in the context of a specific browser tab via Chrome DevTools Protocol.
**Request:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"expression": "document.title",
"returnByValue": true,
"awaitPromise": false
}
```
**Parameters:**
- `clientId` (required): The client ID from `/v1/responses` metadata
- `tabId` (required): The tab ID from `/v1/responses` metadata
- `expression` (required): JavaScript code to execute (string)
- `returnByValue` (optional, default: `true`): Whether to return result by value or as object reference
- `awaitPromise` (optional, default: `false`): Whether to await if the result is a Promise
**Response:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"result": {
"type": "string",
"value": "Example Page Title"
},
"exceptionDetails": null,
"timestamp": 1234567890
}
```
**Response Fields:**
- `clientId`: Base client ID (without tab suffix)
- `tabId`: The tab ID where JavaScript was executed
- `result`: CDP `Runtime.evaluate` result object containing:
- `type`: Result type (string, number, object, etc.)
- `value`: The actual value (if `returnByValue: true`)
- `exceptionDetails`: Error details if execution failed, otherwise `null`
- `timestamp`: Unix timestamp in milliseconds
**Example Usage:**
```bash
# Get page title
curl -X POST http://localhost:8080/page/execute \
-H "Content-Type: application/json" \
-d '{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"expression": "document.title"
}'
# Count elements
curl -X POST http://localhost:8080/page/execute \
-H "Content-Type: application/json" \
-d '{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"expression": "document.querySelectorAll(\"button\").length"
}'
# Execute async code with await
curl -X POST http://localhost:8080/page/execute \
-H "Content-Type: application/json" \
-d '{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"expression": "fetch(\"https://api.example.com/data\").then(r => r.json())",
"awaitPromise": true
}'
```
**Use Cases:**
- Extract specific data from the page (e.g., element counts, text content)
- Verify JavaScript state/variables for evaluations
- Check DOM state programmatically
- Execute custom validation logic
- Interact with page APIs directly
This endpoint complements `/page/content` by allowing precise JavaScript execution rather than just fetching full HTML/text content.
#### `POST /tabs/open`
Open a new browser tab.
**Request:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"url": "https://example.com",
"background": false
}
```
**Response:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "NEW_TAB_ID",
"compositeClientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306:NEW_TAB_ID",
"url": "https://example.com",
"status": "opened"
}
```
#### `POST /tabs/close`
Close a browser tab.
**Request:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "TAB_ID_TO_CLOSE"
}
```
**Response:**
```json
{
"clientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"tabId": "TAB_ID_TO_CLOSE",
"status": "closed",
"success": true
}
```
### Status Endpoints
#### `GET /status`
Get server health and connected clients.
**Response:**
```json
{
"server": {
"running": true,
"uptime": 12345,
"connections": 1
},
"clients": [
{
"id": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"name": "DevTools Client",
"connected": true,
"ready": true
}
]
}
```
#### `GET /clients`
List all connected clients with their tabs.
**Response:**
```json
[
{
"id": "9907fd8d-92a8-4a6a-bce9-458ec8c57306",
"name": "DevTools Client",
"description": "Browser automation agent",
"tabCount": 3,
"tabs": [
{
"tabId": "482D56EE57B1931A3B9D1BFDAF935429",
"compositeClientId": "9907fd8d-92a8-4a6a-bce9-458ec8c57306:482D56EE57B1931A3B9D1BFDAF935429",
"connected": true,
"ready": true,
"connectedAt": "2025-01-15T10:30:00.000Z",
"remoteAddress": "::ffff:172.18.0.1"
}
]
}
]
```
## WebSocket Protocol
Browser agents connect to the WebSocket server and implement JSON-RPC 2.0 protocol.
### Connection Flow
1. **Connect to WebSocket**
```
ws://localhost:8080
```
2. **Send Registration**
```json
{
"type": "register",
"clientId": "unique-client-id",
"secretKey": "your-auth-key",
"capabilities": ["browser-automation"]
}
```
3. **Send Ready Signal**
```json
{
"type": "ready"
}
```
4. **Handle RPC Calls**
Server sends JSON-RPC 2.0 requests:
```json
{
"jsonrpc": "2.0",
"method": "evaluate",
"params": {
"tool": "action_agent",
"input": {"objective": "Click submit button"},
"model": {...}
},
"id": "request-id"
}
```
Agent responds:
```json
{
"jsonrpc": "2.0",
"id": "request-id",
"result": {
"status": "completed",
"output": "Task completed successfully"
}
}
```
## Chrome DevTools Protocol Setup
The browser must be started with remote debugging enabled:
```bash
chromium --remote-debugging-port=9223
```
The CDP endpoint is accessible at:
- HTTP: `http://localhost:9223/json/version`
- WebSocket: `ws://localhost:9223/devtools/browser/{browserId}`
## Usage with Evals Framework
The eval-server is designed to work with the separate `evals/` Python project:
1. **Start eval-server:**
```bash
cd eval-server/nodejs
npm start
```
2. **Run evaluations from evals/:**
```bash
cd evals
python3 run.py --path action-agent/accordion-001.yaml --verbose
```
The evals framework:
- Sends tasks to `/v1/responses` endpoint
- Extracts `clientId` and `tabId` from response metadata
- Captures screenshots via `/page/screenshot`
- Uses LLM judges (LLMJudge, VisionJudge) to evaluate results
- Generates reports and saves screenshots
See `evals/README.md` for detailed evaluation framework documentation.
## Dependencies
Core dependencies:
- **ws** - WebSocket server
- **uuid** - ID generation
- **winston** - Structured logging
- **dotenv** - Environment variable management
## Logging
Logs are written to `logs/` directory (auto-created):
- `combined.log` - All log events
- `error.log` - Error events only
- `api.log` - API request/response logs
## Docker Integration
The eval-server runs inside the `kernel-browser-extended` Docker container. Volume mount for live development:
```yaml
# docker-compose.yml
volumes:
- "./eval-server/nodejs:/opt/eval-server"
```
## Development
```bash
# Install dependencies
npm install
# Start server
npm start
# Check status
curl http://localhost:8081/status
# Test screenshot capture
curl -X POST http://localhost:8081/page/screenshot \
-H "Content-Type: application/json" \
-d '{"clientId":"CLIENT_ID","tabId":"TAB_ID","fullPage":false}'
```
## License
MIT License