docs/roadmap.md

Rama main · réplica de lectura del repositorio original.

# Roadmap — Toward "Agents, but operable"

This roadmap targets the gap between the current single-agent management story
and the fleet-operability promise on openhermit.ai. `plan.md` tracks
already-shipped features; this document tracks what is still missing for
OpenHermit to be a credible platform for running fleets of agents in
production.

## Guiding principle

Every agent is a service. The gateway is the control plane. An operator with
many agents must be able to **deploy, observe, change, and recover** them as a
fleet — not one at a time.

We prefer composition of small primitives over bespoke platform machinery. If
operators can build a behavior themselves with the existing model
(assignments, skills, scopes), the platform should not invent a new concept
for it.

---

## Milestone 1 — Fleet operations

Make the hero animation real: one action affects N agents, with the same
mental model that already works.

### M1.1 — Wildcard assignment as the universal pattern

The skill assignment table already supports `agent_id = '*'`: a single row in
`agent_skills` matches every agent at query time
(`WHERE agent_id IN ($agentId, '*')`). New agents automatically inherit
wildcard assignments at startup. This is the right model — no fan-out at
write time, no partial-failure semantics, no batch concept.

**Current state (verified):**

| Resource | Store wildcard | API accepts `*` | UI accepts `*` | Runtime sync on change |
| --- | --- | --- | --- | --- |
| Skills | ✅ | ✅ | ✅ | ✅ `syncAffectedAgentSkillMounts` |
| MCP servers | ✅ | ✅ | ✅ | ✅ `runner.reloadMcpServers()` |
| Schedules | ❌ per-agent only | n/a | n/a | n/a |
| Channels | n/a (each adapter holds its own token/identity — wildcard not meaningful) | — | — | — |

**Concrete work for M1.1:**

- ✅ **MCP runtime sync** — `AgentRunner.reloadMcpServers()` now
  disconnects and reconnects against the fresh `listEnabled` result;
  the gateway endpoints (`/api/admin/mcp-servers/:id/enable` and
  `/disable`) call it on running agents.
- ✅ **Wildcard convention** — `docs/storage-model.md` documents
  `agent_id = '*'` and lists resources that honor it.

**Out of M1.1 (intentionally):**

- **Wildcard schedules.** Schedules are owned per agent and the schedule
  definition includes prompt content the agent must execute. A wildcard
  schedule would require the scheduler to enumerate agents at trigger time
  and spawn one run per agent — a real feature, not just a query-shape
  change. The same outcome is reachable today by templating identical
  schedules in `agents.yaml` (M3.1) and reconciling, which keeps the
  scheduler simple. Revisit only if a real use case demands centralized
  fleet-wide schedules.
- **Wildcard channels.** Each adapter binds a bot identity / token, so
  "channel for all agents" is not a meaningful operation.

**Acceptance:**

1. Enabling an MCP server with target `*` makes the new server's tools
   available in every running agent's next turn, without restarting any
   agent.
2. `docs/storage-model.md` documents the `agent_id = '*'` convention with a
   table of which resources honor it.

### M1.2 — Fleet overview UI ✅ shipped

The admin UI's default landing tab is now "Fleet": a single table listing
every agent with its operational health.

- Columns: status, last activity, sessions (24h), errors (24h), channels,
  skills count, MCP count.
- Auto-refreshes every 10 s; manual refresh button.
- Multi-select with a "Bulk skill action" dialog. When all agents are
  selected, the dialog targets `agent_id = '*'`; otherwise it iterates
  per-agent using the existing endpoints.
- Backend: `GET /api/admin/agents/fleet` aggregates per-agent stats
  (`fleetStats` on `DbAgentStore`) — sessions and errors from
  `session_events`, skills/MCP counts from assignments including wildcard,
  channels from runtime status.

The "Agents" tab is preserved for per-agent management actions (start/stop,
config, secrets, skills/MCP/channels detail).

### Explicitly NOT in M1

- **No `/fleet/*` API surface.** Existing per-agent endpoints + `*` are
  sufficient.
- **No skill or MCP version field.** Operators who want canary or staged
  rollout publish a second skill (`standup-digest-v2`), assign it to a subset
  of agents, observe, and then either expand the assignment or remove the new
  skill. Two assignments express the intent without any platform-side state
  machine.
- **No Rollout resource.** Same reason — composition of existing primitives
  covers the use case for any reasonable fleet size we care about today.

---

## Milestone 2 — Observability

You cannot operate what you cannot see.

### M2.1 — Metrics ✅ shipped

- Gateway exposes `/metrics` in Prometheus format (no auth — bind to
  localhost or scrape via reverse proxy).
- Per-agent metrics, all labeled by `agent_id`:
  - `openhermit_agent_turns_total` — completed LLM turns
  - `openhermit_agent_turn_duration_seconds` — histogram for p50/p95 latency
  - `openhermit_agent_tokens_total{direction=in|out|cache_read|cache_write}`
  - `openhermit_agent_tool_calls_total{tool}`
  - `openhermit_agent_errors_total{source=runtime|model}`
  - `openhermit_agent_messages_total{source}` — inbound messages by source
    kind (web, telegram, discord, slack, …)
- Default Node process metrics (CPU, heap, event loop) also exposed.
- The fleet overview UI (M1.2) sparklines will source from these metrics,
  not fabricated data.

### M2.2 — Cross-agent audit log

- A query API on top of `session_events` filterable by `agent_id`, `user_id`,
  `channel`, `event_type`, time range.
- Admin UI page "Activity" with filters and a streaming tail mode.
- CLI: `hermit logs --agent one --since 1h --grep error`.

**Acceptance:** From a single page, an operator can find every action a given
user took across all agents in the last week.

### M2.3 — Health endpoints

- Each agent reports a structured health blob: model reachable, exec backend
  reachable, channels connected, scheduler running.
- Gateway `/health` aggregates fleet health into a single OK/DEGRADED/DOWN
  summary.
- Admin UI shows a fleet health pill at the top of every page.

**Acceptance:** Stopping the Docker daemon flips one agent's exec backend to
DOWN, the gateway summary to DEGRADED, and the UI surfaces it without a
manual refresh.

---

## Milestone 3 — Declarative deploy

This is the line between "production service" and "manually-edited JSON."

### M3.1 — `agents.yaml` reconciliation

- New CLI: `hermit apply -f agents.yaml`. The file is the source of truth for
  a set of agents: model, skills (including `*` wildcard assignments), MCP
  servers, channels, schedules, secrets refs.
- Apply diffs against the current DB state and reconciles: create / update /
  delete agents, install / uninstall skills, etc.
- `hermit diff -f agents.yaml` shows a dry-run plan.
- `hermit drift` reports anything in the DB not represented in the manifest.

**Acceptance:** An operator can check `agents.yaml` into git, change a model
version on one agent, run `hermit apply`, and see exactly that one update
applied.

### M3.2 — Org-level secrets

- Per-agent secrets already encrypted at rest (`agent_secrets` table,
  AES-GCM with `OPENHERMIT_SECRETS_KEY`). Next: shared org-scoped
  secrets — one row referenced by many agents, with KMS-backed key
  management for hosted deployments.
- Manifests reference secrets by name (`secretRef: TELEGRAM_TOKEN`); multiple
  agents can share one secret.
- Audit log entry on every secret read.

**Acceptance:** Rotating a Telegram bot token is a single update that takes
effect for every agent referencing it, with an audit trail.

### M3.3 — Supervisor & auto-recovery

- Gateway-side supervisor for agent runners: crash → restart with exponential
  backoff, with a circuit breaker after N failures in a window.
- Channel adapters reconnect on transient failures uniformly across Telegram
  / Discord / Slack and surface state in metrics.
- Postgres reconnect logic in the gateway so a database blip does not require
  a manual gateway restart (we hit this in practice — restoring the DB
  container should be enough).

**Acceptance:** Killing the Postgres container and then restarting it results
in the gateway and all agents recovering automatically within ~30 seconds.

---

## Milestone 4 — Collaboration & multi-tenancy

Operability for teams, not just one root user.

### M4.1 — Multiple owners / admins

- Owner role today is a single account. Allow promoting other users to admin
  with full management rights.
- Audit log entries record actor on every mutation.
- UI: "Members" page to invite / promote / demote.

### M4.2 — Service accounts & API tokens

- First-class non-human principals with scoped tokens
  (e.g. `agents:one:read`, `skills:write`).
- Tokens are issued via CLI / UI, revocable, and show last-used timestamps.

### M4.3 — Per-agent ACLs

- Restrict which users / service accounts can manage which agents.
- Enables a single gateway to host agents owned by different teams.

---

## Milestone 5 — Bridging cloud agents and the user's world

The agent runs in the cloud (or some box that isn't the user's laptop), but
useful work usually involves files and applications that live on the user's
local machine. Today there's no smooth path between the two — users paste
content into chat, copy outputs back out by hand. This milestone closes that
loop with two complementary primitives.

### M5.1 — Shared file drop ("hermit drive")

A small object-store-backed namespace for moving files between users and
their agents in either direction.

Capabilities:

- **User → agent**: drag a file into the chat / web UI, agent gets a stable
  URI it can pass to tools (`drive://<agent>/<id>/<filename>`).
- **Agent → user**: agent produces a report / dataset / image, drops it into
  the same namespace, user gets a shareable link in chat with a short TTL.
- **Per-agent isolation**: each agent has its own bucket prefix; cross-agent
  access requires an explicit grant.
- **Storage backend** is pluggable — local filesystem, S3, MinIO, R2 —
  behind a single interface in `@openhermit/store`.

Open questions:

- Encryption at rest. Same key as agent secrets, or per-file keys?
- Retention. Default TTL vs explicit pin from owner.
- Quotas — per agent, per user, or fleet-wide.

### M5.2 — Computer node (local agent capability host)

A small daemon the user runs on their own machine that **registers with the
gateway** and exposes a narrow, authenticated set of capabilities the cloud
agent can call. Conceptually: a channel adapter, but for "computer" instead
of Telegram or Slack.

Initial capability surface:

- **Filesystem** — read / write within an allowlisted root (e.g. `~/agent-scratch`).
- **Browser** — open URL, take screenshot, scrape DOM (likely via Playwright
  embedded in the node, or by attaching to the user's existing browser).
- **Process** — run a whitelisted command in a constrained working dir,
  stream stdout/stderr back.

Trust model:

- The node ships its own Ed25519 key, registers with the gateway via the
  same auth flow as a web device key. Owner of the agent must explicitly
  link the node to their account before any capability calls work.
- Every capability invocation is **policy-gated and logged**, identical to
  agent tool calls today. The owner sees a stream of "agent X read file Y"
  / "agent X navigated to Z" entries and can revoke at any time.
- Default: prompt-on-each-call. Auto-approve only after the owner
  explicitly trusts a given (agent, capability, scope) tuple.
- Channel-token isolation already gives us the right shape: the node
  authenticates with a per-node token issued to the owner; the agent's
  permission to call those capabilities is independent and revocable.

Open questions:

- Wire format. JSON-RPC over WebSocket (long-lived connection, agent calls
  arrive as messages) is the leading candidate.
- How to surface progress / streaming results — extend the existing tool
  event stream so a "node tool call" looks identical to a built-in tool
  call from the agent's perspective.
- Cross-platform: macOS / Linux first, Windows once the rest is stable.
- Co-existence with the workspace's container-based exec — when does an
  agent reach for the node vs the container?

### Why both, and in this order

Drive (M5.1) is mostly storage plumbing — straightforward but high value
and low risk. Computer node (M5.2) is the bigger architectural lift
(local daemon, OS integration, trust model) and benefits from drive
already existing as the file-transfer primitive between cloud and node.

---

## Out of scope (for now)

These are valuable but deliberately deferred until the milestones above land:

- Hosted / managed offering (cloud control plane).
- Multi-region replication of the gateway.
- Cost / billing primitives.
- Per-agent autoscaling beyond Docker container lifecycle.

---

## Sequencing

The suggested order is M1 → M2 → M3 → M4 → M5. The reasoning:

- **M1** turns the brand promise into reality and is the most visible to
  users evaluating the project. It is also the smallest milestone — most of
  the wildcard mechanism already exists for skills; the work is generalizing
  the pattern and shipping the fleet overview UI.
- **M2** is a hard prerequisite for anyone running this in production —
  without it, the fleet is a black box.
- **M3** is what separates "demo" from "service" and naturally builds on the
  declarative wildcard assignments from M1.
- **M4** unlocks team adoption but assumes the single-operator story is
  already polished.
- **M5** is what makes the agent feel like it's actually *with you* —
  share files, touch your real machine. Best built on top of the
  multi-tenant + audit foundations from M3/M4 because it expands the
  trust surface considerably.