stream.md
Rama main · réplica de lectura del repositorio original.
# agent-stream-hub
Streaming bridge that lets you peek at AI-CLI sessions running on Windows
`agent-term` instances from a MacBook browser or iPhone PWA.
## Architecture
```
[Windows agent-term × N] [MacBook browser / iPhone PWA]
│ HTTPS POST │ HTTPS GET
│ (register + push snapshots) │ (list runs, view one)
▼ ▼
Cloudflare edge (named tunnel → your hub's public URL)
│
▼
cloudflared on the host (supervised, e.g. LaunchDaemon / systemd)
│
▼
localhost:9000 ── agent-stream-hub (this repo, Node)
├── POST /runs (register / update meta)
├── POST /runs/:id/snapshot (push a viewport)
├── POST /runs/:id/heartbeat (liveness + isWorking)
├── POST /runs/:id/input (viewer → source prompt)
├── DELETE /runs/:id (cleanup)
├── GET /runs (list)
├── GET /runs/:id/latest (most recent snapshot)
├── GET /runs/:id/history (older snapshots)
└── GET / (viewer SPA)
│
▼
in-memory state
(ring buffer per runId,
text-similarity compaction)
```
Three tiers:
- **Source** — `agent-term` on Windows (or macOS), multiple concurrent instances.
Outbound HTTPS only. POSTs viewport snapshots to the hub.
- **Hub** — this repo, deployed on an always-on host. Schema-agnostic ring
buffer keyed by `runId`. Also serves the viewer SPA.
- **Viewer** — static HTML+JS served by the hub. Loaded by a browser or
installed PWA via the hub's public URL.
Why three tiers:
- **Corporate firewall** — the source can only outbound HTTPS to a Cloudflare
CDN host, never accept inbound. The hub gives it a reachable endpoint.
- **Clients sleep, the host doesn't** — hub state lives on the always-on host;
a viewer reconnects when its device wakes.
- **Multi-instance** — hub aggregates N sources, presents one list to the viewer.
- **Host stays low-touch** — the relay is schema-agnostic, so adding fields,
styles, or new view types never touches the host.
## Data model — snapshots, not blocks
The hub stores raw viewport snapshots; the viewer stitches them into a
continuous buffer per run. There are no blocks, no per-prompt segmentation,
no marker/divider tracking. The source's terminal viewport is the source of
truth, full stop.
### RunMeta (registration payload, hub-cached for the run's lifetime)
```js
type RunMeta = {
runId: string; // random hex, generated by source at startup
cli: string; // "claude" | "codex" | "copilot" | ...
title: string; // human-readable; the first user prompt
host: string; // hostname of the source machine
startedAt: number; // ms epoch
hue?: number; // 0..360, optional session color for chrome
};
```
### Snapshot (the per-push payload)
```js
type Snapshot = {
runId: string;
cols: number;
rows: Row[];
isWorking?: boolean; // "AI is computing"; drives viewer's working indicator
};
type Row = {
text: string;
styles?: StyleRun[];
};
type StyleRun = {
start: number; end: number;
fg?: string; bg?: string;
bold?: true; italic?: true; underline?: true; dim?: true;
inverse?: true; strike?: true;
};
```
### Hub-side ring entry
```js
type Entry = {
seq: number; // monotonic per run, set by source
ts: number; // ms epoch, hub-assigned on receipt
payload: Snapshot;
};
```
## Endpoints
All under the hub's public base URL. Bodies are JSON unless noted.
### Source → hub (auth-open; sources have no shared secret)
| Method | Path | Body | Returns |
|---|---|---|---|
| POST | `/runs` | `RunMeta` | `204` |
| POST | `/runs/:runId/snapshot` | `{ seq, payload: Snapshot }` | `{ viewerAgeMs }` |
| POST | `/runs/:runId/heartbeat` | `{ isWorking?: boolean, touchedAt?: number }` | `{ inputs: string[], voiceInputs: string[], viewerAgeMs }` |
`voiceInputs` are voice-origin transcripts the source prefixes with the guide
reference before typing (see [voice.md](voice.md)); `inputs` are typed viewer
prompts, written as-is.
`viewerAgeMs` is ms since a viewer last polled this run's detail view
(`/latest` or `/history`); `null` if never. An age, not a timestamp — hub and
source clocks can't be compared. The source uses it to pace its heartbeat by
viewer presence: top pace (2s) while someone is watching — even mid-work — so
interjections land in seconds; back to 30s-while-working once nobody has looked
for ~5 min. Snapshot acks carry it too, which is how a working source (slow
heartbeats, frequent snapshots) learns of a new viewer within a second or two.
`touchedAt` is when the session last drew the user, in ms on the source's
clock: their input, at the desk or from a viewer, or the agent finishing a
turn. agent-term sends the timer its auto-hide ages windows out by, so the
phone ranks sessions the way the desktop keeps them in view. The viewer lists a
machine's sessions by state (your turn first), then most recently touched
first; a run that never sent `touchedAt` ranks by `startedAt`. It is compared
only among one machine's runs, which share a clock. It moves on input and
finished turns, never on a heartbeat, so the list holds still while sessions
wait.
### Viewer / debug → hub (auth-gated: `X-Hub-Secret` required when configured)
| Method | Path | Returns |
|---|---|---|
| GET | `/runs` | `{ runs: RunMeta[] }` (all known), each with `lastSeenAt`, `isWorking`, `touchedAt` |
| GET | `/runs/:runId/latest` | `{ snapshot, lastSeenAt, isWorking }` or `404` |
| GET | `/runs/:runId/history?before=<seq>&limit=<n>` | `{ entries: Entry[] }` oldest→newest |
| POST | `/runs/:runId/input` | viewer-submitted prompt (`source:"voice"` for reviewed transcripts); drained on next heartbeat |
| POST | `/runs/:runId/voice` | raw audio in → transcript + gate verdict; auto-queues on confidence ([voice.md](voice.md)) |
| DELETE | `/runs/:runId` | `204` |
### Auth model
- **Viewer reads and control**: when `STREAM_HUB_SECRET` is set, require
`X-Hub-Secret` on every connection, including loopback. Proxy headers do
not bypass authentication. The viewer prompts once and stores the secret
in `localStorage`.
- **Source POSTs**: `/runs`, `/runs/:id/snapshot`, `/runs/:id/heartbeat`
remain open so source machines do not need the viewer secret. A run ID
acts as a capability to publish snapshots and drain queued replies;
keep it private. Anonymous registration is possible, so this is a hub
for mutually trusted machines and users, not isolated tenants.
- **Local development**: with no secret, API requests require a loopback
Host and no Cloudflare forwarding header. Always set a secret before
exposing a proxy, including proxies that rewrite Host to localhost.
- **Browser boundary**: API requests from other origins are rejected;
there is no wildcard CORS. Serve the viewer from the hub and preserve
Host at the proxy. Native AgentTerm sources do not send Origin headers.
- **JSON POSTs**: require `Content-Type: application/json`; `/voice` accepts
raw audio. Static viewer assets are public and include no session data.
### Resource limits
The hub retains at most 128 runs. Registration metadata is limited to
16 KiB; POST bodies to 4 MiB. Each run keeps up to 400 snapshots or 8 MiB
of serialized snapshot data, whichever is smaller. A 64 MiB global snapshot
budget removes the oldest snapshots under pressure. These are serialized
byte limits, not a bound on total JavaScript heap usage. Runs expire after
24 hours without activity.
Each run queues at most 64 replies totaling 64 KiB across typed and voice
input. A full queue returns `429` without accepting the input; heartbeat
drains it. Registration at capacity returns `503`; metadata over its limit
returns `413`. Snapshot and metadata fields consumed by the viewer are
validated before storage; extra fields remain allowed.
Voice processing permits two concurrent requests, limits decoding to
15 seconds and transcription to 30 seconds, and rejects recordings over
two minutes. Request-rate and connection limits belong at the proxy.
See [Security](SECURITY.md) for deployment and data handling.
## Source-side stream lifecycle
A run does **not** register at agent-term startup. It registers when the
first user prompt is captured by `prompt-capture.js`. Pre-CLI shell activity
and pre-first-prompt CLI welcome output are not streamed.
```
agent-term starts
─► generate runId in memory; wire prompt-capture; no hub traffic yet
User submits first prompt to AI CLI:
─► POST /runs (register; RunMeta.title = first prompt)
─► start heartbeat timer
renderer-watch detects viewport change:
─► POST /runs/<id>/snapshot (current visible rows)
Subsequent prompts:
─► local state updates only (no extra hub traffic)
─► next viewport change pushes a new snapshot as usual
Alt-screen flip (copilot, codex resume picker, ...):
─► POST /runs/<id>/snapshot with new viewport
─► viewer's stitcher inserts a DROPPED marker (alt-screen-replace path)
Periodically:
─► POST /runs/<id>/heartbeat — adaptive pace: 2s while awaiting input or
while a viewer is watching (viewerAgeMs from acks), 30s while working
unwatched; 200ms burst after an input drains
agent-term exits cleanly:
─► heartbeat stops; run becomes stale after 60s; hub evicts after 24h idle
agent-term crashes or net drops:
─► same — heartbeat stops, run ages out
```
### What scrolling and alt-screen mean to the source
The renderer encodes only the **visible viewport** (`buffer.baseY` through
`baseY + terminal.rows`). Content that scrolled off the top is not in the
snapshot — and is not in any later snapshot. The viewer's stitcher captures
rows that pass through the viewport over time, accumulating them into a
buffer; rows that scrolled off **before** the viewer connected are lost.
Alt-screen enter/exit produces a snapshot of the new viewport. The viewer's
stitcher detects the discontinuity (no alignment between old buffer and new
snapshot, similar sizes) and emits a `DROPPED` marker as a visible cue.
## Hub-side compaction
The ring is text-similarity compacted: two adjacent snapshots whose rows
have ≥ `COMPACT_TEXT_MATCH` (default 0.9) of text matching → the newer one
**overwrites** the older in place. Keeps the ring shallow when the source
streams spinner-frame churn.
Net effect: every distinct ring entry represents a meaningful viewport
change. Spinner cycling collapses to one entry. New content (added lines,
text changes) appends.
## Viewer-side stitching
`public/stitcher.js` merges incoming snapshots into a single buffer per
run.
- **Alignment**: for each substantive row of the new snapshot, find all
buffer positions where it matches, then score each candidate by how many
more substantive rows continue to align. Pick max-score (ties → highest
offset = "furthest match"). This handles repeating CLI templates (codex's
per-turn `Done. Rebuilt / AgentTerm / Release` shell) because the correct
offset is the one with the longest extended match, regardless of how
small a sub-window of rows is reused across turns.
- **No alignment + similar sizes** → REPLACE buffer with snapshot, prepend
a `DROPPED` sentinel as visible "previous viewport gone" indicator.
Catches alt-screen flips, picker close, modal dismiss.
- **No alignment + different sizes** → append `GAP` sentinel + snapshot.
Preserves history when the two snapshots are genuinely disjoint pieces
of content the viewer should both show.
- **Cap** at `DEFAULT_MAX_ROWS = 1000` rows. Bounded buffer growth.
## Source connection status
The agent-term stream client tracks its state and surfaces it to the
renderer for UI indication.
| State | Meaning |
|---|---|
| `idle` | no first prompt yet; nothing to send; indicator hidden |
| `disabled` | no hub URL configured; hollow grey dot |
| `connecting` | first prompt fired; register POST in flight |
| `connected` | recent successful POST (register / snapshot / heartbeat) |
| `retrying` | last POST failed; backoff retry in progress |
| `disconnected` | ≥ N consecutive POST failures; retry queue building |
Failure is never silent: persistent dot in the agent-term window, toast on
disconnect, toast on recovery.
### Retry queue
- In-memory ring of pending POSTs (default cap 50).
- On reconnect: drain oldest-first. On overflow during disconnect: drop oldest.
- v1 does not signal gaps to the hub or viewer.
### Async / non-blocking
Network I/O must never block agent-term's UI or PTY work. All hub POSTs are
fire-and-forget from the producer's perspective. A single worker loop owns
the network state. Producers (stream-state, heartbeat timer) never await a
`fetch`.
## URL + secret discovery
Source reads from `~/.agent-term/config.json`:
```json
{
"hubUrl": "https://your-hub.example.com",
"hubSecret": ""
}
```
- `hubUrl` empty → streaming `disabled` (hollow dot, no requests).
- `hubSecret` is unused by the source (all source POSTs are auth-open).
Set when running viewer-side curl debugging.
Viewer prompts for the hub secret on first auth-gated request and persists
it in `localStorage`. Optional seed links use `#s=<URL-encoded-secret>`
so the secret is not sent in the HTTP URL; query-string seeding is unsupported.
## Iterative roadmap — current state
Boxes checked = shipped.
- [x] **Hub + source minimal** — register, snapshot, heartbeat, latest.
- [x] **Viewer SPA** — list runs, click into detail, stitched buffer with
`GAP` / `DROPPED` markers.
- [x] **Adaptive polling** — 1.5s while live, 10s while idle, pause when
tab hidden.
- [x] **Hub-side compaction** — text-similarity collapse of spinner churn.
- [x] **Named tunnel** — stable public URL, no rotation.
- [x] **Daemon supervision** — hub + cloudflared survive a host
reboot without a graphical login.
- [x] **Auth secret** — gated reads + destructive writes.
- [x] **Run grouping** — viewer groups streams by `(host, title)`; per-card
delete; stale tiles dimmed; drill-in for past streams in a group.
- [x] **Viewer → source input channel** — `/runs/:id/input` queue, drained
on heartbeat, written into PTY as typed characters + CR.
- [x] **`/history` endpoint** — older snapshots paginated by `before`+`limit`.
- [x] **Voice input, hub side** — `/runs/:id/voice`: transcribe via resident
whisper-server, confidence-gate, queue as `voiceInputs`. Phone UX and
source-side injection still open — see [voice.md](voice.md).
- [ ] **Hub-side persistence** — survive daemon restart without losing rings.
Deferred until format and schema settle.
- [ ] **History UI in viewer** — frame-replay through `/history` results.
- [ ] **iPhone PWA polish** — manifest, apple-touch-* tags, swipe-friendly
wide-block rendering.
- [ ] **Clickable file/line patterns in viewer** — port from agent-term's
`src/renderer.js` for IDE handoff.
- [ ] **SSE** — `GET /runs/:id/stream`; viewer prefers SSE, falls back to
polling. Only worth doing if polling cost becomes noticeable.
## Locked decisions (don't relitigate without good reason)
- The iOS/iPadOS home-screen icon (`public/icon-180.png`) intentionally keeps
the original terminal artwork at 108×108 (60%), anchored top-left on an
opaque-white 180×180 canvas. The asymmetric field makes the black tile read
as a terminal pane opening into the larger viewer; its smaller prompt mark
and breathing room tested better than an optically enlarged 62% redraw. Do
not crop, upscale, center, or round the inner pane, and do not make it match
the deliberately full-bleed `public/icon.svg` browser/favicon variant.
- Snapshot-based protocol, no blocks, no markers/dividers. Viewer stitches.
- One agent-term process = one run = one `runId`. Resume = new run + new tile.
- Hub keeps last 24h of activity (`EVICT_MS`). User deletes stale tiles
manually via the viewer's per-card `✕`.
- Ring cap = 400 entries per run, post-compaction. Loss beyond that is
acceptable — viewer's accumulated buffer holds anything that passed
through during a viewing session.
- Schema-agnostic relay (Node, ~300 LOC), rarely changes.
- Hub + viewer live in this repo; the source (agent-term) has a one-line
pointer. Deployment specifics are kept separate from this public repo.
- Public URL comes from a named tunnel (stable); never hardcoded as a
constant in agent-term (source reads `~/.agent-term/config.json`).
- `cli` is run-level (in `RunMeta`), not snapshot-level.