docs/REALTIME.md
Rama main · réplica de lectura del repositorio original.
# Realtime architecture
LobbyForge pushes server-to-client state changes over **WebSocket**, on
top of a shared **Redis pub/sub bus**, so the Next.js app stays the only
process that writes to `game_sessions` / `messages`; the WS gateway is a
read-only fan-out. When the socket cannot connect, a panel falls back to
polling the REST read.
The M19 SSE route (`GET /api/servers/{id}/activities/{sessionId}/stream`)
was removed on 2026-10-13: nothing used it since the room page moved to
`RealtimeClient` (M20-bis), and it was the one activity route outside
`withApiSecurity` — it only checked the session cookie, with the first
revocation check 30 s in (security review ACT-001; ACTIVITY_INTEGRITY B-10).
This doc covers the architecture, the wire protocol, the authorization
rules, and the production layout. The SDK-level conventions for
plugins that want to push their own events live in
[`docs/PLUGIN_SDK.md`](./PLUGIN_SDK.md); the per-activity route surface
is in [`docs/ACTIVITIES.md`](./ACTIVITIES.md).
## Architecture overview
```
┌──────────────────────────────────────┐
│ Browser (Next.js client) │
│ RealtimeClient (apps/web/lib) │
└────────┬─────────────────┬────────────┘
│ WS │ REST poll (fallback)
│ │
┌──────────▼──────┐ ┌───────▼──────────────┐
│ apps/ws- │ │ Next.js │
│ gateway (19521) │ │ apps/web │
│ ws + ioredis │ │ GET …/{sessionId} │
└──────────┬──────┘ └───────┬──────────────┘
│ │
│ Redis SUBSCRIBE │ Redis PUBLISH
│ │
└────────┬─────────┘
│
┌───────▼─────────┐
│ Redis pub/sub │
│ lf:{env}:* │
└─────────────────┘
```
The Next.js app publishes to Redis on the action / end / message routes;
the ws-gateway subscribes to the topics and fans them out. A panel whose
socket cannot connect polls the session GET every 5 s instead.
## Topics
| Topic (wire) | Redis channel | Publisher | Consumer(s) |
|---|---|---|---|
| `activity-state:{serverId}:{sessionId}` | `lf:{env}:activity-state:{serverId}:{sessionId}` | activity `actions` + `end` routes (and the GET when it fires timers) | ws-gateway |
| `chat:{serverId}:{channelId}` | `lf:{env}:chat:{serverId}:{channelId}` | messages `POST` route | ws-gateway (chat UI is M20+) |
| `presence:{serverId}` | `lf:{env}:presence:{serverId}` | presence `POST` route | ws-gateway — a content-free "re-fetch" signal |
| `members:{serverId}` | `lf:{env}:members:{serverId}` | `lib/member-bus.ts` (joins, kick/leave, ban, timeout, roles, role edits, nicknames, profiles) | ws-gateway — `members-changed` / `members-removed { userIds }`, `roles-changed`; nothing else passes ([PROFILES.md](./PROFILES.md#live-updates)) |
| `dm:{channelId}` | `lf:{env}:dm:{channelId}` | DM messages `POST` route | ws-gateway — participants with DM access |
| `user:{userId}` | `lf:{env}:user-events:{userId}` | Bot API v2 (ephemeral interaction answers); `dm_activity` (DM list); `mention` (channel badges) | ws-gateway — only the session whose uid matches |
Bots do not use these topics: they connect on `/ws/bot` and get a feed
the gateway builds for them — see [`docs/BOT_API_V2.md`](./BOT_API_V2.md) §4.
Both topics are scoped to `(serverId, resourceId)`. A user can subscribe
only to topics whose `serverId` they have access to.
An `activity-state` event on the bus carries no state. Each consumer loads
the session and projects it for its own viewer: official plugins with
`projectActivityState` (`@lobbyforge/core`); marketplace plugins with their
own `projectState`, which runs in the plugin worker that only the web app
can reach — so the gateway asks web per viewer through the signed internal
`POST /api/internal/activity-projection` (`apps/ws-gateway/src/host-projection.ts`,
`LOBBYFORGE_INTERNAL_WEB_URL`, default `http://web:3000`). If web cannot
answer, the subscriber gets the event without state (ADR-007).
The projections for one socket and topic go out in bus order (each read
waits for the previous one), and each frame carries the `status` and
`revision` of the row it actually projected, not those of the event that
triggered the read. `useActivitySession` ignores a frame whose `revision`
is lower than the one it already shows (a frame without `revision` is
applied as before).
## Wire protocol (WebSocket)
Client → server (JSON text frames):
```json
{ "type": "subscribe", "topic": "activity-state:srv-1:abc" }
{ "type": "unsubscribe", "topic": "chat:srv-1:def" }
```
Server → client:
```json
{ "type": "hello", "ok": true, "uid": "...", "at": "..." }
{ "type": "subscribed", "topic": "...", "at": "..." }
{ "type": "unsubscribed","topic": "...", "at": "..." }
{ "type": "event", "topic": "...", "data": {...}, "at": "..." }
{ "type": "error", "topic": "...", "code": "forbidden", "message": "..." }
{ "type": "access_revoked", "topic": "...", "reason": "...", "at": "..." }
{ "type": "heartbeat", "at": "..." }
```
Errors carry one of:
- `bad_message` — payload didn't match the schema (invalid JSON, missing fields, etc.).
- `forbidden` — the caller isn't a member of the topic's server.
- `unknown_topic` — topic shape is unknown.
- `rate_limited` — subscribe rate limit or subscription cap exceeded.
- `internal_error` — the authorization check failed (a fixed message; the
detail is only logged).
`access_revoked` means the gateway removed a live subscription because the
caller's access changed (kick, ban, role loss, channel policy, block) or a
periodic re-check denied it. `heartbeat` goes out with every protocol ping
(30 s): a browser answers pings without showing them to the page, so this
frame is the page's proof that the socket is alive.
The gateway enforces 30 subscribe messages / 60s / connection. An
`unsubscribe` that arrives while the same topic's `subscribe` is still
being authorized cancels it.
## Authentication
The browser sends the `lf_guest` cookie in the `Cookie` header on the
HTTP upgrade request. The gateway validates it via
`@lobbyforge/core`'s `readGuestSession` and closes the socket with code
`4401` (a non-standard "unauthenticated" code picked for the gateway) on
failure. The cookie's HMAC must verify against `LOBBYFORGE_SESSION_SECRET`
and the payload's `uid` must be non-null (post-`POST /api/auth/guest`).
A revoked session closes with `1008`; in production, a revocation store
that cannot answer closes with `1011`. All three happen right after the
upgrade was accepted, so the browser sees `open` first.
## Authorization
Every `subscribe` is gated by `authorizeTopicSubscribe(...)`:
- The server's owner always passes (no DB lookup beyond `getServerById`).
- Everyone else must be a member of `serverId` (`isServerMember`).
- `chat` and `activity-state` topics also need the channel to exist in
that server and to be visible to the member (role-gated channels;
`administrator` / `manage_channels` skip the visibility check).
- A `chat` topic additionally needs **`read_message_history`**
(`administrator` implies it; `manage_channels` does not) — the same rule
as reading the channel over REST (`authorizeChannelMessageAccess`,
operation `read`). A role that revokes history used to keep the live
feed. Role changes re-run this check on live subscriptions (access
invalidation), so revoking the permission also ends the subscription.
A non-member's subscribe receives `{type: 'error', code: 'forbidden'}` —
the connection stays open so the client can retry with a different topic
or refresh their session.
## Redis subscriber pool
The gateway's `apps/ws-gateway/src/redis-subscriber.ts` keeps ONE shared
ioredis subscriber connection for the whole process, with a refcounted
handler set per topic:
1. First `acquire` for a topic sends `SUBSCRIBE` on the shared connection.
2. Subsequent `acquire`s on the same topic only add a handler to the
fan-out set.
3. `release` decrements the refcount; when it hits zero, the connection
sends `UNSUBSCRIBE` (the connection itself stays open).
ioredis reconnects and re-subscribes what Redis had accepted. A `SUBSCRIBE`
that Redis refused (typically one that waited in the offline queue past its
retries while Redis was down) is retried on the connection's next `ready`,
for the topics someone still holds.
This means a popular voice room with 100 spectators costs **one** Redis
subscription, not 100, and any number of topics costs one connection. The
web app's `apps/web/lib/activity-bus.ts` and `chat-bus.ts` use the same
pattern locally.
## Browser client (`apps/web/lib/realtime-client.ts`)
```ts
import { getRealtimeClient } from '@/lib/realtime-client';
const client = getRealtimeClient();
client.connect();
const unsubscribe = client.subscribe(
`activity-state:${serverId}:${sessionId}`,
(data) => {
// `data` is whatever the publisher JSON.stringified.
// For activity-state: { status, state, at }
// For chat: { id, channelId, userId, content, ... }
}
);
// Later:
unsubscribe();
```
`RealtimeClient` is a module-level singleton — one connection per page
load. It auto-reconnects with exponential backoff (cap 30s), queues
subscribes that arrive while disconnected, and replays them on the
next `open` and again on `hello`. The backoff starts over only on
`hello`: the gateway accepts every upgrade and refuses after `open`.
- `4401` (no or invalid cookie) and `1008` (revoked session) are final:
the client stops reconnecting and calls its `onTerminalClose(listener)`
listeners (`terminalClose` holds the code and reason). An explicit
`connect()` tries again; `wake()` and new subscribes do not.
- `1011` (revocation store unavailable) and every other close back off.
- A subscribe refused with `internal_error` or `rate_limited` is retried
while the topic still has handlers: 1 s, 2 s, 4 s … capped at 30 s, six
times at most (a `subscribed` starts the count over). After
`access_revoked` the client asks once more after 5 s; if access is really
gone the answer is `forbidden`, which ends it.
- Liveness: any frame re-arms a 60 s timer, and the gateway sends a
`heartbeat` frame every 30 s, so only a socket that stayed silent for two
heartbeats is closed (which triggers a reconnect).
- Catching up: the gateway replays nothing, so an event published before a
topic was live is gone — between the page render and the subscription, or
while the socket was down. `subscribe(topic, handler, { onLive })` calls
`onLive` when the gateway confirms the topic on a socket (once per socket,
however many `subscribed` answers arrive: the first time, and after every
reconnect or restored `access_revoked`), and on the next microtask for a
subscriber that joins a topic already live. Anything that shows state
re-reads it there; the lobby's member rows, DM list, open channel and DM,
and the activity panel all do. `onReconnect` remains for listeners that
are not tied to one topic.
The default URL is `ws://{window.location.hostname}:19521` (HTTPS pages use the same-origin `/ws` proxy instead). Override
via `NEXT_PUBLIC_WS_URL` (the env var the build reads).
## Transport selection
The room page and the lobby use `RealtimeClient` exclusively; when the
socket cannot connect (a reverse proxy that strips WS upgrade headers),
they poll the REST read. The SSE route that used to sit beside it was
never used by any client after M20-bis and is gone (see the top of this
page). Chat, presence and activity updates all ride the WS path.
## Production layout
- **Dev**: `pnpm -F @lobbyforge/ws-gateway dev` runs the gateway on
`:19521` with `node --watch`. The Next.js dev server runs on `:19520`.
Both talk to the same Redis instance via `REDIS_URL`.
- **Prod**: The gateway runs as a separate container/pod behind a load
balancer. Reverse-proxy rules forward WS upgrades (the `Upgrade:
websocket` + `Connection: Upgrade` headers) to the gateway. The
Next.js app is unchanged.
Required env vars on the gateway container:
- `LOBBYFORGE_SESSION_SECRET` (32+ chars)
- `REDIS_URL` — required in production; development falls back to the local Docker default only when omitted
- `LF_DB_URL` — Postgres URL for the membership checks
- `WS_ALLOWED_ORIGINS` — optional comma-separated allowlist for browser WebSocket origins. `LOBBYFORGE_APP_ORIGIN` and `NEXT_PUBLIC_BASE_URL` are also accepted when set. Production rejects missing or unlisted origins.
- `WS_HOST` (default `127.0.0.1`)
- `WS_PORT` (default `19521`)
## Future
- **WebSocket auth refresh.** Today the cookie is validated once at
upgrade; a long-lived connection whose cookie expires mid-session
keeps streaming until the next subscribe attempt fails. M21+ should
add a `refresh` message the client can send with a fresh cookie.
- **Server-side broadcast.** Plugins that want to push events outside
the activity-state namespace (e.g. leaderboard updates, world events)
should declare the topic via `manifest.realtime?.topics` and let the
host mediate publication. M20+ design work.
- **Server-initiated pings.** The gateway pings every 30s and sends a
`heartbeat` frame with each ping; the client reconnects after two
missed heartbeats. No change needed unless we add a richer keep-alive
protocol.