docs/services.md
Rama main · réplica de lectura del repositorio original.
```
╔════════════════════════════════════════════════════════════════════════════╗
║ ║
║ BACKEND SERVICES ║
║ Long-lived runtime modules: collab, notifications, email, DB pool. ║
║ ║
╚════════════════════════════════════════════════════════════════════════════╝
```
# Backend Services
Cloud Codex runs in a single Node process. Five services live alongside the
Express routers and own the long-lived runtime: real-time collab,
notifications, the user-scoped WebSocket channel, email delivery, and email
templates. Plus the MySQL pool and the auth/permission middleware that every
route depends on.
```
request path background path
│ │
▼ ▼
┌────────────────┐ ┌────────────────────┐
│ Express routes │ │ services/collab.js │ WS /collab/:logId
│ (HTTP / JSON) │ │ Yjs CRDT sync │ ──► clients
└───────┬────────┘ └─────────┬──────────┘
│ │ debounced ydoc_state save
│ ▼
│ ┌────────────────────┐
│ │ mysql_connect.js │
│ │ c2_query() + pool │
│ └────────────────────┘
│ ▲
▼ │
┌────────────────────────┐ │
│ services/ │ │
│ notifications.js ├───────┘ persist + read prefs
│ user-channel.js (WS) │
│ email.js (SMTP) │ ──► recipient inbox / browser tabs / email
│ email-templates.js │
└────────────────────────┘
```
---
## Collaborative Editing Service
**File:** `cloudcodex/services/collab.js`
Powers real-time multi-user document editing. Runs a WebSocket server attached
to the same HTTP server instance as the Express API (no separate port).
### Technology
- **[Yjs](https://yjs.dev/)** — CRDT (Conflict-free Replicated Data Type)
library. Handles concurrent edits and automatic merge resolution. Think of
it as a shared data structure that multiple clients can modify
independently and then sync without conflicts.
- **[y-protocols](https://github.com/yjs/y-protocols)** — Binary sync protocol
over WebSocket for efficient Yjs state transfer.
- **[Tiptap](https://tiptap.dev/)** (client-side) — ProseMirror-based rich
text editor configured with the `@tiptap/extension-collaboration`
extension. The frontend Tiptap instance maintains the Yjs `Y.Doc` locally
and syncs it with the server.
### Connection Lifecycle
```
client server MySQL
────── ────── ─────
open WS ──────────────────────►
/collab/:logId?token=…
validate session token
check archive read access
load or create Y.Doc ydoc_state
◄────────
sync protocol handshake
◄──── send server state
────► send client state
awareness broadcast (presence)
live edits ─────► broadcast to all other clients (binary update)
│
▼ debounce 3 s of inactivity
write ydoc_state ─────────────► (UPDATE)
client closes ───────────────────► awareness broadcast (gone)
│
│ if last user, 30 s cleanup
▼ destroy Y.Doc from memory
```
1. A client opens a WebSocket to `/collab/:logId?token=<sessionToken>`.
2. The server validates the session token and checks that the user has
**read access** to the archive containing that log.
3. Write permission is determined and attached to the connection. Read-only
clients receive sync updates but their edits are rejected server-side.
4. An in-memory `Y.Doc` is loaded for the `logId` (or created if it doesn't
exist yet). Initial CRDT state is loaded from `logs.ydoc_state` in MySQL.
5. The Yjs sync protocol runs: the server sends its current state to the new
client, and the client sends its state to the server. Yjs merges both
sides automatically.
6. A **presence** (awareness) message is broadcast to all other connected
users showing the new user has joined (name, avatar, a randomly assigned
color for cursor rendering).
### Live Editing
- Every mutation a client makes to the `Y.Doc` is broadcast to all other
connected clients (except the sender) as a binary Yjs update message.
- The server also listens to `Y.Doc` mutations and schedules a **debounced
save** (3 seconds of inactivity) that writes the binary CRDT state back
to `logs.ydoc_state`. This ensures the document can be restored after a
server restart without losing recent edits.
- **HTML content** is *not* written to the database by the WebSocket service.
Only `ydoc_state` is saved via the collab service. `html_content` is
written by the REST `POST /api/save-document` endpoint (triggered by the
frontend's autosave) and on publish.
### Connection Cleanup
- When a user disconnects, an awareness broadcast notifies remaining users.
- If all users leave a document, a 30-second cleanup timer fires that
destroys the `Y.Doc` from memory (after flushing any pending saves).
### Limits & Safety
| Limit | Value |
|--------------------------------|------------------------------------|
| Max WebSocket message size | 5 MB |
| Max HTML content (on save) | 2 MB |
| Debounce before DB save | 3 seconds |
| Cleanup delay after last user | 30 seconds |
| Max simultaneous WS per user | 10 connections |
| Rate limit (messages/second) | 60 messages per 1 second window |
### Presence API
The collab service exposes two functions used by other parts of the server:
- **`getAllPresence()`** — Returns a map of `logId → [{ id, name,
avatar_url, color }]` for all actively connected documents. Used by the
Admin API's telemetry endpoint and by the search/browse APIs to annotate
results with live editing indicators.
- **`getActiveDocCount()`** — Returns the number of documents currently held
in memory.
---
## Notification Service
**File:** `cloudcodex/services/notifications.js`
The single funnel for every user-facing alert: mentions, comments on watched
docs, watched-doc edits/publishes, squad invitations. Owns persistence, push
delivery, optional email, and the rules that prevent floods.
### How a notification fans out
```
source event (mention, comment, publish, …)
│
▼
createNotification({recipientId, actorId, type, …})
│
├─► self-suppression (actor === recipient → drop)
│
├─► coalesce window (same recipient+type+resource within 60 s
│ already exists → drop)
│
├─► sanitize title/body (sanitizeHtml + strip tags + truncate)
│
├─► INSERT notifications ─────────────────► MySQL row
│
├─► broadcastToUser(WS) ─────────────────► open browser tabs
│ (services/user-channel.js)
│
└─► deliverEmail() (look up prefs; default ON for most types
except watched_log_update)
│
▼
buildNotificationEmail(type, data)
│ (services/email-templates.js)
▼
sendEmail({to, subject, text, html})
│ (services/email.js)
▼
recipient inbox
```
### Defaults
A user with `users.notification_prefs IS NULL` receives email for these
types out of the box; a user with a JSON prefs object overrides per-key:
| Type | Default email |
|----------------------------|---------------|
| `mention` | on |
| `comment_on_my_doc` | on |
| `watched_comment` | on |
| `watched_publish` | on |
| `watched_log_update` | **off** |
| `squad_invite` | on |
In-app inbox delivery is always on — preferences only control email.
### Public surface
| Function | Purpose |
|----------------------------------------------|--------------------------------------|
| `createNotification(args)` | The fan-out entrypoint |
| `markRead(id, userId)` | Idempotent; pushes `{type:'read'}` |
| `markAllRead(userId)` | Pushes `{type:'read_all'}` |
| `getUnreadCount(userId)` | Powers the bell badge |
| `listForUser(userId, {limit,before,unreadOnly})` | Cursor-paginated inbox |
| `getPrefs(userId)` / `setPrefs(userId,p)` | Merge with defaults; whitelisted keys|
See [notifications.md](./notifications.md) for the conceptual deep-dive and
[api/notifications.md](./api/notifications.md) for the HTTP surface.
---
## User-Channel WebSocket Service
**File:** `cloudcodex/services/user-channel.js`
Push-only WebSocket distinct from the doc-keyed `/collab` channel. A user
holds **one connection here** regardless of which document they have open;
the notification service uses this channel to deliver inbox updates to all
of that user's open tabs at once.
### Protocol
```
client server
────── ──────
open WS /notifications-ws ──────────────►
start 5 s auth timer
{type:'auth', token:'<sessionToken>'} ───►
validateAndAutoLogin(token)
cap: 10 connections per user
◄────────────── {type:'connected', userId}
◄────────── {type:'notification', notification}
◄────────── {type:'unread_count', count}
◄────────── {type:'read', id}
◄────────── {type:'read_all'}
```
### Hardening
- **Same-origin only** (CSWSH protection): the `Origin` header host must
match the request `Host`; otherwise the upgrade is refused with `403`.
- **Auth timeout**: an unauthenticated client is closed after 5 s.
- **Connection cap**: 10 concurrent WebSockets per user.
- **No client→server messages after auth** — this is a push-only channel.
### Helpers exported
| Function | Use |
|--------------------------------|--------------------------------------|
| `setupUserChannelServer(http)` | Attach to the existing HTTP server |
| `broadcastToUser(userId, msg)` | Send JSON to all of a user's tabs |
| `getConnectedUserCount()` | Diagnostics |
| `isUserConnected(userId)` | Diagnostics |
---
## Email Templates
**File:** `cloudcodex/services/email-templates.js`
Builds the `{ subject, text, html }` payload for each notification type the
Notification Service knows how to email. Renders with a small shared shell
(heading, intro, optional snippet quote, CTA button, footer with manage-
notifications link).
Templates currently registered:
| Type | Subject pattern |
|-----------------------|-------------------------------------------------------|
| `mention` | `[Cloud Codex] {actor} mentioned you in “{doc}”` |
| `comment_on_my_doc` | `[Cloud Codex] {actor} commented on “{doc}”` |
| `watched_log_update` | `[Cloud Codex] {actor} edited “{doc}”` |
| `watched_publish` | `[Cloud Codex] {actor} published “{doc}”` |
| `watched_comment` | `[Cloud Codex] {actor} commented on “{doc}”` |
`buildNotificationEmail(type, data)` returns `null` for unknown types so the
notification service skips sending an email for those (in-app inbox still
fires).
All user-supplied values are HTML-escaped before they reach the template
output.
---
## Email Service
**File:** `cloudcodex/services/email.js`
A thin wrapper around [Nodemailer](https://nodemailer.com) for sending
transactional emails. Configured entirely via environment variables, and
entirely optional: Cloud Codex runs without SMTP configured at all.
**Optional:** `SMTP_HOST`, `SMTP_USER`, `SMTP_PASS` (all three together, or
mail stays disabled), `SMTP_PORT` (default 587), `SMTP_FROM`
A TLS connection is automatically used when `SMTP_PORT` is 465.
### Mail capability
Whether mail is usable is decided once, at boot, not per-send:
- **`isMailConfigured()`** — true when `SMTP_HOST`, `SMTP_USER` and
`SMTP_PASS` are all present.
- **`initMail()`**, called once from `server.js` as a top-level `await`
that resolves *before* the HTTP port opens, so the app never serves
traffic while the answer is undecided. Configured and unconfigured both
resolve (never rejects);
configured also live-verifies the connection. Returns
`{enabled, configured, reason}` and never exits the process. `server.js`
logs the result: `✔ SMTP connection verified` when enabled; when SMTP is
left unset (`configured: false`, a supported way to run), the report line
`✔ Email off (SMTP_HOST, SMTP_USER or SMTP_PASS not set): invitations show a
copyable link; password reset, email two-factor codes and notification
emails are unavailable.`; and when SMTP is set but does not verify,
`✖ Email disabled: SMTP connection failed. Invites will show copyable links;
password reset is unavailable.` All three go to stderr except the first.
- **`isMailEnabled()`** — the cached result of the last `initMail()` call;
what `sendEmail()` and the routes below check. Nothing re-runs
`initMail()`, so fixing SMTP config or reviving a dead host needs a
restart.
The transport sets `connectionTimeout` and `greetingTimeout` to 10s and
`socketTimeout` to 20s. Nodemailer's defaults (2 minutes, 30 seconds, 10
minutes) would let one blackholing host stall the whole boot, since
`initMail()` now runs before `listen()`.
### `sendEmail({ to, subject, text?, html?, from? })`
When mail is disabled, resolves `{skipped: true, reason: 'mail disabled'}`
without touching the network — every caller below is fire-and-forget-safe
with no changes needed. When enabled, sends a single email. Security notes:
- `to`, `subject`, and `from` headers are validated to reject strings
containing newline characters, which would allow
[header injection attacks](https://owasp.org/www-community/attacks/Mail_header_injection_attack).
- The `Precedence: bulk` header is set on all outgoing mail to reduce
auto-responder loops.
### Where it's used
```
trigger caller
────────────────────── ──────────────────────────────────────
user invitation routes/admin.js
password reset routes/auth.js
email 2FA code routes/auth.js
squad invitation routes/squads.js
mention services/notifications.js (mention)
comment on my doc services/notifications.js (comment_on_my_doc)
comment on watched doc services/notifications.js (watched_comment)
watched doc edited services/notifications.js (watched_log_update)
watched doc published services/notifications.js (watched_publish)
```
Three of these have an explicit degraded path rather than just a silent
no-op when mail is off: user invitation always returns its signup link in
the response for the admin UI to show copyably; password reset refuses with
`{success: false, message}` before minting a token instead of sending
nothing; email 2FA is refused 400 at `/api/2fa/enable` (TOTP setup returns
its QR code and secret inline instead). Squad invitation still sends its
in-app notification either way; only the email itself is skipped.
Two more refuse with `503` rather than degrade, because there is no honest
degraded form: `POST /api/login` for an account already using email 2FA, and
`POST /api/2fa/disable` for *any* 2FA method (its confirmation code is
emailed either way). Both return before minting a token or writing a
`two_factor_codes` row. An admin clears the resulting lockout with
`POST /api/admin/users/:id/2fa/reset` (see `docs/api/admin.md`), which works
regardless of mail state.
### `verifyEmailConnection()`
Attempts a connection to the configured SMTP server and returns
`true`/`false`. Called once by `initMail()` above, not directly by
`server.js`; nothing calls it per-send.
---
## Database Connection Module
**File:** `cloudcodex/mysql_connect.js`
Manages the MySQL connection pool and the session token lifecycle.
### Connection Pool
Uses `mysql2/promise` with:
- Pool size: 10 connections
- Infinite queue (`queueLimit: 0`)
- Waits for a connection to become available rather than failing immediately
The pool is created at module load time. `DB_USER` and `DB_PASS` env vars
are required; the process exits if they're missing.
### Key Functions
#### `c2_query(sql, params)`
A simple wrapper around `pool.execute()`. Always uses parameterized queries
— SQL strings are never constructed by string concatenation with user input
anywhere in the codebase. Each call may be handed a different pooled
connection, so it cannot be used for a multi-statement transaction.
#### `withTransaction(fn)`
Runs `fn` against one dedicated pooled connection wrapped in a transaction.
`fn` receives a query executor with the same `(sql, params) => Promise<Array>`
shape as `c2_query`, bound to that connection, so every call `fn` makes
through it participates in the same transaction. Commits and returns `fn`'s
result on success; rolls back and rethrows on any failure. A rollback that
itself fails is logged (`transaction rollback failed`) and swallowed so the
**original** error still reaches the caller, the one that names the
statement that actually broke. The connection is always released back to the
pool. Used by `bootstrapInstance()` in `routes/admin.js` so the first-boot
workspace/squad/archive/document seed either lands completely or not at all.
#### `generateSessionToken(user, ip, userAgent, { provider })`
Mints a new session: one row per sign-in, so each device has its own and
signing out of one leaves the others alone. Returns the raw token and stores
only its digest, `hashSessionToken(token)` from `services/session-token.js`
(SHA-256, lowercase hex). `provider` is `'local'` (the default) or `'google'`
and is recorded in `sessions.auth_provider`. Every row expires 7 days after it
is minted.
Tokens are 64-character random alphanumeric strings generated using
`crypto.getRandomValues`.
#### `validateAndAutoLogin(sessionToken)`
Looks up a session by the digest of its token, checks expiry, and returns the associated `user`
object (`id`, `name`, `email`, `avatar_url`, `is_admin`) or `null`. Called
on every authenticated request by the `requireAuth` middleware.
#### `getSessionProvider(sessionToken)`
Returns the session's `auth_provider` (`'local'` or `'google'`), looked up by
the digest of its token, or `'local'` when the session is gone or the value
cannot be a token. update-account and confirm-email read it before they delete
the caller's session, so the replacement they mint keeps its tag.
#### `touchSession(sessionToken)`
Updates `last_active_at` on the session row. Called asynchronously on every
authenticated request (fire-and-forget — errors are swallowed so they don't
affect the response).
---
## Middleware
**File:** `cloudcodex/middleware/auth.js`
### `requireAuth`
Reads the session token from the `Authorization: Bearer <token>` header, or
falls back to the session cookie (for OAuth browser redirects):
`__Host-sessionToken`, or a lone legacy `sessionToken` while
`LEGACY_SESSION_COOKIE` allows it. Calls
`validateAndAutoLogin` and attaches the user object to `req.user`. Returns
`401` if the token is missing or invalid.
### `requireAdmin`
Must be used after `requireAuth`. Returns `403` if `req.user.is_admin` is
not true.
## Route Helpers
**File:** `cloudcodex/routes/helpers/shared.js`
Shared utilities used across all route files:
- **`sanitizeHtml(html)`** — Runs DOMPurify server-side to strip XSS
vectors. Allows `data:` URIs on `img` tags (for pasted images), denies
all other dangerous patterns.
- **`asyncHandler(fn)`** — Wraps an async route handler so that rejected
promises are forwarded to Express's `next(err)` error pipeline instead
of hanging.
- **`isValidId(id)`** — Confirms a value is a positive integer (guards
against injection via ID params).
- **`canPublish(squadId, archiveCreatorId, user)`** — Async helper that
checks the full publish permission chain.
- **`checkLogReadAccess(logId, user)`** /
**`checkLogWriteAccess(logId, user)`** /
**`checkArchiveReadAccess(archiveId, user)`** — Convenience wrappers
for the archive-level access SQL checks, used in comment, version, and
watch routes.
**File:** `cloudcodex/routes/helpers/ownership.js`
- **`readAccessWhere(alias)`** / **`writeAccessWhere(alias)`** — Returns a
SQL `WHERE` fragment for access checking. Takes 7 positional parameters
via `readAccessParams(user)` / `writeAccessParams(user)`.
- **`isArchiveOwner(user, archiveId)`** — Async check for management-level
ownership.
**File:** `cloudcodex/routes/helpers/images.js`
- **`extractImagesFromHtml(html, saved?)`**: Scans HTML for base64-encoded `<img>`
tags, writes them to disk under `public/doc-images/`, and replaces the
`src` with a served URL. This prevents large binary blobs from being
stored in MySQL and speeds up document loads. The optional `saved` set
receives the hash of each image it decoded.
- **`recordDocImages(logId, html, writer, { saved, introduced })`**: Records in
`doc_images` the images a write stores, so the document's readers can see
them: as the writer's own when `saved` holds the hash, a reference only when
`introduced` holds it (this write added it) and the writer can already see
the image, and nothing else. `introducedDocImages(logId, html, previousHtml,
writer)` computes `introduced` for a save (limited, while a live session is
open, to what that session credits to the writer), and
`recordCreditedDocImages(logId, html, credits)` records the live editor's
saves per credited writer (`DocImageCredits`).
- **`readableDocImageHashes(hashes, user)`**: The one answer to "may this user
see these images": uploader, or reader of a document holding them. The
`/doc-images` handler (`routes/doc-images-serve.js`) and export ask it.
- **`inlineImagesForExport(html, user)`** /
**`inlineImagesForMarkdownExport(markdown, user)`**: Re-embeds served images back
as base64 for self-contained DOCX/Markdown exports, only the ones `user` may see.