docs/LESSONS.md

Rama main · réplica de lectura del repositorio original.

# Lessons

Operational rules paid for by an incident, a benchmark, or a launch. Each one
is here because it is not derivable from the code and would otherwise have to
be relearned the expensive way.

Decisions live in `docs/design/DECISIONS.md`; open work lives in
`docs/ROADMAP.md`; conventions live in `AGENTS.md`. This file is only the
things that bit.

---

## Deploying and cutting over

**A cutover freezes the pipeline first, and the queue must be verified empty,
not merely quiet.** `run_pipeline` defaults `True` and is *not* tied to
read-only mode — read-only unregisters the write **tools**, not the **runner**
— and `VIDTHEQUE_STALE_CLAIM_S` makes a job the old process had claimed
re-claimable minutes later. So a database migrated with live queue rows gives
you the *public* box fetching from YouTube behind the tunnel, from a fresh IP
with no bot-check history. Check both tables:

```sql
SELECT state, COUNT(*) FROM jobs GROUP BY 1;      -- nothing running, nothing queued
SELECT state, COUNT(*) FROM job_items GROUP BY 1;
SELECT index_state, COUNT(*) FROM videos GROUP BY 1;  -- no indexing, no pending
```

At the last cutover this was 1 job running, 4 items queued, 1 item running.
Not hypothetical.

**Cancel terminally. Do not park.** Pushing `not_before` into the future looks
like a pause and is not one: the row still reads `queued`, so it fails the
emptiness check above, and it leaves a live timer pointed at the box you are
about to expose.

**Why it matters beyond tidiness:** the worker has one FIFO GPU queue with no
priority. A whisperX stage in flight makes every visitor's search wait out its
timeout and then return FTS-only — not an error, just a worse answer, which is
the worst failure mode available because it looks like the product being bad.

**Stopping the connector is the rollback, and it is seconds.** No DNS TTL to
wait out. But it does **not** un-publish the keyframes: `/frames/*.jpg` is
`Cache-Control: public, max-age=86400` and `.jpg` is edge-cached by default, so
copies outlive the origin for up to a day. If the rollback is about *content*
rather than an outage, purge the cache too — and accept that already-downloaded
copies are gone for good, as they are for anything ever published.

**Rotate on exposure.** A leaked tunnel token or credentials JSON: rotate,
restart the connector, then force-disconnect existing connections through the
API. A leaked `OPENROUTER_API_KEY`: revoke at OpenRouter **first**, change the
config second — the running process holds it in memory until restart.

**Keep the rollback card next to whichever box currently serves writes.** That
box has changed twice since launch. A rollback procedure written into a dated
launch runbook describes a topology that no longer exists, and the second lever
on the last one pointed at a container that had already been retired the same
day it shipped.

## Running the box

**A persistent `E_RATE_LIMIT` 403 loop on media downloads is a broken yt-dlp
build, not throttling.** vidtheque classifies YouTube's 403 as `E_RATE_LIMIT`,
which is right for the transient block it usually is — so a build YouTube has
broken looks like an endless rate limit: the identical 403 on every retry,
90-minute cool-offs, an attempt ceiling hours away. **Persistence across
cool-offs is the tell.** Test the current nightly standalone binary on the box
before touching anything else. The pin moves as a deliberate commit, separate
from the version bump.

**The real YouTube block is 60–90 minutes against the IP or guest session**, so
a 300 s backoff with 3 attempts burns the entire retry budget in ~12 minutes
and is still blocked at the end of it. This is why the backoff constant is
5400 s.

**The rate limiter's trust boundary is Cloudflare.** It keys off
`CF-Connecting-IP` in preference to the socket address, which is safe only with
an edge in front. Any deployment without one must clear
`VIDTHEQUE_TRUSTED_IP_HEADER`, or every per-IP limit is spoofable with a header.

**A feature that shipped is not a feature that is on.** Following shipped in
0.0.5 with checks defaulting to enabled, and the corpus still stopped growing
— because no channel was ever followed. Before diagnosing the pipeline, check
whether the thing has any rows: `SELECT COUNT(*) FROM follows`.

## Writing things down

**Open work does not live in a dated document.** A "Backlog" section inside a
handoff is a snapshot of one night. Two of the five items in the 2026-08-09
backlog had shipped by the evening of the day it was written, and reading it a
fortnight later cost a day of planning work already done. Open work belongs in
`docs/ROADMAP.md`; a dated record may describe what happened, never what is
left.

**A measurement is worth more than the prose around it.** Nearly every file in
`research/` survives an audit not because it reads well but because something
cites it — 25 source files name a research document, and the GPU and decode
benchmarks cannot be reproduced without the same hardware. Delete narration
freely; never delete a number you cannot re-measure.

**Cite the file, not the memory.** The design docs' habit of naming the
research file and section beside a constant is what makes the constants
auditable years later. `DEFAULT_RATE_LIMIT_BACKOFF_S = 5400` means nothing
alone; with `research/ytdlp-usage-audit-2026-08-10.md` beside it, it is a
finding.

## The web front end

**A header `proxy.ts` sets on a rendered document does not reach the wire.**
The dynamic render overwrites it: the proxy set `Cache-Control: no-store` on
every `/dashboard` page for the whole port, and `curl -D-` showed Next's own
`no-cache, must-revalidate` throughout. Response headers that must stick go in
`next.config.ts` `headers()`, and a test asserts the value, not the comment.

**A development rewrite in `afterFiles` shadows dynamic routes.** `afterFiles`
runs after static pages but before dynamic segments, so a `/dashboard/:path*`
forward sent every detail page (`/dashboard/videos/{id}` and its siblings) to
Python's 404 in development. Forwards go in `beforeFiles`, named exactly.

**`scrollIntoView({ block: "nearest" })` also scrolls sideways.** `inline`
defaults to `"nearest"` as well, so pointing between two OCR boxes slid the
whole line list under the reader (2026-08-10). Scroll the scroller itself, on
one axis.

**`columns: 2` on a box with a bounded height overflows sideways.** The frame
overlay's line list slid under the pointer (2026-08-10). A grid overflows
downward.

**React's `autoFocus` fires on mount only.** A refused sign-in kept the wrong
secret on screen with focus on the button; clear and refocus the field by hand.

**JSX drops whitespace that holds a newline.** Facts glued with a separator
and no explicit `{" "}` have no break opportunity: the video header ran 56 px
off a 390 px screen (2026-09-05). Every strip writes the space out.

**A test that spells out a live countdown after an `await` is a flake.** The
timer can tick between the render and the assertion; six page tests failed
that way under load. Stop the clock before mounting (`test/retry.ts`).

**An effect cleanup that restores `document.title` runs after the next page
named itself.** Leaving an "Unknown video" page for Jobs left the tab titled
"Video". A rendered `<title>` has no cleanup to get wrong.

**A table keyed by its query, fed the previous query's payload, freezes the
wrong baseline.** The jobs poll kept its last reading across a filter change,
so the re-keyed table patched the new listing onto the old rows and claimed a
job had arrived (2026-09-16). A read's data belongs to its request key and to
nothing else.

**A router navigation per search is a scroll to the top.** The public console
pushed `/paris?q=…` through the App Router, so every search, chip and mode
switch re-rendered the page on the server and moved `scrollY` from 1766 to 0
(2026-09-16). Measure `_rsc` requests and `scrollY` around each interaction;
do not infer them from the code.

**Measure router traffic on `next start`, not `next dev`.** Hot reloads
refresh the router, so the dev server showed RSC refetches and scroll jumps
after plain `history.pushState(null, …)` writes. That led to forging Next's
private history state. On a production build the documented writes measured
zero `_rsc` requests and no scroll change (2026-09-16).

**`/api/meta` shares the search bucket, so its 429 is a JSON error body, not
a parse failure.** A visitor who spent the bucket and reloaded got `undefined`
in the MCP line and the paste line, and a hidden ask switch, with nothing
saying why (2026-08-28). Any non-2xx is a state the page names.

**`/tmp` is a 4 GB tmpfs shared by every agent on the box.** When it fills,
headless Chromium and tool output fail with `ENOSPC`, not with a hint. Point
`TMPDIR` at `/var/tmp` or a gitignored cache for test and browser runs, and
delete what you create.

## Design and assets

**Vendored fonts stay under a 200 KB total budget**, OFL-licensed and
latin-subset, with no CDN and no runtime network request. The rule predates
and outlives the specific faces it was written against; the current
Archivo + JetBrains Mono pair spends 75 KB of it.