AGENTS.md
Rama main · réplica de lectura del repositorio original.
# STT — Agent Instructions
## Project overview
Real-time speech transcription platform powered by Faster-Whisper, with a web UI for live event operators (church services, lectures, meetings). Live microphone transcription streams over WebSocket; also supports batch file transcription, real-time translation (Facebook NLLB-200), text-to-speech (Edge-TTS cloud / Piper-TTS local), and PANNs-based music/speech detection.
Stack: Python 3.9+, Flask + Flask-SocketIO, SQLite (per-session databases), Jinja2 server-rendered templates. No frontend build step — `static/` contains vendored jQuery, socket.io, and Font Awesome.
## Commands
The project venv (`.venv/`, Python 3.9) is uv-managed and has no pip — use `uv pip install` or the venv's python directly.
```bash
.venv/bin/python3 speech_to_text.py # Run the server (port 80) — ./start_server.sh / .bat wrappers exist
STT_DEMO=1 .venv/bin/python3 speech_to_text.py # Run as the demo (see Demo build)
.venv/bin/python3 -m pytest # Run tests (testpaths = tests/, config in pyproject.toml)
.venv/bin/python3 -m ruff check . # Lint (line-length 200, target py39)
./install.sh # Install runtime deps (install.bat / install.ps1 on Windows)
uv pip install -r requirements-dev.txt # Dev/test deps (pytest, ruff)
```
Ruff selects the hard-error rules (E9, F63, F7, F82, F811) plus `B` and `RUF`, and excludes `.venv`, `_AUTOMATIC_BACKUP`, `models`, `installer`.
### Demo build
A self-contained demo ships as one executable per OS (~29MB, ~15MB zipped) that runs
with no Python, no models and no ML libraries. It replays a recorded service into the
real UI, so Start/Stop and every page behave exactly as in production.
```bash
python scripts/make_demo_session.py --synthetic -o build/demo.db # a written service
python packaging/build.py --demo --synthetic # build the artifact
STT_DEMO=1 STT_DEMO_DB=<session.db> .venv/bin/python3 speech_to_text.py # run from source
```
`stt/demo_playback.py` replaces the transcription worker: it opens a real session
database and copies the recording's rows into it at their original pace, so the whole
read path (`get_new_entries`, the phase detector, corrections, the file manager) works
unmodified. `stt/demo_api.py` answers the model/TTS/translation routes from
`stt/demo_fixtures.py` — recorded from a real server by `scripts/record_demo_fixtures.py`
and scrubbed through `stt/demo_redact.py`. Those families are **deny-by-default**, so a
route added under `/api/models/` later fails closed rather than reaching an ML import.
The demo reports two things and nothing else: that it ran (a live-map ping carrying
`src=demo`) and that it crashed (a Sentry event tagged `demo`). Both are tagged at the
source so the collector counts a trial as a trial — `stt/worker/index.ts` in the website
repo keeps `demo` out of every install figure, and the anonymous id is carried across
the per-launch data wipe so one person opening the demo twice is not two trials.
Everything else it could send is shut off. The demo binds the network with no password,
so it must not be usable as somebody else's network client. `stt/demo_guard.py` holds
`CHOKE_POINTS` — the few monolith
functions every outbound request passes through — and `audit_choke_points` parses
`speech_to_text.py` to assert each still opens with an `if DEMO:` guard, the way
`stt/peer_auth_audit.py` enforces its own one-door rule. **A new way to reach the
network must be added to `CHOKE_POINTS`.** Routes whose risk is `subprocess` rather than
egress (`/api/tunnel/`, `/api/file-mover/`, `/api/remote-translation/`) are refused by
`demo_api.BLOCKED_PREFIXES` instead, since no network guard would catch them.
Demo mode is `STT_DEMO=1`, set in the shipped artifact by `packaging/demo_rthook.py`. It
redirects all data to `~/.stt-demo` (wiped each launch), so a demo can never write into a
real install. The build refuses to run without `STT_DEMO_DB`: a session database is
congregation speech until someone decides otherwise. Prefer `stt/demo_synth.py` for
anything published; `stt/demo_scrub.py` exists for a real recording, reduces risk, and
does not certify — read the `.review.txt` it writes.
`demo.spec` still refuses to *bundle* a recording nobody named, but nothing else does:
`demo_mode.ensure_session` writes a synthetic service into `~/.stt-demo/sessions` when
discovery finds nothing, and `build.py --demo` on its own generates one. The rule being
protected is "never ship somebody's speech by accident", and a written service is nobody's
speech — erroring on the safe case only taught people to append `--synthetic`.
`stt/demo_window.py` is the demo's only lifecycle control. The artifact is windowed on
Windows and macOS (`console=not (IS_WINDOWS or IS_MACOS)`), ships no watchdog, and
`demo_api` refuses `/api/restart` and `/api/server/restart` — so **without the window the
only way to stop a demo is the task manager**. That is why `tkinter` is deliberately *not*
in `demo.spec`'s excludes and a test asserts so. Tk owns the main thread on macOS, so the
window is the demo's main loop; a machine with no display keeps the old join loop —
**probed in a forked child**, because Tk failing to reach a window server aborts the
process rather than raising, and the cheap withdrawn-root probe answered yes right
before the demo took SIGSEGV. `STT_DEMO_WINDOW=0` settles it without probing. Changing
the port re-execs the process (`relaunch_command`) because `demo_mode.write_config` bakes
the port into config before the server reads it — `STT_DEMO_PORT` / `--port` are the same
lever from outside.
## Architecture
The server is mostly a monolith — most changes land in `speech_to_text.py`.
| Path | Role |
|------|------|
| `speech_to_text.py` (~15,700 lines) | The entire server: Flask routes, SocketIO events, transcription pipeline, translation, TTS, SQLite storage, settings |
| `stt/audio_capture.py` | Microphone capture layer |
| `stt/watchdog.py` | Separate process manager: crash recovery, auto-update, headless mode (`--headless`) |
| `stt/file_mover.py` | SMB/NAS remote file delivery |
| `templates/` | Jinja2 pages: index, live-settings, model-manager, server-settings, translation, corrections, file-manager, word-highlighting, url-builder |
| `static/` | Vendored JS/CSS — no build step, no npm |
| `stt/model_catalog.py` | The shipped faster-whisper catalogue as a floor under the on-disk cache |
| `stt/model_files.py` | Download manifests, file verification, per-family "is this model loadable" |
| `stt/start_watch.py` | Judges a start that is dead or wedged, so STARTING always resolves |
| `stt/session_edit.py` | What may be changed in a recorded session's rows, and what it costs |
| `stt/demo_*.py` | The shippable demo: playback engine, fake backends, egress guards, redaction, synthetic service generator, control window |
| `scripts/` | Dev-only tools: fixture recorder, demo session builder |
| `tests/` | Pytest suite: download state, path safety, staging, text utils, watchdog update |
| `packaging/` | Binary build tooling (build.py, make_icon.py, watchdog.spec, demo.spec) — NOT `build/`, which PyInstaller uses as its workdir |
| `deploy/` | OS service templates: stt-watchdog.service (systemd), com.stt.watchdog.plist (launchd) |
**A recorded session can be edited, but only deliberately.** `stt/session_edit.py` backs
the File Manager's Edit button (`/api/file-manager/db-rows` to read at full length,
`/api/file-manager/db-edit` to write). It accepts exactly `text`, `translated_text`,
`start_time`, `end_time` and `ts_ms` — **never `words_json`**, whose hand-edited timings
would desynchronise the live preview from the text it highlights with nothing on screen to
show it. A batch applies inside one transaction or not at all, the route refuses the
database the recorder currently holds (`transcription_state["db_name"]`, compared by
realpath), and `backup` has no default: rewriting congregation speech is a decision
somebody has to make, so the caller must say `keep` or `none`. The demo's own recording is
just another file this reaches, which is why `Player.begin_session` re-reads its schedule
every session instead of caching it for the life of the process.
**Clearing the screens hides captions and never edits them.** `POST
/api/transcription/clear-display` wipes every display (web viewers, translation, and
ChurchPresenter, which replaces its list from each payload's `segments`) while the service
stays one session file. That is the difference from `reset-session`, which splits the file.
The worker writes a marker row to the session database's `display_clears` table
(`stt/display_clear.py`), and `get_new_entries` shows only rows after it: `id > after_row_id`
**and** `start_time >= audio_cutoff`. The second condition hides what was still in flight
(rolling buffer, live hypothesis, held-back fragment), which finalises later with a higher id
but an earlier start, within `CUTOFF_HORIZON_MS`. Payloads carry `display_epoch` (the marker
id) and the pages treat a change like a new `session_id`. Nothing is deleted or denied, so
exports and the File Manager keep the whole service.
## Configuration
- `config/config.default.json` — defaults/schema. **New settings need a default entry here.**
- `config/config.json` — the user's live runtime settings (gitignored, as are all non-.default files in `config/`); the server writes to it. Don't clobber it with defaults.
## Conventions
- **New logic goes in `stt/` modules, not speech_to_text.py.** The monolith cannot be imported by tests (import-time side effects), so any new pure logic — and logic being touched anyway — belongs in an importable `stt/` module with unit tests shipped in the same commit. `stt/` modules must import clean with stdlib only (CI installs no ML deps), take config/paths as explicit parameters (never read the monolith's globals), and be fully type-annotated (mypy enforces `disallow_untyped_defs` on the logic modules — see pyproject). The monolith re-imports the names via thin wrappers so call sites stay unchanged.
- **Tests**: no fixed-time sleeps (wait on events/poll with deadline), deterministic, `tmp_path` for filesystem/sqlite, assert behavior rather than incidental values. Coverage target is 85% per logic module; the `stt/` package total deliberately includes untested IO-heavy code and understates the logic layer — don't add coverage omits.
- **Service recordings never enter the repository.** A caption is verbatim congregation speech, often naming people present. Code is public — `stt/translation_replay.py` and its tests are in the repo and run everywhere. The material is not: session databases (`tests/fixtures/sessions/`), replay runs (`*.replay.json`), and the caption fixture (`tests/fixtures/real_captions.json`) are gitignored, and the cases needing them `pytest.skip` when absent so CI stays green. When a rule was found by a real caption, put a **constructed** caption of the same shape in the test and describe the real one in prose — the shape is what the test needs, and prose cannot be grepped back to a person.
- **Captions never leave the machine in a support report.** The operator-facing diagnostic report (`/api/diagnostics/report`, built by `stt/diagnostics.py`) is **allowlist-only** in both directions: `LOG_TAGS` names the log tags it may quote and `CONFIG_FIELDS` the settings it may show. `logs/stt.log` is the worker's raw stdout and many tags print verbatim congregation speech, so **a new `print("[TAG] ...")` is invisible to the report until someone adds its tag deliberately** — that is the intended failure mode, the same deny-by-default rule as `stt/demo_guard.py`. Never convert either list to a denylist. The report is never uploaded; it downloads for the operator to read and share themselves.
- **A download is staged, verified, and repairable.** Every transfer goes through
`stt/downloads.py:download_url_to_file`, which writes `<name>.part` and only `os.replace()`s
it into position once it matches the size (and, for LFS weights, the sha256) the Hub
reported. Writing straight to the final name is what made an interrupted download
indistinguishable from a complete one — it listed as downloaded, a re-download skipped it as
"already exists", and the loader either threw in a native reader or hung fetching what was
missing. So: **"already exists" must mean "exists and verifies"** (`stt/model_files.py`
`files_needing_download`), and a completed download writes a `.stt-download.json` manifest
that is what later tells a truncated file from a whole one. A *missing* manifest never means
"not downloaded" — every install predating it has none.
- **The shipped model catalogue is a floor, not a fallback.** `FASTER_WHISPER_MODELS` and
`config/faster_whisper_models.default.json` are two copies of one catalogue and a test asserts
they stay identical — regenerate the seed from the dict, never edit it by hand. The cached
`config/faster_whisper_models.json` has no expiry, so a machine that has written one would
otherwise never read the shipped dict again; `stt/model_catalog.merge_catalog` puts the
shipped entries back on top of whatever the cache or a Refresh discovered. Refresh may add
models, never remove a shipped one. This is why a withdrawn repo could be offered
indefinitely with no code change able to correct it. **`large-v3-turbo` is deliberately not a
Systran repo** — Systran withdrew theirs; it points at whatever
`faster_whisper.utils._MODELS` resolves the name to.
- **A loader checks its files before it opens them.** `faster_whisper_status` requires
`tokenizer.json`, not just a weight file: without it faster-whisper falls back to
`tokenizers.Tokenizer.from_pretrained(...)`, a Rust builtin with its own HTTP client that
honours neither `HF_HUB_OFFLINE` nor any timeout, so a stalled connection blocks for ever.
`local_files_only=True` does not help — faster-whisper only forwards it to `download_model()`,
which a local directory skips. Refusing to call the loader is the only fix.
- **A start always reaches a terminal state.** The worker stamps `init_stage`/`init_stage_at`
at each init step (`_worker_stage`); the status poll runs `stt/start_watch.evaluate_start`
and *writes* an error state when the worker is gone or a stage has not advanced for
`transcription.start_stall_seconds`. The clock is per-stage, so a slow model load is left
alone while a wedged one resolves. **A new init step needs a `_worker_stage` call**, or it
is a window the reaper times as one long stage. The correction has to be written, not just
returned, or Stop and Start keep seeing a phantom `starting`.
- **One place says what a family needs.** `stt/model_files.py` holds `REQUIRED_BY_FAMILY` and
answers through `dir_status(model_dir, family)`; `stt/diagnostics.py` **imports** that map and a
test asserts the two are the same object. A second copy is not a style problem — the copy
applied faster-whisper's `tokenizer.json` rule to every transformers directory and condemned a
working CTranslate2 model on a production machine. Families whose shape does not fit a file
list (piper's `.onnx` + `.onnx.json` pair, the PANNs size floor, supertonic's third-party
downloader) keep bespoke predicates, but each must still reject the zero-byte and short-file
cases, and any "already downloaded, skipping" branch must mean *already whole* or the repair
it skips can never happen.
- **Anything generated into a directory is staged too.** The CTranslate2 conversion builds under
`<dir>.converting` and `os.replace()`s into place, for the reason downloads use `.part`: an
interrupted convert otherwise leaves a directory that exists, is not a model, and is then handed
to a C++ reader.
- **Nothing the server inherits should be the watchdog's.** The graceful-shutdown channel is a pipe the watchdog holds as the server's stdin (`stt/watchdog.py`), and Windows has no cross-process SIGTERM, so losing it turns every stop into a kill with mid-write cuts to the session database. `stt/shutdown_channel.py` reads it from a private `dup` and leaves fd 0 on the null device, so children inherit nothing — and **every failure there falls back to reading the pipe in place**, because an inherited handle is a smaller problem than no shutdown at all. On Windows the CRT's fd 0 and the Win32 `STD_INPUT_HANDLE` children actually inherit are separate; `dup2` moves only the first.
- **A Windows process tree needs a job object.** `start_new_session` + `killpg` sweeps the tree on POSIX and is a documented no-op on Windows, where terminating the server orphaned its multiprocessing children (measured: four strays holding ~1.1 GB). `stt/win_job.py` binds the tree to a job with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`; the `ProcessManager` holds that handle for the server's life and closing it kills whatever is left. Binding is allowed to fail — a leak beats a server that will not start.
- **Worker failures must report themselves.** The transcription worker is a `multiprocessing.Process`, and `BaseProcess._bootstrap` swallows its exceptions to stderr without ever calling `sys.excepthook` — so Sentry does not see them. Anything that can kill the worker belongs inside the `except Exception` in `thread1_function`, which routes through `_report_worker_crash` / `stt/worker_crash.py`. Never assign to `sys.excepthook` or `threading.excepthook` directly either; chain, or Sentry's hook is silently displaced.
- **Root never owns the checkout.** Linux installs made before 2026-09-27 run their service as `User=root` from a checkout the user owns, where git refuses to work ("dubious ownership") or, where it proceeds, leaves root-owned files the user's own updates then trip on. So every git or uv the server or watchdog runs against the checkout goes through `stt/owner_exec.run`: as the owner first, **as root only if that fails**, so no install updates worse than it did. A new subprocess that writes into the checkout must use it too; `update_server.sh` does the same in shell.
- **Commits**: conventional-commit style with a scope, e.g. `fix(translation): …`, `feat(server): …`. **No AI attribution at all** — no `Co-authored-by` line in the message, and no `git notes` attribution either. Commits carry the work, not who typed it.
- **UI work**: follow the design tokens in `DESIGN.md` (colors, typography, spacing, component styles).
- **Do not touch** `_AUTOMATIC_BACKUP/`, `models/`, `panns_data/`, `logs/` — generated/runtime data.