CLAUDE.md

Rama main · réplica de lectura del repositorio original.

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

Webyz is a self-hosted web analytics product (Plausible/Umami shaped). A JS tracker on a customer site posts events to a Fastify API, which writes them to ClickHouse. Postgres (via Prisma) holds accounts, websites, plans, subscriptions and usage; ClickHouse holds the event and session firehose plus aggregates.

## Monorepo layout

pnpm workspaces + Turborepo. Node >= 22, pnpm 10.

| Path | Name in package.json | What it is | Dev port |
| --- | --- | --- | --- |
| `apps/api` | `backend` | Fastify 5 API, tracker ingest, ClickHouse queries, Prisma, cron jobs | 3042 |
| `apps/app` | `app` | Vite + React 19 dashboard (SPA), TanStack Query, shadcn/Tailwind 4 | 3041 |
| `apps/web` | `my-app` | Next.js 16 marketing site + auth pages | 3040 |
| `examples/tracker` | - | static HTML pages that load the tracker for manual testing | - |

All four apps are members of the single root pnpm workspace. `apps/web` used to carry its own `pnpm-workspace.yaml` and lockfile, which made it a competing workspace root; both are gone, so install from the repo root.

`onlyBuiltDependencies` in the root `pnpm-workspace.yaml` allows `sharp` and `unrs-resolver` to run install scripts. Without it pnpm fails the install with `ERR_PNPM_IGNORED_BUILDS`.

## Commands

Root (Turborepo fans out to every workspace that defines the script):

```bash
pnpm dev            # all apps
pnpm build
pnpm lint           # all three apps
pnpm typecheck      # apps/api
pnpm format:check   # no workspace defines this yet, still a no-op
```

Per app, run from the app directory:

```bash
# apps/api
pnpm dev                        # tsx watch src/server.ts
pnpm build                      # clears dist, then tsc -p tsconfig.build.json
pnpm start:prod
pnpm prisma:generate            # regenerates into src/generated/prisma
pnpm prisma:migrate             # prisma migrate dev
pnpm clickhouse:migrate         # applies clickhouse/migrations/*.sql in filename order
pnpm clickhouse:seed
pnpm prisma:seed                # seeds the Free/Growth/Business plans
pnpm typecheck

# apps/app
pnpm dev / pnpm build / pnpm lint

# apps/web
pnpm dev / pnpm build / pnpm lint
```

Local infrastructure:

```bash
docker compose up -d            # postgres 5442, clickhouse 8123/9010, redis 6382
```

Host ports are deliberately non-default because 5432, 6379 and 9000 are commonly taken by other projects; keep `apps/api/.env` in sync with them (see `apps/api/.env.example`).

Ingest writes to ClickHouse inline in the request; there is no queue. A Kafka path existed and was removed as dead code: a hard-coded constant kept it off and nothing shipped it.

Tests use Node's built-in runner, no framework dependency: `pnpm test` in
`apps/api` runs `node --import tsx --test "src/**/*.test.ts"`. Tests sit next to
the code as `*.test.ts` and are excluded from the build. Billing and usage
enforcement (money-path) code must have tests; other areas are untested, so do
not claim a change there is tested. Pure logic is tested directly; database
paths are tested against the Postgres in `DATABASE_URL` when `RUN_DB_TESTS=1`
is set, and skipped otherwise.

A `*.db.test.ts` must not import a module that opens a process-wide connection
(`lib/redis.ts`, `lib/clickhouse.ts`, or anything importing them, such as
`billing-cron.service.ts`). The handle keeps the event loop alive, so the test
file never exits and `node --test` kills it after the timeout, reporting a
failure while every subtest passed. Put the code under test in a module that
takes its dependencies as arguments, which is the convention everywhere else in
`core/`.

Tests never send real email. `core/email/transports.ts` forces the console
transport whenever `NODE_TEST_CONTEXT` is set (Node sets it in every process
the test runner spawns), and the `test` script blanks `RESEND_API_KEY` as well.
Without that guard a DB run sends dozens of messages through whatever key sits
in the developer's `.env`, exhausts the provider's rate limit, and then fails
unrelated tests: `notifyOnce` deletes its row and reports failure when delivery
fails, so assertions about notifications break for a reason that has nothing to
do with the code under test.

## State of the codebase

Everything below builds and runs. `pnpm lint`, `pnpm typecheck` and `pnpm build`
are all green at the repo root, and the compiled API (`node dist/server.js`)
boots and serves.

Verified working end to end against live Postgres, ClickHouse and Redis: signup
and login, tracker ingest, all breakdowns, the timezone-aware graph, realtime,
goals and conversions, public share links, per-site settings, and the usage
sync plus quota enforcement chain in both directions (block and unblock).

Two things are deliberately unfinished, and both need a decision rather than
code:

- **Pricing is final; live purchases open when Paddle verification passes.**
  `catalog/plans.config.ts` carries the approved prices (9 Sep 2026) and
  `BILLING_CONFIG.pricingFinal` is true, so `GET /plans` reports
  `pricing.status: "final"`. A plan is `purchasable` only when the provider is
  configured and its row carries price ids; until then the marketing pricing
  page and the in-app billing page say paid plans are not on sale yet and
  describe the trial. On the hosted stack `REGISTRATION` has been `open` since
  9 Sep 2026 (Paddle's domain review read a closed signup as a login wall);
  purchases still wait on Paddle's verification and domain approval, which
  Paddle itself enforces.
- **Paddle has run live once, end to end, on 10 Sep 2026** (runbook 9a): a
  real overlay checkout on app.webyz.io with a one-use 100% discount (Starter
  monthly, $0 total, real card), `transaction.paid/completed` and
  `subscription.created/activated` delivered and processed, local Starter row
  with the invoice recorded; a provider-side item change to Growth with
  `do_not_bill` synced by `subscription.updated`; a scheduled cancel kept the
  row active with `cancelAt`; an immediate cancel left it CANCELED with a Free
  row backfilled at once and the site unblocked; the discount archived. Not yet
  exercised live: a non-zero payment, overage charging, payment recovery.
- **Paddle sandbox coverage.** Verified against the sandbox
  on 9 Sep 2026: customer creation, the overlay checkout handle, the customer
  portal, signed webhooks (accept, duplicate, reject), a subscription created
  from an invoice-collected transaction with the local row and periods to
  match, a real overage charge with its idempotency scan, and the waiver of
  sub-minimum overage. Not yet exercised: a real payment (the API cannot mark
  a transaction paid), plan changes, cancel and resume, and payment recovery
  (live only). `apps/api/docs/billing-runbook.md` sections 9 and 10 record
  what was seen and what remains. `scripts/paddle/` holds the provisioning,
  probe, deliveries and sandbox scenario scripts.

`npx tsx scripts/paddle/setup.ts` (in `apps/api`) provisions the provider
catalogue from the plan catalog, idempotently: one product and two prices per
paid plan, tagged `custom_data.webyz_plan`/`webyz_cycle`, with the ids written
onto the `plans` rows, plus the notification destination with
`--webhook <url>`. It refuses a live key without `ALLOW_LIVE=1` and refuses
live prices while `pricingFinal` is false. `scripts/paddle/probe.ts` reports
what the account holds, whether the `plans` rows point at prices that exist in
it, and which events the destination carries. Neither prints a secret.

`npx tsx scripts/billing/reconcile.ts` (in `apps/api`) is the read-only
reconciliation of ledger, charging checkpoint, charge transaction and invoice
line for closed periods; it exits non-zero on any finding.
`scripts/billing/comp-plan.ts --email <e> --note "<why>"` (or `--off`) marks a
live, local-only paid subscription as comped (`subscriptions.comped_at`): a
paid plan with no provider behind it is otherwise a hard limit like a trial,
so a plan given away would be paused at its allowance. Comped means no quota
block, no usage emails, nothing billed. The billing page (`apps/app` BillingPage) renders
only from `GET /billing/usage`, `/plans` and `/billing/invoices`; it computes
nothing itself.

`apps/api/docs/billing-runbook.md` is the production checklist for Paddle:
the Retain payment-recovery setting (never cancel on a failed usage charge,
and there is no API for it), price provisioning, the notification event list,
India-seller facts, jobs, overage charging, monitoring queries, reconciliation
steps, the Paddle behaviours the code depends on, and what to exercise in the
sandbox. There is no sandbox harness in the repo.

Billing lives under `apps/api/src/core/billing/`, one folder per concern:
`catalog` (the only place plan numbers, thresholds, trial and spend-cap
settings are written; seeded into `plans`), `entitlements`, `state` (access
derivation), `usage` (billable definition, hourly buckets, period rollover),
`reporting` (one overage charge per closed period, checkpointed in
`reportedEvents`), `subscription` (provider-agnostic lifecycle and period
derivation), `provider` (the `BillingProvider` interface, the Paddle
implementation, a fake for tests; the only place the Paddle SDK is imported),
`webhooks` (idempotent ingress keyed by provider event id), `state` (access
derivation, and the single restriction writer: `state/restriction.ts` decides
NONE / FREE_QUOTA / SPEND_CAP / PAYMENT_FAILED from the whole account state
with PAYMENT_FAILED > SPEND_CAP > FREE_QUOTA, and
`reconcileRestriction` is the only code that writes `subscriptions.restriction`
or `websites.isBlocked`; every payment, cap, quota, trial, plan-change and
site-creation flow changes its fact and calls it, never deciding on its own).

Paddle is the merchant of record, which is why it is the provider: it sells,
charges tax and pays out, so no card detail reaches this server. It also has
neither usage metering nor a provider-side schedule for an item change, and
those two absences shape the domain:

- a plan is exactly **one** recurring price. Overage is billed from our ledger
  as a one-off charge (`chargeOverage`, `reporting/overage-charge.service.ts`)
  when a period closes: units are `ceil(events / 1,000)` at the plan's
  `overagePricePer1k`, `effective_from: immediately`, keyed
  `<periodId>:<cumulative>` and stamped into the charge's `custom_data` as
  `webyz_charge_key` (snake case: the SDK snake-cases custom data on the way
  out) so a retry after a crash cannot bill twice. Paddle refuses any
  transaction under 70c (`BILLING_CONFIG.usage.minChargeCents`), so overage
  worth less is **waived**: a `usage_records` row with status `WAIVED`, the
  checkpoint moved, no transaction; reconciliation counts waived events as
  settled. The charge job takes an optional `scope`, and database tests MUST
  pass it. Paddle bills a one-off item's raw amount in the **subscription's**
  currency regardless of the currency code sent (verified), so the provider
  refuses to charge overage into a subscription not in the plan's currency
  (`assertChargeCurrency`); the account's automatic currency conversion must
  stay off (runbook 1.2a).
- an annual plan is one yearly price. The provider period is the prepaid year
  (the base period); the monthly usage window is derived locally
  (`subscription/periods.ts`, anchored to the base start, clipped to the year)
  and rolled by `rollLocalUsagePeriods` in the usage sync, because nothing at
  the provider moves it.
- Paddle's Retain payment-recovery setting must not cancel on a failed charge,
  or a failed monthly overage charge would revoke a prepaid year. There is no
  API for it: it is a dashboard-only runbook item.
- Webhooks: `provider/paddle-ips.ts` refuses deliveries from addresses Paddle
  does not publish at `GET /ips` (fetched, cached an hour, "unknown" falls
  through to the signature check); `PADDLE_IP_ALLOWLIST` defaults on in
  production, off in development. The billing page initialises Paddle.js on
  load with `pwCustomer` (Paddle Retain) for accounts the provider knows, via
  `GET /billing/checkout-config`, and locks the checkout email
  (`allowLogout: false`) because Paddle attaches a subscription to the email
  typed in. The marketing site has `/refunds`; the terms link to it.

Email and password reset are implemented. With no provider configured the
console transport logs each message, so password reset works in development
straight from the server log. Set `RESEND_API_KEY` to send for real; there is no
SMTP transport, deliberately, because adding a mail library for an unused path
was not worth the dependency.

Geo breakdowns return nothing until a MaxMind database is present. Set
`MAXMIND_LICENSE_KEY` and run `npx tsx scripts/update-geo.ts`; without it
`geo-loader.service.ts` logs a failure and every country field stays empty,
which is why the Geography card can look empty on a working install.

Google Search Console integration (`core/gsc/`, the Search page) is
implemented Plausible-style: OAuth per site, property picked in settings,
data pulled live from Google and cached 10 minutes in Redis, never stored.
It is disabled until the Google login OAuth client is configured
(`GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET`) with the extra redirect URI
`/api/v1/gsc/callback` registered and the "Google Search Console API" enabled
in the Google Cloud project. The OAuth exchange and live queries have not run
against real Google credentials yet; every state the UI can reach without
them (unconfigured, unconnected, no property, reauth) is exercised.

Google Analytics 4 import (`core/imports/`, `routes/v1/imports.ts`, settings
section "Imports & exports") copies a GA4 property's daily aggregates into ClickHouse
`imported_stats` (migration 013: site, import id, date, dimension, name, and
visitors / visits / pageviews / bounces / total seconds / events). The Postgres
`SiteImport` row tracks the job. Flow: OAuth with `analytics.readonly` (own
callback `GA4_IMPORT_REDIRECT_URI`, the login OAuth client) parks the encrypted
refresh token in Redis for an hour; `startImport` clips the range to the
property's creation date and to the day before the site's first tracked session
(`firstTrackedDate`), so imported and tracked days never overlap, then
`processImport` claims the row with a conditional update, deletes the import's
rows, runs `runGa4Import` (report specs in `ga4-reports.ts`, Google behind the
`Ga4Source` seam so tests use a fake) and revokes the grant either way. Up to
`MAX_IMPORTS` (5) imports per site, GA4 and CSV mixed, never with overlapping
dates (`assertImportSlot`), so each imported day has one source and the
dashboard's span (`getImportedWindow`, first to last imported day) is safe; the
minute-long `site-imports` cron resumes PENDING rows and
RUNNING rows whose heartbeat went stale. `ga4-reports.ts` is the only place GA4
values become Webyz labels, written exactly as the breakdowns render them
(channel via `classifyChannel`, "Edge" -> "Microsoft Edge", "Macintosh" ->
"macOS", initcap devices, BCP 47 languages), so reads merge with a plain UNION
on (dimension, name). Not imported on purpose: returning visitors, cohorts and
first-user attribution (they contradict the daily visitor id), demographics and
interests (ad profiling), Google Ads dimensions; GA4 has no exit pages.

CSV import in Plausible's format, for a Plausible export or any tool's files
(`core/imports/plausible-csv.ts` for the format, `csv-import.service.ts`,
route `POST /websites/:siteId/imports/csv`, source `csv`, dashboard
`features/imports/csvFiles.ts`; format documented in `docs/importing.md#csv-format`)
writes into the same table, with no credential and no job: the browser unpacks
a zip if given one (Plausible writes it uncompressed; deflate goes through
`DecompressionStream`; zip64 is parsed but Plausible's exporter does not turn it
on) and PUTs each CSV as raw `text/csv` chunks of at most 240,000 characters,
each with the header row and ending on a record outside quotes, because the
hosted proxy caps request bodies at 1 MB. Each chunk is parsed, mapped and
inserted on arrival; `chunksDone` makes a retry idempotent; `complete` flips it
to COMPLETED; the `site-imports` cron fails uploads silent for an hour and drops
their rows. Column layouts are Plausible's own (`Plausible.Exports`,
`Plausible.Imported.CSVImporter`, read at commit e8f5220), read by header name
so older exports import. Regions ("US-CA") and cities (GeoNames ids) are named
by walking the GeoLite2 search tree once (`geo-names.ts`, about 1.5 s, cached
for 30 minutes), which yields exactly the labels ingest writes. Plausible's own `engagement` and `pageleave` events appear in its
custom events export and are skipped (`PLAUSIBLE_INTERNAL_EVENTS`); a real
export (nihaln.com, 5 Oct 2026) had 46 of them. Custom properties and the path prop of Plausible's own events are stored
(family `props`) but not yet shown.

`imported_stats` keeps one row per day, report family and attribute
combination (families and columns in `core/imports/imported-plan.ts`
`IMPORT_FAMILIES`), so a source row keeps its UTM fields and a country row its
region and city, as Plausible's own imported tables do. `planImported(dimension,
filters)` picks the family that answers a report under its filters, or none
when they span two families; that is Plausible's `decide_tables` rule. Page,
event, search and props rows are rate-neutral (`RATE_NEUTRAL_FAMILIES`): they
add visits but no bounces or seconds, and every rate divides by `rate_visits`
so they do not dilute it (Plausible zeroes imported page bounce and duration
the same way). Hostname filters imported pages but never groups them.

Version labels are major.minor for every source (`utils/version.ts`
`shortVersion`; `VERSION_SQL`, `BROWSER_VERSION_SPEC`, `OS_VERSION_SPEC` in
`db/clickhouse/filters.ts` apply it to tracked rows at read time, and
`version.test.ts` checks the two agree against ClickHouse). Version filter
values are cut the same way so older links still match. Keep any new source's
versions on that rule or its rows split from tracked ones.

Export data (`core/imports/site-export.service.ts`, `db/clickhouse/site-export.ts`,
`/websites/:siteId/exports` routes, dashboard `ExportSettings`) follows
Plausible's flow: Export to CSV prepares the zip in the background, one
`SiteExport` row per site (a new request replaces the last), the file written
to `EXPORTS_DIR/<id>.zip.part` and renamed when complete, an email to the
requester, downloadable for 7 days, deletable. `cleanupSiteExports` in the
minute `site-imports` job deletes expired files, fails a RUNNING row older
than an hour (a restart mid-build) and removes orphan files. `EXPORTS_DIR` is
the `exports` volume in `infra/docker-compose.yml`; the hosted overlay
inherits it. The content is the site's tracked history as Plausible's import
CSVs, one ClickHouse query per file streamed as CSVWithNames into
`utils/zip-stream.ts` (stored entries, data descriptors, `zlib.crc32`, no
zip64), so the zip imports back into Webyz or Plausible. Locations go through JS to turn names
into ISO 3166-2 codes and GeoNames ids (`geo-names.ts` reverse maps); an
uncoded region goes out as its name, which `regionName` in `plausible-csv.ts`
reads back. Owner or admin, any plan, cut to retention, tracked data only.
Exporting test.com and importing the zip into an empty site reproduced every
report; the format's one inherent loss is event visitors per link.

Reads merge through `core/imports/imported-window.ts`: `applyImported` turns a
request window and its filters into an `ImportedContext`, each report calls
`readFor(context, dimension)` for its planned read (null when no family
answers), and `importedStatus` reports `included` / `excluded_filters` / `off`
in `meta.imported`. `db/clickhouse/imported.ts` `importedWhere` turns a planned
read into SQL. Merging happens in top-stats (totals per window, rates over
`rate_visits`), the main graph (day, week and month buckets only, via
`timeseriesComponentsQuery`), every breakdown (`mergedTail` in
`db/clickhouse/breakdown.ts` unions native raw parts with imported rows before
paging), the Pages report (list, summary, sparklines and page detail, merged
before sorting), conversions, custom events, the outbound report (breakdown,
totals, trend; no page grouping) and the CSV export. The Pages report and page
detail show scroll depth and time on page from the `engagements` table
(Plausible's definitions: average of each visit's deepest scroll; engaged
seconds per visit with engagement), plus imported engagement parts. Imported visitors
over a range are a sum of daily visitors, so they overcount repeat visitors;
that is inherent to aggregates and documented in `docs/api.md`.

Three reports added alongside it, live as well as imported: page titles
(`pageBreakdown` with `column: "page_title"`), hostnames (session dimension),
and site search terms (`searchTermBreakdown`, terms read from `url_query` by
`SEARCH_TERM_EXPR` in `filters.ts` over `SEARCH_PARAMS`), each with a filter
(`title`, `hostname`, `search_term`).

## API architecture

Layered, one direction only: `routes` -> `controllers` -> `core/<domain>/*.service` -> `db/clickhouse/*` or Prisma.

- `src/db/clickhouse/*` holds raw SQL only. Always parameterized with ClickHouse `{name:Type}` placeholders and `query_params`. Never interpolate user input into SQL.
- `src/core/<domain>/*.service.ts` shapes and computes (percentages, bounce rate, comparisons). Services take `AppContext` (or a narrower `{ clickhouse }`) as the first argument, never the Fastify request.
- `src/controllers/*` parse `request.params`/`request.query`, normalize via `http/normalize/*`, call a service, return `sendResponse`.

`AppContext` (`src/lib/context.ts`) is `{ clickhouse, prisma, redis }`, attached per request as `request.ctx` by `plugins/external/context.ts`. Pass `request.ctx` down rather than importing singletons inside services.

### Routing and URL prefixes

`src/app.ts` autoloads `plugins/external/*` then `routes/*` with prefix `/api`. `@fastify/autoload` derives prefixes from **directory** names only, never filenames. So everything in `src/routes/v1/` is mounted at `/api/v1/<path declared in the file>`:

- `routes/v1/analytics.ts` declares `/:siteId/browsers` -> `GET /api/v1/:siteId/browsers`
- `routes/v1/tracker.ts` declares `/track` -> `POST` and `GET /api/v1/track`
- `routes/v1/user-auth.ts` declares `/users/auth/login` -> `POST /api/v1/users/auth/login`

Analytics endpoints therefore sit directly under `/api/v1/:siteId/...` with no `analytics` segment. Adding a new domain file to `routes/v1/` does not add a URL segment; declare the full sub-path inside the file.

CORS is driven by `CORS_ORIGINS` in `config/env.ts`, defaulting to `FRONTEND_URL` plus `MARKETING_URL`. It is a strict allowlist because auth rides on a cookie.

### Response and error envelopes

Success goes through `sendResponse` (`src/http/helper/send-response.ts`): `{ success: true, data, meta: { requestId, ... } }`.

Errors go through `registerErrorHandler` (`src/plugins/error-handler.ts`): `{ success: false, error: { code, message, details } }`, where `details` is stripped in production. Expected failures must be created with `createAppError` (`src/errors/app-error.ts`), which sets `isOperational` and `statusCode`; add named factories to `src/errors/domain-errors.ts` rather than throwing raw `Error`. A raw `Error` becomes a 500 with a generic message.

Auth is cookie-session based, not JWT. `SESSION_COOKIE_NAME` is `webyz_session`; `plugins/auth.plugin.ts` validates it against the Postgres `sessions` table (Redis-cached for 5 minutes) and sets `request.session`. Protect a route with `preHandler: [fastify.authenticate]`.

Account self-service (`core/auth/account.service.ts`): `GET /api/v1/users/me/export` returns the account's data as one JSON attachment (profile, sites with goals and funnels, subscriptions, sessions, API key metadata, provider invoices; analytics are pointed at the per-site CSV export, not inlined). `DELETE /api/v1/users/me` needs the password for password accounts, cancels every provider-backed subscription immediately through `BillingProvider.cancelNow` and aborts with 502 `SUBSCRIPTION_CANCEL_FAILED` if the provider refuses (an account must never vanish while a card is still billed), purges ClickHouse for every site via `db/clickhouse/website.ts` `purgeWebsiteAnalytics` (events, sessions, event_data, hourly_aggregates, imported_stats), writes a `deleted_accounts` tombstone (email hash, trial and paid facts) that `startTrial` checks so delete-and-re-register cannot mint another trial, clears caches, then deletes the user row and lets Postgres cascade. The dashboard page is `/settings/account`, which also hosts change password.

The same plugin also accepts `Authorization: Bearer wbz_...` personal API keys (`core/auth/api-keys.service.ts`): only the SHA-256 of a token is stored, the token is shown once at creation, and a key resolves to the same `request.session` shape as a cookie with `request.auth.kind === "api_key"`. Keys are **read-only**: any method other than GET or HEAD is refused with 403 `API_KEY_READ_ONLY` before the handler runs. The plan's `api_access` entitlement is checked on creation and on every use, so a downgrade switches keys off. Management routes are `GET/POST /api/v1/api-keys` and `DELETE /api/v1/api-keys/:keyId`; the dashboard page is `/settings/api-keys`.

`auth.plugin.ts` and `site-access.plugin.ts` are registered explicitly in `app.ts` and **awaited before the route autoload**, because routes reference `fastify.authenticate` at registration time. Only `plugins/external/*` is autoloaded.

### Email and password reset

`core/email/` is a two-file abstraction: `transports.ts` picks a transport
(`resend` over plain HTTP via axios when `RESEND_API_KEY` is set, otherwise
`console`), and `email.service.ts` exposes `sendEmail`, which **never throws**
and returns whether the message went out. Every caller is a background job or a
flow whose success must not hinge on the mail provider.

Templates live in `core/email/templates/index.ts`: password reset, password
changed, quota warning (80/90/100%), and over quota.

Password reset (`core/auth/password-reset.service.ts`):

- Only the SHA-256 of the token is stored, so a database leak yields no working
  links.
- `requestPasswordReset` always resolves, whether or not the address exists and
  whether or not the account is Google-only, so the endpoint cannot enumerate
  accounts. Both API responses are byte-identical.
- Requesting a new link retires any earlier unused one.
- Missing, used and expired tokens all return the same message.
- Completing a reset revokes every session, so an attacker holding one is cut
  off, and sends a notification email.

### Site access control

Analytics reads are guarded by `preHandler: [fastify.optionalAuthenticate, fastify.authorizeSite]`. `authorizeSite` resolves `:siteId` into `request.website` (id, domain, timezone, owner, visibility, share password hash, members; Redis-cached for a minute) and sets `request.siteRole`: `owner`, `admin` or `viewer` for the owner and `WebsiteMember` rows, `public` for anyone when `isPublic` (and, when the share has a password, only with a valid `X-Share-Token` header, else 401 `SHARE_PASSWORD_REQUIRED`), nobody else. It also carries the timezone every period calculation needs. Call `fastify.invalidateSiteCache(siteId)` after any site, sharing or membership mutation. Share tokens (`core/website/share-token.ts`) are HMACs keyed by the site's bcrypt password hash, so no extra secret exists and changing the password revokes them all.

Team members (`core/website/members.service.ts`, `routes/v1/team.ts`): roles are per site; seats are the owner's plan `team_members` entitlement, owner included, with pending invitations holding a seat; invitations are SHA-256-stored tokens emailed as `/invitations/:token`, accepted only by a signed-in account with the invited address. Access helpers in `website.service.ts`: `getOwnedWebsite` (delete only), `getManagedWebsite` (owner or admin: settings, goals, funnels, sharing, notifications, members), `getAccessibleWebsite` (any member: reads). `listWebsites` returns owned plus shared sites with a `role`. Owner-only-looking analytics endpoints (realtime visitors, stream, export) check `request.siteRole !== "public"`. Plan-gated site routes (funnels, Search Console) run `authorizeSite` before `requireEntitlement` so the site owner's plan decides, not the caller's.

Notifications (`core/notifications/`, `routes/v1/notifications.ts`): `EmailReport` rows (weekly after Monday 09:00 and monthly after the 1st 09:00 in the site's timezone, `lastPeriodEnd` prevents a resend) and one `TrafficAlert` per site (live visitors at or over the threshold, 12 hour cooldown), sent by the hourly `email-reports` and five-minute `traffic-alerts` cron jobs. The pure period arithmetic lives in `report-periods.ts` and is tested.

## Tracking and ingest path

1. `apps/api/public/js/script.js` is served by `@fastify/static` from `process.cwd()/public`, i.e. `GET /js/script.js`. It is configured entirely by `data-*` attributes on the script tag (`data-site-id`, `data-endpoint`, `data-track-localhost`, `data-debug`, ...). See `examples/tracker/basic.html`. Optional automatic events, each behind one attribute and documented in `docs/tracker.md`: `data-hash-routing` (hashchange counts as a pageview; the server already keeps the fragment in `url_path`), `data-outbound-links` (`Outbound Link: Click`, prop `href`), `data-file-downloads` (`File Download`, prop `href`, list overridable with `data-file-types`) and `data-track-404` (`404`, prop `page`, fired on pages carrying `<meta name="webyz-404">`, because a script cannot see the HTTP status). `url`, `path`, `title`, `name`, `ref`, `lang`, `screen` are reserved payload keys and never become properties.
2. It is cookieless and stores nothing in the browser (the only exception is the `webyz-disabled` localStorage flag written by `webyz.optOut()`). It sends a short-key payload: `t` (pageview|event), `sid` (website id), `pid` (in-memory pageview id), `url`, `ref`, `title`, `lang`, `screen`, plus custom props. The event time is the server's receive time, never a client value: `ts` is still accepted from old scripts and ignored (`ingest/normalize/timestamp.ts`), because usage is billed by the hour an event carries and a client must not be able to place events in a closed period, past the retention cut or in the future. Visitor and session identity are derived on the server by `src/ingest/identity/visitor-identity.ts`: `user_id = sha256(daily_salt + website_id + ip + user_agent)` with a random salt per UTC day kept in Redis for 48 hours, and `session_id` from a Redis key per visitor with a 30 minute sliding TTL. The same person is therefore one visitor within a day and unlinkable across days, a session never crosses UTC midnight, and the IP is never written anywhere. Old cached scripts may still send `vid`/`ssid`/`new_session`; they are accepted and ignored. If Redis is down, identity falls back to deterministic per-day and per-30-minute values so ingest never stops. The marketing site and the privacy policy (`apps/web/src/app/privacy`) describe this exactly; change them together.
3. `POST /api/v1/track` returns 204 on success; `GET /api/v1/track` always returns a 1x1 GIF (pixel fallback). Both run `checkIngestAllowed` then `publishTracking`. An unknown site id is a 404 so a bad install is visible; an over-quota site gets 202 with nothing written, because a visitor's browser is the wrong place to surface a billing problem.
Both routes carry `schemas/tracking.schema.ts`, validated by the tracker plugin's own Ajv (`removeAdditional: false`, unlike the app-wide validator, so custom event properties survive as extra keys and are bounded by `additionalProperties`; the POST body limit is 16 KB). A malformed POST is a 400; the GET attaches the validation error and still returns the pixel. `checkIngestAllowed` fails open only on connectivity errors (`core/tracker/connectivity.ts`: Prisma init and P1xxx errors, socket codes, pg terminated/timeout messages); anything else is rethrown so a bug can never switch quotas off. `normalizeMeta` keeps at most 30 custom properties of 500 characters.
4. `publishTracking` (`src/ingest/http/publish-tracking.ts`) normalizes and writes straight to ClickHouse in the same request.
5. Normalization (`src/ingest/normalize/normalize-tracking.ts`) does bot rejection (four filters under `core/bots/`, cheapest first, each drop counted per site and reason into the ClickHouse `dropped_events` table by `drops.ts` with `async_insert`, never with the address: `utils/bot-detection.ts`, the `ua-parser-js` bot database plus generic bot words and `HeadlessChrome`; `datacenter-ips.ts`, a sorted-interval match of the client address against `geo/datacenter-ips.txt` minus the allow ranges in `geo/datacenter-ips-allow.txt` (default Apple's iCloud Private Relay egress list: a third of its blocks sit inside the hosting ranges); `referrer-spam.ts`, the Matomo list in `geo/referrer-spam.txt`, matched on the referrer domain and its parents; and `clusters.ts`, Redis flags `bots:cluster:<site>:<screen>|<browser>|<language>` set for 24 hours by the five-minute `detect-scripted-traffic` job when a group of the last hour's sessions has 50+ visits from 20+ visitors, 95%+ bounces and zero engagement, on sites with at least 30 engaged sessions in the window (an absolute count: scripted sessions sit in the denominator of any share, and on the first live site they were 45% of the hour). The three lists are fetched daily by `update-bot-lists` (`list-updater.service.ts`, `scripts/update-bot-lists.ts`) from `DATACENTER_IP_LISTS`, `DATACENTER_IP_ALLOWLISTS` and `REFERRER_SPAM_LISTS`; `list-files.ts` reads the live copy in `geo/` and falls back to `lists-seed/`, written at image build (`BOT_LISTS_DIR`), so every filter works from the first request. `BOT_DATACENTER_FILTER=off` and `BOT_CLUSTER_FILTER=off` switch those two off; a missing file switches a list filter off), visitor identity (above), hostname extraction, URL/UTM/meta parsing, user-agent parsing, and MaxMind geo lookup.
6. `core/tracker/tracking.service.ts` inserts the event, then for pageviews upserts the session row.

`core/tracker/session.service.ts` writes `sessions` for pageviews and `core/tracker/engagement.service.ts` for engagement reports (`t: "engagement"` with `e` visible milliseconds and `sd` scroll percent, sent by the tracker on hide, blur, pagehide and before an SPA route change): an engagement extends `end_time`/`duration_seconds` to the report time, adds to `engaged_seconds`, maxes `scroll_depth`, writes an `engagements` row, and is dropped when no session exists; it is never inserted into `events`, so it is not billed (`billable.ts`) and not shown as activity. Both writers run under `core/tracker/session-lock.ts`, a per-session Redis lock (2 s TTL, 600 ms wait, proceeds unlocked if Redis is down): a route change sends the old page's engagement and the new pageview together, and two read-modify-write cycles on the same ReplacingMergeTree row lose whichever lands first without it. The tracker's engagement arithmetic is tested by running the real `public/js/script.js` in a fake browser under `node:vm` (`src/ingest/tracker-script.test.ts`). Because that table is a ReplacingMergeTree, "updating" a session re-inserts the whole row with a newer `updated_at`, so **every column must be carried forward explicitly or it is lost on the next pageview**. Attribution (`channel`, `referrer_domain`, `utm_*`) is first touch: computed by `ingest/helpers/channel.ts` when the session is created, then preserved. The classifier matches a host by every parent domain and by its registrable domain's name (`domainNames`), so `lm.facebook.com` is Facebook and `google.co.in` Google, but `x.example.com` is not X. The `AI Assistants` channel comes from the host list in `ingest/helpers/ai-assistants.ts` (referrer host or assistant-set `utm_source`); the dashboard reads channel through `CHANNEL_SPEC` in `db/clickhouse/filters.ts`, which regroups older Referral / Other Campaign rows with the same list, so a host added there applies backwards and breakdown and filter labels cannot drift.

Geo data lives in `apps/api/geo/`. `core/geo/geo-loader.service.ts` opens `geo/GeoLite2-City.mmdb` lazily and logs a failure without throwing, so geo silently degrades to empty fields if the DB is missing. `scripts/update-geo.ts` refreshes it with `MAXMIND_LICENSE_KEY`.

## Data model

### Postgres (Prisma)

`apps/api/prisma/schema.prisma`. Generated client is written to `src/generated/prisma` (custom `output`, `prisma-client` generator), so import from `../generated/prisma/client.js`, **not** `@prisma/client`. Regenerate after any schema change. Snake_case column names with `@map` throughout; keep that convention.

Models: `User`, `Session`, `PasswordResetToken`, `ApiKey`, `DeletedAccount`, `Website`, `Goal`, `Funnel`,
`FunnelStep`, `SearchConsoleConnection`, `SiteImport`, `Plan`, `Subscription`,
`BillingPeriodUsage`, `UsageRecord`.

Goals (event or exact page path, XOR) are managed in site settings; funnels
(2-8 ordered steps of the same shape) are managed on the Conversions page.
Funnel analysis is computed live per request with ClickHouse `windowFunnel`
grouped by session (`db/clickhouse/funnels.ts`); goal detail reuses
`sessionBreakdown` with a `conversion` session restriction.

Billing model, because it drives ingest behaviour:

- `Plan.extraPricePer100k` non-null means pay-as-you-go: overage is billed and traffic is **never** blocked. Null means a hard-limit (free) plan.
- The `enforce-limits` job flips `Website.isBlocked`; `checkIngestAllowed` reads it at ingest time. If Postgres is unreachable, `checkIngestAllowed` deliberately fails **open** to avoid data loss.
- `BillingPeriodUsage` is one row per subscription per period, synced from ClickHouse; `UsageRecord` is one row per overage charge sent to the provider, with the idempotency key and the transaction it created.

### ClickHouse

Database `webyz_analytics`. Migrations are plain SQL in `apps/api/clickhouse/migrations/`, applied in filename order by `scripts/clickhouse/migrate.ts`, which records applied filenames in a `migrations` table. Add a new numbered file; never edit an applied one.

- `events` - MergeTree, `PARTITION BY toYYYYMM(timestamp)`, `ORDER BY (website_id, timestamp, user_id, session_id)`. Custom props are the `meta.key` / `meta.value` array pair.
- `sessions` - **ReplacingMergeTree(updated_at)**, `ORDER BY (website_id, session_id)`. Almost all dashboard metrics come from here, not from `events`. Because rows are versioned, queries must deduplicate: either `FROM sessions FINAL` (used by the per-dimension queries in `db/clickhouse/*.ts`) or `argMax(col, updated_at) ... GROUP BY session_id` (used by `top-stats.ts`). Forgetting this double-counts sessions.
- `hourly_aggregates` - AggregatingMergeTree fed by `hourly_aggregates_mv` off `sessions`, holding `AggregateFunction` states for visits, pageviews, bounces, duration and `uniqCombined` visitors. Read with the matching `-Merge` combinators.
- `engagements` - MergeTree, one row per engagement report (website, session, user, timestamp, url_path, engaged_ms, scroll_depth), migration 011 (the session columns are 010), for time-on-page and scroll depth per page.
- `imported_stats` - MergeTree, migration 013, daily aggregates imported from Google Analytics 4 per (website, import, date, dimension, name); merged into reads, never billed (billing reads `events` only).
- `dropped_events` - MergeTree with a 400 day TTL, one row per request the bot filters refused (website_id, timestamp, reason), migration 012; read by `GET /:siteId/filtered-traffic` and the dashboard's filtered-traffic line.
- `event_data` - secondary table. (A `websites` table existed from migration 005 with nothing ever writing to it; migration 008 drops it. Site domain and timezone live in Postgres.)

Time ranges cross the wire as **Unix seconds**. `src/http/normalize/period.ts` turns `period` (`today`, `yesterday`, `last_7_days`, `last_28_days`, `last_91_days`, `this_month`, `last_month`, `this_year`, `last_12_months`, `all_time`, `custom`) plus optional `date`/`from`/`to` into `{ from, to }` seconds, with `to` exclusive (start of the next day). Unknown periods throw `invalidPeriod`.

Query conventions in `db/clickhouse/*.ts`: each dimension exposes a `...BasicQuery`, a `...DetailedQuery` (adds `visit_duration` and `bounce_rate`), and a `...TotalsQuery` used to compute percentages. `?detailed=true` on the endpoint selects the detailed variant.

CSV export: `GET /api/v1/:siteId/export?dataset=<name>&period=...` returns one `text/csv` attachment per dataset (`timeseries` or any breakdown URL segment, listed in `core/analytics/export.service.ts`, which also owns the segment -> dimension map the breakdown routes iterate). It honours the same period, `f.*` filters and retention cut as the dashboard, is owner-only, capped at 10,000 rows, and gated by the `exports` entitlement. Cells are escaped by `http/helper/csv.ts`, which also neutralises formula-leading text. The dashboard's download icon (`features/dashboard/components/ExportMenu.tsx`) fetches it as a blob so errors render in place.

Drill-down filters (Plausible style): overview endpoints (breakdowns, top-stats, main-graph, conversions, custom-events) accept `f.<key>` query params (`f.browser`, `f.page`, `f.country`, ...). `db/clickhouse/filters.ts` is the single allowlist turning them into parameterized SQL; values are matched against the same display expressions the breakdowns render, so the frontend sends the clicked label verbatim ("Chrome 120", "(direct)", "Direct"). Every filter is a `{ op, value }` condition; on the wire the operator is a value prefix (`!` is not, `~` contains, `!~` does not contain, `=` escapes a literal), parsed by `parseFilterValue` on both sides, so a bare value stays equality. `page` and `event` restrict via the events table (`session_id IN`, or `NOT IN` for a negative operator); `f.goal=<name>` is resolved by the analytics controller against the site's goals into one of those two before the query layer sees it. On the frontend they live in the URL as `f.*` params (`features/dashboard/filters.ts`); clicking a breakdown row applies an `is` filter, the Filter form offers the other operators, `FilterBar` shows them as removable pills, and a filtered dimension drills down in place (browser -> versions, country -> regions -> cities). Saved segments (`Segment` rows, `core/website/segments.service.ts`, `routes/v1/segments.ts`) store the wire form per site and are applied from the dashboard header's Segments menu, on shared dashboards too.

Sessions carry `screen` and `language` since ClickHouse migration 009 (rows from before are empty and excluded from those breakdowns); `session.service.ts` must carry them forward like every other column. Custom event properties are read straight from the `meta.key`/`meta.value` arrays with `ARRAY JOIN` (`GET /:siteId/custom-events/properties?event=&key=`, `db/clickhouse/goals.ts`); the Conversions card shows a Properties tab once an `event` filter is set.

Outbound links (`GET /:siteId/outbound-links?by=domain|url|page`, `db/clickhouse/outbound.ts`, `core/analytics/outbound.service.ts`, dashboard `OutboundLinksCard`, on the owner dashboard and the public share page alike since the route is public-readable) is a report over the tracker's `Outbound Link: Click` events, nothing stored separately: the destination is `meta.value[indexOf(meta.key, 'href')]` and its domain `domainWithoutWWW(...)` (not `cutWWW(domain(...))`, which leaves a bare host unchanged), both written once in `filters.ts` (`OUTBOUND_HREF_EXPR`, `OUTBOUND_DOMAIN_EXPR`) so the report's labels and the `outbound_domain` / `outbound_url` filters match the same expression. Those two filters are events-table session restrictions like `page` and `event`, and `buildOutboundRowFilters` additionally narrows the report's own rows to the filtered destination (a domain filter turns the Destinations tab into that domain's URLs and the Pages tab into the pages that link there). Rows carry `percentage` (of visitors who clicked out, for the bars) and `conversion_rate` (of all visitors, shown as CTR). Export datasets `outbound-domains`, `outbound-links`, `outbound-pages` live in `OUTBOUND_DATASETS`, apart from `BREAKDOWN_SEGMENTS` because the route loop mounts one endpoint per segment. Clicks without an `href` are ignored. `outbound-links/summary` (totals plus the preceding window, same comparison rule as top-stats) and `outbound-links/timeseries` feed the Outbound page (`/sites/:domain/outbound`, `pages/OutboundLinksPage.tsx`, nav item "Outbound"), which reuses the breakdown hooks under an outbound filter for its "who clicks out" section rather than having endpoints of its own. In the tracker, `data-outbound-ignore` (hosts, `*.` wildcards) and `data-outbound-strip-query` shape what is sent, the fragment is always dropped, the link label rides as `text`, and `navigateAfter` caps the navigation hold at 300 ms for outbound, download and `data-analytics-event` links alike; `tracker-script.test.ts` simulates anchor clicks through the delegated listener. The card's empty state points at the `data-outbound-links` attribute, which the `TrackingSnippet` tick boxes write, ticked by default for outbound links only. It is not a separate `event_type` on purpose: billing, bounce, goals, funnels and every query key on the two existing types. The same report serves file downloads: `kind=download` on the three endpoints switches the event name to `File Download` (`LINK_EVENT_NAMES` in `filters.ts`), a `type` grouping reads the destination's extension, the `download` filter is the download-side twin of `outbound_url` (`LINK_FILTERS` records which event each link filter lives on, so `buildOutboundRowFilters(filters, params, kind)` applies only that kind's filters row-level and the other kind's stay session restrictions), and datasets `download-files`, `download-types`, `download-pages` feed the export menu. There is deliberately no Downloads tab or page (tried and removed on 25 Sep 2026 as more weight than downloads deserve): downloads are read from the Conversions card, the `download` filter and the CSVs.

## Frontend (apps/app)

The dashboard SPA. Routes: `/login`, `/signup`, `/share/:slug` (with a password
gate and `?embed=true&theme=&background=` for iframes), `/invitations/:token`
public; `/sites`, `/sites/add`, `/sites/:domain`, `/sites/:domain/settings`,
`/settings/billing` behind `ProtectedRoute` + `MainLayout`. Login honours
`location.state.from` for in-app paths so an invitation link survives sign-in.
Site settings sections are filtered by `site.role`: viewers see People only,
admins everything but the danger zone. `/` redirects to `/sites`. On the
hosted nginx, `/pricing`, `/terms`, `/privacy`, `/refunds` and `/docs` on the
app host forward to the marketing site (`infra/hosted/nginx/webyz.conf`). The root element in `index.html` is empty:
placeholder text there flashes before React mounts.

- The dashboard URL carries the site **domain** because it reads better than a
  UUID; `useSiteByDomain` resolves it to the site id from the cached site list.
  Analytics endpoints are always keyed by id.
- The selected period lives in the `?period=` query string so a view is
  linkable and survives reload.
- `lib/axios.ts` unwraps the API envelope: use the `get`/`getPaged`/`post`/
  `patch`/`del` helpers, which return `data` directly. Errors reject as a flat
  `{ status, code, message }`.
- Feature folders: `src/features/<feature>/{api.ts, schema.ts, types.ts,
  hooks/, components/}`. Pages compose feature components. shadcn primitives are
  in `src/shared/components/ui/`, with `Field`/`SelectField` wrappers in
  `src/shared/components/Field.tsx` for labelled, validated inputs.
- `BreakdownCard` is the one tabbed card behind Contents, Acquisition,
  Technology and Geography. It fetches **only the active tab**, and the detailed
  variant only while the expanded modal is open, which keeps the hook count
  fixed regardless of tab count.
- React Compiler is on via `babel-plugin-react-compiler`; avoid manual
  memoization unless profiling says otherwise.
- Site sub-pages (Pages, Conversions, Journeys, Realtime, Search, Outbound)
  open with `shared/components/PageHeader` ("<site> · <title>", one line of
  description, controls on the right). Their summary strips are
  `shared/components/StatGrid` with `StatChange` ("+12%") beside the value.
  The Overview is deliberately separate on both counts: its own larger header
  and `TopStats`, whose tiles are chart tabs with sparklines and an arrow
  badge. Do not merge the two styles.
- The website list and the header's site switcher show each site's own favicon
  (`SiteFavicon`, sizes `md` and `sm`), falling back to the coloured initial
  tile when there is none. The image comes from
  `GET /api/v1/favicon/:domain` (`core/website/favicon.service.ts`), which
  fetches from DuckDuckGo and then Google, caches the bytes in Redis for a
  week and a miss for a day, and answers 404 so the tile stays. The browser
  never calls an icon service itself: that would hand a third party the
  signed-in user's whole list of domains. The route is public because the
  dashboard loads it as an `<img>` and, on the hosted split-host setup, a
  cross-site image request carries no `SameSite=Lax` session cookie.

## Frontend (apps/web)

Next.js 16 App Router marketing site: landing (`/`), pricing (`/pricing`),
the rendered documentation (`/docs`, `/docs/<slug>`), the blog (`/blog`,
`/blog/<slug>`), the changelog (`/changelog`) and the legal pages. Auth and the dashboard live in `apps/app`,
and the CTAs link there via `NEXT_PUBLIC_APP_URL`.

The changelog is the reader's release notes, not the operator's: `src/lib/changelog.ts`
holds dated entries written in product language, each item tagged new, improved
or fixed, rendered as a timeline at `/changelog` and as RSS at
`/changelog/rss.xml` (both prerendered, nothing fetched). The repository's
`CHANGELOG.md` stays the exhaustive Keep a Changelog file an operator scans
before upgrading; a release adds a section there and an entry here, and an
entry's `slug` is a published permalink that must not change.

The blog is Markdown in `apps/web/content/blog`, one file per post, the file
name being the slug; there is no admin panel, so a post is a pull request.
`lib/frontmatter.ts` is a deliberately tiny strict parser (a `---` block of
`key: value` scalars and `[bracketed, lists]`, nothing nested) written instead
of adding a YAML dependency, and `lib/blog.ts` turns a file into a post,
validating as it goes: the slug shape, a real `YYYY-MM-DD` date, a description
under 200 characters, a cover that carries its alt text. Anything it cannot
accept throws with the file named, which fails `next build` rather than
shipping a broken page (verified 18 Sep 2026 with a bad date). A file starting
with `_` is a draft and is not published. The leading `# Title` is stripped
from the body because the page renders the frontmatter title as the only H1.

Every blog page is prerendered, so the Markdown is read at build time only: the
standalone Docker image carries the built HTML, not `content/`. Anything that
starts reading a post at request time has to change `apps/web/Dockerfile`
first, and the repo-root `.dockerignore` excludes `*.md` with one negation per
content folder, so a new content directory needs its own line there or the
image builds without it.

`TableOfContents` is the one piece of client JavaScript a post loads: the
section list beside it marks where you are by reading heading positions on
scroll, directly rather than through an IntersectionObserver (the observer
answers "is this heading on screen", which marks nothing while a section is
taller than the viewport and two things at once when it is short).

SEO is the point of the blog, so each post carries a canonical URL, article
Open Graph and Twitter tags, `BlogPosting` and `BreadcrumbList` JSON-LD, and a
social card generated at build from the title by
`blog/[slug]/opengraph-image.tsx` (no design tool, never stale). The index
carries `Blog` JSON-LD, the sitemap lists every post with its own date, and
`/blog/rss.xml` is the feed.

The pricing page fetches the public `/api/v1/plans` list server side with a
5 minute revalidate, so marketing pricing and in-app billing cannot drift. If
the API is unreachable it renders an honest unavailable message rather than
hardcoded prices.

Page weight is a tracked concern (Lighthouse mobile, 10 Sep 2026: 92, LCP
2.8 s on the hero h1). That LCP is Lighthouse's simulation, not a paint the
h1 waits for: in every trace the h1 paints at first contentful paint, and the
"render delay" is the simulator charging every byte fetched before that paint
(fonts, the React and Next runtime, eager images) at 1.6 Mbps. So the number
moves with total initial bytes, and the runtime JS (about 175 KB gzipped, which
the App Router always ships) is most of it. `next.config.ts` inlines the CSS
(`experimental.inlineCss`, 9 KB gzipped, so no render-blocking request), the
handwriting face is a vendored 16 KB subset of Caveat (`src/fonts/README.md`:
the character set, the licence and how to regenerate it; `.hand` text must
stay within that set), only images in the first viewport are `eager`, and
nothing below the fold is `priority`. Check a change to the layout, fonts or hero against
`npx lighthouse <url> --form-factor=mobile` before shipping it.

Tailwind v4. Tokens live in `src/styles/tailwind.css` under `@theme`; the file
previously used v3 `@tailwind base/components/utilities` directives, which emit
nothing under `@tailwindcss/postcss`. The duplicate root `app/` starter that
used to shadow `src/app/` is gone.

## Open source

The code is licensed AGPL-3.0 (`LICENSE`). Public documentation lives in `docs/` (self-hosting, configuration, tracker, API, development, architecture) and is rendered by the marketing site at `/docs` and `/docs/<slug>` from those same files at build time (`apps/web/src/lib/docs.ts` lists the guides and rewrites GitHub-relative links; the web Docker image copies `docs/` for that reason) plus `README.md`, `CONTRIBUTING.md`, `SECURITY.md` and `CODE_OF_CONDUCT.md`; keep them in step with changes the same way as this file. The legal pages name the operator from `apps/web/src/lib/legal.ts`: an individual (name, city, country) during the beta because no company is registered, switchable to a company entity when payments open; the hosting provider (DigitalOcean, confirmed 10 Sep 2026) and the contact mailbox (`hello@webyz.io`, which must exist and be read) are filled in, and `reviewed` stays false until a lawyer has read the pages. `docs/api.md` is the contract external API-key users read: a change to an endpoint, parameter or error code is not done until that page says so.

## Deployment

`infra/hosted/` is how the hosted product (webyz.io, app.webyz.io,
api.webyz.io) runs: an overlay compose file layered on the open-source stack
(never edit `infra/` for hosting needs; add to the overlay), the marketing
service the open-source stack does not run, split-host URLs for the API, and
the three containers joined to the host's shared `nginx-proxy` container's
`proxy` network, reached by container name with no host ports. Cloudflare
fronts every hostname; `infra/hosted/nginx/cloudflare-real-ip.conf` unwraps
`CF-Connecting-IP` so the API's single trusted hop (`TRUST_PROXY=1`) yields
the visitor's address (verified: nginx logs the client, not the edge).
Certificates are Let's Encrypt through the host's existing certbot webroot and
daily cron (`certs.sh` for first issuance). A release is `git pull &&
./infra/hosted/deploy.sh` in `~/apps/webyz` on the server: builds the three
images there (the GHCR images are same-origin and not used), runs migrate,
starts, probes readiness. Secrets live only in `infra/hosted/environment`
(gitignored); `environment.example` lists every key. Docker's data root on
that host is `/home/docker`, not the 90%-full root partition.
`infra/hosted/backup.sh` is the nightly backup (cron, see the README there):
`pg_dump` of Postgres plus an online ClickHouse `BACKUP DATABASE` to the
`backups` disk the overlay mounts (`infra/hosted/clickhouse/backups.xml`),
kept `BACKUP_KEEP_DAYS` in `BACKUP_DIR` and copied to the rclone
`BACKUP_REMOTE` when set; `deploy.sh` runs the Postgres half before every
migration. The marketing image requires `NEXT_PUBLIC_SITE_URL`,
`NEXT_PUBLIC_APP_URL` and `NEXT_PUBLIC_API_BASE_URL` as build args and fails
otherwise, because the code's localhost fallbacks once shipped a localhost
sitemap and canonical URLs. First deployed
9 Sep 2026; registration opened and the live Paddle keys added the same day.

`docs/self-hosting.md` is the public guide; `infra/DEPLOY.md` is its short form. The self-host model is Plausible-shaped: one server, one domain, one command. `infra/setup.sh <domain>` writes `infra/.env` (domain, `BASE_URL`, generated database passwords, empty optional keys), then `docker compose up -d` in `infra/` loads `docker-compose.yml` (Postgres, ClickHouse, Redis, a one-shot `migrate`, `api`, `app`, no host ports) plus the automatic `docker-compose.override.yml` (Caddy on 80 and 443 with Let's Encrypt for `DOMAIN`). Caddy routes the single domain by path: `/api/*`, `/js/*` and `/health*` to `webyz-api:3042`, everything else to `webyz-app:8080` (`infra/caddy/Caddyfile`); the dashboard's client routes never start with those prefixes, so same-origin means no CORS and no build-time URLs. `docker-compose.expose.yml` replaces the override for an existing reverse proxy (publishes the two services on `127.0.0.1`; `infra/reverse-proxy/nginx.conf.example` is the matching nginx). The stack runs only the product; `apps/web` (the hosted service's landing, pricing, docs and legal pages) is not in `infra/` at all and is deployed by its operators separately with `VITE_API_BASE_URL`/`VITE_MARKETING_URL` baked in for the split `app.`/`api.` hostnames. Images are `ghcr.io/webyz-org/webyz-api` and `webyz-app` (`IMAGE_REGISTRY`/`IMAGE_TAG` in `.env`), published by `.github/workflows/ci.yml` on pushes to `main` and `v*` tags for amd64 and arm64; the compose services carry `build:` too, so `docker compose build` produces the same images from a checkout. `REGISTRATION` (`core/auth/registration.ts`): `open` allows any signup; anything else allows only the first account, checked in both signup paths (password and a new Google account) and reported by `GET /users/auth/providers` as `registration` so the login page hides the signup link and the signup page explains. The compose stack and `setup.sh` default to `disabled`; the hosted product sets `open`. Every compose service caps its logs through an `x-logging` anchor (json-file, 10 MB, 5 files). ClickHouse mounts `infra/clickhouse/config.xml` and `users.xml` (warning-level logs, internal log tables removed, 2 GB per-query memory cap), the Plausible-style small-server profile. The dashboard image is `nginxinc/nginx-unprivileged` listening on 8080, so upstreams are `webyz-app:8080`; the bundled Caddy adds HSTS, nosniff, referrer and permissions headers to dashboard responses (the API sets its own through helmet), with no frame-ancestors rule because shared dashboards may be embedded. Repo hygiene for the public project: `CHANGELOG.md` (Keep a Changelog, with a Breaking subsection operators scan before upgrading), issue and PR templates and Dependabot under `.github/`, and a release process in `docs/development.md` (tag `vX.Y.Z`, CI publishes images, make the GHCR packages public once). The migrate role of the API image runs `scripts/migrate-deploy.sh` (Prisma migrate deploy, ClickHouse migrations, plan seed) without a package manager; `api` waits for it, so every `up` is migrate-then-start. `TRUST_PROXY=1` in `.env` matches the single proxy hop.

## Environment variables

`apps/api/.env` (loaded by `dotenv` in `src/config/env.ts`): `PORT`, `NODE_ENV`, `LOG_LEVEL`, `TRUST_PROXY`, `DATABASE_URL`, `CLICKHOUSE_HOST`, `CLICKHOUSE_URL`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `REDIS_HOST`, `REDIS_PORT`, `MAXMIND_LICENSE_KEY`, `FRONTEND_URL`, `APP_URL`, `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REDIRECT_URI`, `PADDLE_API_KEY`, `PADDLE_CLIENT_TOKEN`, `PADDLE_WEBHOOK_SECRET`, `PADDLE_ENVIRONMENT`, `PADDLE_IP_ALLOWLIST`, `BOT_DATACENTER_FILTER`, `DATACENTER_IP_LISTS`, `DATACENTER_IP_ALLOWLISTS`, `REFERRER_SPAM_LISTS`, `BOT_CLUSTER_FILTER`, `BOT_LISTS_DIR`.

`TRUST_PROXY` decides which upstream hops may set the client address (`config/trust-proxy.ts`): a hop count such as `1` for one load balancer or reverse proxy, a comma list of addresses or CIDRs, or `false` for direct connections. It is required in production (the server refuses to boot without it) and defaults to `false` in development. Every consumer of the client address (rate-limit keys, session IP records, geo, the visitor hash) reads `request.ip`, which Fastify resolves under this setting; nothing parses `X-Forwarded-For` by hand. It was `trustProxy: true` before, which believes the leftmost forwarded value and let any client forge its address.

`CLICKHOUSE_URL` is the single setting; `CLICKHOUSE_HOST` is still accepted as a legacy alias by `config/env.ts`. Add `CORS_ORIGINS` to allow extra front end origins, otherwise it defaults to `FRONTEND_URL` plus `MARKETING_URL`.

`apps/app/.env`: `VITE_API_BASE_URL` and `VITE_MARKETING_URL`, both optional. In `pnpm dev` they default to the local ports; in a production bundle an unset API URL means `window.location.origin`, because a self-hosted install serves the dashboard and the API from one domain and that is what lets one prebuilt image serve any domain. Set `VITE_API_BASE_URL` only when the API is on another origin (the hosted product). The value is always an absolute origin, so the install snippet can never be a relative `/js/script.js`. `src/config/env.ts` is the only reader; nothing else touches `import.meta.env`. There is no `apps/app/.env.example` yet: env files are write-protected for the assistant, so add one by hand with those two names.

`apps/web/.env`: `NEXT_PUBLIC_APP_URL`, `NEXT_PUBLIC_API_BASE_URL`. See `apps/web/.env.example`.

## Gotchas worth remembering

- In `apps/app/src/index.css` the shadcn `@theme inline` block remaps `--color-primary` (and `--color-accent`, `--color-secondary`, `--color-border`) to the near-black shadcn variables, overriding the brand blue in `@theme static`. `bg-primary`/`text-primary` therefore render black. For brand-colored data-UI (chart, bars, selected states) use the `brand` / `brand-ink` tokens, plus `primary-soft`, which are not remapped. Period-over-period changes use `trend-up` / `trend-down` (theme-aware, the Overview's green and red); `success` / `danger` are for status and errors only. The app palette (bg, surface, text-*, brand) is theme-aware: values live on `:root` and `.dark` and are mapped in `@theme inline`, with the `.dark` class managed by `shared/lib/theme.ts` (persisted toggle, OS default). Chart.js cannot read CSS tokens, so `MainGraph` keeps a per-theme literal palette via `useTheme()`.

- Periods and graph buckets resolve in each site's own `Website.timezone`, carried on `request.website` by the site-access plugin. Do not reintroduce a hardcoded zone.
- Graph bucket labels (`bucketLabels` in `main-graph.service.ts`) must name the buckets ClickHouse fills: weeks start on Sunday (`toStartOfWeek` mode 0) and months on the 1st. Starting them at the window's first day once left every weekly graph empty.
- ClickHouse `formatDateTime` uses `%i` for minutes; `%M` is the month name and silently produces labels like `14:August:00`.
- Health: `GET /health` (liveness, always 200) and `GET /health/ready` (Postgres, ClickHouse and Redis each pinged with a 2 second timeout; 503 if any fails), also mounted under `/api`. Both are exempt from the rate limiter. `src/http/health.ts`; registered in `app.ts`, not autoloaded, so it can live outside the `/api` prefix. Security headers come from `@fastify/helmet` in `app.ts`: a deny-all CSP with `frame-ancestors 'none'`, and `Cross-Origin-Resource-Policy: cross-origin` because customer sites load `/js/script.js` from another origin. HSTS is on in production only.
- `tsconfig.json` typechecks `src` and `scripts` with `noEmit`; `tsconfig.build.json` is the one that emits `dist` from `src` only. Build with the latter.
- Signup calls `startSubscriptionForNewUser`: a 30 day Growth trial (config in
  `catalog/billing.config.ts`, one per account via `users.trialUsedAt`) when
  eligible, otherwise the free plan. When email verification is required
  (`EMAIL_VERIFICATION`, `core/auth/email-verification.service.ts`: `auto` means
  required iff `RESEND_API_KEY` is set), a password signup instead gets Free and
  no session; `users.emailVerifiedAt` stays null, login answers 403
  `EMAIL_NOT_VERIFIED` (checked after the password so it cannot be probed), and
  `POST /users/auth/verify-email` redeems the SHA-256-stored single-use token,
  marks the address verified, starts the trial and signs the user in. Google
  signups are verified on creation. The migration that added the column
  backfilled every existing account as verified. A trial is a local `TRIALING` row with no
  provider object and a single usage period; it is a hard limit, since overage
  cannot be billed without a card. The hourly `trials` job sends the 7 and 3 day
  reminders and expires trials to Free, making sites over the free limit
  inactive (oldest stay active) rather than deleting anything. Job-wide trial and
  enforcement functions take an optional scope; database tests MUST pass it, or
  they act on every account in the dev database.
- Plan changes (`core/billing/subscription/plan-change.service.ts`): a higher
  plan, or monthly to annual, is an upgrade and applies now with provider
  proration (`prorated_immediately`, `on_payment_failure: prevent_change`, so
  an upgrade nobody paid for never applies); a lower plan, or annual to
  monthly, is a downgrade. Paddle has no schedule for an item change, so a
  downgrade tells the provider to bill the new price from the next renewal and
  charge nothing now (`do_not_bill`), and the boundary is a local record:
  `pendingPlanId` / `pendingBillingCycle` / `pendingChangeAt`. While that is in
  the future `syncSubscription` pins the local plan to what the customer paid
  for even though the provider already carries the new price; once it passes,
  the provider's price decides and the pending record clears. The hourly
  `apply-plan-changes` job (and any renewal webhook) does that crossing.
  Undoing a pending downgrade puts the provider back on the current plan's
  price. If a due pending change names a plan the provider is not on, the
  provider wins and the mismatch is logged as an ANOMALY: entitlements never
  outrun the money.
  Every change is previewed first (entitlement diff, sites that would go
  inactive, effective date). When a plan change lands, `syncSubscription`
  reconciles active sites to the new limit (active first, then oldest) and
  emails once. Cancellation keeps access to the base period end and is
  previewed the same way. Nothing in these flows deletes data.
- Spending protection (`core/billing/spend-cap/`): the cap is a monthly figure
  including the base, measured against the annual price / 12 for yearly plans.
  It applies only to provider-backed pay-as-you-go subscriptions; free and trial
  have none. Defaults and bounds come from the plan row (seeded from catalog
  multipliers). Raising applies now and lifts a SPEND_CAP restriction; lowering
  below what the period already cost is stored in `pendingSpendCapCents` and
  applied when the next usage period opens. Enforcement restricts at the cap and
  emails once per period through `billing_notifications`.
- Plan entitlements are enforced at the API, never only in the UI.
  `fastify.requireEntitlement(<feature>)` (plugins/entitlement.plugin.ts) is
  the preHandler; it resolves the plan of the account in scope (the site owner
  when `authorizeSite` ran, else the session user) through
  `core/billing/entitlements/entitlement.guard.ts` and rejects with 403
  `FEATURE_NOT_AVAILABLE` (details: feature, plan). Gated today: funnels
  (conversions routes), journeys (analytics route, owner's plan also on public
  dashboards), search_console (every GSC route; the Google callback checks in
  the service), exports (`GET /api/v1/:siteId/export`, owner only) and
  api_access (key creation and every bearer-key request). Retention is an access rule, not deletion: every analytics
  read resolves its period through `resolveRetainedWindow`
  (core/billing/entitlements/retention.ts), which cuts the window at local
  midnight `retention_days` ago in the site's timezone and reports the cut in
  `meta.retention`; the top-stats comparison window is bounded the same way.
  `team_members` is enforced by `inviteMember` (owner's plan, owner included,
  pending invitations count); the plan grids still omit it and can now list
  it.
  The dashboard reads `plan.entitlements` from `GET /billing/usage`
  (`useEntitlements`, `FeatureGate`, `RetentionNotice`) only to hide and
  explain; it is not the security boundary.
- Every new user gets a free-plan `Subscription` at signup, and the hourly `sync-usage` job backfills one for any user without a live subscription. Usage sync and quota enforcement both iterate subscriptions, so a user without one is never metered.
- Ingest drops events whose page hostname does not match the site's domain (or a subdomain, or localhost) with a 202 when `INGEST_HOSTNAME_CHECK` is on (`utils/hostname.ts`); Search Console refresh tokens are AES-256-GCM encrypted at rest under `ENCRYPTION_KEY` (`core/gsc/token-crypto.ts`, legacy plaintext rows pass through until reconnect); request logging is a route-pattern line from an `onResponse` hook in `app.ts` with `disableRequestLogging` and pino `redact` for credentials; the rate limiter uses the Redis store; the console mail transport prints bodies only outside production; the dashboard has an `ErrorBoundary` at the root and around the routed page, a `SessionWatcher` that reacts to the `webyz:unauthorized` event `lib/axios.ts` fires on any non-auth 401, and reads the pause reason from the usage summary's `access.reason`.
- Redis is never the source of truth, so the shared client
  (`lib/redis-client.ts`, options; `lib/redis.ts`, the process-wide instance)
  fails fast: no offline queue and a 2 second command timeout, so a command
  issued while Redis is down rejects at once and one Redis accepted but never
  answered rejects after 2 seconds. Every consumer treats that rejection as a
  miss: sessions, sites, entitlements and API keys fall through to Postgres,
  visitor identity to its deterministic salt and 30 minute buckets, the rate
  limiter lets requests through (`skipOnError`), cron skips the window, the
  Search Console caches refetch. Only the OAuth state keys (Google login,
  Search Console connect) require Redis outright. The realtime pub/sub
  subscriber keeps the offline queue so a SUBSCRIBE issued during a reconnect
  is still sent. Before this the API stalled for the length of a Redis outage
  instead of degrading. A new Redis call site must either `.catch` to a
  fallback or be something that genuinely cannot work without Redis.
- Cron jobs keep their Redis lock for 90% of the interval, so restarting the API does not re-run a job that already ran this window. To exercise a job right after a code change, delete `cron:lock:<name>` in Redis first. A job that resolves `"skipped"` (missing configuration, e.g. `update-geo` without `MAXMIND_LICENSE_KEY`) releases its lock, so adding the setting and restarting runs it; before that fix the weekly geo job held its lock for six days after one skipped run, which is how production had a key set and no database. `npx tsx scripts/update-geo.ts` (inside the api container: `node_modules/.bin/tsx scripts/update-geo.ts`) downloads it immediately.