UPGRADING.md
Rama main · réplica de lectura del repositorio original.
# Upgrading
What a release asks of the operator, from 0.1.50 on. `CHANGELOG.md` has
every change; this page keeps the actions. For releases before 0.1.50, read
the changelog's "Upgrade note" paragraphs.
Still 0.x: a patch release can rename or remove a settings key, and most
releases ask nginx for a reload or a restart. The table says which.
## Every upgrade
- Back up before you start: `unmask backup`.
- Upgrade the unmask packages as one set. `unmask-plugin-nginx` and the
`unmask-web-*` packages are pinned to the daemon's exact version, so the
package manager moves them together. The `unmask` package restarts the
daemon itself.
- nginx: every upgrade re-renders the nginx configuration, so run `nginx -t`
and reload nginx afterwards; the admin pages show a notice until nginx has
loaded it. When the module itself changed, a reload keeps the old module:
the plugin package's output says either "module unchanged -- left in
place; nginx does not need a restart" or "The module file changed: restart
nginx". A testing-channel (rc) build prints the same.
unmask never reloads a running nginx by itself. To automate that reload
(after an upgrade or a settings save alike): `systemctl enable --now
unmask-nginx-reload.timer`. The timer ships disabled with
`unmask-web-nginx`; every five minutes it reloads nginx when a render
changed the configuration and `nginx -t` passes, at most once per ten
minutes, and never restarts it. While the nginx master still maps a
shared library an OS update has replaced, it reloads nothing and writes to
the journal that a restart is needed: the workers a reload forks on that
master crash on the old copy. Without systemd, run
`/usr/share/unmask/nginx-reload-if-stale.sh` from cron.
- Schema: the daemon applies a short schema update when it starts. An index
build that would take long is left for the operator: the package output,
the admin and `unmask doctor` announce it, and `unmask migrate` runs it
while the challenge keeps being served.
- Afterwards, run `unmask doctor` (from a shell, or from the admin): it
reports what the upgrade left to do, and whether nginx has loaded the
rendered configuration.
## Release by release
One line per release; the section of the same name under Details says what
each cell means. "Restart" is a restart of nginx (a reload keeps the module
it has loaded); "Reload" is `nginx -t` and a reload after the package's
render.
| Release | nginx | Sign-in | Settings keys | Schema |
|---|---|---|---|---|
| **0.1.56** | Reload; restart where the module changed | -- | -- | -- |
| **0.1.55** | Restart | Signed out once | 9 new, all optional | 0050 |
| 0.1.54 (tagged, not published) | -- | -- | -- | -- |
| **0.1.53** | Restart | Signed out once | 8 new, all optional | 0046, 0048, 0049 |
| **0.1.52** | Reload | -- | 1 renamed | index builds, on a large database |
| **0.1.51** | Reload | -- | 1 removed, 4 renamed | -- |
| **0.1.50** | Reload | -- | -- | -- |
## Details
### 0.1.56
- **nginx: restart on older systems, reload elsewhere.** The module files
for OpenSSL 1.1, OpenSSL 1.0 and glibc 2.12 hosts (RHEL 8 and its
rebuilds, Debian 11, Ubuntu 20.04, CentOS 7, CentOS 6) are now built with
the stack protector, RELRO and `BIND_NOW` the current build already had,
so the module changes there: restart nginx after the plugin package has
been upgraded. On other hosts the plugin package's output says whether
the module file changed, as on every upgrade.
- **Forward-auth.** The decision endpoint no longer answers on
`/unmask/api/<site>/check`; no shipped configuration used that path, and
through the copy-paste nginx snippet it let a visitor get another address
banned. The daemon upgrade closes it. If your own configuration points
`auth_request` at that path, point it at `/unmask/api/check`. If you
pasted `/usr/share/unmask/snippets/nginx-forward-auth.conf`, paste its
PART 2 again: its locations now set the `X-Original-*` headers themselves.
The packaged `/etc/unmask/forward-auth/server.inc` gains a location for
`/unmask/readyz`; an edited copy is kept on upgrade, so add it by hand
(copy the healthz block) to reach readyz through nginx.
- **Monitoring.** `unmask_event_total` changes meaning: it is the number of
events this daemon has written since it started (0 at every start), no
longer the last 24 hours of the event table. `rate()` and `increase()`
over it need no change -- the shipped dashboard uses them -- but a rule
that read the raw value as "events in the last 24 hours" must become
`increase(unmask_event_total[24h])`, and on nodes sharing one MariaDB each
daemon now exports its own writes (`sum()` is the total).
`unmask_event_unique_ip` is an estimate from the hourly rollup and has no
sample while the daemon has no database or has not finished its first
rollup pass. `unmask_db_query_seconds` is a summary. New:
`unmask_db_up`, `/unmask/readyz` (200 with the database, 503 without;
keep a load balancer's health check on `/unmask/healthz`, which is
unchanged), two alert rules (`UnmaskDaemonDown`, whose `job` label you set
to your scrape job's, and `UnmaskDatabaseUnavailable`) and a dashboard
tile: import the dashboard again.
- **A daemon without its database.** On an install that is set up, a daemon
that cannot open its database at start no longer sends the admin to the
setup wizard: admin pages answer 503, the daemon retries, and it restarts
itself in place (same PID) when the database answers. Nodes sharing one
MariaDB take a named lock (`unmask_aggregate:<database>`) while they fold
events into the hourly statistics; counts an earlier release doubled are
not corrected, and leave with the rollup's 32-day window.
- **Admin pages.** The script policy that 0.1.54 published report-only is
enforced: `Content-Security-Policy: ...; script-src 'nonce-...'
'report-sample'; report-uri <base>/admin/csp-report`. Nothing in a stock
install is affected. Anything that adds a script to the admin's pages
from outside is now refused by the browser -- a proxy in front that
injects or rewrites scripts (an optimiser, an analytics snippet added with
`sub_filter`), a userscript, an extension that writes a script into the
page: exclude `<base>/admin/` from it. Each refusal is one line in the
daemon's log (`admin csp violation: ...`, at most 20 a minute); on a stock
install such a line is a bug in the admin to report, or an injected script
that was stopped. `POST <base>/admin/csp-report` is new, takes no session
(the browser sends it on its own) and writes nothing but that line; a
front proxy that lets only known admin paths through has to let it
through as well, or the reports are lost. The install wizard's pages now
send the admin's security headers too.
- **Passwords.** Every page that sets an admin password now refuses fewer
than eight characters, as the setup wizard always did. Existing shorter
passwords still sign in.
- **config.yml.** A pattern the settings page would not have taken -- one
that does not compile, or that nginx's regex engine could take exponential
time on -- is now named in the daemon's log at start and by `unmask
doctor`. Nothing is changed in the file or in what is rendered.
- **Alpine.** The plugin now places `load_module` in
`/etc/nginx/modules/50-mod-unmask.conf`; an install that has the line in
nginx.conf keeps it there. To go back to 0.1.55 or older, remove the
plugin package and install the older one (an in-place downgrade would load
the module twice). Alpine 3.22 and older: Alpine's own nginx package is
built without `--with-compat` and has never loaded the module; use
nginx.org's package there, or Alpine 3.23.
- **Uninstalling.** Removing the packages now lists the `include` lines
left in your own server blocks and prints what `nginx -t` answers
(README, "Uninstall"). With dnf, leave out `-q` to see it.
### 0.1.55
0.1.55 is 0.1.54 with one fix: the module file is built without fortified
libc calls, so Alpine's nginx (musl, loading the glibc build through
`gcompat`) loads it again. 0.1.54 was tagged but never published; every
item under 0.1.54 below applies to an upgrade from 0.1.53 to 0.1.55.
### 0.1.54 (not published)
- **Sign-in.** The admin session, its CSRF tokens and the sign-in form's
token are signed with a new `secret.admin_secret` that only the daemon
holds; every operator signs in again once after the daemon restarts.
Nothing to configure: an install whose `config.yml` has no `admin_secret`
gets one generated and kept at `/var/lib/unmask/admin_secret` (0600, the
daemon's user). To carry the key in `config.yml` instead (a backup, or a
second daemon host behind one admin address) set `secret.admin_secret` to
any random string; `unmask config-init` writes one on a fresh install. The
key must never reach nginx's files; `unmask doctor` reports it if it does.
- **nginx: restart, and re-render.** Besides the module changes above, the
plugin's ban lookup now reads `$unmask_ban_ja4`, which the new `http.inc`
defines on `$effective_ja4` (an older plugin ignores the map), so JA4 bans
hold behind a balancer that forwards the fingerprint. With
`nginx.https_redirect` on, the rendered `server.inc` changed too: re-render
(`unmask render-nginx`, or any save) and reload.
- **A posture that ends in a CAPTCHA.** On a site whose operating-mode
bucket (`known_browser_action` or `unknown_ua_action`) is
`pow_then_captcha` or `captcha_only`, a visitor holding only a
proof-of-work pass clears the CAPTCHA once after the upgrade; a
CAPTCHA-grade pass carries over. Clearing such a posture already mints a
CAPTCHA-grade pass, so an ordinary visitor on it is unaffected: what is
asked again is a pass obtained another way (a `pow_only` bucket, a rate zone
or a protected path set to `pow`, or a cookie a client minted itself).
Known browsers on `pow_only` are not touched.
- **Challenge in flight.** A challenge page served by the previous version
carries a token the new verify refuses as stale; the page reloads itself
once and the visitor sees the check again. The token is now bound to the
client and names the chain, and a `pow_then_captcha` page sends its
proof-of-work with the CAPTCHA answer.
- **Runtime directory.** `/run/unmask` is 0750 and owned by the web
server's group (the sockets in it already were); the service unit, the
init script and the container set it.
- **nginx: restart.** The module changed: it reads the ban list on a change
instead of on every request, accepts a quoted `_bv` cookie, keeps the
main-request marker across an internal redirect, encodes the compose
redirect's query, and is built with a stack protector and full RELRO.
Restart nginx once the plugin package has been upgraded; a reload keeps the
module it has loaded, and the admin says so.
- **Challenge page.** The challenge is served with a Content-Security-Policy
of its own (a nonce for its inline scripts, `blob:` workers, the CAPTCHA
provider's origin). A site that adds its own policy in the same `server`
block has to allow the same; README's nginx section lists it.
- **Access-log socket.** `/run/unmask/log.sock` changes from 0666 to 0660
owned by the web server's group on the first start of the new version.
The service unit already puts the daemon in that group
(`SupplementaryGroups=`), so a package install needs nothing. Where the
daemon is not a member of the worker's group, or no `nginx` / `www-data` /
`http` group exists, the socket stays at 0666 and the daemon log and
`unmask doctor` say so; set `nginx_log.socket_group` (and add the daemon
user to that group) to share it by group.
- **Outbound connections.** A webhook, feed or AI endpoint on the host's
loopback address needs `outbound.allow_loopback: true` from this version;
the SMTP relay and a local AI model keep loopback without it. The cloud
metadata addresses are refused everywhere. `outbound.deny_private: true`
closes the private ranges as well.
- **Settings keys.** New and optional: `secret.admin_secret` (see
Sign-in), `nginx_log.socket_mode`, `nginx_log.socket_group`,
`outbound.deny_private`, `outbound.allow_loopback`, `server.access_log`
(`"off"` silences the line per request), `db.max_conns` (MariaDB, custom
profile), `db.mysql_tls` (TLS to MariaDB) and the challenge branding's
`privacy_url`. Nothing is renamed or removed.
- **Schema.** Update 0050 adds the table the login lockout counts in; it is
small and runs at start.
- **Admin pages.** Every response carries a report-only script policy
(`Content-Security-Policy-Report-Only`); nothing is blocked, and the
browser console lists what an enforced policy would stop. The sign-in
form now carries a CSRF token: a sign-in page left open from before the
upgrade must be reloaded once.
- **Plugin.** The placer puts in only the module built for exactly the
host's nginx version; nginx loads no other, and the "closest match" of
the same branch it used to pick never loaded. With no module for the
host's version it prints the version, the versions bundled and the ways
out, and exits 1 (the package install still succeeds; nginx starts
without the module, as before). A host whose nginx the bundle does not
cover should pin the nginx package to a bundled version.
### 0.1.53
- **nginx: restart.** The plugin and the daemon must move together: the
pass cookie is bound through nginx's new `$unmask_bind_key`, the reuse
cap's per-pass tier is keyed on its new `$unmask_bv_id`, and the module
also changed its ban-file check and the `_bv` cookie budget. Restart
nginx once the plugin package has been upgraded. forward-auth: the nginx
and Apache snippets changed too; reload nginx / restart httpd so the
updated snippets are read. The Apache snippet needs mod_setenvif, which
the distributions' httpd packages load by default.
- **Sign-in.** Schema update 0046 gives each account a session generation;
every admin session is signed out once by the upgrade. Password-reset and
invite links sent by an earlier build stop working; send them again. A
reset link is now built on `nginx.admin_public_url`; without it, on the
first hostname in `nginx.admin_allowed_hosts`; with neither, the mail
carries the path alone.
- **Settings keys, new and optional.** `nginx.trusted_lb_realip` (the
visitor's address from X-Forwarded-For behind the trusted LB / CDN set);
`nginx.cdn_egress_extra` (your own CDN origin-facing ranges);
`nginx.cdn_egress_ban` (ban CDN egress after all);
`nginx.admin_public_url` (the admin's own address, used for the links in
mail); `notifications.critical_url`,
`notifications.critical_min_severity` and `notifications.critical_format`
(a second webhook for the critical alerts, with a format of its own);
`secret.sig_hash` (`sha1`, the default, or `sha256`; below).
- **Schema.** 0046, a session generation per account. 0048 widens
`cookie_bv` on MariaDB; on a large table it is left for the operator
(`unmask migrate -status`). 0049, the pass cookies each address presented
in a day.
- **Defaults and behaviour.** New checks on by default, held for review on
an install that reviews its upgrades: the UA-TLS consistency tier, the
proof-of-work pass budget, and the reuse cap's per-pass budget. The
default bypass presets pass GET, HEAD and OPTIONS only. A crawler passed
on its name alone meets traps, rules and limits, and a per-address budget.
With a third-party CAPTCHA configured, a verify without its token is
refused. Changing the SMTP host or the AI endpoint clears the stored
credential. Forwarded headers count only from a trusted proxy, for
`/metrics` too. The data directory is 0751 so nginx's worker reads the
ban list (no ban held in nginx before). No automatic ban lands on a CDN's
origin-facing address (CloudFront, Cloudflare, Fastly); a CloudFront
preset joins the trusted LB / CDN set. `nginx.admin_allowed_hosts`, when
set, guards every admin page, not only sign-in and password reset: list
every name you reach the admin under before upgrading. Logging out ends
every session of the account, on every device. A pass re-bound to a new
address ends with the window of the solve behind it (seven days for a
proof-of-work by default); a visitor whose pass was being carried past
that window meets a challenge again. A browser whose Accept-Language
names neither ja nor en gets the challenge, deny and admin pages in
English; they were Japanese. The access log carries the pass cookie
(`bv=`) once the plugin verified it, and the stats page ranks the passes
behind each address -- how many in a day, and the busiest hour of one --
from the moment nginx runs the rendered configuration.
- The pass-cookie reuse cap counts in two tiers now. The new first tier is
per pass: one pass (one browser on one network) holds 500 requests an hour
by default (`rate_limit.reuse.pass_per_hour`), so the people behind an
office proxy or a carrier NAT each have a budget of their own, and a
solved CAPTCHA is a new pass with its own. The address's budget (10,000 a
day, 2,000 at once) stays as the second tier; a new pass does not reset
it. An install that reviews its upgrades runs the address tier alone
until 0.1.53 is acknowledged. A site whose pages call its API many times
a minute from one browser can raise the hourly figure; the stats page's
"busiest hour of one pass" column shows what the visitors reach.
- The admin's "nginx is still running the previous configuration" notice
now tells a module upgrade apart: every access-log line carries the hash
of the module file nginx loaded (`mo=`, the module's `$unmask_module_id`),
and while the file on disk is newer, the notice asks for a restart in
place of a reload -- a reload keeps the old module. It needs the new
module to say so; this one upgrade still relies on you to restart.
- Cookie signatures and the proof-of-work seed can be HMAC-SHA256. The
daemon and the module now verify both HMAC-SHA1 (what every release so
far signed with) and HMAC-SHA256; what is issued stays SHA-1 until you set
`secret.sig_hash: sha256`, which is safe once every nginx that verifies
the site's cookies runs the 0.1.53 module (restarted onto it). A later
release makes sha256 the default, and a later one still stops accepting
sha1. Cookies signed before the switch stay good for their window.
`unmask doctor` warns when sha256 is set while nginx still logs lines
without a module id (an older module, or an older configuration).
- nginx's error log, and `unmask doctor`, now say when the module's TLS
callback was replaced by another module (every request then runs without
a JA4): one callback per SSL context is OpenSSL's rule, not a bug to fix
on either side; load one of the two.
### 0.1.52
- **nginx: reload.**
- **Settings keys.** `audit_retention_days` became
`audit_log_retention_days` (the retention tab; 0 keeps the log), with
`notice_retention_days` for the notification list. The old key is ignored
and named at startup; delete it from config.yml to silence the warning.
- **Schema.** On a large database the upgrade leaves the index builds to
you -- the stats page's per-visitor card indexes, and 0039 (the Community
Bans count): run `unmask migrate` when convenient, or start it from the
admin's notice; the challenge keeps being served meanwhile. Bans work
during the build; account and profile edits wait.
- **Behaviour.** The settings page takes the admin role; a viewer gets the
dashboard, statistics and bot hunt.
### 0.1.51
- **nginx: reload.** Native installs need it for path normalization,
challenge counting and monitor mode.
- **Settings keys.** `global.passthrough` (pass-through mode) is gone and
ignored. If it was on, enforcement is back on; monitor mode
(`challenge.default.observe_only`) is the switch, and on a module install
it is rendered into nginx (`render-nginx` + reload after turning it on).
`search_bots.upstream_disabled` and `upstream_ua_enabled` are replaced by
`search_bots.upstream_pattern_mode`: crawlers still listed in
`upstream_disabled` pass again until each is set to Normal checks in its
group's Detail on the UA filter tab (stored as
`upstream_pattern_mode: {<pattern>: none}`); the startup log names them.
`upstream_ua_enabled` needs nothing. The over-block alarm's `min_serves`
/ `max_serves_per_ip` became `min_stuck_ips` / `stuck_percent`.
- **Behaviour.** Crawler modes read Pass / Normal checks / Stop (were
White / Neutral / Black). The admin shows a notice while nginx runs an
older configuration.
### 0.1.50
- **nginx: reload.**
- **Schema.** `unmask migrate` ends at the last index and can be
interrupted; `unmask db-vacuum` builds the query statistics, and
`migrate -status` and the upgrade notice point to `unmask db-vacuum -plan`
for a fragmented database file.
- **Behaviour.** The SQL-injection honeypot preset is on by default, held
for review on an install that reviews new enforcement; its patterns now
match encoded probes. A site whose search echoes SQL itself can switch it
off.
## Rolling back
The schema only moves forward: a release's schema updates are not undone,
and an earlier daemon is not run against a later schema. Going back to an
earlier version therefore starts from the backup taken before the upgrade
(`unmask backup`; `unmask backup -h` has the restore steps). Without one,
the database stays as the later version left it, and the earlier daemon
starts against it at your own risk.
1. Stop the daemon: `systemctl stop unmask` (`rc-service unmask stop` on
OpenRC).
2. Put back `config.yml` and, on SQLite, the database file from the
archive, after removing the `-wal` and `-shm` files next to it; the
`branding/` files to `/var/lib/unmask/branding/`; `chown` all of them to
the daemon's user (`unmask`). On MariaDB, restore the `mysqldump` taken
with the backup. Settings keys the earlier version does not know are
named at its startup and ignored.
3. Install the earlier version, the three packages as one set (the plugin
and the web snippets are pinned to the daemon's exact version). The
repository carries the current release only; the package files of every
release are on its release page,
https://github.com/unmask-sh/unmask/releases, under the version's tag.
- rpm: `sudo dnf downgrade ./unmask-<version>-*.rpm ./unmask-plugin-nginx-<version>-*.rpm ./unmask-web-nginx-<version>-*.rpm`
- deb: `sudo apt install --allow-downgrades ./unmask_<version>-*.deb ./unmask-plugin-nginx_<version>-*.deb ./unmask-web-nginx_<version>-*.deb`
- apk: `sudo apk add ./unmask-<version>-*.apk ./unmask-plugin-nginx-<version>-*.apk ./unmask-web-nginx-<version>-*.apk`
- the gateway container: pin the image to the version in the compose
file (`image: unmask.sh/unmask:<version>`; the registry keeps every
released tag, `latest` moves) and `docker compose up -d`. The
`/etc/unmask` volume holds the config and the database: put the
backup's files back there before starting the container.
The `unmask` package starts the daemon again; otherwise `systemctl start
unmask`.
4. nginx: the plugin package placed the earlier module file, and a reload
keeps the one loaded: `nginx -t`, then restart nginx. The daemon's
start re-rendered the configuration for the earlier version.
5. `unmask doctor` reports what is left: the rendered configuration nginx
runs, the module it loaded, and the schema.
Sessions signed out by the later version stay signed out; sign in again.