CHANGELOG.md
Rama main · réplica de lectura del repositorio original.
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
New entries go in [`changes/unreleased/`](changes/unreleased/), one file per PR; see [`changes/README.md`](changes/README.md).
## [1.12.0] - 2026-11-10
### Added
- **A chunked upload into a Maven, Gradle, npm, PyPI or RPM repository is indexed by the format, as its own upload route indexes it** (#1846, #4992). `PUT /api/v1/uploads/{id}/complete` now dispatches on the repository format and runs that format's parsers and records: a Maven jar or POM gets its artifactId, version and POM metadata and appears in the generated `maven-metadata.xml`; an npm tarball's `package.json` becomes its packument version document; a PyPI wheel or sdist is validated against its `METADATA`/`PKG-INFO` and listed in `/simple/<project>/`; an RPM is named and catalogued by its NEVRA. Before, those artifacts were committed as generic rows named after the file, so the Maven, npm and PyPI indexes did not list them. A package whose embedded metadata names another package or version than its path is refused with 400, as the native route refuses it. Incus and Debian already had this since 1.11.0. Existing artifacts keep their storage keys and stay readable; they are not re-indexed. A completion that sends `Prefer: respond-async` gets 202 with the session as the poll location, for packages whose parse outlasts a proxy timeout. Chunked uploads now also get the repository's upload quarantine hold, as the native routes do, and the storage quota applies to them unless the session is a trusted replication session (an administrator or service account). See `docs/chunked-uploads.md`.
- **ISO live and installer media are scanned for the packages of their root filesystem** (#1966). An uploaded ISO9660 image, detected by its volume descriptor rather than its extension, used to be staged as one opaque file: nothing inside it was cataloged and the scan read as a clean grade A. The scanner now locates the embedded squashfs rootfs (`live/filesystem.squashfs`, `casper/filesystem.squashfs`, `LiveOS/squashfs.img`, `images/install.img`), extracts only its package databases (dpkg, rpm, apk) and release files with `unsquashfs` under the scan-workspace ceilings (`MAX_SCAN_EXTRACTED_BYTES`, 200,000 entries, checked against the declared sizes before anything is written), and grades and inventories them like a container image. An image whose rootfs cannot be read (none embedded, layered Ubuntu casper media, a corrupt or oversized squashfs, no package database) is recorded `not_cataloged` with `inventory_status = 'partial'` and flags the repository instead of reading clean. See `docs/iso-image-scanning.md`.
- **Lifecycle policies take a `match.format` scope** (#2024). `"match": {"format": "docker"}` (or an array such as `["npm", "yarn"]`) limits a policy to repositories of those formats: the policy is skipped entirely in a repository of any other format, in the dry-run preview, the live run and the proxy-cache eviction alike, so one global policy can clean up build images without touching Maven or npm repositories. It combines with `match.path_prefix`, `match.version_pattern`, `exclude` and `min_keep`. Format names are the repository format keys (`docker`, `helm_oci`, `conda_native`, ...); an unknown name is rejected with 400 when the policy is created or updated. A 1.11.x server refuses a policy that carries `match.format` rather than ignoring the scope: in a mixed-version fleet a run, preview, create or update on a 1.11.x replica answers 400 `unknown key 'match.format'` and a scheduled run there fails before deleting anything (logged, `last_run_at` unchanged) and is retried on a later tick, so the policy only runs on upgraded replicas; see `docs/lifecycle-policy-assignments.md`.
- **Headless clients can sign in with the RFC 8628 device authorization grant** (#3461). A CLI, CI runner or other browserless client calls `POST /api/v1/auth/device/code`, shows its user a short code, and polls `POST /api/v1/auth/device/token`; the user enters the code on the `/device` page of a signed-in browser session, sees the requested scopes and when and from which IP address the request was started, and approves or denies. The client then redeems the device code exactly once for an AK access token and refresh token. This is a plain Artifact Keeper grant, not an OpenID Connect flow: the approver signs in however AK already allows, and no `id_token` is issued. The grant is off by default; set `DEVICE_AUTH_ENABLED=true` to turn it on (see `docs/device-authorization.md` for the settings).
The issued pair is capped at the requested scopes that the approver may grant, never carries `*` or `admin`, and its refresh family takes part in rotation, replay detection and logout like any other session. Only an interactive session can approve: API tokens, service accounts, download tickets, Basic credentials and scope- or repository-restricted JWTs are refused. A redeemed code cannot be redeemed again, polls inside the interval get `slow_down` and a longer interval, codes expire after 10 minutes, and wrong user codes are throttled per user and per client IP. Code issuance, approval, denial, failed attempts, replayed codes, unredeemed approvals and token issuance are all audited, and neither code is stored in plaintext or written to the audit trail. Contributed by @vivekdsr.
- **Package-manager clients can authenticate with LDAP/AD credentials** (#3830). Format and repository-download endpoints now accept a directory username and password in the same `Basic` (`curl -u`) credential every package client already supports. The passthrough is opt-in through `LDAP_BASIC_AUTH_PASSTHROUGH=true` and runs only after local bcrypt login, a short-lived JWT in the password field, and an API token in the password field have all been tried, so existing `.npmrc`, `auth.json`, `nuget.config` and token-based setups are unchanged. Binds share the `/v2/token` per-`(username, IP)` budget, and a per-username cap stops binding after three distinct failed passwords so an Artifact Keeper client cannot walk an AD account into lockout. Successful and failed passthrough logins are audited. The success cache is keyed by a per-process HMAC of the credential, re-reads `users.is_active` on every hit, and expires after five minutes, so a changed AD password works immediately and its old value stops working within that window; deactivation clears both caches at once.
- **`DB_QUERY_SPANS=all` exports a database span for every SQL statement without logging every statement** (#3954, #4959). Database `CLIENT` spans used to come only from the statements `RUST_LOG` let through, so a span per statement needed `sqlx::query=debug`, which also printed every statement on stdout. With `DB_QUERY_SPANS=all` the filter is applied to each output separately: every statement becomes a span while stdout and exported log records still follow `RUST_LOG`. `log` (the default) keeps the 1.11 behaviour and `off` turns database spans off. The startup line reports the mode as `db_query_spans`. See `docs/operations/logging.md`.
- **The repository upstream check, the curation sync and the curation popularity lookups now trace their upstream requests** (#3954, #4959). `POST /api/v1/repositories/{key}/test-upstream`, the curation sync's RPM and Debian index fetches and PyPI provenance lookups, and the pypistats / npm download-count lookups now send through the same path as the proxy, so each request exports an OpenTelemetry `CLIENT` span with the HTTP client attributes (`url.full` without credentials or query, `http.response.status_code`, `error.type`) and forwards `traceparent`. The scheduled curation sync runs outside any request, so each of its fetches starts a trace of its own. See `docs/operations/logging.md`.
- **`GET /api/v1/groups/{id}/members` now lists a group's members instead of returning 405** (#4027). The path served only `POST` and `DELETE`, so a client that had learned the projects API (`GET /api/v1/projects/{id}/members`) got a bare 405 here even though the data was already readable through `GET /api/v1/groups/{id}`. The new endpoint returns the same `members` and `members_total` fields as the group detail endpoint, paginated with `member_limit` / `member_offset` (default 50, max 200) in join order, works for SSO and LDAP-synced groups, and enforces the same visibility rule as the group detail endpoint, so a caller outside the group still gets a 404 rather than the member list.
- **Scan-on-proxy now gates Go module zips from Go Remote and Virtual repositories** (#4103, part of #4097). With `scan_on_proxy` on, a proxied `@v/{version}.zip` goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable zip is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The zip must be the module the request names (every entry under `{module}@{version}/`, and a `go.mod` that declares that module); one that is not is withheld under `fail_closed` (423) without being scanned. A zip that agrees is graded as the requested `module@version` (through a synthetic `go.mod` pin, #4952) and on its contents, chiefly the requirements in its own `go.mod` (see "Archive formats graded on their contents" in `docs/security/scan-on-proxy.md`). A Virtual repository applies the stricter of its own and each Remote member's policy, and a member's block or fail-closed lock is final. On Remote and Virtual Go repositories (whether or not scan-on-proxy is on), a module path or version containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400` on every route: a `?` in the version let the `.mod` and `.info` routes fetch the module zip from upstream and serve it unscanned. `.mod`, `.info`, `@v/list` and `@latest` are otherwise unchanged, and `GET /api/v1/formats` now reports Go as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates `.gem` downloads from RubyGems Remote and Virtual repositories** (#4104, part of #4097). With `scan_on_proxy` on, a proxied gem goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable gem is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The gem is graded as the `{name}-{version}` its file name names, which its own gemspec must declare (the same pin hosted gems are scanned with); a gem whose gemspec names something else is withheld under `fail_closed` (423) without being scanned. A Virtual repository applies the stricter of its own and each Remote member's policy, and a member's block or fail-closed lock is final. On Remote and Virtual RubyGems repositories, a gem file name containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. Spec indexes and the JSON API are unchanged, and `GET /api/v1/formats` now reports RubyGems as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates Composer dist archives from Composer Remote and Virtual repositories** (#4105, part of #4097). With `scan_on_proxy` on, a proxied `dist/` download goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable archive is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The archive's own `composer.json` must name the requested package (and any `version` it declares must match); one that does not is withheld under `fail_closed` (423) without being scanned. A dist that agrees is graded as the requested package version (through a synthetic `composer.lock` pin, #4952) and on its contents (a bundled `composer.lock`, vendored packages). A Virtual repository applies the stricter of its own and each Remote member's policy; a scanning Remote member now fetches the dist from the URL its own upstream metadata names, so a Virtual over a Packagist-style Remote serves through the gate. On Remote and Virtual Composer repositories, a `dist/` vendor, package, version or reference containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. Package metadata is unchanged, and `GET /api/v1/formats` now reports Composer as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates package archives from Pub Remote and Virtual repositories** (#4107, part of #4097). With `scan_on_proxy` on, a proxied `packages/{name}/versions/{version}.tar.gz` goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable archive is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The archive's root `pubspec.yaml` must declare the requested name and version (the scanner then grades that package version from it); an archive that does not is withheld under `fail_closed` (423) without being scanned. A Virtual repository applies the stricter of its own and each Remote member's policy, and a member's block or fail-closed lock is final. On Remote and Virtual Pub repositories, an archive name or version containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. The JSON API is unchanged, and `GET /api/v1/formats` now reports Pub as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates Conan recipe and package archives from Conan Remote and Virtual repositories** (#4108, part of #4097). With `scan_on_proxy` on, every proxied recipe or package file except `conanfile.py`, `conanmanifest.txt` and `conaninfo.txt` (the `conan_export.tgz`, `conan_sources.tgz` and `conan_package.tgz` archives, and any other file name, so an archive under another name is never streamed unscanned) goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable archive is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. A Conan archive carries no identity of its own, so it is graded as the requested reference's `name/version` (through a synthetic `conan.lock` pin, #4952) and on its contents (a bundled `conanfile.txt` or `conan.lock`, binaries the engine recognises); bytes that are not a readable gzipped tar are withheld under `fail_closed` (423) without being scanned. On Remote and Virtual Conan repositories, a recipe or package file path containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. `conanfile.py`, `conanmanifest.txt`, `conaninfo.txt`, revision lists and search are unchanged, and `GET /api/v1/formats` now reports Conan as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates pod archives from CocoaPods Remote and Virtual repositories** (#4109, part of #4097). With `scan_on_proxy` on, a proxied `pods/{name}-{version}.tar.gz` goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable archive is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The pod's own `.podspec.json` must declare the requested name and version; an archive that does not is withheld under `fail_closed` (423) without being scanned. On Remote and Virtual CocoaPods repositories, a pod archive file name containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. The bundled scanner has no CocoaPods advisory data, so a pod is graded on its contents and is only found vulnerable for a vendored component of an ecosystem the scanner does grade. `GET /api/v1/formats` now reports CocoaPods as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates source archives from Swift Remote and Virtual repositories** (#4110, part of #4097). With `scan_on_proxy` on, a proxied `{scope}/{name}/{version}.zip` goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable archive is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest, and `fail_open`, `fail_closed` and `record_only` behave as they do for the other enforced formats. The archive must be a readable zip with a `Package.swift` at its root or under its one top-level directory; one that is not is withheld under `fail_closed` (423) without being scanned. The bundled scanner has no Swift advisory data, so a source archive is graded on its contents and is only found vulnerable for a vendored component of an ecosystem the scanner does grade. On Remote and Virtual Swift repositories, a source archive scope, name or version containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with `400`. Release metadata and `Package.swift` are unchanged, and `GET /api/v1/formats` now reports Swift as `scan_on_proxy: "enforced"`.
- **Scan-on-proxy now gates Debian packages from Debian Remote and Virtual repositories** (#4111, part of #4097). With `scan_on_proxy` on, a proxied `pool/` file, and a `.deb`/`.udeb`/`.ddeb` of a `dists/` or flat layout, goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable package is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest graded on the same release, and `fail_open`, `fail_closed` and `record_only` behave as for the other enforced formats. The package's own `control` must name the package, version and architecture of the requested file name, and is then graded as itself through a synthetic installed system (an `etc/os-release` and a one-entry dpkg status) on the release the remote's own cached `Release` names, or failing that the release a `+deb12u1` / `~bpo12+1` version names. A package whose metadata disagrees, can be read in more than one way (two `control.tar` members or `control` entries, a field given twice), or whose release cannot be determined (or is one the bundled scanner has no feed for) is withheld under `fail_closed` (423) without being scanned; source tarballs and diffs are inconclusive the same way, while `.dsc`, `.buildinfo`, `.changes` and every index stream as before. Bytes that disagree with the SHA-256 a cached `Packages` index records are refused with 502. On Remote and Virtual Debian repositories, a pool, `dists/` or flat path containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with 400. `GET /api/v1/formats` reports Debian as `scan_on_proxy: "enforced"`, and the generic download route refuses a scanning Debian Remote with 403 `scan_on_proxy_route_required`.
- **Scan-on-proxy now gates RPM packages from RPM Remote and Virtual repositories** (#4112, part of #4097). With `scan_on_proxy` on, a proxied file on the `/packages/` route or the catch-all (the yum layout dnf fetches) goes through the shared inline scan-and-block gate instead of streaming unscanned, except repository metadata and signing material (`repodata/`, `repomd.xml` and its signature and key, `.treeinfo`, `media.repo`, `RPM-GPG-KEY*`, `.asc`/`.sig`/`.gpg`/`.key`): a vulnerable package is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, and `fail_open`, `fail_closed` and `record_only` behave as for the other enforced formats. The package's own header must name the name, version, release and architecture of the requested file name, and the package is then graded as itself on the release its dist tag names (`el9` as RHEL 9, `fc40` as Fedora 40, `amzn2023` as Amazon Linux 2023) through a synthetic `etc/os-release`. A package whose header disagrees, lists an identity tag twice, or carries no such dist tag (or one naming a release the bundled scanner has no feed for), and any other file that is not an RPM (a `.drpm`, an installer image), is withheld under `fail_closed` (423) without being scanned. On Remote and Virtual RPM repositories, a request path containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with 400. `GET /api/v1/formats` reports RPM as `scan_on_proxy: "enforced"`, and the generic download route refuses a scanning RPM Remote with 403 `scan_on_proxy_route_required`.
- **Scan-on-proxy now gates Alpine packages from Alpine Remote and Virtual repositories** (#4113, part of #4097). With `scan_on_proxy` on, a proxied `.apk` goes through the shared inline scan-and-block gate instead of streaming unscanned: a vulnerable package is refused with 403 `scan_blocked`, a clean one is served with `X-AK-Scan: clean`, the verdict is reused for the same digest graded on the same release, and `fail_open`, `fail_closed` and `record_only` behave as for the other enforced formats. The package's own `.PKGINFO` must name exactly the requested `{pkgname}-{pkgver}.apk` for the requested architecture (or `noarch`); it is then graded as itself, and through its `origin`, on the release the requested branch names (`v3.19`, `edge`) through a synthetic installed system (an `etc/os-release` and a one-entry `lib/apk/db/installed`). A package whose `.PKGINFO` disagrees, appears twice or names a field twice, or that is requested through a branch naming no release (`latest-stable`) or one the bundled scanner has no feed for, is withheld under `fail_closed` (423) without being scanned. A Remote that re-fetches a package because its cached copy is unreadable now goes through the gate too. On Remote and Virtual Alpine repositories, a branch, repository, architecture or file name containing `?`, `#`, `;`, `\`, a stray `%`, a space or a control character after decoding is now refused with 400. `GET /api/v1/formats` reports Alpine as `scan_on_proxy: "enforced"`, and the generic download route refuses a scanning Alpine Remote with 403 `scan_on_proxy_route_required`.
- **Every credential-typed setting can now be read from a file via a `<NAME>_FILE` variable instead of the environment** (#4142). Following the `INITIAL_ADMIN_PASSWORD_FILE` (#2803) and `DEPENDENCY_TRACK_API_KEY_FILE` (#2084) precedent, `DATABASE_URL`, `JWT_SECRET`, `ADMIN_PASSWORD`, `OIDC_CLIENT_SECRET`, `LDAP_BIND_PASSWORD`, `OPENSEARCH_PASSWORD`, `PEER_API_KEY`, `SMTP_PASSWORD`, `GITHUB_TOKEN`, `MIGRATION_ENCRYPTION_KEY`, `SSO_ENCRYPTION_KEY`, `AK_WEBHOOK_SECRET_KEY`, `S3_SECRET_ACCESS_KEY`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`, `S3_PRESIGN_SECRET_ACCESS_KEY`, `AZURE_STORAGE_ACCESS_KEY` and `AZURE_CLIENT_SECRET` accept a `_FILE` companion naming a mounted secret file (Docker/Kubernetes secrets, Vault agent templates), so those values no longer have to be readable from the process environment. The direct value wins when both forms are set, file contents are trimmed of surrounding whitespace, and a `_FILE` pointing at an unreadable or empty file is an error naming the variable and the path (never the contents) rather than being silently treated as unset. Documented in `.env.example` and `docs/operations/file-based-secrets.md`.
- **Hosted conda repodata renders are bounded per process, stream the catalog, and shed with `503` past a wait** (#4176, #4172). A render of a hosted subdir's documents now walks the catalog with a cursor and writes each record straight into the served encodings, so its memory is about the size of its output instead of the whole row set. At most `AK_CONDA_REGEN_CONCURRENCY` renders (default 2) run at once per process, and a request that has waited `AK_CONDA_REGEN_WAIT_SECS` (default 30, `0` for no bound) for its render is answered `503` with `Retry-After: 5` while the render finishes for the requests that follow. New metrics `ak_conda_repodata_regenerations_in_flight` and `ak_conda_repodata_requests_shed_total`. See `docs/operations/load-shedding.md`.
- **Remote conda repositories whose upstream publishes no CEP-16 shard index serve shards synthesised from the cached upstream repodata** (#4178). Such a remote answered 404 on `repodata_shards.msgpack.zst`, so pixi, rattler and other CEP-16 clients downloaded the whole `repodata.json` on every solve, and a virtual channel with such a member served no shard index at all. The remote now stream-parses the upstream `repodata.json.zst` (or `.bz2`, or plain) its proxy cache holds, without holding the plain document, stores one shard per package name alongside hosted shards (migration 306, `origin = 'synthesized'`), and serves the index and the shards itself; a virtual's shard index includes those members. When the cached upstream document changes past the remote's metadata TTL the shards are marked stale and rebuilt before the index is served again; an unchanged document is a no-op. Synthesised shards are checked against their address before they are served, like hosted ones.
- **Conda package requests that find the database pool saturated are shed at once with `503` and `Retry-After: 1`** (#4182). A package request used to queue on the pool for up to `DATABASE_ACQUIRE_TIMEOUT_SECS` (5 s) before it was answered 503, so a burst from a fleet of installers (rattler fetches up to 50 packages at once per installer) stalled every queued request before telling it to back off. The package route's lookup now waits at most `CONDA_PACKAGE_DB_WAIT_MS` (default 500, `0` turns the gate off) for a connection and otherwise answers 503 with `Retry-After: 1` straight away, which conda clients retry; shed requests are counted in `ak_conda_package_requests_shed_total`. `docs/operations/load-shedding.md` has pool, Postgres and concurrency sizing for an installer fleet.
- **The generic artifact PUT honours `If-Match` and `If-None-Match`, so a shared index document can be updated with compare-and-swap** (#4186). `PUT /api/v1/repositories/{key}/artifacts/{path}` (and the multipart upload routes) now evaluate RFC 9110 preconditions against the live artifact's ETag, which is its quoted SHA-256: `If-Match: "<sha256>"` writes only when the stored artifact is that one and then replaces it, even on a repository without versioning; `If-None-Match: *` creates only when the path is free. A failed precondition is `412 PRECONDITION_FAILED`, checked before the body is read and again with the row locked when the upload commits, so of several conditional uploads racing on one ETag exactly one succeeds and the rest get 412 (the single-winner contract unconditional uploads got in #4832). The upload's `201` and the generic download now carry `ETag`. Unconditional uploads are unchanged, a released coordinate (a versioned path) still cannot change bytes, and a malformed precondition header is a 400. The object a conditional replace leaves without a row is handed to storage GC. See `docs/generic-conditional-writes.md`.
- **Service-account token diagnostics name the virtual-repository members a token cannot read, and flag repository restrictions that deny everything** (#4215). `GET /api/v1/service-accounts/{id}/tokens/{token_id}/scope-analysis` lists, per virtual repository in the token's scope, the leaf members the service account cannot read through it and why (`no_grant`), and the token list gains `unreachable_member_count` per token. `POST /api/v1/service-accounts/repo-selector/preview` accepts an optional `service_account_id` and reports the same before a token is minted. The analysis uses the read paths' own helpers (recursive member walk, parent visibility, owner grant and read-action checks, and the #4559 rule that a token naming the virtual reaches its members through it), so it cannot drift from what a download does.
Token rows and the analysis also carry an optional `scope_error`: `invalid_selector` (a stored selector that no longer parses), `selector_matches_nothing`, or `deleted_pinned_repositories` (every pinned repository was deleted). All three already deny every repository at authentication; authentication and the diagnostics now share one stored-scope resolver. The preview rejects a selector with unknown keys instead of ignoring them. All of these routes are admin-only, and nothing here changes what a token may access.
- **Repository admins can operate their own repository's age gate, and age-gate decisions name who made them** (#4238). The age gate is configured per repository but could only be operated by a full instance admin: a principal holding the repository-scoped `admin` verb on its own repository could not turn the gate on or off for it, could not see which of its own packages were pending review, and could not approve one — so every approval became a platform-team ticket, and the instance-wide queue meant whoever approved a package also saw every unrelated team's pending reviews. `GET`/`PUT /api/v1/repositories/{key}/age-gate` now authorize through the same `require_repo_write_access` + `require_repo_admin` chain as every other repository-configuration subresource (`set_cache_ttl`, `set_routing_rules`, `set_upstream_auth`, `update_repo_security`), and a new repository-scoped review queue — `GET /api/v1/repositories/{key}/age-gate/reviews`, `GET .../reviews/{id}`, and `POST .../reviews/{id}/{approve,reject,reopen}` — exposes the same five operations as `/api/v1/admin/age-gate/*` narrowed to that one repository.
**The grant resolves the same way whether it was given to a person, to a service account, or to a group.** `require_repo_admin` asks `PermissionService::check_permission`, which resolves a `repository:admin` action held directly by the principal, held by any group the principal belongs to, or inherited from the repository's owning project — so delegating repository administration to a team through a group gives that team exactly the operation an individually-granted admin gets, on exactly the repositories the grant covers and no others. Holding `write` still does not suffice: operating the gate is repository configuration, not artifact publishing.
**Scope is taken from the authorized path segment, never from the query string.** The repository-scoped list ignores any `repository_key` a caller puts in the query, and a review id belonging to another repository answers `404` rather than `403`, so a repository admin cannot widen the queue or probe which review ids exist elsewhere on the instance. `/api/v1/admin/age-gate/*` is unchanged — it remains the unscoped, instance-wide queue behind `admin_middleware` — and an instance admin keeps unrestricted access to every repository-scoped route.
**A repository administrator can tighten what an instance administrator decided, but only an instance administrator can relax it or hand the policy back.** The age gate is a supply-chain control, and without a guard a repository admin could re-approve or reopen a package an instance admin rejected, or disable a gate an instance admin turned on — which also stops its rejections being honoured, for every virtual that consumes the remote. Migration 290 adds two locks that only an instance administrator's action sets or clears; nothing a repository administrator does changes either one.
- **Reviews** (`age_gate_reviews.instance_locked`). An instance admin's approval or rejection locks the review: a repository admin may still **reject** it, but not approve or reopen it (a reopened review is pending, and a pending review is honoured once the package ages past the threshold). An instance admin's **reopen releases** the lock — a reopened review is undecided, so the repository's admins may decide it. The lock is part of the decision's compare-and-set, so an instance decision landing mid-request becomes a retryable conflict instead of being overwritten.
- **Policy** (`repositories.age_gate_instance_locked`). An instance admin's `PUT /api/v1/repositories/{key}/age-gate` locks the policy unless it sends `"instance_locked": false`, which **hands it back** to the repository's admins; a repository admin sending the field at all is refused. An instance-admin `PUT` that omits the field re-locks a handed-back policy, and the web UI's age-gate form does not send it yet (artifact-keeper/artifact-keeper-web#1038), so after saving that form as an instance admin, hand the policy back again with `"instance_locked": false`. Against a locked policy a repository admin may still enable the gate or raise its minimum age, but not disable it, lower the minimum age, or switch the age-source mode. The check runs under the row lock that already serialises policy writes. `GET .../age-gate` now returns `instance_locked`, so a client can explain a refused write before attempting it.
- **Why a lock and not "last decided by".** A marker would be escaped by tightening first — reject an instance admin's approval to become the last decider, then re-approve. The lock survives the tightening.
- **Upgrade.** Every decision and configured policy made before this release went through instance-admin-only endpoints, so the migration locks them; hand a policy to its team with `"instance_locked": false`. A repository nobody has configured stays unlocked, and its own admins may set and change its policy freely.
- The tier follows the actor, not the route: an instance admin acting through a repository route is still an instance admin. The review response carries `instance_locked` too.
**Decisions now say who made them.** `age_gate_reviews.reviewed_by` was returned as a bare UUID, so the queue could not answer "who promoted this package" without a lookup per row, and a CI service account was indistinguishable from a person; the audit entries for approve, reject and reopen also carried no `actor_name` or `resource_name`, unlike the repository-configuration write next to them, so an age-gate decision reached a SIEM without naming its actor. The review response now carries `reviewed_by_username` and `reviewed_by_is_service_account` alongside `reviewed_by` (hydrated by a `LEFT JOIN users`, the way `promotion_history` already hydrates `promoted_by_username`), and the three audit entries carry the actor name, the repository key and `actor_tier` (`instance_admin` / `repository_admin`). A policy change's `REPOSITORY_UPDATED` entry now also carries `age_gate_change` — the policy it replaced (`previous_enabled`, `previous_min_age_days`, `previous_mode`), the actor's tier, and `relaxed: true` when the change weakened the gate — so disabling the gate or dropping the minimum age is visible in the trail as exactly that, not only as the values it ended at; `backend/schemas/audit-event.v1.schema.json` documents the field. Setting and releasing a lock are distinct actions, `AGE_GATE_LOCK_SET` and `AGE_GATE_LOCK_RELEASED`, each naming the actor, its tier, the repository and the policy or review at that moment — so any relaxation by either tier is identifiable from the log alone: a release, an approve or reopen, or a policy change with `relaxed: true`. The promotion-approval queue gains the same attribution — `requested_by_username` and `reviewed_by_username` on `ApprovalResponse` — shown only to an instance admin or an administrator of the approval's source repository. Those read paths are gated on the source repository's visibility alone, so on a public or `internal` source repository any signed-in user can read its approvals; everyone else still gets the ids, as before, rather than a way to enumerate the instance's admin and CI accounts. A permission-lookup failure hides the names rather than failing the read. All the new response fields are additive. The four username fields are nullable: `null` for an automatic (unreviewed) row, for an account that has since been deleted (the audit log remains the record of the decision), and — on the approval endpoints — for a caller who does not administer the source repository.
- **Signing keys can store an out-of-band OpenPGP trust attestation** (#4282). An admin fetches a challenge from `GET /api/v1/signing/keys/{id}/trust-attestation/challenge`, signs it with an external issuer key (for example a root key on a hardware token), and submits the detached signature to `POST .../trust-attestation/verify`; `GET .../trust-attestation` returns the stored record. A valid signature is recorded as `signature_valid`, a cryptographic check only and not a trust-anchor decision (#2462): nothing in signing or verification reads it yet. Replacing an existing issuer is refused (409). SHA-1/MD5 signatures, non-binary signatures, RSA issuer keys under 2048 bits and revoked or expired issuer keys are rejected, and each recorded attestation writes a `SIGNING_KEY_TRUST_ATTESTED` audit event.
- **Remote and virtual Protobuf repositories reverse-proxy BSR module metadata reads** (#4284). `POST /proto/{repo}/{Service}/{Method}` forwards an explicit allowlist of read-only `buf.registry.module.v1` Connect RPCs that have no local handler (for example `ModuleService/ListModules` and `LabelService/ListLabels`) to the remote repository's configured upstream, or to the readable remote members of a virtual repository in priority order. Only Connect headers are forwarded, so Artifact Keeper credentials never reach the upstream; responses carry only Connect headers, an upstream error is answered with a Connect error rather than the upstream body, and responses are capped at 16 MiB and 60 seconds and are not cached. `Download` is not proxied by this route. These calls still need a write-capable token, as every `POST` under `/proto` does today.
- **Admins can pre-provision OIDC-managed groups before the first user login** (#4297). `POST /api/v1/admin/sso/oidc/{provider_id}/groups` with `{"name": "..."}` creates the group in that provider's namespace (or returns the existing one), so projects and permissions can reference it before anyone in it has signed in; a later login whose group claim names it joins the same group. This enables declarative and IaC-driven RBAC configuration while keeping the IdP authoritative for membership; the new admin API reuses the existing ownership-safe federated-group provisioning path and does not create users or memberships.
- **Proxy cache write failures are counted in `ak_proxy_cache_publish_failures_total`** (#4319). A proxy repository serves the upstream body and writes it to the cache at the same time; when the cache write failed, the client still got its bytes and the failure was only logged, so a storage backend that refused every cache write looked healthy while every request missed the cache. The new counter is labelled by `backend` (`filesystem`, `s3`, `gcs`) and `class` (`put_failed`, `copy_rejected`, `publish_mismatch`, `commit_failed`) and is incremented where the failure is logged. Upstream errors, the per-object size ceiling and client disconnects are not counted. See `docs/operations/proxy-cache-metrics.md`.
- **The proxy cache can live on Azure Blob storage with `AZURE_PROXY_CACHE_ENABLED=true`** (#4320). Since #3923 an Azure deployment kept its proxy cache on the local filesystem under `STORAGE_PATH`, so remote repositories pinned to Azure were refused and a multi-replica deployment needed `STORAGE_PATH` on shared storage. With the new setting and `STORAGE_BACKEND=azure`, remote repositories cache in the Azure container, default to `storage_backend: "azure"`, and every replica shares one cache. The Azure backend gained List Blobs (`list`, and the storage reindex's `list_keys`, which used to answer that Azure cannot enumerate its blobs), and its server-side copy (Put Blob From URL) now authorizes the destination with a Shared Key header instead of a SAS URL when an access key is configured, and falls back to a streamed copy when the endpoint answers 501 Not Implemented, as the Azurite emulator does. Off by default because the existing local cache is not migrated: after turning it on each remote repository starts cold. See `docs/operations/proxy-cache-on-azure.md`.
- **Logs and metrics can be exported over OTLP next to traces** (#4337). Set `OTEL_LOGS_EXPORTER=otlp` to send the diagnostics log to the collector as OpenTelemetry log records, each carrying the `trace_id` and `span_id` of the span it was emitted under, and `OTEL_METRICS_EXPORTER=otlp` to push the `/metrics` instruments with the same names and labels. Both are off by default, use the protocol and collector of `OTEL_EXPORTER_OTLP_ENDPOINT` (or `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` / `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`), and leave stdout logging and the Prometheus `/metrics` endpoint unchanged. SQL text stays stdout-only (#4455). Every OTLP signal now carries `service.instance.id`; `OTEL_TRACES_EXPORTER=none` turns span export off. See `docs/operations/logging.md`.
- **Hosted Terraform repositories can serve as an OpenTofu/Terraform origin provider registry** (#4356). Before, the package document had no `shasums_url`, `shasums_signature_url` or signing keys, so `tofu init` / `tofu providers lock` failed with `checksum list has no SHA-256 hash`, and `/.well-known/terraform.json` was only served per repository. Hosted repos now serve a deterministic `SHA256SUMS` document and a binary detached OpenPGP signature (`SHA256SUMS.sig`) made with the repository's existing signing key, and advertise both plus the public key in the package document. Authenticated callers get single-use, path-bound download tickets on `download_url`, `shasums_url` and `shasums_signature_url`, so private repositories work too (the clients fetch those URLs without credentials). Signatures are reused for five minutes per key and checksum list. Without a configured signing key the new routes return 404 and the package document is unchanged. Protocols come from the `X-Terraform-Protocols` upload header or the ZIP's `terraform-registry-manifest.json` (default `5.0`). Setting `TERRAFORM_DEFAULT_REPO` makes `GET /.well-known/terraform.json` resolve host-level source addresses to that repository; see `docs/terraform-registry.md`.
- **An instance administrator can lock a repository's proxy scan policy so a repository administrator cannot weaken it** (#4363). Mirroring the age-gate lock, any `PUT /api/v1/repositories/{key}/security` by an instance administrator now locks the proxy scan policy (migration 300, `scan_configs.proxy_scan_instance_locked`) unless the body sends `"instance_locked": false`. While locked, a repository administrator may still tighten the policy but gets `403` for turning `scan_on_proxy` off, moving `proxy_scan_action` towards `record_only`, or loosening the severity gate, and for sending `instance_locked` at all. The lock is shown as `instance_locked` on the security config GET and on `GET /api/v1/security/configs`. Existing configurations start unlocked. During a rolling upgrade, a replica still on 1.11 does not know the lock and accepts a repository administrator's write; finish the upgrade before relying on it. See docs/security/scan-on-proxy.md.
- **`GET /api/v1/admin/holds/policy-blocks` lists what a scan policy refuses right now, and the quarantine queue shows where each hold came from** (#4510). The new queue is derived from the download gate's own decision, not from the stored quarantine stamp: proxied paths whose stored `vulnerable` verdict that repository's proxy scan gate refuses (verdicts are shared by digest, so a record-only repository caching the same bytes does not list them), then hosted artifacts that the gate's scan-policy evaluation refuses (`block_unscanned`, `block_on_fail`, severity, predicate policies), each with its violations. It pages by cursor because the hosted half is evaluated, not counted. `GET /api/v1/admin/holds/quarantine` items gain `hold_source` (`upload`, `admin`, `scan_policy` or `null`) and the queue takes a `hold_source` filter. Instance administrators with an unrestricted credential only. See docs/security/download-holds.md.
- **Download statistics can be trimmed on a schedule with `DOWNLOAD_STATS_RETENTION_DAYS`** (#4541). Every download writes one row to `download_statistics` (hosted) or `proxy_download_statistics` (proxy), and since proxy history outlives cache eviction (#4540) neither table shrank except on repository delete. Set `DOWNLOAD_STATS_RETENTION_DAYS` to a positive number of days and a scheduler pass deletes older rows from both tables every `DOWNLOAD_STATS_RETENTION_INTERVAL_SECS` (default one hour), on one replica at a time under a scheduler lease, in batches of 5000 rows. Off unless set. Download counts, trends, the `no_downloads_days` lifecycle policy and blast-radius attribution forget what the period drops; `docs/operations/download-statistics-retention.md` explains how to choose it.
- **`repodata_mode` and a rebuild endpoint for hosted RPM repositories** (#4544, #5067). A CI job that publishes an RPM and installs it from the same repository right away can call `POST /api/v1/repositories/{key}/rpm/repodata/rebuild?root=<root>&wait=true` (the repository `write` permission, like an upload): it answers `200` with the revision once the precomputed repodata includes every write committed before the call. Without `wait=true` it answers `202` and the background builder builds the root at its next tick, skipping the debounce. To keep 1.11's behaviour for one repository (render on request and validate on every request, read-your-writes), set `"repodata_mode": "on_request"` on create or `PATCH`; `"precomputed"` is the default. Repository create, get, update and list responses include `repodata_mode` for local RPM repositories, and `GET /api/v1/repositories/_/capabilities` reports `rpm_repodata_mode`. See `docs/rpm-repodata-depth.md`.
- **Maven downloads from remote and virtual repositories carry `X-Checksum-SHA1`, so Maven Resolver skips the separate `.sha1` request** (#4573). Resolver reads the SHA-1 or MD5 of a download from these headers when they are present ("Remote Included" checksums, on by default) and only requests the `.sha1` file when they are not. Hosted repositories have sent them since #568, but a jar served through a remote repository carried only `X-Checksum-SHA256`, which Resolver ignores, so a build from an empty local repository made two requests per artifact. A released artifact served by a remote repository now carries the SHA-1 of its cached `.sha1` file, read from the cache alongside the body and never fetched upstream for the header (the first download of an artifact, before its `.sha1` is cached, still goes without it). A jar that passed the scan-on-proxy gate carries the SHA-1 and MD5 of its exact bytes, and a virtual repository's serve from a hosted member carries that artifact's stored checksums. `-SNAPSHOT` files and content-coded bodies are left without the headers.
- **Admins can hide web UI navigation entries the instance does not use** (#4574). The web sidebar lists every feature, so teams that only host a few package types get a long menu full of entries nobody uses. `PATCH /api/v1/admin/settings/system` now accepts `hidden_nav_items`, a list of navigation entry identifiers (the web UI sends route paths such as `/peers`), and `GET` on the same endpoint returns it. The list is stored in `system_settings` under `ui.hidden_nav_items`, every change is written to the audit log as `SETTING_CHANGED`, and `GET /api/v1/system/config` publishes it to every caller, guests included, as `ui.hidden_nav_items`. The server treats the identifiers as opaque and only checks their shape (at most 200 entries of 1 to 128 characters from letters, digits, `/`, `-`, `_` and `.`); an invalid list is refused with 400 and nothing is written. This is navigation only: no request is allowed or refused because of it. Nothing is hidden by default, and an unreadable or malformed stored value reads as nothing hidden. Needs no migration.
- **Conda package downloads answer conditional requests with `304 Not Modified`, and the hosted package route reads the database once** (#4579, #4181). Package responses ignored `If-None-Match` and `If-Modified-Since` and carried no `ETag` or `Last-Modified`, so an artifact manager in front of Artifact Keeper that revalidates its cache (Nexus, for example) re-downloaded every package on each refresh. Every package response now carries `ETag: "<sha256>"` and `Last-Modified`, and a matching validator is answered `304` with no body on hosted channels, cached Remote channels and Virtual channels (served by a hosted member or by a cached Remote member), under the plain and both `/t/<token>/` URL layouts. Access, quarantine and scan-policy checks still run first, so a held or blocked package is refused rather than revalidated. A Remote package that is not yet in the proxy cache keeps the upstream's `ETag` and is always served in full, so a conditional request never cancels a cache fill. The hosted package route also resolves the repository, its visibility, the artifact, its quarantine state and whether a scan policy applies in one statement instead of five sequential round trips, which matters for fleets of installers fetching thousands of packages each.
- **Responses served from a stale cache entry because a remote's upstream is unreachable are now marked, and the grace window is configurable** (#4581). Under the RFC 5861 `stale-if-error` grace the proxy served the cached body byte-identically to a fresh one, so neither clients, a proxy in front, nor operators could tell an upstream outage from normal operation. Such responses now carry `Warning: 110 - "Response is stale"` and `X-AK-Stale-Since` (the instant the entry's freshness lapsed), including virtual repository merges that include a stale member. The grace window, previously fixed at 3600 seconds, is set instance-wide with `PROXY_STALE_IF_ERROR_GRACE_SECS` (default 3600) and per repository with `PUT /api/v1/repositories/{key}/stale-if-error-grace` (`0` disables serving stale bodies; maximum 30 days). The window now also bounds a refill whose upstream fetch fails, on the buffered and streaming paths, which previously served any expired body however old; operators who relied on that can raise the window to its 30-day maximum. A copy served under `on_upstream_not_found = keep` (#4640) is not a stale-if-error serve and carries no marker. Each stale response increments `ak_proxy_stale_if_error_responses_total{repo}`, and `GET /api/v1/repositories/{key}/stale-if-error-grace` reports `last_stale_served_at` for the repository.
- **A remote (proxy) repository can now keep serving its cached copy when upstream answers 404 or 410, instead of deleting it, via `on_upstream_not_found = keep`** (#4640). Until now a stale mutable entry whose upstream revalidation answered 404/410 had its cached body deleted and the path negative-cached — correct when upstream is the source of truth, but destructive for deployments that use the proxy cache as a continuity copy of an external registry that may retract content. The new per-repository policy, settable with `PUT /api/v1/repositories/{key}/on-upstream-not-found` and readable with `GET`, defaults to `delete`, which is today's behaviour, so nothing changes until an operator opts in. With `keep`, a definitive 404/410 serves the cached bytes on both the buffered and streaming fetch paths, on the conditional `HEAD` revalidation probe, and to the npm computed-packument cache; a path that was never cached still returns 404 and is negative-cached, and the Package Age Policy, quarantine and lifecycle gates still apply to a kept copy. The TTL is deliberately left unextended so a later upstream 200 re-publish is picked up, and `ak_proxy_kept_on_upstream_not_found_total{repository}` counts every kept serve so content that no longer exists upstream is listable.
- **A remote repository's proxy cache can be capped in bytes, with least-recently-used eviction** (#4660). `PUT /api/v1/repositories/{key}/proxy-cache-cap` with `{"cap_bytes": N}` sets the cap, `GET` returns it with the bytes and entries the cache holds now, and `DELETE` removes it (repository `admin` action). Before, a remote's cache grew until a lifecycle age window expired: `quota_bytes` only skips a single oversized object and lifecycle `size_quota_bytes` is refused on a remote. After every cache fill, the least recently used entries are evicted until the cache is back at or under the cap; their download history is kept. An object larger than the cap is served without being cached, and if eviction cannot get under the cap (storage refusing deletes) new fills are served without being cached until it can. Setting a cap below current usage evicts at once. New counters `ak_proxy_cache_cap_evicted_entries_total`, `ak_proxy_cache_cap_evicted_bytes_total` and `ak_proxy_cache_cap_refusing_total`. See `docs/proxy-cache-cap.md`.
- **Admins can permanently delete an artifact, so a release can be re-uploaded without waiting for storage GC** (#4700). A deleted release stayed in the trash as a tombstone, and every upload path refused different bytes at that path until GC hard-deleted it, which with `GC_TRASH_RETENTION_DAYS` set meant the whole retention window. `DELETE /api/v1/admin/trash/{id}` now purges one trashed artifact, and `DELETE /api/v1/repositories/{key}/artifacts/{path}?permanent=true` deletes and purges in one call. Both are admin only and record `ARTIFACT_PURGED`.
The stored object, and a Maven artifact's checksum sidecars, are deleted only when no other artifact or revision in the same storage scope still references the key; otherwise GC reclaims them later. OCI manifests and blobs are refused (409), as restore refuses them, because their tags and shared layers belong to storage GC and blob GC. Proxy-cache objects are never deleted.
- **The publish-age gate covers Maven and Gradle repositories** (#4702). A Maven or Gradle Remote can now enable the age gate (`PUT /api/v1/repositories/{key}/age-gate`), in either mode, as npm, PyPI, Go, Cargo and VS Code Remotes already could. Every file of a version younger than the minimum age (jar, POM, Gradle module, sources, classifier archives) and its checksum and signature sidecars answer the same `451 age_gate_blocked` refusal and record the same pending review the other formats do; the GA-level `maven-metadata.xml` drops the version and re-points `latest` and `release`, so `LATEST`, `RELEASE` and version ranges never resolve to a version whose download would be refused, and the metadata checksum always matches the served body. A Virtual applies the gate of the Remote member a file or listing comes from. In `upstream_publish_time` mode a version's publish time is the upstream `Last-Modified` of its POM, read through the proxy cache; `-SNAPSHOT` versions are not gated. See `docs/security/maven-age-gate.md`.
- **The security dashboard can now show what scan-on-proxy blocked, and who asked for it** (#4777). `GET /api/v1/security/dashboard` gains `proxy_blocked`, the distinct proxied digests a repository's proxy scan gate refuses right now (`policy_violations_blocked` keeps its meaning). `GET /api/v1/security/proxy-blocked` lists them across repositories with path, digest, severity counts and CVE ids, filterable by repository and minimum severity (instance administrators only). Every refused proxied pull now writes a `PROXY_SCAN_BLOCKED` audit entry with the principal, client IP, repository, path, digest and verdict, aggregated per digest and principal per minute so a CI job retrying a blocked pull writes at most two rows a minute per replica. The download gate's listing decision is also exposed as a batched, cached service call for generated indexes (#4288), which conda repodata generation will use. See docs/security/scan-on-proxy.md and docs/audit-events.md.
- **The remote-instance relay accepts `PATCH`, so the web UI can forward partial updates to a remote instance** (#4795). The relay handled GET, POST, PUT and DELETE only, so repository settings edits and chunked-upload session updates relayed from the UI were answered with `405 Method Not Allowed`; PATCH now shares POST/PUT's auth checks, path validation, query-string forwarding and content-type/body handling, and its route is documented in the OpenAPI spec.
- **`GET /api/v1/admin/downloads` takes a `downloader=all|anonymous|authenticated` filter** (#4839). Anonymous downloads are recorded with a null `user_id`, which the UUID-typed `user_id` filter could not select, so an admin reviewing unauthenticated traffic before turning off `allow_anonymous_download` had to page through every download. `downloader=anonymous` returns only the rows without a user and `downloader=authenticated` only the attributed ones, on hosted and proxy rows alike; the filter combines with the artifact, repository, IP, source and date filters, also applies to `/downloads/by-ip/{ip}` and `/downloads/by-user/{user_id}`, and is honoured by anything that pages the listing (the web export). `downloader=anonymous` together with a `user_id` is rejected with 400. Omitted, the listing is unchanged.
- **Repository metadata renders are now observable: `ak_repo_metadata_regenerations_total`, `ak_repo_metadata_regeneration_seconds` and `ak_repo_metadata_cache_requests_total`** (#4923, #4174). The RPM repodata cache, which renders `repomd.xml` and its indexes once per repository state, moved into a format-agnostic metadata revision cache that conda repodata will also use. Its behaviour for RPM is unchanged; it now reports each render with `format` and `scope` labels and each request as a `hit`, `coalesced` or `miss`. See `docs/operations/repo-metadata-cache-metrics.md`.
- **Scan-on-proxy now grades a proxied Go module, Composer package or Conan reference as the version requested, not only on what the archive bundles** (#4952, part of #4097). The bundled scanner does not catalog a Go module zip, a Composer dist or a Conan archive as itself, so an advisory against the requested coordinate (for example `golang.org/x/text v0.3.0` pulled as its own module zip) was not found and the pull read `clean` unless something inside the archive was vulnerable. Once the archive passes the identity check, the scan workspace now gets a pin for the request coordinate: a one-line `go.mod` that requires `{module} {version}`, a `composer.lock` naming the package and version, or a `conan.lock` requiring `name/version`. The scanner must catalog that coordinate before a `clean` verdict counts, and a coordinate the pin file cannot carry exactly is inconclusive (`423` under `fail_closed`) and not scanned. What the archive bundles is still graded too. Pub, CocoaPods and Swift are unchanged. Because the grade now depends on the requested coordinate, a stored verdict for these formats is reused only for the coordinate it was graded as (recorded on the verdict, migration 301); the same bytes requested as another version or reference are scanned again.
- **`record_only` now has a view of what enforcement would refuse** (#5232). `GET /api/v1/security/proxy-blocked`, the dashboard's `proxy_blocked` and `GET /api/v1/admin/holds/policy-blocks` list only what a repository's own settings refuse, so all three are empty under `record_only`. `GET /api/v1/security/proxy-blocked?would_block=true` applies each repository's severity gate to its recorded verdicts whatever its action, through the predicate the gate's other listings use, and the dashboard adds `proxy_would_block`, read from the same query as `proxy_blocked`.
Files with no usable verdict are not listed. The preview response counts them in `would_block_summary` (`no_verdict`, `stale_verdict`, `clean_graded_nothing`), because `fail_closed` scans them inline on the next pull and answers `423` if the scan is inconclusive. Without `would_block` the list's response is unchanged, and `proxy_blocked` and `policy_violations_blocked` keep their meaning. See `docs/security/scan-on-proxy.md`.
- **Agent Plugins repositories host AI coding-agent plugin bundles for Claude Code, Cursor, Codex, Copilot / VS Code and the Agent Plugins Standard** (#5251). A new hosted-only `agentplugins` format stores one zip per plugin version at `/agentplugins/{repo}/{name}/{version}/{name}-{version}.zip` (streamed `PUT`, optional `X-Checksum-Sha256`, `409` on an existing version) and generates `marketplace.json`, `claude-marketplace.json`, `cursor-`/`codex-`/`vscode-marketplace.json` and `{name}/index.json` from the database on every request, with `ETag`/`304`, the latest downloadable SemVer version per plugin in each document, and versions the download gate refuses (quarantine, holds, scan policies) left out. Claude Code adds a repository directly with `claude plugin marketplace add <url>/claude-marketplace.json`; a private repository is read either through `extraKnownMarketplaces` with an `Authorization` header, or with a read token in the path (`/agentplugins/t/{token}/{repo}/...`, `Cache-Control: private, no-store`), and archives are always streamed from the server's own origin so the configured headers reach them.
Publishing refuses a zip with no harness manifest, an unsafe or symlinked entry, an ambiguous plugin root, a non-SemVer version, a version with build metadata, or a manifest whose `name` or `version` differs from the path: Claude Code resolves a plugin's version from `plugin.json` first, so a stale manifest version would freeze every install. A deleted version can only be re-published with identical bytes. Remote and virtual repository creation, and an Agent Plugins repository keyed `t`, are refused with `400`. See `docs/agent-plugins.md`.
- **MCP registry: one set of gates on every write path, provenance in served entries, and generic API fields** (#5253). The admin edit (`PUT .../versions/{v}`) now runs the publish's `mcp_server_deny`, namespace-grant and lint gates and refreshes the version's lint, declared capabilities and catalog row; soft delete and MCPB bundle upload run the namespace write gate (and the deny list, for uploads). Remote ingest lints every stored version and records its declared capabilities, and every reported or observed capability snapshot has its tools linted onto the server version. `POST /validate` (`/v0` and `/v0.1`) works without a credential on a public registry, as `mcp-publisher validate` expects, and never resolves packages for a caller who cannot read every backing repository. A NuGet package pulled through a Remote whose upstream is a V2 feed now pins. `_meta["com.artifactkeeper.registry/source"]` gains `origin` (`hosted`, `remote`, `mirror`) and `upstreamVerified` (the upstream URL moves to `upstreamUrl`); `mcp_auto_approve_namespaces` entries are globs over the full server name (`io.github.modelcontextprotocol/*`); `/api/v1/packages` rows of MCP servers carry a server.json summary in `metadata`; `/api/v1/formats` advertises `repository_types`; MCP repository responses carry `mcp_caller_can_lock` and `mcp_caller_is_repo_admin`. See `docs/mcp-registry.md`.
- **MCP capability snapshots, tool diffs and an isolated introspection worker** (#5253). Each MCP server version now records what it advertises (tools, resources, prompts, instructions) with its provenance: declared in the server.json (`_meta["com.artifactkeeper/capabilities"]`) or an MCPB manifest, reported by CI or a gateway (`POST .../versions/{v}/capabilities`), or observed by the new `ak-mcp-introspector` worker. `GET .../capabilities` and `GET .../capabilities/diff` give the Tools tab and the approval queue their data, and a changed list under the same version raises a `tool_drift` finding (`MCP_TOOLS_DRIFT`) that the findings UI and scan policy treat like any other.
Probing is opt-in per repository (`mcp_probe_remotes`, `mcp_sandbox_introspection`, both off by default) and never happens in the backend: the separate `ak-mcp-introspector` binary claims jobs over the API with a dedicated `mcp:introspect` token, probes `remotes[]` over public addresses only (SSRF-guarded resolver, no redirects, 1 MiB cap, never `tools/call`, never the publisher's headers, legacy `initialize` fallback), and can run stdio packages installed from this instance's own repositories in a gVisor or microVM sandbox with placeholder secrets and a 120 s cap. Docker MCP Gateway and ToolHive inventories can be imported with `ak-mcp-introspector push`. See "Capabilities and introspection" in `docs/mcp-registry.md`.
- **MCP registries lint every server version and roll up its health** (#5253). Each publish and status change of an `mcp` repository is linted offline for secrets pasted into defaults, headers and URLs, credential-named inputs not marked `isSecret`, prompt-injection phrases and hidden characters in descriptions and declared tools, launcher arguments that re-route installs or escape the container (`--registry`, `--index-url`, `--privileged`, `-v /:/host`), suspicious remote URLs, OCI tag/digest and server-name label mismatches, MCPB bundles without a hash, and destructive or shell-like tool names. A `block` finding refuses the publish with `422 mcp_lint_blocked`; every other finding is stored as an ordinary scan finding (scan type `mcp_lint`) at a severity the scan policy reads (`review` = high, `warn` = low), so the findings UI, acknowledgements and the scan-policy gate apply unchanged. `mcp_require_oci_label: true` refuses an image whose stored config lacks a matching `io.modelcontextprotocol.server.name` label (`422 mcp_oci_label_mismatch`). `GET /api/v1/repositories/{key}/mcp/servers/{name}/versions/{version}/health` rolls the pinned packages' scan results, backing gates and lint verdict up to `ok`, `warning`, `critical` or `unknown`. See "Lint and health" in `docs/mcp-registry.md`.
- **MCP Registry repository format: host a private catalog of Model Context Protocol servers** (#5253). A Local or Staging `mcp` repository speaks the MCP registry API v0.1 (and `/v0`) at `/mcp/{repo}/`, so VS Code's MCP gallery (`chat.mcp.gallery.serviceUrl`), GitHub Copilot's private registry policy and `mcp-publisher` can use it in place of the public registry. server.json documents (schema 2025-12-11) are validated strictly, write-once per version, and listed, searched and paged with upstream's envelope; `PATCH .../status` deprecates or deletes a version.
By default (`mcp_package_policy: own-repos`) every package a server runs must already be in one of this instance's repositories, mapped per `registryType` with `mcp_package_repos`: the publish resolves it in the database (nothing is fetched), pins the exact version or image digest, and every read serves the Artifact Keeper coordinates (`registryBaseUrl` for npm, PyPI, NuGet and Cargo; `host/repo/path:tag@sha256:...` for OCI). A package from anywhere else is refused with `422`, as are `:latest` and tagless images. A pinned OCI tag that later moves keeps serving the pinned digest and is audited as `MCP_PIN_DRIFT_DETECTED`. A server version whose server.json or pinned package is quarantined, refused by scan policy (or by the stored proxy-scan verdict of the Remote it was pulled through), younger than that Remote's age gate, or gone is left out of every listing, and an exact read answers the same `404` as an unknown name (the reason only for callers who hold the repository), because VS Code keeps a server enabled on any other refusal. `open` is an explicit opt-out that stores packages verbatim. Remote and Virtual MCP repositories are refused until a later release. See `docs/mcp-registry.md`.
- **Remote MCP registries: an approval-gated allowlist populated from an upstream MCP registry** (#5253). A Remote `mcp` repository syncs the upstream's `/v0.1/servers` listing hourly (and on demand, `POST /api/v1/repositories/{key}/mcp/sync`) into a catalog the web UI browses, and serves on `/v0.1` only the names an administrator approved (`mcp_remote_posture: approval_required`, the default and instance-lockable; `allow_with_gates` is the opt-in relaxation). Names can be approved one by one, in bulk by full-name glob (`POST .../mcp/catalog/bulk-approve`, `io.github.acme/*`; a bare namespace is refused with `422`) or by id (`POST .../mcp/catalog/bulk-decide`, which also blocks), blocked in bulk by glob (`POST .../mcp/catalog/bulk-block`), with a `dryRun` preview of either glob decision, automatically on sync (`mcp_auto_approve_namespaces`), or as single `name@version` pins; `mcp_server_deny` always wins, on every repository type, and a hosted publish of a denied name is refused with `403 mcp_server_denied`. Approved versions are stored write-once with every package pinned at sync through the Remote's own `mcp_package_repos` (an upstream `latest` or tagless image is pinned to its digest with a warning, or hidden with `mcp_oci_floating_tag_policy: hide`). An upstream edit of an ingested version keeps the stored document, revokes that version's approval and re-queues it (`MCP_SERVER_CONTENT_DRIFT`); lookalike names are flagged for review at ingest; the Remote's age gate holds new server versions by first sighting; and any gate that cannot be evaluated answers `503 mcp_gate_unavailable` (fail closed, audited `MCP_GATE_UNAVAILABLE`). `mcp_remote_mode: mirror` ingests and indexes every upstream name for offline search while serving stays curated. The first sync asks upstream for deleted versions too, so a version already deleted upstream is recorded as `deleted`. Catalog items carry a cached risk `summary` (transports, package registry types, secret, package and remote counts, and the latest ingested version's lint counts), and `GET /api/v1/mcp/pending-summary` counts the approval queue across every Remote MCP repository the caller can read. See "Remote repositories" in `docs/mcp-registry.md`.
- **Virtual MCP registries, GitHub Actions login for `mcp-publisher`, and per-namespace publish grants** (#5253). A Virtual `mcp` repository merges its MCP members on the same `/mcp/{repo}/v0.1` routes: a version is answered by the first member (hosted members first, then Remotes, each by priority) that serves it under that member's own gates, a name held by any hosted member is never served from a Remote (dependency confusion), listings page across members with a composite cursor, and writes answer `405`. `mcp-publisher login github-oidc --registry https://ak/mcp/{repo}` works through `POST .../v0/auth/github-oidc`, bridged to the CI OIDC exchange; the other upstream logins answer `501`.
Permission rules accept `conditions.mcp_namespaces` (patterns such as `com.acme/*` or `io.github.{repository_owner}/*`, the latter filled from the CI token's claims), audited as `MCP_NAMESPACE_GRANTED`/`MCP_NAMESPACE_REVOKED`; with `mcp_require_namespace_grant` plain repository write no longer suffices. Administrators can record DNS TXT or HTTPS well-known namespace proofs (`/api/v1/admin/mcp/namespace-proofs`, SSRF-guarded, audited). Also new: `GET .../x/com.artifactkeeper/holds` lists held versions with their reason for repository holders, `GET .../openapi.yaml` serves the upstream API specification, and `MCP_DEFAULT_REPO` optionally serves one repository at the host's `/v0.1` and `/v0`. See `docs/mcp-registry.md`.
- **MCP registries resolve packages through Remote repositories, keep deleted versions reserved, validate, edit and soft-delete versions, and host MCPB bundles** (#5253). A server whose npm, PyPI, NuGet or Cargo package was pulled through a Remote backing repository now publishes against the Remote's proxy cache (pinned by sha256 and gated by the cache entry's hold, stored scan verdict and age gate); a package not pulled yet is filled through the Remote's own proxy path by a background resolver while the version is held as `resolving` (`202`), then pinned, or failed on a definitive refusal. Every `name@version` a registry ever stored stays reserved after a delete and a trash purge (`409 mcp_version_exists`). `POST /validate` runs the publish checks without writing (no credential needed; package resolution only for callers who can read the backing repositories). Repository administrators can replace a version's document (`PUT`), and `DELETE` soft-deletes a version. MCPB bundles are uploaded beside a version, their manifest is recorded and their `mcpb sign` signature verified strictly (a signer is `valid` only when its certificate fingerprint is listed in `MCP_TRUSTED_BUNDLE_SIGNERS`, otherwise `self_signed`; `mcp_require_bundle_signature` refuses unsigned and untrusted bundles, including bundles pinned through the `mcpb` mapping), and they resolve as `mcpb` packages served with ticketed download URLs. A by-digest delete of an OCI manifest a server version pins is refused, a Local OCI backing repository refuses images named on another host (`mcp_external_registry`), package names and versions match each backing format's equivalences (npm percent-encoding, NuGet `1.0` and `1.0.0`, Cargo `-` and `_`), and the age-gate `451` now carries `Cache-Control: no-store`.
- **Remote repositories can be declared cache-only, serving what is cached and never contacting upstream** (#5282). An operator who seeded a Remote and then disconnected the instance had no way to say so: a miss still tried the upstream (a 60-second stall when a firewall drops packets) and an expired index entry still revalidated. The closest workaround, an upstream filter excluding everything, hid seeded index files once their TTL passed and answered with the filter's reason. `PUT /api/v1/repositories/{key}/upstream-mode` with `{"upstream_mode":"cache_only"}` (Remote-only, repository admin) now makes the Remote serve every cached entry, fresh or stale, with no revalidation; stale serves carry `Warning: 110` and `X-AK-Stale-Since`. A miss answers `404 not in cache; repository '<key>' is cache-only` at once, without upstream contact, and is counted as `ak_proxy_cache_lookups_total{result="cache_only_miss"}`. The mode is enforced at the shared upstream gate, so the npm `/-/` passthroughs and `npm audit`, the curation upstream sync and the `test-upstream` probe honour it too, and a virtual repository skips a cache-only member that lacks the path. Each change is recorded as `REPOSITORY_UPDATED` with `upstream_mode_change`. `PROXY_UPSTREAM_MODE=cache_only` puts every Remote in cache-only mode at once. See `docs/proxy-cache-freshness.md`.
- **A Remote repository's proxy cache can be seeded from a list of paths** (#5284). `POST /api/v1/repositories/{key}/cache/seed` with `{"kind": "paths", "content": [...]}` (a JSON array or newline-separated text, at most 10,000 paths) starts a background job and answers `202`; `GET /api/v1/repositories/{key}/cache/seed/{job_id}` returns its inventory, one row per path in request order: `cached`, `already_cached` (a fresh entry existed and the upstream was not contacted), `refused` (with the reason) or `failed` (with the status a client would have seen), with size and SHA-256 where known. Each path is fetched through the same proxy entry as a client download, so the upstream filter, origin policy, Package Age Policy, quota and proxy-cache cap, and the scan-on-proxy refusal of the generic route apply to a seed too. Concurrency (default 4, at most 16) and an optional `max_total_bytes` bound the job. Jobs are stored and resumed by any replica after a restart; each completed job writes one `CACHE_SEEDED` audit row. Requires `write:repositories` and repository `admin`. The typed manifests (`pypi-requirements`, `npm-lockfile`, `oci-images`, `hf-models`) and seeding a Virtual repository answer `400` for now. See `docs/cache-seed.md`.
- **Model files uploaded to `huggingface` and `mlmodel` repositories are now scanned for executable pickle payloads and malformed headers** (#5285, #3013). A new in-process `model` scanner, which needs no sidecar or network access, walks the pickle opcode stream of `.bin`, `.pt`, `.pth`, `.ckpt`, `.pkl`, `.pickle` and `.joblib` files (and the `data.pkl` of a zip-wrapped PyTorch archive) without executing it, and classifies every `GLOBAL` / `STACK_GLOBAL` / `INST` import: a code-execution primitive such as `os.system`, `subprocess.*` or `builtins.eval` is a critical `malicious` finding, and any import outside the weights-only allowlist is a `policy` finding naming it. It also reports malformed, truncated or overlapping safetensors headers and malformed GGUF headers, implausible counts and oversized chat templates. Every finding carries a stable rule id (for example `model.pickle.dangerous-global`) in `affected_component`. Parsing is bounded throughout and never reads tensor data. A `.safetensors` file stored in a `huggingface` repository through the generic or chunked upload API now also records its header summary in `artifact_metadata`, as `mlmodel` uploads already did. Proxied files are not judged yet. See `docs/security/model-file-scanning.md`.
- **The audit log can now record every request the proxy sends to an upstream** (#5286). With `AUDIT_UPSTREAM_FETCHES=all`, each cache-miss fetch, revalidation and OCI token exchange writes an `UPSTREAM_FETCHED` audit entry with the repository, caller, method, host, port, path (never credentials or the query string), status, result (`ok`, `not_modified`, `not_found`, `auth_challenge`, `error`, `timeout`), announced size, duration and the principal of the client request that caused it. A cache hit writes nothing, and a single-flight fill writes one entry however many clients wait on it. `AUDIT_UPSTREAM_FETCHES=errors` records only failed requests; the default `off` records nothing. The entry is written by a bounded background task, so it adds no latency to the request and a failed write never fails it. This lets an air-gapped deployment list what came in during its seed window, and show that nothing was fetched after the cut, from AK's own audit log. See docs/audit-events.md.
- **The scanner-adapter runs offline and imports a trivy vulnerability DB over HTTP, with no restart** (#5287). On a disconnected install the adapter tried to download the trivy DB at startup and stayed not-ready, and the only way to seed its cache was shell access to the volume. `SCANNER_OFFLINE=true` now skips every DB and Java-DB download and scans with `--offline-scan`. `PUT /api/v1/vulndb` takes the `db.tar.gz` that `oras pull ghcr.io/aquasecurity/trivy-db:2` produces, checks that it holds exactly `trivy.db` and `metadata.json` with the expected schema version, refuses a DB older than the installed one unless `?force=true`, swaps it in atomically without disturbing in-flight scans, and makes the adapter ready. `GET /api/v1/vulndb` reports the installed DB's build date and age. Both routes require the new `SCANNER_ADAPTER_TOKEN` bearer token and are off without it. Connected installs behave as before, except that `SCANNER_TRIVY_SKIP_DB_UPDATE` now also applies to filesystem scans. `docker-compose.yml` gains commented air-gapped settings for the adapter and the trivy server. Scanner-adapter 1.7.0.
- **One switch pauses every scheduled job that contacts an upstream** (#5288). Set `UPSTREAM_JOBS_PAUSED=true`, or send `PATCH /api/v1/admin/settings/system` with `{"upstream_jobs_paused": true}` (no restart, audited as `SETTING_CHANGED`). While it is on, the curation upstream sync and its popularity lookups, the environment advisory re-evaluation (OSV), the npm replication-feed consumer, peer replication transfers and the Remote MCP registry sync skip each tick before any network work. The backend logs the paused jobs once at startup, and each job logs when it pauses or resumes. Turning the switch off resumes each job on its next tick from where it stopped. The env var pins the switch on, and an unreadable setting counts as paused. `GET /api/v1/admin/settings/system` and the npm feed status report the state. Local jobs (GC, lifecycle, retention, metrics) and request-time proxy fetches are unaffected. The default is off. `CURATION_POPULARITY_PYPI_URL` and `CURATION_POPULARITY_NPM_URL` point the popularity lookups at an internal mirror, or turn them `off` so each lookup answers unknown at once instead of waiting out a connect timeout. See `docs/operations/disconnected-operation.md`.
- **Agent Plugins Remote repositories proxy another Artifact Keeper's hosted Agent Plugins repository** (#5289). Creating an `agentplugins` repository with `repo_type: "remote"` and `upstream_url` set to the central repository (`https://central.example.com/agentplugins/<key>`, or the `/agentplugins/t/<token>/<key>` form) is now accepted. The Remote serves `marketplace.json`, the per-harness marketplaces and `{name}/index.json` fetched through the proxy cache, with every archive URL rewritten to point at the Remote and every `sha256` kept exactly as the upstream lists it, so `claude plugin install` and `ak agent plugins install` verify the same bytes on every instance. Archive downloads are checked against the `sha256` in the upstream index before they are cached or served: a mismatch is answered with `502` and never cached, and a matching archive is cached as immutable and served without contacting the upstream again. Upstream credentials, the upstream filter, the origin policy and the proxy cache settings apply as for every other Remote. Virtual Agent Plugins repositories are still refused, now with a message naming the follow-up; hosted repositories are unchanged.
- **The Hugging Face guide now explains how to restrict which models a Remote can fetch, and a new page documents upstream filters in general** (#5290, #840). A Hugging Face Remote requests model info as `api/models/<org>/<name>...` and files as `<org>/<name>/resolve/...`, so a filter written for one shape (`^acme/`, or `^api/models/acme/`) silently breaks `snapshot_download` and `hf download` on their first call. `docs/huggingface.md` gives a pattern that matches both shapes, `^(api/models/)?(org-a|org-b)/[^/]+(/|$)`, and explains anchoring, look-alike organisations, case and how to check the result. `docs/upstream-filter.md` covers the subject rules for every format, the npm and Hugging Face two-shape cases, and how already-cached content behaves. Unit tests read the documented patterns out of both pages and check them against the paths the Hugging Face handler sends upstream.
The guide also says to set the cache TTL when the Remote is created (a raised TTL applies to an entry from its next revalidation), that a user outside the group a private repository is granted to gets 404 rather than 403, and that `snapshot_download` in `huggingface_hub` 2.x lists the tree recursively instead of using the model-info `siblings` (a Remote forwards that query since #5321).
- **An administrator can export the instance's repositories as one config document, and plan or apply it on another instance** (#5292). `GET /api/v1/admin/config/export[?repositories=a,b]` returns a versioned JSON document (`schema_version: 1`) with every repository keyed by `key`, sorted, with its per-repository settings: name, description, format, type, visibility, upstream URL, quota, promotion-only and versioning flags, custom User-Agent, curation flags, virtual members with priorities, and for remotes the cache TTL, `on_upstream_not_found`, the stale-if-error grace override, the upstream filter, routing rules, the egress proxy, upstream auth and the npm scope policy. No credential is exported: upstream passwords and tokens, and userinfo in upstream and egress proxy URLs, appear as `{"secret_ref": "<kind>/<key>"}` placeholders. `POST /api/v1/admin/config/import?mode=plan|apply` takes the document plus a `secrets` map: `plan` reports what would be created, updated, left unchanged and what the target has that the document does not mention, and lists missing secrets and changes an import cannot make; `apply` creates and updates through the regular repository routes (so validation, SSRF checks and per-repository audit rows are the usual ones), non-virtual repositories before virtual memberships, is idempotent, never deletes (`prune=true` answers 400 for now) and records one `CONFIG_IMPORTED` audit row. A `schema_version` other than 1 is refused. Scan policies, curation rules, the age gate, lifecycle, promotion and sync policies and permissions are not covered yet. See `docs/config-as-code.md`; a JSON Schema is at `docs/schemas/config-document.v1.json`.
- **An MCP Remote can be seeded with named server versions without a full catalog sync** (#5408). A Remote learned a name only from its sync, which pages the upstream registry in name order: the official registry is over 2,000 pages, so a name late in the alphabet took several passes (hours) to reach the catalog, and `POST .../mcp/catalog/pins` answered `404` for it until then. `POST /api/v1/repositories/{key}/mcp/seed` takes a list of up to 500 `{name, version}`, answers `202` with a `statusUrl`, and seeds them in the background. For each name it fetches the upstream's version list (`/v0.1/servers/{name}/versions`) through the sync's outbound path (SSRF guard, credentials, egress proxy, upstream gate, `UPSTREAM_FETCHED`). It records the name in the catalog as a sync page would (deny list, lookalike check, auto-approval), then pins and ingests the version as `catalog/pins` does. `GET .../mcp/seed/{jobId}` reports each item as `pinned`, `already_pinned`, `refused` (denied, blocked or review-flagged name, unlisted version) or `failed`. The request is refused with `409` while scheduled upstream jobs are paused or the Remote is cache-only. This is the air-gap workflow "seed exactly these servers"; see `docs/mcp-registry.md` and `docs/air-gapped.md`.
### Changed
- **A remote repository's upstream filter now also gates the curation upstream sync and the admin `test-upstream` probe** (#840). The filter (`include_patterns` / `exclude_patterns`) already gated every proxy fetch and the npm `/-/` and audit passthroughs, but curation sync walked its upstream index, and downloaded PyPI distributions for attestation verification, without consulting it, and the admin connectivity probe sent its HEAD regardless. The RPM (`repomd.xml`, `repomd.xml.asc`, `primary.xml`) and Debian (`Packages.gz`, `Packages`) index fetches, and the PyPI provenance, JSON and distribution fetches, now match the filter against the same subject the proxy would use (the upstream-relative path, or the full URL for a fetch outside the upstream base): a refused fetch is skipped without a request, the staging repo is left unsynced rather than failed and stays unstamped so a filter change applies on the next tick, and a refused attestation lookup leaves the package unverified instead of falsely verified. `POST /api/v1/repositories/{key}/test-upstream` requests the upstream root (subject: the empty path), so an include-only filter refuses it with 404 unless a pattern admits the root. A package the filter refuses is also not ingested into the curation catalog even when the index listing it is admitted, so the catalog never advertises packages the remote will not serve (existing catalog rows are left as they are). A repository with no filter is unaffected. The npm upstream change feed remains outside the filter: it is a single global subscription, not a per-repository upstream fetch.
- **A remote repository's upstream filter now also covers the npm `/-/` passthroughs and npm audit** (#840). The 1.11.0 upstream filter (`include_patterns` / `exclude_patterns`) gated every fetch made through the proxy cache, but the npm `/-/` namespace (search, attestations, ping and the rest) and `npm audit` called the upstream directly. They now match the filter against the path they would send (`-/v1/search?text=...`, `-/npm/v1/security/advisories/bulk`): a refused `/-/` request answers 404 and a virtual repository skips that member, and a refused audit answers the empty advisories body, all without contacting the upstream. A repository with no filter is unaffected; one whose filter lists only `include_patterns` for package names now also refuses search, ping and audit unless a pattern admits them, for example `^-/`.
- **OCI manifest reads by digest, the referrers API and storage GC now key on the `oci_manifests` existence table** (#1683). On a hosted repository, `GET`, `HEAD` and `DELETE /v2/<name>/manifests/<digest>` resolve through the repository's record of the manifest instead of inferring it from tags and edge rows, and the #1681 content-addressable fallback is gone. The untagged child of a tagged index keeps pulling by digest after its own tag is deleted. The referrers API lists a referrer while the repository records the referrer itself, whether or not its subject exists. Deleting the last tag of an index that a tagged index of indexes still references no longer drops that index's own child edges, which stranded the grandchildren while the top index still served them. Storage GC keeps a manifest object, and the blob refs of an index's child, while the manifest is recorded, so the child of an untagged index that still pulls by digest is no longer reclaimed. Upgrading from 1.11: a rolling upgrade is supported; while 1.11 replicas still run, keep `BLOB_GC_ENABLED` unset and `GC_TRASH_RETENTION_DAYS` above the rollout length on them, as `docs/operations/oci-manifests-reconcile.md` explains with a table of what a mixed 1.11/1.12 fleet answers.
- **Storage GC no longer holds a database transaction across object-store deletes, and a pass can be budgeted** (#2524). Every sweep (soft-deleted artifact keys, OCI blob sweep, objects left by repository deletes, abandoned OCI uploads, orphaned Maven objects) now claims a candidate in a short transaction that removes its rows and records the delete in flight, commits, and only then deletes the object, with up to `STORAGE_GC_DELETE_CONCURRENCY` (default 4) deletes at once. A failed or interrupted delete stays recorded and is retried by a later pass; a key that gained a reference in between is kept. The check before each delete also counts revision history, so the newest revision of a trashed artifact in a versioning-enabled repository keeps its bytes instead of being left pointing at a deleted object. An OCI push of a digest whose blob is being deleted at that moment is answered 503 ("retry the upload") instead of committing a row for deleted bytes. New per-pass budgets `STORAGE_GC_MAX_KEYS_PER_RUN`, `STORAGE_GC_MAX_BYTES_PER_RUN` and `STORAGE_GC_MAX_RUN_SECS` (default 0, unlimited) stop a pass early and report it as `budget_exhausted` in the GC result, and new `ak_storage_gc_*` metrics count deletes, failures, skips, budget stops and the retry backlog. The GC result's `bytes_freed` now counts the soft-deleted artifact keys it reclaims; it read 0 for them before because the per-key byte sum was decoded with the wrong type. See `docs/operations/storage-gc.md`.
- **Chunked upload completion no longer needs a full-size local scratch file on object storage** (#3916). On S3-, GCS- and Azure-backed repositories, and for any format that does not need the body as a local file, the staged chunks now stream in order into one `upload-staging/<session-id>/assembled` object, are verified there, and are promoted to the content-addressed key with a server-side copy. Local `.uploads` assembly remains only for filesystem repositories, backends without native streaming, and RPM or Incus/Debian finalizes that parse the body from disk.
- **Hosted conda channels store their CEP-16 shards when packages change, and a shard request is a lookup by address** (#4173, #4172). Every shard request used to list the whole repository and rebuild shard after shard until one matched the requested hash, so resolving a 3,000-package environment cost O(names) catalog walks per shard. Shards now live in the new `conda_shard_index` table: database triggers mark the package names a write touches (upload, withdrawal, delete, quarantine change, metadata edit) in the writer's own transaction, only those names are rebuilt, and readers rebuild anything still marked before they serve. A stored shard is checked against its address before it is served. Channels that existed before the upgrade are backfilled in the background in resumable batches; until a channel is done its shards are built per request as before. See `docs/operations/conda-hosted-repodata.md`.
- **Hosted conda repodata is rendered once per change and served from the metadata revision cache** (#4174, #4172). `repodata.json`, `.zst`, `.bz2`, `current_repodata.json`, `run_exports.json`, the JLAP bootstrap, the shard index and `channeldata.json` were rebuilt from a full catalog scan on every request. They are now rendered together once per subdir revision (which moves with every upload, withdrawal, delete, quarantine change and lapsed hold) and kept, so concurrent cold requests share one render and warm requests read no catalog rows. Virtual channels key their merges on their hosted members' revisions instead of reading every hosted record per request. Renders report through `ak_repo_metadata_regenerations_total{format="conda"}`, and other replicas are told on a notification channel of their own (`ak_conda_repodata_invalidation_v1`), which 1.11.x replicas do not listen on. The served bytes are unchanged.
- **Hosted conda channels serve a real JLAP, so conda and rattler update a cached index with a patch instead of downloading it again** (#4175). The JLAP was served only as `repodata.json.jlap`, while conda and rattler request `repodata.jlap` (the name conda-forge serves), and it named only the current index hash, so JLAP clients always fetched the whole `repodata.json`. It is now served under both names. Each new revision of a subdir is now recorded with its BLAKE2b-256, an RFC 6902 patch from the previous document is appended to a stored JLAP whose checksum chain continues from the stored bytes, and range requests from the client's saved position return only the new lines (`416` past the end). The last `AK_CONDA_JLAP_REVISIONS` (default 8) documents are kept per subdir (migration 305). `repodata.json.zst` and `.bz2` now decompress to exactly the bytes `repodata.json` serves, which the JLAP chain hashes; before, they held a compact rendering. Clients that read them see the same records, but the files are slightly larger (measured on a 30,000-record channel and on a 387,000-record one: about 4 % for `.zst`, under 1 % for `.bz2`), and decompress to about a third more bytes. The JLAP is recorded with the revision before its documents are served, so a withdrawal is absent from the JLAP tail as soon as it is absent from `repodata.json`.
- **Deleting a repository answers 202 with a job instead of running the delete in the request** (#4311). `DELETE /api/v1/repositories/{key}` now records a job, renames the repository out of the way at once (its key no longer resolves, it leaves the repository listing and the key can be reused) and answers `202 Accepted` with the job and a `Location` of `GET /api/v1/repository-deletions/{id}`, which reports progress in the shape of a migration job. A background worker deletes the rows in batches of 500 and reclaims the storage under a cluster-wide row claim, so one replica runs each job and another resumes it if that replica dies. Before, the key collection and the database cascade ran inside the request, and a repository of 10,000 or more artifacts behind a proxy timed out (504 or `SERVICE_UNAVAILABLE`). The repository delete also hands its objects to the GC journal and re-checks references before each delete. Clients that expected `200` from the delete should accept `202`. The `repository.deleted` event is emitted when the job completes. Migration 304 adds `repository_delete_jobs`. See `docs/operations/repository-delete.md`.
- **Hosted OCI blob pulls verify the stored bytes against their digest** (Part of #4323). A whole-blob pull from a hosted repository, or from a Virtual's hosted member, whose stored object no longer hashes to its digest is refused with `502 STORED_OBJECT_INTEGRITY` and no body when the blob is at most 4 MiB, and cut short before its last chunk when it is larger, as the generic download route already does. Range pulls and blobs streamed from an upstream are served as before. `DOWNLOAD_VERIFY_CHECKSUMS=false` turns the check off.
- **Stored bytes are verified on more download routes, and a small corrupted object is answered `502` instead of a cut-short `200`** (#4323). The serve-time SHA-256 check that #3919 added to the generic download route now also covers every virtual-repository member serve, the format routes that read hosted artifacts through the shared fetch (Terraform, Go, Composer, pacman, VS Code, sbt and others), and Incus image downloads (checked against the digest the response advertises). An object up to 4 MiB is checked before any byte is sent, so a mismatch is `502` with `code: STORED_OBJECT_INTEGRITY` and nothing of the bad object; a larger one keeps streaming and is aborted before its last bytes. A virtual repository whose member fails the check answers 502 rather than falling through to another member's bytes. `DOWNLOAD_VERIFY_CHECKSUMS=false` still turns verification off everywhere. OCI blob pulls and the Maven, npm, PyPI, Cargo, NuGet and Helm hosted downloads read storage through their own paths and are not covered yet. See `docs/operations/download-integrity.md`.
- **A proxy cache hit on a filesystem cache reads the body alongside its metadata sidecar instead of after it** (#4331). A first-touch hit used to make three sequential storage round trips: read the sidecar, measure the body, open it. On network storage (NFS) each costs tens of milliseconds, which is most of the latency gap a Maven build sees against local disk. The body is now measured and opened at the same time as the sidecar is read (the streaming path), and the buffered path reads the body alongside the sidecar too; the size and checksum checks against the sidecar are unchanged, and a body opened before a concurrent re-publish of the entry is reopened. S3, GCS and Azure caches keep the sequential reads, since opening a body there starts transferring it. `docs/operations/proxy-cache-ha.md` answers the question in the issue about running remote repositories on several replicas.
- **Repository security grades report what was assessed instead of an F floor** (#4336). A failed scan (#2167) or an archive no scanner could catalog (#4036, #4154) used to force a repository's grade letter to F while its score stayed 100, which mixed "known bad" with "not assessed" and changed no gate. Repository scores now carry `grade_status` (`rated`, `incomplete` or `unrated`) and `coverage` (artifacts, scanned, cataloged, partial, failed, uncataloged, `coverage_pct`); `grade` is the letter over the assessed artifacts and is `null` when the repository is `unrated` (nothing assessed, including no scans at all). For a `rated` repository the letter is exactly the one earlier releases showed. Each scan row carries `grade_status` and `grade_status_reason` in the same vocabulary, so a `partial` scan reads `incomplete` with its reason. The security dashboard adds `repos_rated`, `repos_incomplete` and `repos_unrated`; `repos_grade_a` and `repos_grade_f` now count shown letters, so a floored repository is no longer counted as F. Deleting an artifact now recalculates its repository's score, so removing the archive that could not be cataloged makes the repository `rated` without waiting for the next scan.
Client impact: `grade` on `GET /api/v1/security/scores`, `GET /api/v1/repositories/{key}/security` and the external-findings ingest response is now nullable and never the F floor. Clients that treated an F as "not assessed" should read `grade_status`. Migration 303 adds nullable columns only; during a rolling upgrade 1.11 replicas keep reading and writing the `grade` column exactly as before (it still carries the floor), and a score a 1.11 replica wrote is read by 1.12 with its status derived from the stored flags until the next recalculation.
- **A download refused by a scan-policy hold now answers 403, not 409** (#4510). A scan policy that refused an artifact after a scan stored the refusal as a quarantine hold, so every download route answered the quarantine `409` ("pending security review"), and a client or CI job that retries on `409` waited for a release that waiting never brings. Hosted holds now record where they came from (`artifacts.quarantine_source`, migration 299: `upload`, `admin` or `scan_policy`), and the gate answers a scan-policy hold with the same `403` and neutral body the scan-policy gate already used. A hold that recorded no source (written before 1.12.0, by a 1.11 replica during a rolling upgrade, or by the scanner) answers `403` exactly when a scan policy refuses the artifact now, so the change needs no backfill. Upload-period and administrator holds still answer `409`. **Client impact:** treat `403` on a held artifact as final until the findings are resolved; see docs/security/download-holds.md. During a rolling upgrade, replicas still on 1.11 keep answering `409` for these holds until they are upgraded.
- **The security blast-radius report keeps proxy pulls attributed after the cache entry is purged** (#4542). Proxy downloads were attributed to a vulnerable digest only through the live cache entry they were served from, so evicting or purging the compromised object, the usual remediation, also erased who had pulled it. Each `proxy_download_statistics` row now records the `checksum_sha256` that was served (a cache hit records the cached digest, a cache fill fills the rows of the serves it fed when it commits), and the report counts pulls by that digest; affected digests and repositories now include ones that were pulled and later purged. Rows from before the upgrade are still attributed through their cache entry while it exists. Migrations 295 to 297 add the column and build its index online (`CREATE INDEX CONCURRENTLY`).
- **Hosted RPM repodata is built in the background and served from storage** (#4544, #5067). A hosted RPM repository used to render a root's `repomd.xml`, `primary.xml.gz`, `filelists.xml.gz` and `other.xml.gz` on the first request after every change, on every replica: about 35 s of CPU for a 2,500-package root, and nearly every request during a bulk import. Now a change marks the root dirty (migration 308, triggers on `artifacts` and `artifact_metadata`), and a background builder renders it once the root has been quiet for 3 s, or after at most 60 s of continuous writes (`RPM_REPODATA_BUILD_QUIET_SECS`, `RPM_REPODATA_BUILD_MAX_DELAY_SECS`). One replica builds each root under a cluster lease. The documents are stored under `repodata/<repository id>/<root>/<revision>/` in the repository's storage, and every replica serves them from there; `repomd.xml.asc` is still signed per request, so a key rotation applies at once. The stored bytes are the ones the on-request render produces for the same packages. A root that has not been built yet (every root right after the upgrade) renders on request as before until its first build. The previous revision is kept so a download in flight during a swap completes, and older ones are deleted. **Behaviour change:** right after an upload or a delete, a hosted repository's repodata can describe the previous state for a few seconds, until the builder catches up. Virtual repositories, curated publications (`@N`) and remote repositories are unchanged.
- **A hosted conda subdir's `repodata.json.bz2` is rendered on its first request and kept for the revision** (#4882). bzip2 is the slowest encoding by far (1.5 s for a 26 MB document, 26 s for conda-forge `linux-64`) and the one clients ask for least, so it is no longer rendered with every revision; the first `.bz2` request renders it from the kept `.zst` document under the regeneration bound, and later ones are served the kept bytes. The file is unchanged: single-stream bzip2 at level 9. A lower level was measured to save under a fifth of the time for a tenth more bytes, and parallel multistream output was not adopted: pixi 0.81 (rattler), which reads `.bz2` when a channel has no `.zst`, fails on a multistream file, while conda 26.9, mamba 2.9 and micromamba 2.0 never request `.bz2`.
- **The admin repair endpoints have an operator page** (#4918). `docs/operations/admin-repair-endpoints.md` covers `POST /api/v1/admin/repositories/{key}/reindex-storage` (Maven and Gradle ghost objects) and `POST /api/v1/admin/repositories/{key}/backfill-debian` (hosted Debian packages migrated without a control document): when to run each, what they read and write, the admin-only access, per-repository lock and audit actions (`STORAGE_REINDEX_RUN`, `DEBIAN_BACKFILL_RUN`), the `limit` cap and the backfill's `after` / `next_after` continuation with curl loops, every report field, including the rows the backfill reports but does not move, and the cost on a very large repository. Documentation only; no behaviour change.
- **The OCI manifest existence table is now reconciled at startup and every six hours** (#4955, #1683). `oci_manifests` rows were written only by the push and delete handlers, so a tag removed by the lifecycle tag prune, by the startup orphan-tag reconciliation or by a 1.11.x replica during a rolling upgrade left a row for a manifest that answers 404 by digest, and a push served by a pre-268 replica left none. A reconcile pass now adds the missing rows and removes rows for manifests with no tag, no live index edge and no live `artifacts` row, re-checking each under its row lock. It runs at startup (replacing the one-shot backfill, which it includes) and every `OCI_MANIFESTS_RECONCILE_INTERVAL_SECS` (default 21600, `0` for startup only), and reports `ak_oci_manifests_reconcile_*` metrics. Nothing reads the table yet; this is the prerequisite for moving manifest reads onto it. See `docs/operations/oci-manifests-reconcile.md`.
- **The scan-on-proxy guide now says how the repository `artifacts` routes behave for a scanning Remote** (#4958). `docs/security/scan-on-proxy.md` gains a section covering the formats gated in 1.12.0 (Go, RubyGems, Composer, Pub, Conan, CocoaPods, Swift), which behave as the earlier formats do: the `artifacts` listing and the `artifacts/*path` metadata route serve no bytes and run no gate, the generic download route refuses proxied files with `scan_on_proxy_route_required`, the format route gates them, and an `artifacts` row a Remote holds is served from storage without a scan on both download routes. Documentation only; no behaviour change.
- **A Maven deploy into an S3-, GCS- or Azure-backed repository streams to object storage instead of a local scratch file** (#4990, #3916). `PUT /maven/{key}/{path}` (jars, POMs, checksum sidecars, `maven-metadata.xml`) now stages its body on the repository's backend with bounded memory, computing SHA-256, SHA-1 and MD5 on the way, and copies it server-side to its Maven key inside the publish lock, as the generic upload routes have done since 1.11.0. A multi-GiB jar no longer needs local disk of its size on the replica that receives it, and a full or read-only `AK_UPLOAD_STAGING_DIR` no longer fails the deploy. Filesystem repositories keep the local spool. Whether uploads stage on the backend is now decided by the backend's native streaming capability rather than its name. An upload that wrote its content-addressed object but did not commit its row (a conditional upload that lost its compare-and-swap after writing, a quota denial) now hands that object to storage GC instead of leaving it with no record.
- **An asynchronous chunked completion holds a ten-minute lease instead of six hours** (#5002). `Prefer: respond-async` now claims the session with a shorter completion lease and renews it with the same TTL while the detached commit is alive. A replica that restarts mid-commit no longer strands the session in `committing` for six hours: the lease lapses after ten minutes without a renewal and the same completion request can be retried with the staged chunks still present. The in-request completion keeps the six-hour lease, and a live renewal still blocks a duplicate completion and fences a stale owner.
- **A chunked upload into a format without a native finalize now dispatches its format handler, and every format metadata document is written in the artifact transaction** (#5002). Completion previously ran a format parser only for Maven, Gradle, npm, PyPI, RPM, Incus and Debian; other formats stayed generic even though their handler derives metadata from the upload path. The completion now calls the handler's `parse_metadata` with an empty body and records its document, while paths a native route declines and the generic handler keep the old behaviour. The document is written in the same transaction as the `artifacts` row, so a crash cannot leave an indexed artifact without its metadata; catalog rows, RPM scriptlet analysis, cache invalidation and the Debian/RPM auxiliary tables remain after the commit. See `docs/chunked-uploads.md`.
- **Format-native single-PUT uploads into S3-, GCS- and Azure-backed repositories no longer spool the whole artifact to local scratch** (#5002). NuGet, PyPI, Terraform, JetBrains, Swift, Git LFS, Helm, Ansible, Chef and Pub now stage the request body on the repository's own backend, computing SHA-256, SHA-1 and MD5 in the same pass, and promote it with a server-side copy (NuGet and PyPI through the content-addressed upload path). A route that has to open the archive to parse it — a NuGet `.nuspec`, a wheel or sdist, a Terraform module archive or provider manifest, a Swift manifest fallback, a Helm `Chart.yaml`, an Ansible collection manifest, a Pub `pubspec.yaml` — downloads the staged object to one temporary file just for that read and removes it. Filesystem repositories, backends without native streaming, and formats whose processing reads the body as a file keep the local spool, as #4990 requires. See `docs/operations/upload-scratch-disk.md`.
- **A remote Terraform repository's network mirror redirects cached provider archives to presigned object-storage URLs** (#5015). With `PRESIGNED_DOWNLOADS_ENABLED` and a backend that can sign, a fresh cache hit no longer streams the archive through the backend; a miss, a `HEAD` and backends that cannot sign still stream.
- **Every outbound HTTP client now sends `User-Agent: artifact-keeper/<version>`** (#5110). Follow-up to #5036, which covered only the SSO/OIDC client. The upstream/proxy, webhook, internal service, large-object and server-side-copy clients, the S3 `object_store` client, and the clients for cloud metadata endpoints, presigned storage URLs and AWS token fetches sent none, which web application firewalls (for example the AWS WAF `NoUserAgent_HEADER` rule) commonly refuse or rate-limit. A repository's custom User-Agent still takes precedence on proxied requests.
- **Hugging Face files fetched by commit sha are cached as immutable instead of being revalidated against the Hub every five minutes** (#5283). The proxy cache classifier had no Hugging Face rules, and the Remote download path handed it a generic format anyway, so every proxied Hugging Face path got the conservative 300-second mutable TTL. A multi-gigabyte `model.safetensors` that `snapshot_download` fetched as `<model>/resolve/<sha>/...` was revalidated every five minutes after first use, although content addressed by a commit can never change. Now `<model>/resolve/<40-hex sha>/...`, and model info and tree listings at a commit sha, are immutable; branch, tag and `refs/pr/N` paths and model info without a revision stay mutable with the usual TTL. Model ids of one or two segments are both recognised, and a path that parses both ways is immutable only if every parse says so. Hosted Hugging Face coordinates are unaffected: a re-publish of the same file still answers 409.
- **Hosted Hugging Face uploads stream to storage instead of being held in memory** (#5291). `POST /huggingface/{repo}/api/models/{id}/upload/{rev}` took the whole body as one in-memory buffer, hashed it, and only then wrote it, so a 5 GB GGUF needed 5 GB of backend heap per concurrent upload and one large upload could OOM a pod sized for normal traffic. The body now streams through the shared staging path the Git LFS, agent-plugin and Maven uploads use (local scratch, or the repository's own object store when it streams natively), with SHA-256 computed as it arrives, and is then written to its final key. A body over `MAX_UPLOAD_SIZE` is refused with 413, up front from `Content-Length` or as soon as the limit is crossed, and leaves no artifact row, stored object or staging file. A re-publish to a path that already holds a file is now refused with 409 before its body is received. Status codes, the stored artifact, catalog registration, push replication, the scan-on-upload hook and the response are otherwise unchanged.
- **A Go Remote now caches the checksum database it answers under `sumdb/<host>/...`, so `go` verifies seeded modules with the checksum database out of reach** (#5344). The route used to forward every request to `sum.golang.org` uncached, so a module missing from `go.sum` failed verification offline even when the module itself was cached. Answers now go through the Remote's proxy cache: tiles and `lookup/<module>@<version>` are immutable entries, and the signed tree head `latest` is mutable (revalidated after the cache TTL, served stale within the stale-if-error grace when the checksum database is down, and served as it is in cache-only mode). The upstream filter (matched against the absolute `https://<host>/...` URL) and cache-only mode apply as for other Remote fetches, the fetches are recorded as `UPSTREAM_FETCHED` audit events, and a Virtual repository caches through its Remote members. The host allowlist is unchanged and the Remote's upstream credentials are never sent to the checksum database. On Remote and Virtual repositories a sumdb path containing `?`, `#`, `;`, `\`, a stray `%` or a control character is refused with `400`. A Hosted Go repository keeps the uncached pass-through. See "Format notes" in `docs/air-gapped.md`.
- **The MCP registry, air-gapped, Hugging Face and disconnected-operation pages now match the code on main** (#5369). `docs/mcp-registry.md` said an anonymous read of a private registry gets the plain `404`. It actually gets `401 mcp_unauthorized` with the `WWW-Authenticate` challenge from the repository visibility middleware; only an authenticated caller with no grant gets the `404`. The page also said a pinned OCI digest is freed by trashing the server version's artifact through the artifact API, which the generic artifact `DELETE` has refused on MCP repositories since #5257. It now says plainly that a digest stays pinned unless the whole MCP repository is deleted, and links #5368, which proposes an admin path. `docs/air-gapped.md` now describes the merged cache-only Remote mode (`PUT /api/v1/repositories/{key}/upstream-mode`, `PROXY_UPSTREAM_MODE`), cache seeding (`POST /api/v1/repositories/{key}/cache/seed`, `paths` only), the scanner-adapter's `SCANNER_OFFLINE` and `PUT`/`GET /api/v1/vulndb` (with `SCANNER_ADAPTER_TOKEN`), the `UPSTREAM_JOBS_PAUSED` switch and its runtime setting, `UPSTREAM_FETCHED` with `AUDIT_UPSTREAM_FETCHES=all` in the verification checklist, config export and plan/apply for the high side, and the MCP Remote sync, instead of calling them planned. `docs/huggingface.md` drops the limitation that a Remote tree listing loses its query (fixed by #5321) and notes that commit-pinned paths are immutable cache entries (#5283). Documentation only; no behaviour change.
- **A Maven GET against a virtual repository issues fewer database queries** (#5489). Serving a virtual repository read its member list twice per request, once for the dependency-confusion guard and again for per-caller authorization, and ran the `artifacts` exact-path and SNAPSHOT lookups that a virtual repository can only miss. The list is now read once and shared (the guard keeps the unnarrowed list), and the lookups run only for hosted repositories. No behaviour change.
- **Dependency, build and audit housekeeping with no runtime behaviour change** (#5555, #5556, #4375, #5411, #5413, #4809, #4810, #1429, #4925, #5087). `cargo audit` ignores RUSTSEC-2026-0459 with its justification in `.cargo/audit.toml` (#5555, #5556): the advisory is in the C SQLite that `libsqlite3-sys` 0.30.1 builds, which is in `Cargo.lock` only because the `sqlx` feature tables name `sqlx-sqlite` through weak `sqlx-sqlite?/...` entries. The backend enables only the PostgreSQL driver, so the crate is never built or linked, and no `sqlx` release allows the patched `libsqlite3-sys` 0.39 yet; the ignore comes out once one does or the lock stops naming the crate. quick-xml moves to 0.42, and the NuGet, PyPI, VS Code, SAML, S3, curation-sync and Artifactory-import XML readers follow its string API (#4375). The amd64 leg of Docker Publish builds on GitHub-hosted `ubuntu-24.04` again, because the self-hosted builder's 16 GiB memory cgroup OOM-killed `dockerd` in the release-profile compile once the MCP Registry merged (#5411, #5413); what the image contains is unchanged. The signing-key trust attestation migration was renumbered to 291 before any release, so it no longer shares 290 with the age-gate instance lock (#4809, #4810). The Incus scanner runs its extraction through the shared `ScanWorkspace` lifecycle like the other filesystem scanners (#1429, #4925), and a stale code comment on the `test-upstream` error classification was corrected (#5087).
- **Release CLI binary builds start from a warm cache** (#4927, #4928). `release.yml` builds the five CLI binaries on the tag ref, and their Rust caches were saved on that ref, which no later tag can restore, so every release compiled each leg cold (27 to 54 minutes a leg). The caches are now warmed from the release candidate on `main` (`release-cli-caches.yml`), where the tag run can read them. Nothing in the shipped binaries changes.
- **Test suites hardened against the flakes and silent skips seen during the 1.12.0 cycle** (#5186, #4256, #4801, #4803, #5011, #4814, #4900, #4911, #4930, #5008, #5127, #5214, #5216, #5239, #5240, #5348, #5365, #5416, #5566, #5569, #4963, #5192). The migration handler tests no longer skip silently when the database is unreachable, `LogCapture` is reliable under a parallel `cargo test`, two storage-total tests no longer race over global sums, and a protobuf queued-push assertion no longer depends on the database collation (#5186, closing #4256, #4801, #4803 and #5011). The streaming cache test keys its coordinates apart from the sidecar LRU (#4814, #4900), the replication echo tests wait for the spawned push queue (#4911), and the OpenAPI route-consistency test checks documented routes, parameter-first paths included, against the production router (#4930, #5008). The RPM revision GC test is isolated (#5127), the pool-exhaustion test no longer fails on a slow connect (#5214, #5216), a warm 9 MiB read no longer hangs in the mock server's `Drop` (#5239, #5240), the cancelled-extraction and wheel-parity tests are deterministic (#5348, #5365, #5416), and the #2264 age-gate integration pin expects the unknown-key `404` that non-readers now get (#5566, #5569). Ignored live tests prove Azure Put Blob From URL signed with a Shared Key against a real storage account; no signature fix was needed (#4963, #5192).
- **Documentation** (#4960, #4926, #5189, #5190, #5527, #5528). An operator guide for inbound `traceparent` and a span and attribute reference in `docs/operations/tracing.md` and `docs/operations/logging.md` (#4960); a design note for ISO live-media rootfs scanning (#4926); a brand banner in the README (#5189, #5190); and `REDTEAM.md` names and pins the HTML UI component red-team scenario RT-15 runs against (#5527, #5528).
### Deprecated
- **The Incus native chunked upload endpoints are deprecated in favour of `/api/v1/uploads`** (#5002). Every response from `POST /incus/{repo_key}/images/{product}/{version}/{filename}/uploads` and the `PATCH`, `PUT`, `GET` and `DELETE /incus/{repo_key}/uploads/{uuid}` routes now carries `Deprecation: @1793404800` and `Link: </api/v1/uploads>; rel="successor-version"`, including error responses. The routes remain functional for existing Incus clients; new integrations should use the shared resumable API with an Incus repository. See `docs/chunked-uploads.md`.
### Fixed
- **The `oci_manifests` reconcile no longer removes the rows of manifests that still pull by digest** (#1683). The stale predicate added earlier in this release removed the row of the leaf of an index of indexes, of the child of a migrated index kept only by checksum, and of the old digest after a tag moved while its blob refs remained, all of which a 1.11 replica still serves. It now keeps them, adds rows for such manifests that have none, and sweeps again in the same pass when a removed index leaves its children stale. The storage GC's blob-ref prune retires the row of a manifest whose refs it prunes, so a 1.11 and a 1.12 replica stop serving it together.
- **A `clean` proxy verdict that graded no components is no longer reused under `fail_closed`** (#3396, #5043). Proxy scan verdicts are stored per content digest and shared by every repository that serves those bytes. A `clean` verdict with no findings, no cataloged components and no recorded coordinate (for example one written by an on-demand rescan) says only that the scan saw nothing it could grade, so a `fail_closed` download no longer serves from it: the bytes are scanned again as for a first pull. A download whose identity cannot be established no longer reuses any `clean` verdict recorded without a coordinate, so it stays withheld (423) under `fail_closed` and is served `pending` under `fail_open`. A `POST /api/v1/repositories/{key}/security/proxy-scans/rescan` in a repository whose format does not scan on proxy (for example `generic`) in which the scanner catalogs nothing and finds nothing is now inconclusive: it answers 503 `RESCAN_SCAN_FAILED` and records no verdict, instead of recording `clean`.
- **A proxy rescan that cannot grade a package archive leaves the stored verdict unchanged** (#3396, #5030). `POST /api/v1/repositories/{key}/security/proxy-scans/rescan` has no request coordinate to pin, so for the objects the download gate grades only through a pin or a staged image (npm tarballs, PyPI sdists, gems, crates, NuGet packages, Go module zips, Composer dists, Conan archives, `.deb`, `.rpm` and `.apk` packages, and OCI image manifests) a rescan whose scanner cataloged no components is now inconclusive: it answers 503 `RESCAN_SCAN_FAILED` and keeps the stored verdict, instead of recording `clean`. For every format, a rescan that finds no vulnerabilities no longer replaces a stored `vulnerable` verdict when it cataloged fewer components than that verdict was graded on; that is inconclusive too.
A pinned proxy download (npm, PyPI, RubyGems, Cargo, NuGet, conda, Maven and the other pinned formats) no longer reuses a `clean` verdict that was recorded without a pin, such as one written by a rescan; it is scanned again as for a first pull, and a `vulnerable` verdict still refuses it.
- **Upstream `CLIENT` spans export `http.response.status_code` and `server.port` as integers** (#3954, #4959). They were recorded as unsigned integers, which the OpenTelemetry tracing bridge exports as strings, so a trace backend could not compare them numerically (`http.response.status_code >= 500`) or group them with the integer values the request span already exports.
- **A remote conda channel serves a plain `repodata.json` by decoding the upstream `.zst`/`.bz2` document instead of buffering the 160-200 MiB plain index** (#4180). Classic conda clients and `CONDA_USE_ONLY_TAR_BZ2` setups request the uncompressed index, which conda-forge publishes far above the 128 MiB compressed-metadata ceiling; the request used to fall through to a slow streaming read of the plain document, and before that answered 502. The plain response is now produced by streaming the compressed upstream document through a decoder, so the ceiling applies to the 20-60 MiB compressed bytes, the decoded document is never held whole, and the compressed bytes stay in the proxy cache for later plain, `.zst` and `.bz2` requests. Channels that publish no compressed encoding still fall back to the plain streaming path. An upstream compressed document that is empty or does not decode answers 502 and is evicted from the proxy cache, so the next request fetches it again.
- **Maven checksums of a quarantined or scan-policy blocked artifact now answer 404** (#4288, #4967). A `.sha1`, `.md5`, `.sha256` or `.sha512` request for a hosted artifact answered with the digest even while the artifact itself was refused, from a stored sidecar or computed from the stored bytes, on the direct hosted route, for a `-SNAPSHOT` alias and through a virtual repository. The checksum now gets the artifact's own download gate (quarantine, then scan policy): while the artifact is refused every checksum of it answers 404, and once it is released the checksums are served again. Checksums of proxied (remote) artifacts come from the upstream and are unchanged.
- **A proxied file the scan gate cannot vouch for is no longer committed to the proxy cache** (#4514). On a Remote that scans on proxy, the format route now fetches a package into memory, runs the scan gate, and only then writes it to the proxy cache. A body that is inconclusive under `fail_closed` (answered `423`: no scanner, a failed or over-budget scan, an identity that cannot be established) is no longer cached, so no reader can serve it from the cache; it is fetched from the upstream again on its next pull, and concurrent pulls of one file on one replica wait for the first pull's decision (up to 35 seconds) instead. A body that fails the format's own check before the gate (a Cargo index `cksum`, a Maven `.sha1`) is not cached either. Served bodies are cached as before, and so are blocked (`403`) ones, which every reader of the scanning Remote refuses from the stored verdict and which stay listed, counted and rescannable. A fill refused by a Package Age Policy hold is still committed with its hold. No migration and no change to the cache sidecar format. See "The proxy cache and the gate" in `docs/security/scan-on-proxy.md` and the design note `docs/security/proxy-cache-commit-after-gate.md`, now marked shipped.
- **Presigned redirects to a cached package of a Remote that scans on proxy now consult the stored scan verdict** (#4514, #5068). The redirect fast path of PyPI distribution downloads, and the shared redirect helper for cached package archives, hand out the cached object without passing its bytes through the scan gate. For a Remote that scans on proxy, in a format that enforces it, they now look up the verdict stored for the cached body's SHA-256 under the Remote's own scan configuration: a blocking verdict is not redirected to, and under `fail_closed` neither is a body without a reusable `clean` verdict graded as a coordinate the bytes carry themselves, so the request is served by a path that reads the body itself. `record_only` repositories, Remotes that do not scan and formats that do not enforce scan-on-proxy are unaffected. Verdict reads on these paths are memoised per digest, so a new verdict reaches them within 30 seconds. See "Proxy-cache fast paths" in `docs/security/scan-on-proxy.md`.
- **Package rows held by a Remote that scans on proxy are served through the scan gate on the Go, RubyGems, Composer, Pub, Conan, CocoaPods and Swift routes** (#4514, #5071). A Remote can hold `artifacts` rows of its own (from replication, a publish into the remote, or recorded before proxied content stopped being stored as rows), and the format route served those from storage without the gate while gating every file it proxied. Now the stored bytes go through the same gate, graded as the coordinate the request names by the same identity rule a proxied file of that format is: a blocking verdict for their SHA-256 is `403` `scan_blocked`, `fail_closed` scans inline when there is no reusable verdict (`423` if inconclusive), `fail_open` serves `pending` and scans in the background, and a row over the scan cap is `423` under `fail_closed` and served `pending` otherwise. The response carries `X-AK-Scan`. Hosted repositories, Remotes that do not scan and the generic download route are unchanged. See "The artifacts routes" and "Known gaps" in `docs/security/scan-on-proxy.md`.
- **A secondary S3/Azure/GCS storage backend skipped at startup is now visible instead of a bare 500** (#4530). When a configured secondary backend fails to initialize (for example a malformed `S3_PUBLIC_ENDPOINT`), the skip is logged at ERROR, repositories on it answer `503 SERVICE_UNAVAILABLE` naming the backend instead of `500 STORAGE_ERROR`, `/health` lists it under `skipped_storage_backends` with its configuration error, and the `ak_storage_backends_skipped` gauge counts such backends (0 when none). Behaviour change: while a configured secondary backend is skipped, `/health` reports `unhealthy` with HTTP 503, so the image's Docker `HEALTHCHECK` (which polls `/health`) marks the container unhealthy; `/livez` and `/readyz`, which the Helm chart probes, are unchanged. A GCS configuration error other than an unset `GCS_BUCKET` now also counts as a skip.
- **Two writers can no longer deadlock on precomputed RPM repodata bookkeeping** (#4544, #5073). Marking a hosted RPM root dirty used to lock that root's row in `rpm_repodata_roots` until the writer committed, so two transactions that changed two roots in opposite order (for example two metadata updates, which take no other per-repository lock) could deadlock, and Postgres aborted one of them. Writers now only append to `rpm_repodata_dirty` (migration 309), which never waits; the background builder moves those marks onto the roots.
- **`PATCH /api/v1/repositories/{key}` can clear `description` and `quota_bytes` back to `null`** (#4729). Both fields were bare `Option<_>` request fields, so an explicit `null` deserialized exactly like an omitted field, and the service's `COALESCE($n, column)` UPDATE read the resulting NULL bind as "leave alone": `{"description": null}` or `{"quota_bytes": null}` answered 200 and changed nothing. Both now use the three-way `Option<Option<_>>` semantics `project_id` (#4026) and `trusted_gpg_key` (#2568) already use — omit to leave unchanged, `null` to clear, a value to set — and the service clears the column with a targeted write in the same transaction, ahead of the main UPDATE. Clearing a quota means unlimited; `0` remains the legacy unlimited sentinel. No first-party client sends `null` for these fields today: the web UI strips a null quota and sends `""` for a cleared description, and the generated SDKs type both as non-nullable optionals, so the behaviour change does not wipe values on an untouched form.
- **A Debian repository imported before 1.11.2 can recover its package registrations without the source system** (#4740). Re-running the migration job was the only documented repair for a repository whose `.deb` rows had no `debian` control document, catalog entry, or maintainer-script analysis, which needs access to the source. A new admin backfill, `POST /api/v1/admin/repositories/{key}/backfill-debian`, walks the live `.deb` rows that lack that metadata, reads each `control.tar` member from the repository's own storage with bounded ranged requests, and records the same control document, catalog row, and script analysis a native upload writes. A package outside `pool/<component>/` (a flat `pool/<file>.deb`, or the `pool/<letter>/<name>/` layout of a Nexus apt repository or an Ubuntu pool mirror) is moved under `pool/main/`, the component the migration importer picks without the source's `deb.component`. It also soft-deletes the migration-origin `dists/` index copies from the artifact list, since AK renders and signs its own, while leaving hosted rows alone. The run is idempotent and bounded per call (a truncated call returns `next_after` to pass as `after` on the next), and reports unreadable or inconsistent packages without aborting the rest.
- **OCI/Docker images pushed to hosted repositories are now replicated to push/mirror peers** (#4790). A `docker push` queued no replication at all, because an image's config and layer blobs live in `oci_blobs` rather than `artifacts` and the sync worker moved one `artifacts` row at a time. A pushed manifest now queues a push task, and the sync worker sends the manifest's blob closure (config and layers, and child manifests for a multi-arch index) to the peer before the manifest itself, through new receive routes `PUT /api/v1/repositories/{key}/oci/blobs/{digest}` and `.../oci/manifests/{digest}`. The peer verifies every digest and commits the manifest last, so it never serves an image with missing blobs, and a replicated write is not fanned back out. The receive routes take the same checks as generic replication uploads: the peer's API key needs the `write:artifacts` scope and `write` on the target hosted repository. Both nodes must run this release; a peer on an older release answers the new routes with an error and the task is retried.
- **Chunked uploads through `/api/v1/uploads` now replicate to subscribed push peers** (#4791). The generic finalize path wrote its own `artifacts` row and never queued a push sync task, so a completed chunked upload only reached a peer after a manual sync; it now fans out like every direct and format-native publish, without re-queueing peer replication writes.
- **Peer replication no longer echoes NuGet and nested-layout RPM writes, and a pacman signature or conda attestation added after its package now reaches push peers** (#4792). Follow-up to #4725. The NuGet push handler and the `repodata_depth > 0` branch of the RPM upload queued push sync tasks even for a write that arrived from a peer (`X-Artifact-Keeper-Replication: true`), so the write was pushed out again to every push peer of the repository; both now skip the fan-out like every other format. A pacman `.sig` and a conda CEP-27 attestation are stored in their package's metadata, and uploading one queued nothing, so a peer that had already received the package never got it. Attaching one now delivers the package again with its new metadata (the finished push task is re-armed, and a push that was in flight when the signature or attestation arrived is re-armed when it finishes).
On the receiving side, a trusted peer pushing the exact bytes that are already live at a path is now accepted as a replay that refreshes the artifact's metadata, instead of being refused with 409 as an immutable republish. Client uploads of identical bytes to an occupied coordinate are still refused, and a peer still cannot change the bytes of a released coordinate. A peer running an older release keeps refusing the replay, so the signature or attestation reaches it only after it is upgraded.
- **The migration jobs list reports `pagination.total` for the rows its `status` filter matched** (#4800). `GET /api/v1/migrations?status=failed` already narrowed the rows on the page, but the `COUNT(*)` behind `pagination.total` was computed without the filter, so a filtered view reported every job the caller could see and `total_pages` — the page count and "N of M" figure the UI derives from it — was too large. The count now carries the same filter as the listing; unfiltered totals are unchanged.
- **Streamed uploads above ~4.88 GiB no longer fail on S3-compatible stores that cap a multipart upload at 1,000 parts** (#4805). Part sizes assumed AWS's 10,000-part cap, so a large OCI layer pushed to such a store failed at part 1,001. The new `S3_MAX_PARTS` (default 10,000, clamped into 1 to 10,000) sets the part budget that multipart part sizes and part-count checks derive from; `S3_MAX_PARTS=1000` streams objects up to ~500 GiB. The default behaviour is unchanged.
- **A token scoped to a virtual repository can read its Packages catalog through that virtual repository** (#4808, #4130). The format endpoints and virtual artifact reads already support request-bound member reads (#4559), but `GET /api/v1/packages?repository_key=<virtual>` still intersected the member rows with a token naming only the parent, returning an empty page. Package detail had the same mismatch; package versions now also accept the optional `repository_key` context. The list, count, detail and versions use the same authorized leaf-member set, including nested virtuals, after checking that the caller can read the named parent and each member. Unknown or unreadable contexts reveal no membership, and a package outside the named repository does not match.
The token itself is unchanged: no selector expansion flag, read-expansion JWT claim, or implicit direct-member access is added. Reads without a virtual context keep their existing scope checks, member writes remain unavailable to a virtual-only token, and native index/download policy enforcement is unchanged. This is catalog metadata authorization, not a replacement for the format-specific package admission or scanning gates.
- **The admin trash listing and restore now honour a repository-restricted admin token** (#4813). `GET /api/v1/admin/trash` lists only the token's repositories, and `POST /api/v1/admin/trash/{id}/restore` answers 404 for an artifact outside them, the same as an unknown id, as `DELETE /api/v1/admin/trash/{id}` already did since #4701.
- **Per-repository settings reads now require read access to the repository** (#4815). `GET /api/v1/repositories/{key}/cache-ttl`, `/on-upstream-not-found`, `/stale-if-error-grace`, `/routing-rules` and `/pypi-tracks` answered any caller, including an anonymous one, for any repository. They now apply the same visibility check as `GET /api/v1/repositories/{key}`: a repository the caller may not read answers 404, exactly like a key that does not exist.
- **NuGet V2 upstream translation no longer drops XML entities and text before them in description/authors** (#4816). When parsing V2 OData feeds, character/entity references (`&`, `ö`) and CDATA sections were treated as separate events that overwrote field values; the parser now accumulates text across `Text`, `GeneralRef`, and `CData` events before committing to the field.
- **S3 storage reindex no longer reports artifacts with non-ASCII or reserved characters in their path as missing** (#4817). On S3 storage, object_store writes such a path under a percent-encoded object key (`uni-1.0-café.jar` is stored as `uni-1.0-caf%C3%A9.jar`), and listing returned that stored key unchanged, so `POST /api/v1/admin/repositories/{key}/reindex-storage` reported the row's object as missing and counted the object as an unregistered ghost that a non-dry run would register a second time under the encoded path. S3 listings, including the `S3_PROVIDER=oss` REST fallback, now return keys in the form reads and writes use, which also lets proxy-cache purges and RPM version retention delete such objects. Existing object keys are not rewritten. An object written by another tool under a key that Artifact Keeper can neither read nor write (for example a raw `~`) is left out of the listing with a warning.
- **Members of the `repository-owner` role pass the repository `admin` check again** (#4818). Since 1.11.0 the repository administration routes (email subscriptions, the staging release target, upstream auth, egress proxy, routing rules, upstream filter, conda allowlist, PyPI tracks, the age gate and per-repository security settings) answered 403 to a `repository-owner` member, including the user who created the repository, because the check read fine-grained permission rules only and ignored role assignments. It now uses the same resolver as the write and delete checks, so a role carrying `admin` on the repository is honoured; `developer` and `reader` members are still refused.
- **Make DATABASE_URL_FILE take effect in the published backend image** (#4819). The baked-in `DATABASE_URL=set-database-url-at-runtime` placeholder no longer prevents the `_FILE` form from being used; `DATABASE_URL` can also be left unset entirely.
- **Creating a repository with `upstream_username`/`upstream_password` but no `upstream_auth_type` is refused instead of silently dropping the credentials** (#4825). The create used to answer 201 and store nothing, so the Remote fetched upstream unauthenticated while the operator believed it was configured, and the public-remote credential check (#4662) never saw the credentials. It now answers 400 `VALIDATION_ERROR` naming `upstream_auth_type` and creates nothing. The type is not inferred, matching `PUT /api/v1/repositories/{key}/upstream-auth`, which also requires `auth_type`. Empty strings still count as absent.
- **`ak_db_pool_connection_failures_total{stage="startup"}` is now exported** (#4828). The startup connect retry added in 1.11.2 (#4549) counted every failed initial connect attempt, but the database pool was built before the Prometheus recorder was installed, so those increments went to the `metrics` crate's no-op default recorder and the `startup` series never appeared on `/metrics`. The recorder is now installed before the pool is built, so a replica that needed several attempts to reach the database reports them once it is up. The `request` series was not affected.
- **A remote conda channel serves a CEP-16 shard again once its upstream does, after refusing it for not matching its address** (#4829). Since 1.11.0 a shard whose upstream bytes do not hash to its address is answered with 502, but those bytes had already been written to the proxy cache under the shard's path, which is cached as immutable and never revalidated. After the upstream recovered, the remote, and every virtual over it, kept answering 502 for that shard without asking the upstream again until the cache entry was evicted by hand.
A shard is now checked against its address before it is committed to the proxy cache, and only a shard that matches is stored. A cached shard that does not match (one stored by an earlier release) is evicted on the next request and fetched again, so an affected cache heals without operator action. Both cases count in `ak_proxy_cache_checksum_mismatch_total`.
- **Logout now revokes the session's refresh token even when the refresh token is not in the request** (#4831). `POST /api/v1/auth/logout` revoked the refresh-token family only when the request carried the refresh token, in the body or the `ak_refresh_token` cookie. A bearer logout with an empty body, and a browser logout from the web UI, carry neither, so logout answered 200 while the session's refresh token kept minting new access tokens until it expired. Access tokens minted with a refresh token now carry that session's family id, and logout revokes the presenting session's family whether or not the refresh token is included. Other sessions of the same user stay signed in, as before. Access tokens issued before upgrading carry no family id; a logout with one of them behaves as before until the session next refreshes.
- **Concurrent uploads to one artifact path now agree on a single winner** (#4832). An upload to a path that already holds a live artifact at the same version is refused with 409 `CONFLICT` ("Artifact version already exists and is immutable") unless the repository has versioning enabled. That check ran before the bytes were stored and outside the transaction that writes the artifact row, so several `PUT /api/v1/repositories/{key}/artifacts/{path}` requests racing to one free path could all pass it: each was answered 201, and the last to commit silently replaced the content the others had been acknowledged for. The same rule is now part of the row write itself, for direct uploads and for chunked-upload completion: exactly one of the racing uploads is answered 201 and keeps its bytes, and every other one gets the documented 409. A deleted artifact can still be replaced, and a versioning-enabled Generic or Mlmodel repository still appends a revision. The 409 is now listed in the OpenAPI description of the PUT.
- **Deleting a repository on filesystem storage now removes its files even when another repository holds the same bytes** (#4833). Generic and other content-addressed artifacts share a storage key across repositories that hold the same content, and the delete spared any key another repository's row also named. On filesystem storage each repository keeps its own copy in its own directory, so the other repository's copy wrongly shielded this one, the cascade then dropped the only row that named it, and garbage collection could never find it again. The check now counts only a repository that resolves the key to the same object (same backend, and on filesystem the same storage path), including an older revision there. A key the delete still spares because it is genuinely shared is handed to the storage-GC candidate set, so the next GC pass reclaims it once no repository references it, which also covers two repositories sharing a key on cloud storage being deleted together. GC likewise reclaims a trashed filesystem copy while another repository serves its own copy of the same key.
- **A `repository-owner` role member passes every inline repository-administration check** (#4834). Repository update and delete, cache TTL, upstream-not-found policy, stale-if-error grace, npm scope policy (read and write), proxy cache invalidation and virtual-member mutation gated on fine-grained `permissions` rows only, so a member holding the `admin` action through a role assignment (the `repository-owner` role granted to a repository's creator, and to the developers migration 172 promoted on older rule-less repositories) was refused 403. They now use the same resolver as `require_repo_admin` (#4818). Behaviour change: a `repository-owner` member, or a user with a global role assignment carrying `admin`, can now update and permanently delete that repository (`DELETE /api/v1/repositories/{key}`, token scope `delete:repositories`), not only a global admin or a user with a fine-grained `admin` rule. `developer` and `reader` members are still refused.
- **A failed hosted Maven upload no longer leaves its bytes behind** (#4837). A hosted upload stores the file before the transaction that records it, so when that transaction failed (a metadata write error, a quota refusal, a failed POM read) the client got an error while the object stayed in storage. On object storage those orphaned bytes were then downloadable through the repository's storage fallback, serving a deploy the server had rejected. The upload now removes its object when the publish does not commit, on every storage backend, and the object-storage fallback no longer serves an artifact coordinate that has no catalog row (checksum sidecars and `maven-metadata.xml` are still served from it). The cleanup is serialised against an overlapping upload of the same coordinate, so a failed upload can never delete the bytes of a concurrent one that succeeds.
- **Server-side S3 copy of a large object with a non-ASCII or reserved character in its path no longer fails with `NoSuchKey`** (#4838). object_store stores each key segment percent-encoded (`café` is stored as `caf%C3%A9`), and S3 URL-decodes the `x-amz-copy-source` header once before looking the source up. The multipart `UploadPartCopy` path, which every object above `S3_MAX_SINGLE_COPY_BYTES` takes, encoded the caller's key instead of the stored one, so the header decoded to a key that does not exist. The header is now the URL encoding of the stored key, the same value object_store's single-request `CopyObject` sends for a small object.
- **An invalid `upstream_auth_type` on repository create is refused before the repository row is written** (#4843). An unknown auth type, `basic` without a password, or an AWS auth type without an `aws` block returned 400 `VALIDATION_ERROR` only after the repository had been inserted, leaving a half-configured Remote behind (and a retry failing with 409 on the key). The credentials are now built and validated up front, alongside the `upstream_username`/`upstream_password` check (#4825), so a refused create leaves nothing behind.
- **The repository list and the repository update response now report a Remote's upstream authentication** (#4851). `GET /api/v1/repositories` and `PATCH /api/v1/repositories/{key}` answered `upstream_auth_type: null` and `upstream_auth_configured: false` for every repository, even one with upstream credentials configured, while `GET /api/v1/repositories/{key}` reported them correctly. The web UI's edit dialog for a Remote reads the list row, so it showed no authentication configured. The list already read the upstream auth rows (for `public_with_upstream_credentials`) but never projected them; the list, create and update responses now set both fields from those rows, exactly as the detail response does. Only the auth type and whether one is configured are reported, never the credentials.
- **NuGet repositories translating a V2 upstream now list the versions on every page of the upstream feed** (#4856). A Remote over a V2 OData feed answers its V3 registration index and flat-container version list from the feed's `FindPackagesById()`, and a virtual repository merges a V2 member the same way into its registration, flat container and V2 feed. Only the first page of that feed was read: the feed-level `<link rel="next">` continuation (with its `$skiptoken`) was ignored, so every version the upstream served on a later page was missing and restores that needed one failed. The continuation is now followed, up to 20 pages per query, with rows repeated across pages listed once. A continuation must stay on the configured upstream origin (scheme, host and port), the only place the repository's upstream credentials are sent; one that points elsewhere is not requested, a warning is logged, and the pages already read are served. Continuation pages are cached per URL next to the first page. The raw V2 feed of a Remote was not affected: its next link is rewritten onto AK and the client follows it. Contributed by @PauloHMattos.
- **Removing the `LDAP_*` environment variables now disables the LDAP provider they created** (#4857, #4863). The `LDAP_*` bootstrap seeds an enabled LDAP provider, but a later boot without `LDAP_URL` / `LDAP_BASE_DN` left that provider enabled, so the login page kept offering only the LDAP form and local login stayed hidden for a directory the operator had removed. The boot now disables (never deletes) the provider the environment manages, matching what removing `OIDC_*` already does (#2819), and logs a warning naming it. Providers created through the admin API are never touched, and an administrator who re-enables the provider through the admin API takes ownership of it, so the next boot without `LDAP_*` leaves it enabled. Re-setting the variables does not re-enable a provider the boot disabled; re-enable it through the admin API.
- **Pushing an OCI blob of about 2 GiB to an Azure-backed repository no longer fails with 500 "Azure Put Blob From URL failed: error sending request"** (#4858). Every OCI blob finalize copies the uploaded bytes to their content-addressed key, and on Azure that copy is a synchronous Put Blob From URL for any source up to 5000 MiB. Azure sends no response until the copy is done, but the request went out on the data-plane client, whose 30-second read timeout aborted any copy that took longer while Azure was still copying. Monolithic and chunked uploads both end in that copy. Put Blob From URL now has a client of its own with no read timeout and a per-request deadline sized to the source: 60 seconds plus the time to copy it at 8 MiB/s. The source SAS now also outlives that deadline. Every other Azure request keeps the 30-second read timeout. The same copy finishes staged generic uploads (the resumable `/api/v1/uploads` path) and proxy-cache staging-to-cache moves, which failed the same way and are fixed with it. `AZURE_COPY_TIMEOUT_SECS` sets a minimum for the copy deadline, for accounts that copy slower than 8 MiB/s.
- **Deleting a versioned Generic or MLModel repository now removes the objects of its older revisions as well** (#4865). The delete collected storage keys only from the repository's live `artifacts` rows. Older revisions live in `artifact_versions`, which no `artifacts` row names, and those rows cascade away with the repository, so the history objects stayed on filesystem and object storage with nothing left to derive them from, and the storage GC orphan scan, which starts from `artifacts`, could never find them. The key set now unions the repository's live artifacts and its own revisions under the same location-scoped sharing guard, and a revision key the delete spares because another repository still names it is handed to the storage-GC candidate set so the next GC pass reclaims it once no reference remains.
- **Storage error messages carry the operation and the transport error, not the request URL** (#4866). When a request to Azure Blob Storage failed in transport (connection reset, timeout), the error returned to the API client and written to the log repeated the full request URL, including its query string. They now name only the operation and the transport error. The network errors of a Google Cloud Storage resumable upload, which repeated the upload session URL, are trimmed the same way.
- **A plugin install or reload whose git source redirects to a blocked address is now refused with a 400 instead of a 500** (#4869). The outbound SSRF policy refuses two things under two different wordings: a host that resolves to an address the policy blocks ("blocked by SSRF policy") and a redirect hop that points at one ("redirect target rejected by SSRF policy"). Only the first was recognized when a git clone failed, so a source that redirected onto a blocked address fell through to the generic internal-error branch and the API answered 500 for what is a rejected request. Both refusals are now recognized through one shared vocabulary, so the plugin install and reload paths answer 400 with the same policy wording as the connect-time refusal, and an upstream fetch whose redirect the policy refuses is reported as the 503 the policy refusal deserves instead of a generic 500. No security change: the redirect is still aborted before the target is contacted.
- **A CRAN repository of type `remote` now proxies the upstream `PACKAGES`/`PACKAGES.gz` index, so R clients pointed directly at it can discover and install packages** (#4877). The source index was built from locally-catalogued rows alone, and a proxy repository only ever has rows for packages already fetched through it, so `GET /cran/{repo}/src/contrib/PACKAGES` answered an empty `200` even though the upstream (e.g. `https://cran.uib.no/`) publishes a full index; only individual package downloads were proxied, so `install.packages()` had nothing to resolve against. A remote repository now fetches the upstream `src/contrib/PACKAGES` (falling back to `PACKAGES.gz` for mirrors that ship only the gzipped form) and merges it over the locally-held rows, preserving every upstream DCF field — `Depends`, `Imports`, `LinkingTo` and the rest — because R resolves the dependency graph out of those. A locally-held package still wins its `(Package, Version)` slot, since its row carries the coordinates the download route serves from storage. An upstream that publishes no index (`404`) is not an error, but a genuine upstream failure with nothing held locally is now surfaced instead of being masked as an empty `200`.
- **A virtual conda channel merges each subdir once for every encoding and keeps serving the previous merged repodata while it rebuilds** (#4879). `repodata.json`, `.zst` and `.bz2` of one subdir were three independent merges (three member fetches, three merges, three encodings), and when a remote member's cached repodata expired the kept merge was dropped, so every request waited on a full rebuild: about 45 s for a conda-forge `linux-64` `.bz2`, which ended in `merge shed` 503s and client timeouts when a dozen subdirs refreshed at once. The merged `repodata.json` is now kept once, zstd-encoded, and `.json` and `.bz2` are derived from it on first demand and kept with it (each encoding keeps its own ETag; the plain `.json` of a virtual is now compact JSON). A kept document that is no longer fresh is served at once, with `Warning: 110`, while exactly one background merge rebuilds it, and the next request after the rebuild gets the new document, with the encodings the previous one had been asked for already built. The new `CONDA_VIRTUAL_MERGE_MAX_STALE_SECS` (default: the merge cache TTL) bounds how long after it was first found stale a document may still be served; `0` restores waiting on the rebuild. A merge with a failed member is never served stale. Stale serves count as `ak_conda_virtual_merge_cache_hits_total{via="stale"}`. See `docs/operations/conda-virtual-merge-memory.md`.
- **`AK_PROXY_METADATA_BUDGET_BYTES` above 4 GiB is now honoured instead of being silently clamped, so the conda virtual merge cap grows with it** (#4881). The buffered-metadata budget counted one semaphore permit per byte, and a reservation is taken through tokio's `acquire_many`, which takes a `u32`, so the budget was clamped to 4294967295 bytes with no log line: an operator who set 32 GiB got a 4 GiB budget, and the default `CONDA_VIRTUAL_MAX_CONCURRENT_MERGES`, derived from it, stayed at 3. The budget is now accounted in 4 KiB units (a reservation rounds up to a whole unit), so any value the platform can address is represented and 32 GiB yields `budget_bytes: 34359738368` and 28 default merges at the default member ceilings. A blank, non-numeric or zero value still falls back to the 1 GiB default, and that, or a value too large to represent, now logs a warning at startup naming the configured and effective values.
- **`PUT /api/v1/peers/{id}/network-profile` now routes to the peer network-profile handler instead of returning a 500 with a raw axum extractor error** (#4884). The router had been mounted at `/peers/network-profile` without the `:id` segment; it now requires the peer ID, rejects malformed IDs with a normal 400, and returns 404 for an unknown peer.
- **Label moves and late format metadata now reach push peers** (#4889). The protobuf `_labels` index was a zero-byte row that the sync worker had nothing to transfer, and the NuGet, PyPI, Debian and nested-layout RPM routes queued their push before writing the format `artifact_metadata`. The label index is now stored as a real JSON object (with a refreshable, content-addressed row) and queued like any other upload, and those four routes queue after their metadata is committed; the metadata-diff re-arm now covers every format, so a worker that finished a transfer before the document landed redelivers it. Client-uploaded `maven-metadata.xml` remains intentionally row-less and unpushed: peers regenerate it from the versioned artifacts they receive rather than accepting one deploy's snapshot over their own generated view.
- **Concurrent first Maven uploads of one immutable release coordinate can no longer leave the winner's row pointing at the loser's bytes** (#4892). The object is now written inside the publishing transaction while holding the per-key publish lock, and an upload that finds the coordinate already published under that lock returns 409 without touching the winner's storage key; previously both uploads wrote the same object before either row was inserted, and a loser whose write landed last replaced the winner's bytes while its own row was rejected. The write happens before the repository-wide quota and usage-ledger row locks are taken, so other uploads into the same repository do not wait on it. In Artifactory migration path mode a hosted Maven artifact upload now writes its object once instead of twice (#4893).
- **A retried OCI blob completion no longer fails with 409 "upload session is already being modified"** (#4902). When a reverse proxy timed out the final `PUT ?digest=` of a large layer (Envoy and Envoy Gateway cut at 15 s by default) or the client dropped the connection, the backend abandoned the completion mid-way and left the upload session locked for six hours, so the identical `PUT` that containerd 2.x re-sends after a 504 was refused and the push failed. The completion now runs to its end even when its request is dropped. A retry with the same digest waits for it (up to `OCI_UPLOAD_COMPLETION_RETRY_WAIT_SECS`, default 30 s) and gets the same result, or 503 with `Retry-After` while it is still running; a retry after it finished gets 201; a retry with a different digest is still 409. A session held by a replica that died is reclaimable after five minutes instead of six hours. See `docs/operations/oci-reverse-proxy-timeouts.md` for the proxy timeouts to set.
- **Accept an API token presented as a Bearer credential at `/v2/token`, so Renovate and other clients that configure a bare token can list Docker tags** (#4903). The exchange previously recognized only JWTs in that slot and answered 401, which surfaced as a failed `tags/list` request.
- **The PyPI age gate no longer blocks old releases whose filename spells the version differently from the Warehouse release** (#4907). The gate looked up a distribution's publish time by the exact version string in its filename, but Warehouse keys `releases` by the release's own spelling, so `cuda_toolkit-13.0.3.0-py2.py3-none-any.whl` (release `13.0.3`, uploaded months earlier) found no publish time and was withheld with a 451 indefinitely, which broke `pip install torch==2.14.1` on Linux. The lookup now falls back to PEP 440 equality on the download path and in both simple-index listing filters; a version that is not PEP 440-equal to a release still has no publish time and stays gated.
- **The rollback guide now warns that a 1.11.x database cannot serve a 1.10.x backend for OCI pushes** (#4915). Migration 243 (1.11.0, #3851) replaced the unique key on `oci_upload_cleanup_keys` that the 1.10.x upload path names in `ON CONFLICT (storage_key)`, so after a rollback to 1.10.x with `SKIP_MIGRATIONS=true` every `docker push` failed with "there is no unique or exclusion constraint matching the ON CONFLICT specification". `docs/operations/rollback.md` now says so up front, lists migration 243 in a new table of releases that cannot be rolled back across, and widens its boundary condition to dropped or replaced unique constraints.
- **Protobuf label replication no longer loses moves, leaves superseded documents behind, or fails on pre-#4889 rows, and Maven plugin-prefix metadata now reaches peers** (#4916). Each `_labels` write reclaims the previous content-addressed document once no live row names it and carries a monotonic revision that a receiving peer compares before overwriting (equal revisions are broken deterministically by checksum). The sync worker rebuilds an empty legacy label row from its inline label map before transferring it. A group-level plugin-prefix `maven-metadata.xml` (`<plugins>`) gets an artifact row and travels through the Maven upload route like a normal artifact, and a receiving peer merges it into the mapping it already holds (keeping its own prefixes) instead of replacing it, while artifact-level version indexes remain row-less and peer-regenerated.
- **Protobuf commits now replicate to push peers** (#4921). A protobuf (BSR) commit row recorded the commit digest as its `checksum_sha256`, so the receiving peer, which verifies the pushed bundle against that value, refused every commit push with 409 "checksum mismatch". New commit rows record the sha256 of the stored bundle (the commit digest stays the row's version and is still what every lookup uses). Rows written before this release need no operator step: the sync worker recomputes the sha256 from the stored bundle before pushing such a row and stores the corrected value. A push that the peer refuses for that one artifact (a checksum mismatch, or an immutable path already occupied) now fails only that task and no longer puts the whole peer into retry backoff; connection errors and 5xx responses still back the peer off as before.
- **A cold `repomd.xml` request for a large hosted RPM repository no longer runs the backend out of memory** (#4924, #4544). Rendering repodata loaded every package's stored metadata, file lists included, and built each uncompressed document in full before compressing it, so a root of about 2,500 packages with 2.5 GB of metadata was OOM-killed at a 10 GiB limit and `dnf` saw 503. The render now reads the packages from one consistent database snapshot in batches of about 32 MiB of metadata and compresses and hashes `primary.xml`, `filelists.xml` and `other.xml` incrementally; peak memory follows the batch size instead of the repository size. The documents and checksums are unchanged for repositories whose documents stay under 32 MiB uncompressed; larger documents are compressed as they are written, so their compressed bytes (and the checksums `repomd.xml` gives for them) can differ once from what an earlier release served for the same packages.
- **Remote and virtual conda channels fall back to an upstream plain `repodata.json` when the `.zst`/`.bz2` probes answer 401/403, evict an undecodable cached member document, and keep a proxied `repodata.json.sig` verifiable** (#4929). S3 buckets without list permission answer 403 for a missing key, so a channel that only publishes the plain index used to fail the compressed probe instead of serving it; 401/403 now mean "this encoding is absent" for the compressed candidates on both the single-remote and virtual-member paths, while a refusal on the plain document is still reported. The same rule applies to a remote's upstream CEP-16 shard index and to the documents its shards are synthesised from, so such a channel serves a shard index instead of a 502. A virtual member that reads an undecodable, empty or truncated cached `.zst` now evicts it and answers 502 so the next request refetches, matching the single-remote path. A remote channel that publishes `repodata.json.sig` serves upstream's plain `.json` for a plain request, because the signature covers those bytes rather than the decoded `.zst`/`.bz2`; unsigned channels such as conda-forge keep the compressed transcode.
- **OTLP export starts, and stops, cleanly when the collector setting is wrong or the collector is down** (#4931). An unparseable `OTEL_EXPORTER_OTLP_ENDPOINT` no longer aborts startup: span export is turned off with a message on stderr, as log and metric export already were. An empty `OTEL_EXPORTER_OTLP_ENDPOINT` now counts as unset instead of turning the export on with nowhere to send. On shutdown the span, log and metric pipelines flush at the same time, so an unreachable collector holds shutdown for about 5 s instead of about 15 s, inside the 10 s a container runtime grants before it kills the process.
- **OTLP log records and shutdown now carry enough context and fit inside a container stop grace period** (#4931). The `Request completed` event copies `method`, redacted `uri` and `status` onto its own fields, so it remains identifiable in an OTLP log backend even when logs are not joined to traces. Shutdown flushes spans, logs and metrics concurrently against one shared 5 s deadline instead of letting each provider apply its own timeout, and `artifact_keeper.virtual.member_count` is exported as an integer rather than a string.
- **The guest-access guard no longer binds against the directory, a rejected package login writes one `LOGIN_FAILED` row, and web LDAP login no longer undoes an administrator's deactivation** (#4933). With guest access off, the guard now answers `/api/v1` and `/v2` from local credentials and leaves the single directory bind to the package-route middleware, so management and OCI requests cause no directory traffic and one rejected package request no longer writes two audit rows. LDAP accounts now keep `is_active` across a federated login, matching the passthrough: an admin-disabled account must be explicitly reactivated.
- **Composer Virtual repositories now serve Packagist-style Remote members' metadata and dist archives** (#4951). Metadata proxied through a Virtual served the member's `dist.url`s verbatim, sending the Composer client to the off-registry host and bypassing the Virtual; and a non-scanning Remote member's dist was resolved by synthesizing `dist/<vendor>/<package>/<version>/<ref>.zip` against the member's upstream base, which Packagist-style upstreams never serve and answered 404. The Virtual metadata now rewrites dist URLs to the Virtual key like the single-repo Remote path, and the Virtual download walk resolves each Remote member's real dist URL from its own cached metadata before streaming or scanning it.
- **A download statistics retention pass no longer scans the whole table when it runs out of rows to delete** (#4963, #4541). With `DOWNLOAD_STATS_RETENTION_DAYS` set, the last batch of every pass read all of `download_statistics` and `proxy_download_statistics`, because nothing indexed the download time, so the hourly pass cost a full table read even with nothing to trim. Migrations 313 and 315 add an index on `downloaded_at` to each table, built online with `CREATE INDEX CONCURRENTLY` so downloads keep being recorded during the upgrade, and each batch now reads through it. Every recorded download writes one more index entry.
- **Maven jars served through a virtual repository from a remote member now carry `X-Checksum-SHA1`** (#4963, #4573). Maven Resolver skips its separate `.sha1` request when the download carries this header. Direct remote serves already sent it, but a virtual repository's unscanned serve from a remote member did not. The value is the cached `.sha1` of the member that served the bytes, never another member's, and `-SNAPSHOT` files still go without it. A `HEAD` for a released file on a remote or virtual repository now also carries it on a path nothing has cached yet, by fetching the `.sha1` from the upstream once; a `GET` still reads only what is cached.
- **NuGet V3 search now aggregates package versions across virtual repository members and reports the latest stable SemVer** (#4976). A virtual NuGet repository combining multiple V2 remotes previously kept whichever member's result arrived first and applied V3 package paging to V2 version rows before grouping, so search could show an arbitrary older version and omit versions from other members. Search now groups version rows before paging, merges duplicate package IDs and their version lists, recalculates the root version with the requested prerelease policy, and preserves the available version entries.
- **An upload of content that storage GC is deleting at the same moment no longer commits a row whose object is gone** (#4980). GC's orphan-key sweep re-checks a key's references and then deletes the object; an upload that deduplicated against that object (or wrote it) and committed its row between the two left a live artifact whose downloads failed. The transaction that commits an upload's row now first takes the key's GC journal slot: while a GC pass holds a delete lease on the key (taken in the last 15 minutes, recorded in the new `oci_gc_candidates.leased_at` column, migration 302) it waits up to 5 seconds, writes the object again if a delete removed it, and otherwise answers a retryable `503` and commits nothing. Covers the generic PUT and multipart routes, the native uploads that go through the shared upload service, and chunked completion. See `docs/operations/storage-gc.md`.
- **The storage scrub's run accounting and bookkeeping are accurate** (#4985, #4323). `objects_checked` no longer counts an object twice when recording its finding fails; bytes read from a repair donor and the re-read of a repaired object now count toward `bytes_read`, so `max_bytes` bounds a repair run too; each run removes findings whose artifact or OCI blob row was hard-deleted and the resumable cursors of deleted repositories; and a scrub requested for an unknown repository key is audited (`started`, then `failed`) before the 404.
- **A repository renamed to `~deleting~<job id>` is unreachable through every by-key path, not only its original key** (#5002). Repository deletion renames the row in the request transaction and lets a background job remove it, but a direct lookup of the renamed key could still resolve the repository while the job ran or after it failed. The `~deleting~` prefix is now a shared predicate rejected by `RepositoryService::get_by_key`, the proxy and format resolvers, the auth visibility middleware, repository listings and every search path, so no route, virtual member, search result or stale job can expose a repository that is still being deleted. A repeated `DELETE` of the renamed repository still returns the job in flight. See `docs/operations/repository-delete.md`.
- **A repository's security score no longer goes stale when two of its scans finish at the same time** (#5006). Each score recalculation read the repository in its own snapshot with no coordination, so when two scans completed together one recalculation could store a score computed before the other scan had committed, and the other failed with a serialization error that nothing retried. The score, `grade_status` and coverage then kept counting the finished scan as missing until the next scan or delete in that repository (on 1.11 replicas this could show as a false F). Recalculations of one repository now run one at a time under a database advisory lock and are retried on a serialization failure, so the stored score always includes every completed scan. This covers every path that recalculates a score: scan completion, artifact deletes, finding triage and SBOM ingest.
- **SSO and CI OIDC requests send a User-Agent and report identity-provider HTTP errors** (#5036). Requests to identity providers (SSO discovery, token, JWKS and userinfo; CI OIDC discovery and JWKS) now carry `User-Agent: artifact-keeper/<version>`, so a web application firewall that refuses requests without one (such as the AWS WAF core rule set) no longer blocks them. A non-2xx answer from discovery or JWKS is now recorded in the server log as `... returned HTTP <status>` instead of an invalid JSON error, and so is a token endpoint answer that is non-2xx and not JSON (OAuth JSON error responses are handled as before). Login responses still show only a generic error; the start of the response body is written to the server log only (at debug level for the token endpoint).
- **Proxied package archives with two entries at one path are refused before the scan-on-proxy identity check** (#5050). The check that ties a proxied package to the coordinate it is served as reads the first matching manifest in the archive (`package/package.json`, `Cargo.toml`, `*.nuspec`, `pom.properties`, `info/index.json`, `metadata.gz`, `go.mod`, `composer.json`, `pubspec.yaml`, `*.podspec.json`, `Package.swift`), while the extraction that unpacks the archive for the scanner keeps the last entry at a path, and a zip entry repeated as `./name` or `a//b` was read under one name and extracted under the other. An archive whose non-directory entries repeat a path, in the container or, for a conda `.conda`, across its payload tars, no longer establishes an identity: under `fail_closed` it is withheld (423) and under `fail_open` it is served `pending`, unscanned and unrecorded, as an unreadable archive is. A survey of 1,282 published packages across npm, PyPI, RubyGems, crates.io, NuGet, Maven Central and conda-forge found no outer archive with a repeated path.
- **Full backups now read artifact bytes from each repository's own storage path instead of the deployment-global storage root** (#5053). Filesystem repositories store objects under `<STORAGE_PATH>/<repo_key>`, so the backup's single global storage handle missed every artifact and the run failed with `backup is missing artifact content`. Backups now resolve each artifact through its repository's configured storage backend and location, and capture an object shared by several repositories once. Restores write the bytes back to every repository that references them, at the location of the repository as it exists on the restoring server. A repository location taken from the archive is used only when that repository does not exist, and only if it lies below the configured storage root. Legacy archives without repository rows still use the global handle.
- **CI OIDC exchanges mint a token that lives the provider's configured TTL instead of dying with the CI platform's ID token** (#5054). `POST /api/v1/auth/ci/token` capped the minted access token at the presented assertion's own `exp` (#3820). GitHub Actions ID tokens live about five minutes, so every token exchanged from GitHub Actions stopped working after about five minutes, whatever TTL was configured, and longer build steps failed with 401. The assertion now only has to be valid when it is exchanged, as with AWS STS, GCP Workload Identity Federation, Azure, PyPI trusted publishing and Vault. The minted token lives the provider's new `token_ttl_secs` (default 900, between 60 and 3600, set with `POST`/`PUT /api/v1/admin/ci-oidc`; migration 307). An expired assertion is still refused, the mapping's repository ceiling and group binding still apply, a CI service account still cannot mint API tokens, and its session cannot be refreshed (the exchange returns no refresh token, and `/api/v1/auth/refresh` and the registry refresh grant now refuse CI accounts). `kubernetes` providers are unchanged: their pull-only credential still expires with the ServiceAccount token. Jobs that run longer than the TTL exchange a fresh ID token again; see `docs/ci-oidc.md`.
- **A virtual conda channel's package allowlist now applies to its hosted members too, so the channel no longer serves packages outside its list** (#5055). The allowlist (#4576) filtered only what remote members contributed: hosted (local and staging) members passed through unfiltered in the merged index and on downloads, so anyone who could publish to a hosted member could put a package on every channel containing it, list or not. The list is now the channel's boundary for every member, in every index form a virtual serves (`repodata.json` plain, `.zst` and `.bz2`, the CEP-16 shard index and its shards, `current_repodata.json`, `run_exports.json`, `channeldata.json`) and on package and attestation-sidecar downloads through the virtual, with the same matching rules as before (name glob, version spec, subdirs; filename and record identity must agree). A hosted package the list does not admit is dropped and still owns its name, so the upstream's packages of that name stay excluded. Reads of the hosted repository itself are unchanged. A new `scope` field on `PUT /api/v1/repositories/{key}/allowlist` restores the old behaviour per channel: `"scope": "remote_only"` (the default is `all_members`); a request that omits `scope` keeps the stored scope.
**Upgrade impact:** on a channel with an enabled allowlist, hosted packages that are not on the list disappear from that channel (its index and downloads through it) until they are added to the list, or until the channel is set to `"scope": "remote_only"`. Check each virtual conda channel with an allowlist before upgrading. The hosted repositories themselves keep serving those packages directly.
- **npm: the version route resolves dist-tags** (#5063). `GET /npm/{repo}/{package}/{version}` (and the scoped form) answered 404 for a dist-tag such as `latest` or `next`, on Local, Remote and Virtual repositories. When no version of that name exists, the segment is now resolved through the package's `dist-tags`, as the npm registry does; an exact version still wins over a tag of the same name, and an unknown tag is still 404. On a Local or Staging repository the stored tags are used, with `latest` derived from the versions when none is stored. On an age-gated Remote or Virtual repository a tag resolves through the gated `dist-tags`, as the package document serves them, so `latest` names the newest version old enough to download.
- **Download verification now holds a bounded amount of memory: small stored objects checked before their status is sent share one 256 MiB budget, and a download that finds it full is checked while it streams instead** (#5064, #4989, #5004). Verifying stored bytes before the response status (#4323) reads an object of up to 4 MiB whole and keeps it until the client has read it. That covers the generic download route, OCI blob pulls, every virtual-member serve and the hosted routes on the shared fetch. The global concurrency limit does not bound those buffers, because a request's permit is released when its response starts and the body is sent afterwards. The total was therefore set only by how many connections clients kept open. The buffers now draw on one process-wide budget, `AK_SERVE_VERIFY_BUFFER_BUDGET_BYTES` (default 256 MiB), separate from the proxy-metadata budget. A serve reserves its object's size in 4 KiB units and gives it back once its body has been read or dropped. A serve that finds no room never waits: it is verified while it streams, as an object over 4 MiB always is, so a corrupt object is still never delivered complete, but the transfer is aborted instead of answered `502`. `ak_serve_verify_buffer_bytes_reserved` and `ak_serve_verify_buffer_fallback_total` report the budget's use. A `HEAD` that the generic route answers through a virtual repository's hosted member now reads nothing, as a `HEAD` answered from the artifact row already did.
- **Precomputed RPM repodata revisions no longer stay behind when a repository's storage location changes** (#5072). The builder recorded one location per root, the current revision's, and deleted superseded revisions through it, so revisions written to the previous `storage_backend` / `storage_path` were never deleted there. Each revision's claim now records where it is written (`rpm_repodata_revisions`, migration 311), and builds, emptied roots and deleted repositories delete every revision through its own location. That also covers a build that died before its swap. A delete that fails during a build stays recorded and is retried by the next build. Served repodata is unaffected. Objects stranded in an older location before this release are not known to the builder and are not removed.
- **PyPI: the age gate's last-known-good substitute is served through the scan gate on a Remote that scans on proxy** (#5075). When the download age gate withholds a young version, a PyPI Remote answers with the newest older version it holds. That substitute was served through the plain proxy path (presigned redirect, cache stream or upstream fetch) while every other distribution download of the repository went through the scan-on-proxy gate. It now goes through the same gate as a requested distribution, on the direct route and for a Remote member of a Virtual repository (under the stricter-of-two policy): a stored blocking verdict refuses it with `403`, `fail_closed` withholds it with `423` until it is scanned, and `fail_open` serves it `pending`. Repositories that do not scan on proxy are unchanged.
- **The admin repair endpoints now audit a run before it starts** (#5078). `POST /api/v1/admin/repositories/{key}/reindex-storage` and `.../backfill-debian` wrote their audit row only when the call returned, so a call cut off by the request timeout or a client disconnect left no record of the rows it had already registered or moved, and an unknown repository key was not audited at all. Each call now writes a `started` row before any work, then `completed` (with the report counts) or `failed`, or `cancelled` when the call is dropped mid-run, tied together by a `run_id`, like the storage scrub. Existing fields (`repository`, `dry_run`, the counts, `error`) keep their names.
- **The repository `test-upstream` probe now reaches the upstream the way proxy fetches do** (#5079). `POST /api/v1/repositories/{key}/test-upstream` now applies the upstream URL validation before it sends anything (a URL that validation refuses answers `400`) and goes out on the repository's egress route (`direct` or `explicit` proxy), so its result reflects the path real fetches take. Its `502` message, and the curation sync failure messages built from HTTP client errors, no longer include the request URL.
- **The Debian backfill dry run now checks where each misfiled package would move** (#5082). `POST /api/v1/admin/repositories/{key}/backfill-debian` with `dry_run: true` counted a package whose `pool/<component>/` destination holds a different package or a trashed row as registered and relocated, while the real run refuses it. The dry run now reads the destination (without locking or writing), including one an earlier package of the same call would move into, and reports a taken destination in `artifacts_failed` with the same reason as the real run.
- **The Debian backfill counts a merged package once when its destination row is also a candidate** (#5088). When a misfiled copy merges into a live row with the same bytes, the real run writes the control document on that row, so a later candidate page no longer selects it. The dry run writes nothing, so it counted the destination again and could report one more `packages_registered` per merge than the real run. A real run also registered the destination a second time when it sat later on the same page. Both modes now remember such destination rows for the rest of the call and pass over them. A dry run resumed with `after` no longer remembers rows from the earlier call.
- **Curation sync now reaches each source Remote the way proxy fetches do** (#5089). The sweep built one HTTP client for every staging repository, so a Remote's `direct` or `explicit` egress proxy setting was not applied to its index fetches and attestation downloads, and the stored upstream URL was not checked by the upstream URL validation. Each Remote now gets a client built from its own egress configuration, and a Remote whose upstream URL the validation refuses is skipped with an info log, without a request, and is not marked as synced.
- **Google Cloud Storage error messages no longer carry the request URL** (#5093). When a GCS request failed in transport (for example a refused connection or an unresolvable `GCS_ENDPOINT`), the storage error, the `/health` storage check message and API error bodies repeated the full request URL, naming the endpoint host, the bucket and the object key. They now name only the operation and the transport error, as #4866 did for Azure. A few S3 error paths that printed the raw object-store error are trimmed the same way, and the health check strips any URL left in a backend's message.
- **A remote repository now applies the upstream outbound address policy to IP-literal destinations in every egress routing mode** (#5095). URLs a Remote reads from upstream content, such as the chart URLs in a Helm `index.yaml`, were checked against the policy only when the repository routed through an explicit egress proxy; with direct or inherited routing the connect-time resolver check was relied on, and an IP-literal host never reaches the resolver. Every upstream fetch now checks an IP-literal destination (including numeric, octal and hex IPv4 spellings, IPv6, IPv4-mapped IPv6 and NAT64 `64:ff9b::/96` addresses) against the same policy that validates a saved upstream URL before any request is sent, and a refused destination answers `502` with code `UPSTREAM_DESTINATION_REFUSED`. The configured private-address exceptions (the private-CIDR allowlist, also accepted as `UPSTREAM_PRIVATE_IP_ALLOWLIST`, and `UPSTREAM_ALLOW_PRIVATE_IPS`) apply as they do when an upstream URL is saved; loopback, link-local and cloud metadata addresses stay refused. Host names keep the connect-time check, and explicit egress proxies are unchanged. The VS Code gallery query (a JSON POST) now also goes through the repository's egress routing, and the CI OIDC token exchange checks the JWKS URL the issuer's discovery document names.
**Upgrade note:** a Remote whose stored upstream URL is a private IP literal saved before upstream URLs were validated stops fetching until that address is admitted by the private-CIDR allowlist or `UPSTREAM_ALLOW_PRIVATE_IPS`. A stored loopback or link-local upstream (for example a sidecar mirror on `127.0.0.1`) cannot be admitted; point it at a non-loopback address. When the CIDR allowlist is set, it takes precedence over `UPSTREAM_ALLOW_PRIVATE_IPS` for upstream fetches, as it already did when saving an upstream URL.
- **Image inspect and base-image info resolve a digest only within the requested repository and image** (#5096). `GET /api/v1/repositories/{key}/image-inspect` and `GET /api/v1/repositories/{key}/image-builds/base-info` resolved a tag through the image's tags but accepted any `sha256:` digest as given and read the manifest, config and provenance by digest. On storage shared by several repositories, that described a manifest stored for another repository or image. A digest now resolves when a tag of the image names it (a push by digest records one), when the image's catalog records it, or when the repository holds it and it is a platform or attestation child of an index that belongs to the image. The platform manifest and attestation an index lists are likewise read only when the repository holds them. Any other digest answers exactly like one that does not exist: 404 for inspect, `found: false` for base-info. A Virtual repository records no manifests of its own, so inspecting one by digest now answers 404, as inspecting one by tag already did.
- **OCI scan-on-proxy withholds a manifest it cannot classify under `fail_closed` instead of serving it unscanned** (#5097). The inline scan gate on a Remote OCI repository skipped every manifest it did not recognise as a runnable image, including ones it could not parse or classify. Container clients decode manifests more leniently than the server (case-insensitive keys, last duplicate key wins), so such a manifest could still run on the client. The gate now classifies positively: a manifest skips the scan only when no container runtime would run it (an index or manifest list, an artifact manifest, no config, no layers, the empty config, or no layer a runtime would unpack, which covers Helm charts, cosign signatures and attestations, SBOMs, in-toto, WASM and Flux artifacts). A manifest that does not parse, uses case-variant, non-ASCII or duplicate field names, or pairs a config that is not a recognised image config with a layer a runtime could unpack, is handled like an inconclusive scan: `423` under `fail_closed`, served with `X-AK-Scan: pending` under `fail_open` and record-only, and counted under its own `unclassifiable` verdict label. The same applies to a Remote member served through a Virtual repository, which now also refuses Docker schema1 manifests under scan-on-proxy as the Remote itself does. The scanner applicability check likewise keeps such manifests scannable. An artifact that pairs its own config with OCI image layers (a BuildKit registry cache exported with `image-manifest=true`, or an artifact pushed by older ORAS releases with `application/vnd.unknown.config.v1+json` over `application/vnd.oci.image.layer.v1.tar`) is in that last group, so a `fail_closed` scanning Remote now withholds it; proxy such artifacts through a Remote without scan-on-proxy, or with `fail_open`. Dashboards or alerts that enumerate the verdict label of the proxy scan verdict metric should add `unclassifiable`.
- **Image inspect reads config and provenance only from the requesting repository** (#5111). `GET /api/v1/repositories/{key}/image-inspect` and `GET /api/v1/repositories/{key}/image-builds/base-info` read the config blob and the provenance attestation layer that a manifest names by digest alone. A manifest can name any blob digest, so on storage shared by several repositories that could describe a blob recorded only for another repository. Both blobs are now read only when the repository records them, as its hosted blob downloads require: an unrecorded config answers exactly like a missing one (404 for inspect, `found: false` for base-info), and unrecorded provenance is left out like missing provenance. A missing object's 404 message no longer carries the storage backend's error text.
- **Hosted members of a PyPI virtual are streamed instead of read into memory when no presigned redirect is available** (#5126). On filesystem storage, or S3 without presigned downloads, serving a wheel or sdist through a virtual repository buffered the whole stored object per concurrent request. The shared local serve now opens the object as a stream, keeping memory bounded, with the existing stored-object integrity verification applied before the headers for small objects and while streaming for large ones.
- **Stop a racing repodata fold from rebuilding an unchanged RPM root** (#5128). A dirty mark that a concurrent fold read before a build cleared the queue could be applied after the build, asking for a redundant revision even though the mark was already in the build's snapshot. The fold now compares marks with the build's claim instant, and the reconcile no longer dirties a root whose build is still in flight.
- **The published backend image now builds with jemalloc, so bursts of proxy downloads no longer grow RSS until the container is OOM-killed** (#5130). The image is UBI 9/glibc, whose malloc keeps freed memory in per-thread arenas: a proxy-only mirror serving bursts of concurrent Maven downloads went from 77 MiB to 3919 MiB and stayed there while idle, then was killed at its 4 GiB limit. Both compiling stages of `docker/Dockerfile.backend` now default to `CARGO_FEATURES=jemalloc`, and `docker-publish.yml` passes it explicitly; jemalloc returns freed pages after its decay interval instead of holding every arena warm. The Linux release tarballs are built with jemalloc too, including a 64 KiB-page aarch64 build so they also start on larger-page kernels. Source builds use `cargo build --release --features jemalloc`; a glibc build that cannot switch can set `MALLOC_ARENA_MAX=2` as a mitigation. Sizing, verification and the allocator's effect on idle RSS are in `docs/operations/backend-memory-allocator.md`.
- **Finalizing a whole-body OCI layer push no longer reads the layer back** (#5133). When a client sends the whole layer in the final `PUT ?digest=` (containerd does), the backend already has its digest from streaming it, so it no longer re-reads, rehashes and rewrites the layer before answering. Filesystem storage now moves staged uploads into place with a rename for this path, for `POST ?digest=` and for the staging object of a multi-part upload; object stores keep their server-side copy. This shortens the wait that containerd 2.3 and later limit to 30 s, which surfaced as `net/http: timeout awaiting response headers` on large layers. On object storage the copy still grows with layer size. `docs/operations/oci-reverse-proxy-timeouts.md` now covers client-side limits.
- **Small downloads no longer take about 44 ms on a keep-alive connection** (#5139). Accepted connections left Nagle's algorithm on, so when a response head and a body smaller than one TCP segment went out as two writes, the body waited for the client's delayed ACK of the head. This capped a connection at about 22 requests per second for small files such as Maven `.sha1` and `.md5` sidecars, with or without a reverse proxy. The HTTP accept loop now sets `TCP_NODELAY` on every accepted connection, on the API and metrics listeners alike.
- **A crate whose published `Cargo.toml` says `readme = false` or `readme = true` keeps its identity, so a `fail_closed` Cargo Remote no longer withholds it with `423`** (#5140, #5141). Cargo writes `readme = false` into the normalized manifest it publishes for any package without a README, and the manifest parser accepted only a path there, so reading the crate's own `Cargo.toml` failed, the proxy could not confirm the archive was the crate and version requested, and a `fail_closed` Remote answered `423` on every pull of it. One reported build hit 12 common crates, including `parking_lot_core`, `thiserror-impl`, `time-macros` and `percent-encoding`. The field now accepts a path, `false` or `true`, as the Cargo manifest reference allows.
- **A scanned `.crate` is no longer graded on the packages in its own bundled `Cargo.lock`** (#5143). Many published crates ship the author's `Cargo.lock`, and both engines cataloged it, so a crate such as `async-ctrlc@1.2.0` was blocked (403 on a `scan_on_proxy` Cargo remote) for advisories against `crossbeam-deque@0.7.3` and others that no consumer installs: Cargo never resolves a dependent against a dependency's lockfile. When a crate is scanned with its pinned coordinate, any `Cargo.lock` it ships is now removed from the scan workspace, so the crate is graded on its own name and version only. `cargo install --locked` still fetches each locked crate through the registry, where each is graded on its own coordinate. A vulnerable verdict already stored for a crate's digest is still reused until it expires (30 days) or the CVE scanner version changes.
- **Hugging Face `resolve` downloads honour HTTP `Range`, so an interrupted `hf download` can resume** (#5145). The route ignored the header and answered `200` with the whole file, which `huggingface_hub` appended to its partial copy before failing its size check. It now answers `206` with `Content-Range`, or `416` for a range past the end of the file, and advertises `Accept-Ranges: bytes`, on the hosted, Remote (cached or not) and Virtual paths. Hosted files are also streamed from storage instead of being read into memory.
- **An unrelated empty hosted member no longer blocks below-threshold remote image blobs through a Docker virtual repository** (#5146). Hosted block-on-any applies only when that member references the requested blob; proxy policies, hosted referrers, and fail-closed reference lookup errors remain enforced.
- **NuGet virtual repositories resolve imported mixed-case package names** (#5147). Local member downloads now match IDs case-insensitively, as direct local downloads already do, while retaining download gates and the stored path. Other package formats keep their existing matching rules.
- **npm proxy identity lookup supports late metadata and DefinitelyTyped archive roots** (#5148, #5149). It prefers `package/package.json` and accepts a root manifest in a single alternatively named directory; ambiguous fallback roots remain withheld. Its separate entry budget defaults to 100,000 and is configurable with `MAX_NPM_IDENTITY_ARCHIVE_ENTRIES`; decompressed-byte and metadata-size limits remain enforced. A fail-closed identity refusal now returns `scan_identity_unestablished` instead of claiming a scan is pending, retaining HTTP 423 and withholding the bytes.
- **An age-gated Remote that does not host a package no longer refuses it through an npm or PyPI virtual** (#5150, #5151). The member used to answer 451 and write a pending review with no publish time, which stopped the member that does host the package from being asked. A member now steps aside when its upstream answers 404 or 410, or its package document does not list the version; that answer is read from the upstream, not the proxy cache, and is reused for at most 3 seconds. For PyPI only an index that is about the requested project can give it, so a login page or other 200 from the upstream still blocks. It writes no review and serves no bytes for that request, so if no member hosts the version the virtual answers 404. A member that hosts the version and finds it too young still blocks the virtual, as does an unreadable upstream, an upstream that lists the version with no time, a rejected review, and a `first_seen` observation still inside the window. A direct Remote answers as before.
PyPI upstreams without the Warehouse JSON API can now be dated: the gate and the HTML listing read `data-upload-time` from the simple index, and the gate also reads PEP 700 `upload-time` from a PEP 691 response, using the Remote's configured credentials. `first_seen` mode can now start its clock from the simple index on these upstreams. `min_age_days = 0` no longer blocks on an unknown publish time for any format, and on npm, PyPI and Maven it no longer depends on upstream metadata being readable; the sweep also releases pending reviews with no time at 0. A rejected review still blocks at 0. Above 0, a version with no time from any source stays blocked until approved, which is the documented rule for every format: use `first_seen` for an upstream that publishes no times.
- **Acknowledging the last open finding releases a scan-policy hold** (#5153). The scanner now records `quarantine_source = scan_policy` on the hold it writes, and an acknowledgement that clears the policy releases that hold and audits the release, instead of the download answering 409 until an administrator released it by hand. An administrator's hold, an upload-period hold, a rejection, and a hold written before this version (which records no source) are not released.
- **An acknowledged finding stays acknowledged across rescans, dedup copies and promotions** (#5153). An acknowledgement was a flag on one finding row, so every path that wrote new rows brought the finding back, and the download gate also counted rows from every scan the artifact ever had. It is now a decision about an advisory, component and version on specific bytes in one repository, with a reason, an actor and an optional `expires_at`, applied by every reader. One decision covers every engine reporting the same component, so six CVEs reported by two scanners take six acknowledgements rather than twelve. A new CVE, the same CVE on another component or version, a new build under the same version, and a re-grade above the accepted severity all stay open.
The download gate now grades the artifact's current scans (per scan type, the latest complete scan and anything after it), so a finding only an older scan reported no longer blocks; a later crashed or incomplete scan does not clear it. A promotion copies the source's verdict into the target repository when the target would grade exactly what the source grades, and its live decisions when the promoter could acknowledge in the target, so a promoted copy with a current, complete verdict is no longer treated as unscanned. Anything less copies no verdict, and a `promote:artifacts` token carries no decision. A clean verdict keeps dedup's 1-day limit, so one older than that does not travel; the promotion response and audit row say so. Promotion thresholds and `max_cve_severity` rules now count the findings open for the target, so a decision opens those gates only if it applies there or the promoter could acknowledge there and the promotion carries it. An approval now re-reads the policy and is refused where a promotion would be, unless the reviewer sets `skip_policy_check`. Deleting an artifact or repository whose scan had been copied no longer fails on the copy's `source_scan_id`, which is now cleared instead. A revoke reaches every copy a promotion made. The same bytes in another repository are not covered unless the request says `"scope": "all_repositories"`: this is narrower than copying decisions on every dedup, on purpose, so acknowledging in one repository cannot open the gate of another. When a finding's scan has copies in other repositories an acknowledge or revoke must name its repository, and a caller who can see only a copy of a scan now gets that copy instead of a 404. External scanner submissions never supersede each other. New: `expires_at`, `scope` and `repository_id` on `POST /api/v1/security/findings/{id}/acknowledge`, `GET /api/v1/security/acknowledgements`, `DELETE /api/v1/security/acknowledgements/{id}`, and the audit actions `FINDING_ACKNOWLEDGED`, `FINDING_ACKNOWLEDGEMENT_REVOKED`, `FINDING_ACKNOWLEDGEMENT_CARRIED`, `FINDING_ACKNOWLEDGEMENT_NOT_CARRIED` and `FINDING_ACKNOWLEDGEMENT_EXPIRED`.
On upgrade, an acknowledged row becomes a decision only when every row of that finding in the artifact's current scans is acknowledged. A finding acknowledged only in an older scan, or by one engine while another engine's row is open, stays open and needs acknowledging again.
- **A hosted conda subdir no row has ever named answers `repodata.json.bz2` without a regeneration permit, and two renders of one revision no longer step the JLAP tail back** (#5167, #4978, #5005). The `.bz2` of such a subdir was the one encoding that waited for a permit to compress an empty document, so it could answer 503 while real renders held every permit; it is now filled with the other encodings. Two renders of the same revision that decided visibility at different instants took turns appending each other's document to the stored JLAP; a render of the revision already recorded now carries the stored JLAP instead. Until the revision next moves, a client holding the second render's `repodata.json` finds no patch from it and downloads the index.
- **A Postgres deadlock now answers a retryable 503 instead of a 500** (#5168, #4989). Postgres breaks a lock cycle by rolling one transaction back, so a `40P01` that reaches a client means that transaction changed nothing and running it again succeeds. It answered `500 DATABASE_ERROR`, which clients read as a server bug and do not retry. It now answers `503 DATABASE_DEADLOCK` with `Retry-After`, and is still logged at error level.
- **A failed storage GC delete no longer blocks uploads of the same content for 15 minutes** (#5168, #4989). When the object-store delete failed, the `oci_gc_candidates` row stayed leased, so the upload commit guard answered 503 for that content until the lease aged out, although the delete was over and the object still stored. The failed delete now clears its own lease, as the repository-delete purge already did. The row stays and the candidate sweep still retries the delete on the same schedule with its reference re-check. A lease another GC claim or repository delete has taken since is left alone, because that delete may still land.
- **Storage reindex on the filesystem backend now finds objects whose names contain `.tmp.`** (#5168, #4323). The listing skipped every such file, so a Maven file like `x.tmp.jar` was never listed or recovered, and a live row for one was reported as a missing object. It now skips only the backend's own atomic-write staging names, `.tmp.<uuid>` and `<name>.tmp.<uuid>`. RPM version retention uses the same listing, so it now also collects such a file under a version number it is removing.
- **Storage scrub repair now checks `max_bytes` between donor copies** (#5168, #4989). The repair loop read up to five donor copies without looking at the budget, so a run of corrupt donors could overshoot it by several object sizes. It now stops between donors once the budget is spent. The first donor is always tried, so an object as large as the budget can still be repaired, and a copy that verified is always written. A repair stopped this way records that it hit the byte budget, not "no verified good copy available", since the copies it did not read may be good; a later run retries it.
- **A Remote OCI repository now repairs a corrupt stored blob from its upstream instead of failing every pull of that layer** (#5169, #5004). A stored copy that no longer hashes to its digest answered `502 STORED_OBJECT_INTEGRITY` until an operator removed it by hand. It is now treated as a cache miss: the blob is refetched, checked against its digest and recorded size, the corrupt object is overwritten, and the pull is answered from the store. If the upstream fails or its bytes do not match, the pull keeps the `502` and the object is left alone. A blob over 4 MiB is only found corrupt after its `200` went out, so that transfer is still aborted and the repair runs behind it for the client's retry. Hosted repositories keep the `502`. See `docs/operations/download-integrity.md`.
- **Scan-on-proxy grades a Debian package against its own suite's release, and a signed `.apk` keeps its stored `.PKGINFO` metadata** (#5170, #4988). A package whose version marker names another Debian release than its suite (`+deb11u1` in a Debian 12 suite) is now inconclusive, so `fail_closed` withholds it (423) instead of grading it against either feed. When a package backported from a stable update carries two markers (`+deb13u1~bpo12+1`), the rightmost is the one read; it used to be graded as Debian 13 in a Debian 12 backports suite. A marker on an Ubuntu or derivative package is not compared.
The gate no longer falls back to "the one release every cached suite agrees on", which could be another suite's. It now finds the suite the way apt's traffic leaves it findable: indexes fetched under `by-hash/`, `_all` packages in an architecture's index, the security archive's `pool/updates/`, Ubuntu's two-stream `Packages.xz`, a `Release` or index past its cache TTL, and, when a client with current lists fetched no index, the index the cached `Release` names, fetched and verified against it. A package no such index lists is graded on its version marker, or is otherwise inconclusive. `bookworm-updates` and its like (`Version: 12-updates`) now name their release, which also stops them leaving every unmarked package of a Remote that caches them withheld.
Debian's own archive currently has packages this withholds under `fail_closed`: `designate` (`+deb11u1` in `bookworm-security`), `curl` (`+deb13u2~bpo13+1` in `bookworm-backports`) and a few `+deb13uN` versions in `testing`. See `docs/security/scan-on-proxy.md`.
`AlpineHandler::extract_pkginfo` now reads every gzip member, through the reader the scan gate uses, so `parse_metadata` no longer drops the `.PKGINFO` of a signed package. Only the core format-handler test path reads that document with a package body, and it does not read `pkginfo`.
- **The pre-push hook can pass on a clean checkout again** (#5182). It ran plain `cargo test`, which many unit tests cannot pass because they rely on `cargo nextest` running one process per test, and it said no database was needed when the presigned-redirect tests need one. So it failed on every checkout and contributors had to bypass it. The hook now runs `cargo check` by default, which needs only cargo. Setting `AK_PRE_PUSH_TESTS=1` adds the unit tests under `cargo nextest` with a required database, and the hook stops with instructions if either is missing. CI still runs the unit tests on every PR, and `CONTRIBUTING.md` and `TESTING.md` now say what is needed.
- **A protobuf `Download` naming a module with no commit or label no longer returns a 500 for a module with one commit** (#5197). The latest-commit query could pick the module's `_labels` index, which is written right after the first commit under the same name, and the handler then failed reading that JSON document as a gzipped commit bundle. The query now skips the index, so the request returns the module's latest commit.
- **A Debian repository whose signing key cannot produce OpenPGP now answers 409 instead of failing or serving the wrong key** (#5198). With a `key_type=rsa` key, the default, `InRelease` and `Release.gpg` returned 500 and logged an ERROR on every anonymous request, and `gpg-key.asc` returned 200 with an X.509 PEM labelled `application/pgp-keys`. All of them now return 409 with a WARN and a message naming `key_type='gpg'`, as RPM does. This also covers a Virtual repository that signs its own `Release`; a Remote or Virtual repository serving an upstream's release files is unchanged.
`POST /api/v1/signing/repositories/{id}/config` now returns 400 when it is asked to bind such a key to a Debian, RPM or pacman repository, or to turn metadata signing on with one already bound, and 404 for an unknown key. Key creation already refused this for a repository-scoped key, but a global key or an older one could still be bound. An existing binding keeps working as before until it is changed: signing can still be turned off, and `SIGNING_AUTO_MIGRATE_LEGACY_KEYS=true` still replaces the key.
- **`GET /api/v1/search/advanced` now applies `repository_key`, `path`, `version`, `min_size`, `max_size`, `created_after` and `created_before`** (#5200). The endpoint documented and parsed all seven, but built its query from `query`, `format`, `name` and the sort fields only, so `?repository_key=sa` (and `ak artifact search --repo`) still returned artifacts from every repository the caller could read, and the size and date filters narrowed nothing. Each now narrows both the items and `pagination.total`, including the `per_page=0` count-only form, on top of the caller's existing repository visibility: naming a repository the caller cannot read returns nothing. `repository_key` and `version` match exactly; `path` is a literal prefix of the stored path (a leading `/` is ignored); `min_size` and `max_size` are inclusive byte bounds; `created_after` and `created_before` are exclusive and take an RFC 3339 timestamp or a `YYYY-MM-DD` date (midnight UTC). An unparseable date is now a 400 instead of being ignored. The facet counts are unchanged: as before, they describe everything the caller can see rather than the filtered result.
- **A `PATCH /api/v1/repositories/{key}` answered with a 400 or 422 changes nothing** (#5201). The handler saved the repository row (visibility, name, description and the other columns) and the earlier `repository_config` settings before it checked the later fields, so a rejected request could still, for example, switch a repository from `private` to `internal`. Every field is now checked before the first write: the `apt_*` fields, `custom_user_agent`, `virtual_metadata_partial`, `quarantine_enabled`, `quarantine_duration_minutes`, `release_repository_key` (including the release repository lookup), the npm scope policy and the `debian` filter config.
- **`POST /api/v1/signing/artifacts/{artifact_id}/sign` now signs for a repository configured with `sign_packages: true`** (#5208). The route used the metadata-signing key lookup, which needs `sign_metadata: true`, so a repository set up only for package signing got a 409 claiming it had "no active signing key or signing config". Artifact content signing now accepts `sign_packages` (and, as before, `sign_metadata`); metadata signing still needs `sign_metadata`. When signing is refused, the 409 names what is missing: no signing config, no `signing_key_id`, an inactive key, or `sign_packages: false`.
- **`GET /api/v1/promotion/repositories/{key}/promotion-history` now honours its `artifact_id` query parameter** (#5209). The parameter was documented and parsed but never applied, so the endpoint returned every entry for the repository and `pagination.total` counted them all. Both the page and the total are now filtered by `artifact_id` when it is given, alongside the existing `status` filter.
- **A buffered proxy fetch that its client gives up on now still fills the cache, and the cancelled request is logged** (#5220). A cold upstream fetch used to run inside the request of the client that started it: when that client timed out and disconnected (yarn after 60 seconds, for example) the fetch was cancelled, nothing was cached, and every retry started the same fetch from scratch. No line was logged for the abandoned request, so the only trace was `Fetching artifact from upstream`. The single-flight leader's fetch and cache write now run on their own task, a first step of the #4533 design: a client that disconnects no longer cancels them, and a retry joins the running fill or reads the cache it wrote. Fills tied to the request stay in it: a caller-supplied identity check (#4829), and a fill that the scan gate commits (#4514). At most `PROXY_UPSTREAM_MAX_DETACHED_FILLS` (default 16) fills run detached per replica; past that a fill runs in its request as before. A detached fill is abandoned 60 seconds after its fetch deadline, so a stalled cache write releases the single-flight slot and the cross-replica lock. A request that outwaits another request's fill now gets a 502 and, inside the grace window, the stale cached copy, instead of a 500. A request whose future is dropped before the handler answers now logs `Request cancelled` (WARN) with the `method`, `uri`, `correlation_id`, `user`, `ip` and `latency_ms` fields of `Request completed` (a panic unwinding through the middleware logs `Request panicked` at ERROR instead).
The upstream timeouts are now settings: `PROXY_UPSTREAM_CONNECT_TIMEOUT_SECS` (default 10, was a fixed 60), `PROXY_UPSTREAM_READ_TIMEOUT_SECS` (default 60, unchanged: the longest wait for the response headers or the next body chunk) and `PROXY_UPSTREAM_FETCH_DEADLINE_SECS` (default 600), a new deadline on a whole buffered upstream fetch, so an upstream that trickles bytes cannot hold a fill for as long as it keeps the socket open. A buffered object that cannot be fetched within it (128 MiB at under about 220 KB/s) is no longer cached; raise it for slow upstreams. Streamed downloads are not bounded by the deadline. The upstream client, and each Remote's egress-proxy client, which until now had no timeouts at all, also send HTTP/2 keep-alive PINGs every 30 seconds, idle connections included, and close a connection that leaves a PING unanswered for 10 seconds. Idle pooled connections are closed after 90 seconds. A stalled multiplexed connection is therefore replaced, not reused for every new fetch.
- **npm: a scoped tarball requested with an encoded slash is served** (#5221). `GET /npm/{repo}/@scope%2fname/-/name-1.0.0.tgz`, which yarn sends for some scoped packages, answered 400 ("Scoped package name must have exactly one '/'") while the unencoded `@scope/name/-/…` form returned 200. The encoded name is a single path segment, so the request matched the scoped version route with `-` as the package. That route now recognises the shape and serves the tarball, the same as the unencoded path, on Local, Remote and Virtual repositories.
- **A proxied download refused under a Package Age Policy hold can be released straight away** (#5223). The streaming proxy answered the hold's `409` at once and finished the upstream fetch in the background, but the catalog row that `POST /api/v1/quarantine/proxy-cache/{repo_key}/release` reads was only written when that fetch committed, so a release in between answered 404 "No cached artifact at that path" for as long as the fetch took. The held row is now recorded before the `409` goes out, so the release succeeds immediately, and a release that lands while the fetch is still running is kept when it commits: the committed entry is served instead of being held again. The same fix stops a second in-flight fetch of the same path from putting back a hold that was released while it ran. A new fetch of an entry that was released earlier (for example after the cached copy expires) is still a new evaluation and is held again if it falls inside the window.
- **A crate with a renamed dependency builds from a Cargo registry** (#5227). The sparse index now lists a dependency declared as `renamed = { package = "rbase", ... }` as `"name": "renamed", "package": "rbase"`, the index format, instead of copying the publish API's `explicit_name_in_toml` field, which cargo ignores in the index; cargo then looked for the crate under its real name and failed with `cannot find module or crate`. The entry is built from the metadata stored at publish, so crates already published are served correctly without being published again.
- **`puppet module install --module_repository` now installs from a Puppet repository** (#5229). The client resolves a module through `GET /v3/releases?module=<owner>-<name>`, and that route only accepted `POST` (publish), so every install failed with `405 Method Not Allowed`. The route now lists the module's releases in the Forge v3 shape (`pagination` with `next`, and `results[]` carrying `version`, `file_uri`, `file_sha256` and the module's `metadata.json` as `metadata`), honouring `limit`, `offset`, `sort_by` and `exclude_fields`. A Remote repository forwards the upstream Forge's list, and a Virtual one merges its hosted members' releases with its Remote members' (a hosted version shadows the same version upstream). The advertised `file_uri` (here and on the module, release and publish responses) is now relative to the repository root, as the Forge's is: the client appends it to the `--module_repository` URL, so the old `/puppet/<repo>/v3/files/...` form was fetched as `/puppet/<repo>/puppet/<repo>/v3/files/...` and 404'd.
- **A first install through a scanning Remote no longer loses scans to the queue** (#5231). The 30-second budget for an inline proxy scan started when the scan began waiting for a permit, not when it began running. Under `fail_closed`, a file could answer `423 scan_pending` with its scan barely started, which failed `npm install` on a host with little CPU to spare. Under `record_only` and `fail_open`, everything still queued 30 seconds after an install's burst of requests was dropped: in an `express` install, about half of the 67 files got no verdict on 24 cores, and 53 of them with two CPUs. An over-budget scan was killed too, so a retry started from nothing.
A file scan now runs in its own task and requests wait on it. The 30 seconds count from when the scan starts, and a `fail_closed` request waits at most 60 seconds for that (`AK_PROXY_SCAN_QUEUE_WAIT_SECS`). Either way it is answered within three quarters of `GLOBAL_REQUEST_TIMEOUT_SECS` of arriving. A request that stops waiting is still refused, and `fail_closed` still serves no byte without a verdict, but a running scan goes on for up to 120 seconds, and a retry joins it or finds its verdict. `record_only` and `fail_open` scans stay queued until they run; a queued `fail_closed` scan leaves the queue 30 seconds after the last request waiting on it has gone. A verdict recorded while a scan ran is no longer overwritten by it, and a verdict records the scanner version the scan started with.
The limit of four concurrent scans is unchanged. New limits bound what waits: 4,096 scans in flight and 1,024 per repository, 1 GiB of held bytes and half of that per repository (`AK_PROXY_SCAN_QUEUE_BUDGET_BYTES`), and 256 waiting requests per repository and 16 per scan. A file past one of them gets no scan for now, and counts in the new `ak_proxy_scan_queue_refused_total`. Behind a proxy with a 60-second read timeout, as nginx has by default, a long wait now answers `504` rather than `423`: see "Timeouts in front of the gate" in `docs/security/scan-on-proxy.md`. Image manifest scans are unchanged.
- **The legacy `X-Webhook-Signature` header is now `sha256=HMAC(secret, body)`, so a receiver can verify it** (#5234). This changes a value on the wire. Since the v2 wire contract (#919) the header was meant to sign the raw body alone, as the webhooks documentation and the release-gate suite describe ("`sha256=<hex>`, no timestamp"), but it was computed over `"<t>.<body>"` and so held the same hex as the `v1=` token in `X-ArtifactKeeper-Signature`. The header carries no timestamp, so a receiver that read only `X-Webhook-Signature` and followed the documented form could never verify a delivery. `X-ArtifactKeeper-Signature` (`t=`, `v1=` over `"<t>.<body>"`, and `v2=`) is unchanged, and during a rotation overlap the legacy header still uses only the current secret. **Am I affected:** only if a receiver verifies `X-Webhook-Signature`. A receiver that follows the documented body-only form starts passing. A receiver that was written against the observed value, recomputing it over `"<t>.<body>"` with `t=` taken from `X-ArtifactKeeper-Signature`, fails after the upgrade: switch it to verify the `v1=` token in `X-ArtifactKeeper-Signature` (same inputs, and it adds replay protection) or to HMAC over the body alone. The legacy `X-Webhook-*` headers are deprecated and slated for removal; new receivers should verify `X-ArtifactKeeper-Signature`.
- **`POST /api/v1/webhooks/{id}/rotate-secret` now returns the new signing secret, once, so a rotated webhook stays verifiable** (#5235). The endpoint described the new secret as delivered out-of-band, but no route returned it: a receiver could only verify with the previous secret, and when the 24-hour overlap ended it could not verify at all unless the webhook was recreated. The 200 response now adds a `secret` field, sent with `Cache-Control: no-store`, the same contract `POST /api/v1/webhooks` has at create. Capture it from that response. It is stored only encrypted and no later call returns it: `GET` and `LIST` still expose only `secret_digest`, a repeat rotation inside the window still answers 409 without it, and it is not written to the audit log or to request logs. If the response is lost, the previous secret keeps verifying until the overlap ends, and a new rotation is accepted after that; to get a new secret sooner, delete and recreate the webhook.
The admin requirement from GHSA-qcmj stays: non-admin callers and admin-owned tokens without the `admin` scope get 403. Rotation also takes the same token-scope check as `GET`, so a credential gets a secret only for a webhook it can read; any other webhook answers 404 and is not rotated.
- **An instance administrator with an unrestricted credential can list, make and revoke acknowledgement decisions in every repository** (#5250). `GET /api/v1/security/acknowledgements` and `"scope": "all_repositories"` no longer depend on which repository roles the administrator holds, so a fresh install's built-in `admin` can use them. A credential restricted to some repositories still reaches only those.
- **Alpine `APKINDEX` now lists every package under its own name and version and carries its `k:` (provider priority) and `i:` (install_if) fields** (#5254). The index generator ignored `provider_priority` and `install_if` in a package's `.PKGINFO`, so `apk add postgresql` stopped with "please select one of the 'provided by' packages explicitly" and `-doc`/`-openrc` style subpackages were never auto-installed. A package uploaded through the generic repository endpoint was also listed under the first two path segments (`P:v3.24` / `V:main`), because the metadata derived at index time never recorded `pkgname`/`pkgver`. The index now takes `P:`/`V:` from the package's own `.PKGINFO` whenever it has been parsed and emits `k:` and `i:` in the order `apk index` writes them.
Packages stored by an earlier release are repaired automatically: the next `APKINDEX.tar.gz` request re-reads each one once (at most 32 per request, packages missing from the index first) to record the new fields, and corrects the artifact's stored name and version, which listings and search read, to the package's own identity. A package whose bytes cannot be re-read keeps its existing index entry and is not retried.
- **Artifact listings now report an exact `pagination.total` and `total_pages` on `page=N` requests, so a client that pages up to `total_pages` gets every item** (#5255). Without `?count=exact`, `GET /api/v1/repositories/{key}/artifacts` reported a lower bound (the rows seen so far, plus one when another page exists) that looked exact: with 10 cached items in a remote repository at `per_page=3`, page 1 said `total=4, total_pages=2`, and a client that looped to that `total_pages` fetched 6 of the 10. The flat hosted, virtual and remote listings now count by default on `page=N` requests, including the first page of any walk. `?cursor=` pages keep the cheap total by default, so a cursor walk of a large repository stays O(page) per request, and the new `?count=lower_bound` restores it everywhere.
Every artifact listing response, flat and grouped, now carries `total_is_exact`, so a lower-bound total is machine-detectable; the `group_by=maven_component` and `group_by=docker_tag` listings keep their lower-bound default and say so. A page past the end of any listing no longer inflates the total (page 5 of that 10-item listing said `total=12`): an empty page past the start now runs the count and reports the real total.
- **Agent Plugins token URLs now really send `Referrer-Policy: no-referrer`** (#5259). The `/agentplugins/t/{token}/...` routes carry a credential in the path and set `no-referrer` so a browser never leaks the URL in a `Referer`, but the global security-headers middleware unconditionally replaced it with the default `strict-origin-when-cross-origin`. The middleware now fills in the default Referrer-Policy only when a route has not set one, the same way it already treats Content-Security-Policy (#1773), so a route can tighten it. Deployments that expose the backend directly (without a reverse proxy restoring the header) were affected; every other route keeps the same default.
- **Scanning a Maven JAR no longer reports the `provided` and `test` dependencies declared in its embedded build POM as shipped components** (#5262). The scan workspace unpacks the archive before running grype on the directory, which exposed `META-INF/maven/<groupId>/<artifactId>/pom.xml` to syft's POM cataloger; every declared dependency was then graded as if bundled, so a Maven plugin declaring `maven-core` as `provided` could be blocked on `maven-core` CVEs. Directory scans now pass `--exclude '**/META-INF/maven/**/pom.xml'`. The archive itself stays in the workspace, so its own identity and any bundled or shaded components are still cataloged from their `pom.properties`.
- **Two Agent Plugins repositories on one shared bucket can publish the same plugin archive** (#5267). Archives were stored under a content-only key, `agentplugins/{sha256}/{name}/{version}/{name}-{version}.zip`, with no repository in it, so when a staging and a production repository (or a private and a public one) on the same S3, GCS or Azure namespace published byte-identical zips, the second publish was refused with 409 "storage key ... is already owned by another repository". New archives on a shared cloud namespace are written under `agentplugins/{repository_id}/{sha256}/...`, the repository-scoped shape npm and Conan already use, so each repository keeps its own object; the content hash stays inside that prefix, so a losing concurrent publish of different bytes to the same version still cannot overwrite the winner's object. Filesystem backends and `STORAGE_KEY_SCHEME=flat` keep the previous key, and archives already published stay readable because downloads resolve the object through the key their artifact row recorded.
- **A Remote Helm repository serves upstream indexes whose chart dependencies have no `version`** (#5277). Upstreams such as `charts.rancher.io` and `rke2-charts.rancher.io` list local subcharts (`repository: file://...`) without a `version`, which Helm allows. `GET /helm/{key}/index.yaml` answered 502 `Failed to parse upstream index.yaml` for the whole repository, and a strict virtual repository with such a member answered 502 for its merged index. Dependency `version` and `name` and maintainer `name` are now optional, as in Helm, and the proxied index leaves out any field the upstream left out. An upstream index entry that still cannot be read, such as a `null` entry or one with no chart `name` or `version`, is now skipped with a warning, as `helm repo add` does, so the other charts in the index are still served.
- **RPM repodata lists source RPMs as `src`** (#5278). An uploaded `.src.rpm` was described in `primary.xml`, `filelists.xml` and `other.xml` under its header's ARCH, which in a source package is the build host's architecture (e.g. `x86_64`), so dnf treated it as an installable binary package that provides every subpackage it builds, resolved dependencies on those subpackages to it, and failed the transaction check. A source package (RPMTAG_SOURCEPACKAGE set, the rule `createrepo_c` uses) is now recorded and listed with arch `src`, so dnf never installs it or uses it to satisfy a dependency. Source packages uploaded before the fix need no re-upload: their repodata block is re-derived from the stored header on the next render (only packages that may be source packages are re-read), and a migration marks the precomputed repodata of every repository holding a `.src.rpm` for a rebuild. The `arch` field of such a package's stored artifact metadata keeps the old value until it is re-uploaded; the repodata does not depend on it.
- **`snapshot_download` works against a hosted Hugging Face repository** (#5280). With `HF_ENDPOINT` pointed at a Local `huggingface` repository, `HfApi().model_info` and `snapshot_download` failed: the model-info document had no `id` key (which `huggingface_hub`'s `ModelInfo` requires) and a revision name where a timestamp belonged, its `siblings[].rfilename` were the stored `{model}/{revision}/{file}` paths of every revision instead of the requested revision's paths inside the repository, and the synthetic commit it reported as `sha` was not understood by the tree and resolve routes, so every file `snapshot_download` requested at `/resolve/<sha>/<file>` answered 404. Model info now has the Hub's shape (`id`, `modelId`, Hub timestamps, relative `rfilename`s of the requested revision only), the model-info, tree and resolve routes accept the synthetic commit in place of the revision it stands for, model info without a revision describes `main`, and an unknown revision answers 404 with `X-Error-Code: RevisionNotFound`. `list_models` entries also gained `id` and a real `lastModified`. Remote (proxy) repositories are unchanged. README and the format registry no longer claim dataset support, which the format never had.
- **A peer subscription or sync policy can no longer be created with the `pull` replication mode, which was accepted, was the default, and never replicated anything** (#5293). Only `push` and `mirror` subscriptions ever got sync tasks, so a subscription made with `pull`, or with no mode at all, silently never moved a byte. `POST /api/v1/peers/{id}/repositories` and the sync-policy create and update routes now refuse `pull` with 400 and a message that names `push` and `mirror`; an omitted mode is `push` on both entry points, and the column default is `push` too (migration 330). Sync policies also reject an unknown mode at write time instead of failing later at evaluation. Existing `pull` subscriptions and policies are left unchanged, logged once at startup, and subscriptions report a new `effective` field (`active` or `inactive`), so a row that never replicates says so.
The new [air-gapped deployment guide](docs/air-gapped.md) collects what a disconnected install needs: topologies, images, the Remote settings, the scanners' databases, the background jobs that reach out and how to turn each off, format notes, and a verification checklist. A unit test checks that every Artifact Keeper env var and API route the guide names exists in the tree.
- **A package uploaded through the generic `PUT /api/v1/repositories/{repo}/artifacts/{path}` endpoint is now named, described and indexed exactly as its format's own upload route records it** (#5320, #5254). The endpoint used to take the artifact's name and version from the first two path segments and record no format metadata (except an RPM's header), so an Alpine package at `v3.24/main/x86_64/nginx-1.30.6-r0.apk` was listed in `APKINDEX` as `P:v3.24` / `V:main`, a Maven jar was missing from `maven-metadata.xml`, and npm and Debian packages lost their dependencies. A package at its format's native layout path now gets the same parse the native route and the chunked upload completion run, before anything is stored: Alpine (`APKINDEX`, identity from the package's `.PKGINFO`), Debian (`Packages`), Maven (`maven-metadata.xml`), npm (the packument), PyPI (the simple index), RPM and Incus (`images.json`). A package the native route refuses (a control file, `package.json` or wheel `METADATA` that names another package or version than the path) is refused with 400 and nothing is stored; an unreadable `.apk` is stored unindexed, as the native route stores it. Any other file, and any package of another format, uploads exactly as before. Chunked uploads to an Alpine repository get the same Alpine parse. Packages uploaded before this change keep their rows; upload them again (or through the native route) to re-index them.
- **`snapshot_download` through a Hugging Face Remote now gets the files in subdirectories** (#5321). `huggingface_hub` builds a snapshot from `api/models/<id>/tree/<commit>?recursive=true`, and the Remote proxied the tree request without its query, so the Hub listed only the top level and files such as `1_Pooling/config.json` were left out while `snapshot_download` still reported success; offline or on a disconnected site the model was incomplete. The tree route now forwards `recursive`, `expand` and `cursor` in one canonical spelling, so each distinct listing is cached on its own (and is the path the upstream filter sees), and follows the Hub's `Link: rel="next"` pages, returning them as one list. A page shorter than the Hub's page size is the last one and needs only the cached fetch; a full page whose continuation cannot be read fails the request rather than being served as the whole listing. Local repositories honour `recursive` too: without it, subdirectories are listed as `directory` entries, as on the Hub.
- **A `HEAD` on a Remote no longer downloads the whole file from upstream and throws it away, so a new file is fetched once instead of twice** (#5322). `hf_hub_download` sends `HEAD` and then `GET` for every file it resolves. On a cache miss the `HEAD` ran the download path: it opened an upstream `GET` of the whole file and dropped the body unread, which cached nothing, so the `GET` that followed downloaded the file again. For multi-GB model weights that doubled egress and time. A cold `HEAD` is now answered from an upstream `HEAD` (same upstream filter, origin policy and Package Age Policy hold as a `GET`; a cached file is still answered from the cache), with the `ETag`, `Content-Length` and `X-Repo-Commit` `huggingface_hub` needs. An upstream that refuses `HEAD` gets the `GET` path as before. This covers every format served through the shared Remote download path (Hugging Face, RPM, RubyGems, Puppet, Hex, CRAN, Ansible, Bazel) and the Remote members of Virtual repositories of those formats.
`ak_proxy_cache_lookups_total` now counts the streaming proxy path too, `HEAD` and `GET` alike, with the `result` values `hit`, `miss` and `negative_hit`. It used to count only buffered metadata fetches, so file and `HEAD` hits were missing from the hit ratio. See [proxy cache metrics](docs/operations/proxy-cache-metrics.md).
- **A gated model on a Hugging Face Remote without access now raises `GatedRepoError` instead of being retried as a server error** (#5323). When the Hub refused the Remote with `401`/`403` and `X-Error-Code: GatedRepo`, the client got a `502` (or a `400` when the Hub's `WWW-Authenticate: Bearer realm="Authentication required"` was mistaken for an OCI token challenge), so `huggingface_hub` retried five times with backoff. Resolve, model info and tree now answer with the Hub's status and an `X-Error-Code` `huggingface_hub` recognises: `GatedRepo` for a gated model, and `RepoNotFound` for a `401` on a missing or private model (`RepositoryNotFoundError`). `X-Error-Message` names the Remote and `PUT /api/v1/repositories/{key}/upstream-auth`. The Hub's message and the Remote's upstream token are never passed on, the refusal is not cached, and a Remote with an upstream credential that has access downloads as before. An Artifact Keeper user who cannot read the Remote still gets `404`. `docs/huggingface.md` describes the three cases.
- **A `mirror` peer subscription's `replication_schedule` now runs** (#5342). The cron schedule was stored and shown, but nothing acted on it, so `mirror` behaved exactly like `push` and existing content moved only when someone called `POST /api/v1/peers/{id}/repositories/{repo_id}/sync`. The sync worker now checks enabled `mirror` subscriptions about once a minute and, when a schedule is due, queues every artifact the peer has no task for (the same work as the `sync` route), setting `last_replicated_at` in the same transaction so it advances only when the pass completes. A subscription that has never completed a pass runs one on the first check, and missed occurrences add up to one pass. Scheduled passes honour the upstream-jobs switch (#5288): none runs while it is on. An invalid cron expression is now refused with 400 instead of being stored and never firing.
- **A `HEAD` on a Maven, npm, PyPI, OCI or other Remote no longer downloads the file from upstream and throws it away** (#5343). The fix for #5322 covered the formats served through the shared Remote download path; the handlers that stream their own Remote downloads still answered a cold `HEAD` by opening an upstream `GET` of the whole file and dropping the body unread, which cached nothing, so the `GET` that followed downloaded the file a second time. Clients that probe before they download (Maven Resolver, Gradle, `docker pull`, containerd, skopeo) paid double egress for every new artifact. A cold `HEAD` is now answered from an upstream `HEAD` with the headers the `GET` would send (`Content-Length`, `Content-Type`, `ETag`, `Content-Encoding`, and the format's own: `Docker-Content-Digest`, Maven's `X-Checksum-SHA1`, the attachment file name), and a cached file is answered from the cache as before. An upstream that refuses `HEAD` gets the `GET` path. Covered: Maven, Gradle, sbt, npm (Remote tarballs and a Virtual's last-known-good substitution), PyPI, Cargo, NuGet, Conda, Debian, Alpine, Helm, Go, Composer, Conan, CocoaPods, Swift, Pub, Chef, JetBrains, VS Code, Git LFS, Terraform providers and mirror archives, RPM packages, Bazel, the generic repository download route, and OCI blobs on a Remote and through a Virtual.
- **An npm Remote that cannot reach or may not contact its upstream now says why on the `/-/` meta endpoints, instead of answering `501 "... is not implemented for local repositories"`** (#5345). `npm audit signatures` (`GET /-/npm/v1/attestations/...`), `npm search` and the other `/-/` reads fell through to the Local repository stub whenever the upstream request failed. A Remote now answers `503 upstream unreachable` when its upstream cannot be reached, and `404` with the gate's own reason when the upstream filter refuses the path or the repository is cache-only (the cache-only reason was previously reported as an upstream filter refusal). An attestation answer still held in the attestation cache is served instead. A Virtual repository whose Remote members cannot answer returns `503` (a member was unreachable) or `404` listing each member's refusal. `/-/ping` and `/-/whoami` are still answered locally when the upstream is unreachable, Local repositories keep their `501`, and the audit POST endpoints keep their empty-advisories fallback. See `docs/air-gapped.md`.
- **Packages and package versions report their real download counts** (#5346, #5260). `download_count` on `GET /api/v1/packages`, `GET /api/v1/packages/{id}` and `GET /api/v1/packages/{id}/versions` read the `packages` / `package_versions` columns, which nothing ever incremented, so every package in every format showed 0. The counts are now computed when the catalog is read, from the same download history the artifact download stats use: a version counts the downloads of its files in `download_statistics` (the artifact with the version's checksum and the files recorded under the package's name and version, such as every file of a Hugging Face model revision), plus, on a Remote repository, the pull-through serves in `proxy_download_statistics` that carry the version's digest. A package reports the sum of its versions. Nothing is added to the download path, and a listing page costs one extra query. A Remote version cataloged without a digest (a streamed Maven pull-through) cannot be tied to its serves and reports `null` instead of a misleading 0; `download_count` is therefore nullable in the OpenAPI schema. The stored columns are left in place, unused.
- **The upstream `HEAD` that answers a cold `HEAD` is recorded as an `UPSTREAM_FETCHED` audit event** (#5347). With `AUDIT_UPSTREAM_FETCHES` set to `all` (or `errors`, for a failed probe), every request the proxy sends upstream is recorded, but the `HEAD` probe added for #5322 used the plain sender and was missing, so an egress audit did not see cold `HEAD`s. It now records one event with method `HEAD` and caller `proxy`. A `HEAD` answered from the cache contacts nothing and records nothing, as before.
- **The age-gate `451` is no longer cacheable** (#5356, #5257; 1.11.x #5371). An age-gate block now carries `Cache-Control: no-store`, so a CDN, client cache or proxy cannot keep refusing a version after a review approves it or it ages past the gate (RFC 7725 makes a `451` cacheable by default).
- **MCP capability snapshot history no longer grows without bound, and a stale or duplicate introspection completion no longer stores a snapshot** (#5386, #5389). Every remote probe added a new snapshot row, even when nothing had changed, and the scheduler re-queued every remote of every version of a server every `max(ttlMs, 6 h)`. An unchanged remote therefore added four identical rows a day per version, forever. A snapshot that repeats the newest one of the same version, source and origin (same content hash, outcome and auth state) is no longer stored again: the existing row's new `lastSeenAt` is bumped and it is returned, with no audit, lint or `tool_drift` (`POST .../capabilities` reports this as `created: false`). The scheduler pass now keeps the newest 20 snapshots per version, source and origin and prunes older ones. It never prunes the current snapshot, the drift baseline, a snapshot a `tool_drift` finding names, or the one the newest job of a target points at. Re-probes are queued only for a server's latest version and for versions published in the last 30 days. Migrations 332 (`last_seen_at` column) and 333 (a concurrent index on `mcp_introspection_jobs.snapshot_id`).
Completing an introspection job also stored the snapshot (and audited it, linted it and raised drift) before ending the claim, and ignored whether the claim was still current. So two concurrent completions of one claim, or a completion racing the lease reaper or a re-claim, could both store a snapshot and a drift finding. The claim check and the snapshot insert now run in one transaction, the side effects run only after it commits, and a late or duplicate completion (or retry) stores nothing and gets `409 mcp_claim_invalid`.
- **One MCP publisher can no longer starve the introspection queue** (#5387). Introspection jobs were handed out oldest first across every repository, so a publisher that pushed many versions with several stdio packages filled the queue with `sandbox_run` jobs, and each one that timed out held a worker for up to three 120 s attempts while every other repository's probes waited. A claim now picks the repository with the fewest jobs claimed at that moment, then its oldest due job, in the same single `FOR UPDATE SKIP LOCKED` statement; and a `sandbox_run` that times out is retried once instead of twice, so its second timeout is final.
- **An MCP resolver fill now has one deadline over the whole fill, response body included** (#5395). The in-process request that fills a Remote backing repository was bounded only until the response head arrived; the body was read afterwards with no deadline, so a slow upstream trickling a package could hold a resolver job, and its worker slot, indefinitely. Every request of a fill (head and body, both PyPI requests) now has to finish within one 300 s deadline, and a fill that misses it is recorded as a retryable failure and retried with backoff like any other transient fill error.
- **A Remote can be created with its upstream mode and egress proxy, and an MCP Remote no longer starts its first sync before its settings are stored** (#5402). An MCP Remote is synced by the scheduler as soon as it is due, and a Remote never synced was due at once, so its first pages of the upstream registry could go out before a following `PUT .../egress-proxy` or `PUT .../upstream-mode` applied; the only safe order was to pause every scheduled upstream job instance-wide around the create. `POST /api/v1/repositories` now takes `upstream_mode` (`online` or `cache_only`) and `egress_proxy` (`mode`, `proxy_url`, `no_proxy`, the body of the `PUT` route) for Remote repositories, validates them before the repository exists (400 on any other repository type) and stores them before anything else; a `cache_only` creation is recorded in the creation audit row. MCP Remotes gain `mcp_remote_sync_paused` (create and `PATCH`), which keeps the scheduled sync off one Remote until it is cleared, and a Remote that was never synced is not due until it is one minute old, so a scheduler tick that lands during the create cannot start a pass without its create-time settings. A Remote created `cache_only` makes no upstream request at all; while scheduled upstream jobs are paused no Remote syncs on its own, as before. `docs/mcp-registry.md` and `docs/air-gapped.md` describe creating a Remote in its final state.
- **A manual MCP Remote sync runs in the background and answers `202`, instead of being cut off by the request timeout** (#5404). `POST /api/v1/repositories/{key}/mcp/sync` ran the whole pass inside the request. Against the official MCP registry a pass often takes longer than `GLOBAL_REQUEST_TIMEOUT_SECS` (120 s), so the caller got `503 Server overloaded, please retry: request timed out`, and the pass was dropped wherever it was, without recording its summary. The trigger now takes the repository's sync lock, starts the scheduler's pass on a task of its own and answers `202 {"repository", "status": "running", "statusUrl"}` at once; the pass finishes whatever happens to the request. `GET .../mcp/sync` adds `running` (a pass of the repository holds its lock, on any replica), and `lastFinishedAt`, `lastError` and `lastSummary` report the pass once it ends. A trigger while a pass runs still answers `409` and starts no second pass. A trigger while scheduled upstream jobs are paused (`upstream_jobs_paused`, #5288) now answers `409` without contacting the upstream, as the scheduled sync does. An unreadable first upstream page, which the trigger answered `502`, is now recorded as the pass's `lastError`.
- **A config export now carries a Remote's cache-only upstream mode, and plan and apply carry it to the target** (#5407). `GET /api/v1/admin/config/export` (#5292) left out `upstream_mode` (#5282), so re-applying a disconnected site's export to a rebuilt instance gave Remotes that contacted their upstream again. A Remote stored as `cache_only` now exports `"upstream_mode": "cache_only"` (read through `GET .../upstream-mode`; `online`, the default, is omitted), `mode=plan` reports a difference under `changes`, and `mode=apply` writes it through `PUT .../upstream-mode`. The instance-wide `PROXY_UPSTREAM_MODE` override is not exported. `docs/schemas/config-document.v1.json` and `docs/config-as-code.md` describe the field.
- **`POST /api/v1/curation/repos/{key}/sync` starts a Remote MCP registry's sync in the background, instead of being cut off by the request timeout** (#5422). The route ran an MCP Remote's catalog pass inline, so a pass of the official registry outlasted `GLOBAL_REQUEST_TIMEOUT_SECS` (120 s): the caller got `503`, and the pass was dropped mid-run, the bug #5404 fixed on `POST .../mcp/sync`. For an MCP Remote the route now starts the same background pass. It answers `202` with `triggered: true`, `succeeded: true` and a `status_url` (`GET /api/v1/repositories/{key}/mcp/sync`). On a `202`, `succeeded` means only that the pass started; before, it meant the pass had read its first page. A pass already running or paused upstream jobs answer `409`, as on the MCP route; before, the route answered `200` with `succeeded: false`. The `CURATION_SYNC_TRIGGERED` audit row records `background: true` for these. Staging repositories are unchanged: `200` once the pass ran, with `succeeded` its result.
- **A fail-closed proxy pull withheld with `423` because its scan was inconclusive now logs why** (#5473). No scanner configured, a scan that failed and a scan that ran out of its inline budget all answer the same `423` "retry shortly", but only the budget case resolves on a retry, and nothing recorded which one a client had hit. The gate now writes a `warn` line with the repository, file, digest and the reason (`NoScanner`, `ScanFailed` or `BudgetExceeded`). The response is unchanged.
- **Proxy load can no longer exhaust the main database pool through download recording or cross-replica locks** (#5475). The inline download recorder took a pooled connection on every proxy serve with no ceiling, and with `PROXY_SINGLEFLIGHT_ADVISORY_LOCKS_ENABLED=true` every advisory lock detached a connection the pool then replaced, also unbounded. The recorder is now capped at the same ceiling as the deferred recorder (#3778) and skips the download count past it, and `PgAdvisoryLock` caps its detached connections, releasing the key before refusing so the caller falls back to per-process single-flight. `PROXY_SINGLEFLIGHT_LOCK_MAX_CONNECTIONS` sets the lock cap and defaults to `PROXY_SINGLEFLIGHT_MAX_LOCK_WAITERS` + 1.
- **An anonymous pull answers a retryable 503, not 401, when the repository lookup hits a database outage** (#5476). Format routes and `/repositories/{key}/download/*path` read the repository row and treated any lookup error as "no such repository", so a pool timeout or a database restart gave anonymous Bazel and cargo clients a 401, which they treat as fatal, and whole CI jobs failed on a few seconds of pool exhaustion. An unavailable database (the `is_db_unavailable` cases from #4750) now answers 503 with `Retry-After`. A missing or private repository still gets the same 401 as before, so the answer is no existence oracle (#1808), and a lookup error is still not negative-cached (#3750).
- **`.apk` downloads honour `Range`, so a client that fetches an archive as separate spans gets the bytes it asked for** (#5478). The alpine package download ignored the `Range` request header and answered `200` with the whole archive. An apk consumer that fetches a package's signature, control and data sections as three ranged `GET`s and checksums each span therefore checksummed the whole file three times and failed all three, which makes an Alpine mirror unusable to those clients. The alpine package download now answers `206` with the requested window and `416` for an unsatisfiable one, reusing the existing `parse_byte_range` / `slice_byte_stream` / `ranged_stream_response` helpers (#1847) on the hosted, proxy-cached and cold-cache arms alike, and it advertises `Accept-Ranges: bytes` on an unranged serve so a client that probes first knows it may range at all.
- **5-field cron schedules run on the weekday they name** (#5487). Backup schedules and lifecycle policies take standard 5-field cron, but the weekday field went to the `cron` crate unchanged, and it numbers weekdays 1-7 from Sunday where standard cron numbers them 0-7 with 0 and 7 both Sunday. So `0 3 * * 1` ran on Sunday instead of Monday, every numeric weekday ran one day early, and `0 3 * * 0` was refused as invalid. Numeric weekdays, ranges, lists and steps in a 5-field expression are now translated to day names before parsing; `N/step` runs from N through Sunday as in standard cron, and long day names (`monday`, `tues`, `thurs`) are read in the same field. Day names were already read correctly, and 6- and 7-field expressions keep the crate's own numbering. Upgrading: a schedule written with numeric weekdays moves to the day it names, one day later than it has been running.
This affects backup schedules, lifecycle policies and lifecycle policy assignments, a peer's `replication_schedule`, and a 5-field `GC_SCHEDULE` or `STORAGE_STATS_SCHEDULE`; `*/n` steps and day-name schedules run on the same days as before. If you compensated for the bug (for example wrote `2` to get Monday), change the schedule back to the standard number, or it will now run a day later than you intend. A backup schedule runs once more at its stored `next_run_at`, on the old day, and moves to the new day from then on.
- **A virtual repository whose member fails answers a retryable server error instead of a definitive "not found"** (#5495). Two shapes of the same bug. The Maven virtual checksum walk treated every unsuccessful member fetch alike, so an upstream member answering 5xx or timing out produced the same `404` as a member that genuinely does not have the coordinate; the walk now distinguishes a miss from a failure and answers `503` when a member failed and nothing else served the file (a permanent upstream `4xx` stays a miss, and the artifact walk answers the same way, see #5504). Separately, the virtual member authorization on the cargo, conda, Debian pool, Go module, npm tarball, PyPI metadata/file and VS Code gallery paths went through the helper that flattens a failed visibility query into the empty member set, which those callers then report as `404` (or, for the gallery, an empty listing); they now use the fallible form, so a database failure surfaces as a server error. Both directions still fail closed: a member the caller may not read is still dropped silently, so member existence is never disclosed.
This matters because `404` is a definitive answer that package clients cache. Gradle, Maven, Cargo and npm will not re-ask for a coordinate they have been told does not exist, so a few seconds of upstream or database trouble turned into a build failure that persisted long after the cause had cleared. A structural test now requires every client-facing virtual path to authorize through the fallible helper; the OCI blob/manifest walk is the one remaining exemption and names its reason in the source.
- **A delete made while a peer was offline is now replicated once the peer returns** (#5497). The delete fan-out filtered on `pi.status IN ('online','syncing')`, but the sync worker already skips offline peers when it claims a task, so the second gate only meant the task was never queued at all. The peer kept the artifact forever, and a backfill could not repair it because backfill only pushes. The push fan-out never had this filter, so a peer that went offline replicated later uploads but none of the deletes in between.
- **The generic download route honours `Range` on proxied serves** (#5502). A Remote repository's proxy-cache hit or upstream fetch, and a Virtual repository's serve through its members (Remote members included), answered a ranged GET through `/general/{key}/{path}` or `/{key}/download/{path}` with `200` and every byte, so clients that fetch byte windows and check each against its own digest (apk segment fetches) failed and fell back to the origin. Both arms now cut the client's range from the serve with the helper the Hugging Face `resolve` route uses since #5145, which moves next to `ranged_stream_response`: `206` with `Content-Range`, `416` for an unsatisfiable range, `If-Range` honoured, HEAD unchanged. `Range` is still never forwarded upstream, so a cold serve keeps filling the cache with the whole object.
- **Virtual repositories answer `404` for a member's permanent upstream `4xx` and `503` only when a member is temporarily unavailable, on artifacts and checksums alike** (#5504). The Maven virtual checksum walk counted every non-`404` member answer as a failure, so a Remote member whose upstream answers `403` for a missing key (S3-hosted Maven buckets do) or `401` for stale credentials turned every `.sha1`/`.md5` no member has into a `503` that clients retried forever. A permanent upstream `4xx` or a rejected path is now a miss. A member counts as unavailable for an upstream `5xx`, for the retryable `408`, `425` and `429` (rate limiting), for a timeout or transport failure, or for a fault on the server's own side. On the checksum walk only, a hosted member that has the artifact row but whose storage read fails also counts as unavailable, where it read as a miss; hosted members on the artifact walk are left to a follow-up issue. The shared artifact walk behind most formats' virtual downloads (Maven, sbt, npm, Cargo, Conda, Debian, Go, Composer, Hex, Alpine, Chef, Git LFS, JetBrains, Protobuf, Terraform, VS Code, the generic `/download/` and `/general/` routes, and the scan-aware walks of Cargo, Conda, VS Code, Conan, CocoaPods, Pub, Swift and RubyGems, including members that scan on proxy) used to read a member's upstream `5xx` as a miss, so `lib-1.0.jar` answered the `404` Gradle caches while `lib-1.0.jar.sha1` answered `503`. It now passes over an unavailable member exactly as over a missing one, still serves from a lower-priority member that has the file, and answers `503` instead of `404` only when nothing served and a member was unavailable. A member's quarantine or policy block, and a Maven or Go member's age refusal, still take precedence.
- **Restoring a trashed artifact now cancels its queued peer deletes and pushes it again** (#5505, #5497). A delete task queued for a peer that was offline when the artifact was deleted stayed pending after an admin restored the artifact from trash, so the peer deleted the restored artifact when it came back and nothing re-pushed it. `TrashService::restore` now queues a push for every eligible push or mirror peer, forcing a finished push back to `pending` because the peer may have applied the delete before the restore, and cancels those peers' pending and failed delete tasks for the artifact. A delete for a peer the restore does not push to (a pull-mode subscriber, a peer a sync policy filter now excludes, or a disabled subscription) is kept, so the peer still drops the artifact. A delete a worker already claimed is left to finish and the push follows it. The cancellation and the push are written in the restore's own transaction, so a crash cannot leave a live artifact with a pending delete.
A peer that stays offline longer than the trash retention still keeps the artifact: the trash purge and storage GC hard-delete the row, and `sync_tasks.artifact_id` cascades, so the queued delete is dropped with it.
- **A ranged download from a cold Remote cache now fills the cache** (#5509, #5554). The proxy's write into its cache is driven by the client reading the body, and the HTTP server stops reading a body once its `Content-Length` bytes are out. A ranged `GET` answered `206` from a cold fetch with a window ending before the end of the file (the signature and control spans of an `.apk` that rules_apko fetches as three ranged `GET`s, or any mid-file range on the generic and Hugging Face routes) therefore cut the cache write short: it was rejected as truncated, the upstream fetch was aborted, and requests sharing that fetch started over, so every repeat went upstream again and only a range ending at the end of the file was ever cached. A streaming fill whose client stops reading after the first bytes now reads the rest of the object on its own task and commits it, so the next request is a cache hit. These fills share the `PROXY_UPSTREAM_MAX_DETACHED_FILLS` limit with the buffered fills that already outlive their request (#5220), and the same deadline (`PROXY_UPSTREAM_FETCH_DEADLINE_SECS` plus 60 seconds), after which an unfinished one is abandoned; with no slot free a fill ends with its client as before. The Alpine package route now ranges its proxy-cache, cold-fetch and Virtual serves with the same helper as the generic and Hugging Face routes, so it also honours `If-Range` (on a proxied serve a stale validator gets `200` with the whole file; a hosted package has no ETag, so any `If-Range` gets the whole file) and answers a `HEAD` with `Range` with `200` and no body, as those routes do. `Range` is still never sent upstream.
- **A Basic credential with a multibyte character across byte 8 of the password no longer drops the connection** (#5525). The API-token lookup sliced the first 8 bytes of the credential as a `token_prefix` after checking only its byte length, so a Basic credential of `x:1234567é` on a format route such as `GET /pypi/x/simple/` panicked with "byte index 8 is not a char boundary" and the client saw a closed connection instead of a response; a user whose password had such a character could never use Basic auth on the format routes. `AuthService::validate_api_token`, `TokenService::validate_token` and `TokenService::has_scope` now treat a credential whose first 8 bytes are not a character boundary like any other malformed token: a 401, still burning the dummy bcrypt so the shape of the credential is not leaked by timing.
- **A scan whose findings carry long version or source strings is stored instead of failing** (#5534). `scan_findings.affected_version`, `fixed_version` and `source` were limited to 100 characters (`affected_component` to 255, `title` to 500, `source_url` to 512), and findings are written in one batch, so a single advisory listing several fixed versions in one string failed the whole scan with `value too long for type character varying(100)` and dropped every finding it reported. Migration `339_scan_findings_text_columns.sql` changes those six columns to `TEXT`; on PostgreSQL 16 and later this is a catalog-only change (no table rewrite or index rebuild) that briefly takes an exclusive lock on `scan_findings`, waiting at most 5 seconds per attempt and retrying, and it drops and recreates the `scan_findings_effective` view in the same transaction. Two operator notes: if every lock attempt times out (about 35 seconds, for example behind a long-running transaction on `scan_findings`), the migration fails and the server exits at startup, leaving nothing half-applied, and the next start retries it; and because the column types change, a database connection that prepared a findings query before the migration answers `cached plan must not change result type` on every later execution, so during a rolling upgrade replicas still on the previous version can fail findings reads (findings list, policy evaluation, SBOM) until they are replaced or their pooled connections recycle (up to 30 minutes). A finding that still cannot be stored (a NUL byte, or a `cve_id` over 30 characters) now fails only its own scan, with an error message naming the finding and the column, and the remaining scanners for the artifact still run; before, the scan stayed `running` until the stuck-scan janitor failed it and the later scanners were skipped. The `source` findings filter now accepts values up to 1024 characters.
- **CI and gateway capability reports now apply the worker result's observation and cache metadata bounds and error code** (#5543). A report's `ttlMs` was only checked for being non-negative, with no upper bound, and its `observedAt` and `cacheScope` refusals answered `mcp_invalid_capabilities`, unlike the same metadata on job completion. Reports now share `validate_metadata` with job completion: an `observedAt` more than the 5-minute clock-skew window ahead, a `cacheScope` other than `public` or `private`, or a `ttlMs` outside `0` to `2592000000` (30 days) is refused with `400 mcp_invalid_result_metadata` (previously `400 mcp_invalid_capabilities`) and the offending `field`, and nothing is stored. A malformed report body still answers `mcp_invalid_capabilities`. Gateway operators that parse the error code must accept the new one.
- **Purging an artifact, deleting it with `?permanent=true`, or deleting its repository now removes it from push and mirror peers too** (#5624). `sync_tasks.artifact_id` cascaded on the artifact row (migration 008), so removing the row also removed the delete tasks queued for its peers, and the peers kept serving the artifact; with `?permanent=true` the delete was queued and then dropped within the same request. Migration 340 changes that foreign key to `ON DELETE SET NULL` and gives each delete task the repository key and path it removes on the peer (`target_repository_key`, `target_path`), so the task outlives the row and the sync worker sends `DELETE /api/v1/repositories/{key}/artifacts/{path}` from it. `DELETE /api/v1/admin/trash/{id}`, `?permanent=true` and each batch of a repository delete job queue those deletes for every push/mirror subscriber in the same transaction that removes the rows; a repository delete addresses them by the key the repository had before the job renamed it. When the same key and path hold a live artifact again (re-uploaded, restored, or in a repository re-created under that key), the delete is cancelled only for a peer that artifact is pushed to, and the push waits for an in-flight delete of that path; a peer with a disabled subscription or a policy that excludes the new artifact still gets the delete. The sync worker sweeps the push tasks of removed artifacts that the cascade used to delete.
Upgrade notes. **340** takes `ACCESS EXCLUSIVE` on `artifacts` and `sync_tasks` together, in the order the application uses, and holds both through the backfill of unfinished delete tasks (about 15 µs per row) and the catalog swap; nothing is rewritten or scanned for validation (the columns are nullable without defaults, the foreign key is re-added `NOT VALID`). Requests touching either table wait for that window. Each attempt waits at most 5 s for the locks; a lock timeout or a deadlock is retried, five attempts in about 35 s. If all fail, the server exits with nothing of 340 applied, and the next start retries it. **341** builds `idx_sync_tasks_artifact_id` with `CREATE INDEX CONCURRENTLY`, which blocks no reads or writes; without it every deleted artifact row scanned `sync_tasks` for its foreign-key action. Like 297, the build first waits for transactions already open on `sync_tasks`, so a long-running transaction delays the start; an interrupted build is dropped and retried at the next boot.
- **Git LFS objects and MCP server versions are stored under repository-scoped keys on cloud storage, and an MCP approval that cannot store the version no longer answers `200`** ([#5659], [#5660], #5672). On S3, GCS or Azure, Git LFS objects were written to the flat key `gitlfs/<oid[..2]>/<oid>` and MCP server versions and bundles to `mcp/<name>/<version>/...`, keys every repository shares. Pushing an LFS object another repository already held (a fork or mirror pushing the same object) therefore failed with `409 CONFLICT` "refusing cross-repository overwrite" (a regression in this release: 1.11.12 wrote the shared key without the cross-repository guard), and approving or publishing an MCP server version another MCP repository already held failed the same way. New objects now go to `gitlfs/<repository-id>/...` and `mcp/<repository-id>/...`; objects stored before keep serving from the key their row records, nothing is migrated, and filesystem storage and `STORAGE_KEY_SCHEME=flat` keep the flat keys. `POST /api/v1/repositories/{key}/mcp/catalog/{id}/approve` (and the pin route) also answered `200` when storing the ingested version failed, logging the error and never serving the version; it now answers the storage error (`409 CONFLICT` with the guard's message, `507 QUOTA_EXCEEDED`, or `500 STORAGE_ERROR`). The decision is still recorded and the next sync retries; an unreachable upstream still answers `200` with `ingestError`. See `docs/mcp-registry.md`.
- **Maven metadata (`maven-metadata.xml` and its checksums) through a Virtual where no member serves and at least one is unavailable (5xx, 408/425/429, DNS/connect/TLS failure or connect timeout) now answers 503 with Retry-After and is not cached, instead of a cacheable 404; a member that misses the 20 s deadline still gets the 502** ([#5669], #5670). This is the maven-metadata subset of #5621.
- **Maven SNAPSHOT aliases, checksum sidecars and SNAPSHOT version metadata now always agree with the stored files** ([#5664], [#5665], [#5667], #5677). The `-SNAPSHOT` alias of a main jar (`…/1.0.0-SNAPSHOT/shop-core-1.0.0-SNAPSHOT.jar`) could serve the newest `-sources` or `-javadoc` jar of the build, with a matching alias `.sha1`, so clients could not notice; it now resolves to the newest build of exactly the requested classifier and extension. A `.md5`, `.sha1` or `.sha256` upload for a stored artifact must now name that artifact's digest (otherwise `400 VALIDATION_ERROR`, "Checksum mismatch"), and once a released (non-SNAPSHOT) artifact has a sidecar stored, replacing it answers `409 CONFLICT`, like re-uploading the released file; a normal deploy, which uploads each sidecar once after its file, is unaffected. Deleting a timestamped SNAPSHOT build now also drops the stored version-level `…/<version>-SNAPSHOT/maven-metadata.xml`, so it is regenerated from the remaining builds (or answers `404` when none is left) instead of pointing Maven at the deleted build.
### Security
- **Plugin installs and reloads fetch git sources through the SSRF-guarded HTTP client, so the connection goes only to an address the outbound policy admits at connect time** (#4827, GHSA-p28r-8m4q-w9pg). Previously libgit2 resolved the source host again after the URL check. Plugin git fetches now also honor the `HTTP_PROXY`/`HTTPS_PROXY` environment and `CUSTOM_CA_CERT_PATH` like the other outbound requests.
- **SAML responses are read with entity references, character references and CDATA decoded, so the NameID, attributes and groups AK consumes are exactly the values the IdP signed** (#4827, GHSA-cp5x-hq47-4mg3). A Response that uses an entity other than the five predefined ones, or that carries a DOCTYPE, is refused, and whitespace inside a value is kept as signed rather than trimmed.
- **WASM plugin runtime updated to wasmtime 36.0.19** (#4842; RUSTSEC-2026-0379, RUSTSEC-2026-0380). Two Cranelift code-generation defects in 36.0.18 could miscompile plugin code: a large Wasm `br_table` instruction could jump to the wrong target on aarch64, and a bit-packing overflow in the compiler could silently produce wrong machine code. Both are fixed upstream in 36.0.19. The `wasmtime` and `wasmtime-wasi` workspace pins move together as in #4679, and no configuration change is needed.
- **The Go toolchain floor for every binary Artifact Keeper builds with Go moves to go1.27.3, and the scanner adapter's bundled trivy is rebuilt on go1.27.3, published as scanner adapter 1.6.0** (#4852). This fixes CVE-2026-111943 (HIGH, Go stdlib). `GO_MIN_PATCH` was 1.27.2 in the backend, the Alpine backend and the scanner adapter, and the adapter's bundled trivy was built on stdlib v1.27.2. The floor is raised in all three Dockerfiles at once rather than one image at a time, because the stdlib is a property of the compiler, not of the image that happened to be reported first; the watch that found it only named the adapter. A new `scripts/ci/check-go-floor-agreement.sh` now fails any PR that leaves the floors disagreeing, so the class cannot come back one image at a time. The bundled trivy moves from `ghcr.io/artifact-keeper/trivy:0.76.0-r1` to `0.76.0-r2`, the same upstream Trivy v0.76.0 rebuilt on go1.27.3 (artifact-keeper/trivy#19); the trivy release is unchanged, so a self-run trivy server stays on 0.76.0.
- **TLS certificate verification updated to rustls-webpki 0.103.16 and webpki-roots / webpki-root-certs 1.0.10** (#4861; RUSTSEC-2026-0381, RUSTSEC-2026-0382, RUSTSEC-2026-0383). The previous versions could accept a certificate chain through a distrusted intermediate, and the bundled root set still trusted the distrusted emSign intermediates. No configuration change is needed.
- **Access tokens are invalidated at logout** (#4883). Signing out now ends the session's access token as well as its refresh token, and so does refresh-token replay detection. Before, the access token stayed valid until it expired (30 minutes by default). The replica that handled the logout refuses the token at once; other replicas refuse it within `SESSION_REVOCATION_CACHE_TTL_SECS` (default 5 seconds, `0` to 300). Other sessions of the same user are not affected. Adds migration 294 (`session_family_revocations`, additive). See `docs/session-sign-out.md`.
- **`ctutils` bumped 0.4.0 -> 0.4.5 for RUSTSEC-2026-0385** (#4910). `ctutils` 0.4.0 generated masks and compared `u128` values in non-constant time. It reaches the backend through `crypto-bigint` under the `bergshamra` XML signature crates. Lockfile-only; no behaviour change.
- **`crypto-bigint` bumped 0.7.5 -> 0.7.6 for RUSTSEC-2026-0387** (#4941, #4940). `crypto-bigint` 0.7.5 could take secret-dependent branches in some operations intended to be constant-time. It reaches the backend through the `bergshamra` XML signature crates. A second copy, 0.5.5, stays in the tree through `elliptic-curve` 0.13 (`pgp` and `bergshamra`) because the 0.5 line has no patched release; the advisory is ignored in `.cargo/audit.toml` until those crates move off it (#4940). Lockfile-only; no behaviour change.
- **zstd moved to 0.14.3 / zstd-safe 8.2.0 for RUSTSEC-2026-0391, RUSTSEC-2026-0398 and RUSTSEC-2026-0403** (#5023). `async-compression` and `rattler_package_streaming` (now 0.27.16 with `rattler_conda_types` 0.58) decode through the patched line. The one remaining zstd 0.13 copy belongs to wasmtime's compilation cache, which the backend never enables.
- **zip is now 9.1.2, which fixes GHSA-m665-v7pm-7grj (critical)** (#5026; shipped in 1.11.x as #5441). zip 9.0.4 and earlier had a parser differential between the local file header and the central directory record (Zip Cloaking), so one archive could show different entries to different readers and get around checks made on the other view. The backend parses uploaded NuGet, VS Code extension, PyPI wheel, Composer, Go, Swift, Terraform and conda archives, WASM plugins and Artifactory imports with this crate. The change is lockfile-only.
- **pulldown-cmark moved to 0.13.5 for RUSTSEC-2026-0414** (#5038). The crate is a build-time dependency of `prost-build`, which formats doc comments of the checked-in `.proto` files; it is not linked into the backend and never parsed untrusted markdown, so this clears the audit finding rather than a runtime exposure.
- **The bundled grype and the scanner adapter's bundled trivy now link go-jose v4.1.6, published as scanner adapter 1.6.1** (#5041). This fixes CVE-2026-116587 (GHSA-mgcc-cww5-fp2q, HIGH, denial of service when parsing JWKs that contain RSA keys), which failed the image publish gate on both binaries at go-jose v4.1.5. No grype or Trivy release carries the fix yet, though both upstreams have made the bump on main. The grype build stage in the backend and Alpine backend images adds a guarded go-jose override next to its x/net override, and the adapter's trivy moves from `ghcr.io/artifact-keeper/trivy:0.76.0-r2` to `0.76.0-r3`, the same upstream Trivy v0.76.0 with the same override. The Trivy release is unchanged, so a self-run Trivy server stays on 0.76.0.
- **libz-sys moved to 1.1.31 for RUSTSEC-2026-0418** (#5107). The advisory concerns the crate's build script output when it is built with `cc` 1.6.0 or later; the backend builds with `cc` 1.2.55, so this clears the audit finding rather than a runtime exposure.
- **wasmtime moved to 36.0.20 for RUSTSEC-2026-0424, pulldown-cmark to 0.13.6 for RUSTSEC-2026-0433, and RUSTSEC-2026-0429 ignored on utoipa-swagger-ui's build-time zip** (#5162). The wasmtime advisory lets a component that receives a borrow of a resource it defines itself grow host memory that is never charged to the store, so it bypasses the plugin host's store limits; AK's plugin host runs component-model plugins with WASI, so an installed plugin could have exhausted backend memory. Installing a plugin requires an administrator. All `wasmtime-*`, `cranelift-*`, `pulley-*`, `wiggle-*` and `winch-codegen` crates move together. pulldown-cmark is a build-time dependency (via `prost-build`) and is not linked into the backend. zip 2.4.2 is only a build dependency of utoipa-swagger-ui 8.1.0, whose build script unpacks the bundled Swagger UI with its own `by_index` loop and never calls `ZipArchive::extract`; the backend's own zip is 9.x.
- **A refused Azure signature no longer puts the source SAS into error messages and logs** (#5181). When Azure refused the signature on a Put Blob From URL copy, its error body quoted the string it signed, including the source URL and its SAS token, and the backend copied that body into its error. OCI blob uploads and the chunked upload API return that text to the client, and the storage spans and proxy cache log it, so a signed read URL for the source blob could reach authenticated API clients and the logs until it expired. Azure errors now report the HTTP status, the Azure error code, the copy source's code and status, and the request id, and never the response body. This covers every Azure blob operation and the List Blobs parse error, not only copy.
- **`POST /api/v1/repositories` now applies `quarantine_enabled` and `quarantine_duration_minutes` instead of silently dropping them** (#5324). The two fields existed only on the update request, so a repository created with the upload-time quarantine period turned on answered 200, stored neither setting, and served its first uploads unheld until a `PATCH`. Create now validates and stores them through the same code as update: the create response reports the stored values as `GET` does, and enabling quarantine on a Virtual repository is refused with 400 on create as it already was on update. `virtual_metadata_partial` is applied on create the same way. The remaining update-only settings that create does not apply (a non-empty `release_repository_key`, `curation_enabled: true`, a `curation_default_action` other than `allow`) are refused with a 400 naming the field instead of being ignored; set them with `PATCH` after creating the repository.
- **MCP registries: search, the storage tree and listing totals no longer reveal server versions a reader is not served** (#5338, #5253). A caller who does not hold an MCP registry already got the registry's own `404` and no listing row for a held, refused, resolving, unresolvable or soft-deleted server version on the generic artifact routes. `/api/v1/search/*` (items, totals, facets, suggestions, trending and recent) now leaves those versions out too. So does the repository storage tree, which is rebuilt from the versions the caller is served. The artifact listing's total now counts only served versions on every page, with `total_is_exact: false` past 10,000 matching rows. Holders keep full access.
- **MCP registries get an administrator purge for a deleted server version, and the generic and lifecycle side doors are closed** (#5368). A service account sending the client-set `x-artifact-keeper-replication: true` header could write a server.json into an MCP repository through the generic artifact upload routes (claiming a `name@version` in another team's namespace with an unvalidated, unpinned document) or trash another namespace's server.json through the generic `DELETE`, and a lifecycle policy attached to an MCP repository could trash server.json rows, releasing their OCI pins without the MCP delete's audit or `isLatest` recompute. On an MCP repository the replication exemption now needs an instance administrator's token (how peer links are set up; peer replication keeps working), a service account is refused with `400`, and lifecycle policies skip MCP repositories (a preview says so). Releasing a pin is now a supported, audited step: `POST /mcp/{repository}/v0.1/servers/{serverName}/versions/{version}/purge` (instance administrators only, on a version already soft-deleted, `409 mcp_version_not_deleted` otherwise) trashes the version's server.json and bundles, recomputes `isLatest`, keeps the tombstone so the version can never be republished, and audits `MCP_SERVER_PURGED` with the released digests. The OCI `405` for a pinned digest names this route.
- **MCP sandbox: the introspector now enforces the sandbox's time, memory, disk and failure bounds** (#5384, #5253). When a sandboxed run ends, whatever the outcome, `ak-mcp-introspector` stops the engine client (`SIGTERM`, then `SIGKILL`) and removes the container through the engine (`rm -f`): before, only the Podman/Docker client was killed and the guest kept running past the 120 s wall clock. Each attempt gets a fresh container name (a retry no longer fails with "name already in use"), guests carry an owner label and the worker removes its leftovers at startup, and Podman guests get `--timeout` (and `--http-proxy=false`). A stdio line is read only up to the 1 MiB budget, so one endless line no longer grows the worker by gigabytes. Guests run with `--log-driver=none`, so Docker no longer copies their output into an uncapped JSON log. An engine exit of 125-127 (runtime, network or image missing, pull failed, name in use) is reported `start_failed` with the engine's message, and `start_failed` is now retried, instead of being stored as the server's `needs_config`. The worker applies the backend's limits (256-byte names, 64 KiB `instructions`, 1 MiB body) and posts `refused` with the reason instead of a result the backend rejects, which used to hang until the claim lease expired. `AK_MCP_SANDBOX_RUNTIME=runc`, `crun` or any runtime that is not gVisor or Kata is refused without `AK_MCP_SANDBOX_ALLOW_DEFAULT_RUNTIME=true`. The default runner images are pinned by digest, an OCI package's image is removed after its run, and the worker warns about unpinned runner images. The engine client or launcher starts with a cleared environment (an allowlist plus `AK_MCP_SANDBOX_*` and `AK_MCP_SANDBOX_PASS_ENV`), so it never sees the worker's token. The Kubernetes launcher names each pod afresh, deletes it on `SIGTERM`, deletes pods an earlier attempt left behind, and with `AK_MCP_SANDBOX_AK_IP`/`AK_MCP_SANDBOX_AK_HOST` gives guests a `hostAliases` entry and no DNS server. See `docs/mcp-registry.md`, "The `ak-mcp-introspector` worker".
- **The MCP sandbox reference deployment now reaches Artifact Keeper and nothing else** (#5385). The `ak-gateway` forwarder in `deploy/mcp-introspector/compose.sandbox-network.yaml` carries the AK hostname as a network alias, and `nginx-ak-gateway.conf` forwarded to that same hostname, so inside the forwarder it resolved to the forwarder itself: every connection looped back until nginx ran out of connections, and npm and PyPI installs from a guest failed. The forwarder now reaches AK as `ak-upstream`, an `extra_hosts` entry for AK's address. On Kubernetes, `networkpolicy.yaml` no longer lets `ak-only` guest pods query kube-dns, which forwards outside names and so let a guest send data out in DNS queries; guests reach AK through a `hostAliases` entry built from `AK_MCP_SANDBOX_AK_HOST` and `AK_MCP_SANDBOX_AK_IP` (new placeholders in `deployment.yaml`).
`docs/security/mcp-sandbox.md` now warns that `runsc --ignore-cgroups` removes the memory, pids and CPU limits and says how to delegate cgroups v2 instead, says what isolation holds with gVisor and without it, and says that "install only from AK" is enforced by the network, not by the worker's `--registry` / `--default-index` arguments. `docs/mcp-registry.md` says that a server can tell it is being probed and answer differently, so a capability snapshot is evidence of what the server showed AK, not of what it shows clients.
- **MCP capability snapshots now lint resources, prompts and `instructions`, refuse hidden characters in names and URIs, and flag lookalike names** (#5388). A stored capability snapshot (`ci_reported`, `gateway_reported`, `remote_probe`, `sandbox_run`) was linted on its tools only, and `CapabilitySet::from_doc` applied no Unicode filtering, so resource URIs, names and descriptions, prompt names, descriptions and arguments, and server `instructions` were stored and shown with bidi overrides, zero-width characters or tool-poisoning phrases unflagged, and a tool named `rеad_file` (Cyrillic `е`) passed as `read_file`. The `hidden_text`, `injection_phrase` and `secret_literal` rules now cover every string of resources, prompts and `instructions` (also in a publisher-declared block and MCPB manifest prompts); a capability document whose tool or prompt name, resource URI or name, or prompt argument name carries a control, bidirectional, zero-width or tag character is refused with `400`, as the publish path already refused them; and a new `confusable_name` rule (`review`) flags tool, prompt and resource names that mix scripts with ASCII lookalikes or read as ASCII through UTS #39 confusables. The lint version is now `ak-mcp-lint/2`. There was no XSS: the web UI renders these strings as text.
- **MCP sandbox: the introspector's token is no longer readable by a guest that escapes gVisor** (#5390, #5253). In the reference deployment (Option A) the worker, rootless Podman and the gVisor sentry all run as `ak-sbx`, so a sentry escape could read the `mcp:introspect` token from the worker's `/proc/<pid>/environ`. `ak-mcp-introspector run` now takes `--token-file <path | ->` (`AK_MCP_INTROSPECTOR_TOKEN_FILE`) and makes itself non-dumpable at startup (its `/proc` files become root-owned, `ptrace` by its own user is refused). The systemd unit passes the root-only `/etc/ak-mcp-introspector/token` as standard input (`StandardInput=file:`, `--token-file -`), so no file or environment the worker's user can open holds the token, and the Kubernetes Deployment mounts the Secret as a file. `--token` / `AK_MCP_INTROSPECTOR_TOKEN` still work but log a warning. See `docs/security/mcp-sandbox.md`.
- **The tree browser no longer shows or serves MCP server versions a reader is not served, and no longer lets shared caches keep private content** (#5391, #5253). `GET /api/v1/tree` left a held, refused, resolving, unresolvable, denied or soft-deleted server version in the listing, and `GET /api/v1/tree/content` returned its server.json to anyone who could read the repository, anonymous callers of a public registry included. Both now apply the registry's serve decision: such a version has no node, and its content is the same `404` as a missing path. `/api/v1/tree/content` sent `Cache-Control: public, max-age=3600` for every repository. It now sends `no-store` for an MCP registry and `private, max-age=3600` for any non-public repository. Successful `/mcp/` reads now send `Cache-Control: no-store` for a version and `private, no-cache` with `Vary: Authorization, Cookie` for a listing. Holders keep full access.
- **The event stream, peer and transfer reads, and OCI replication writes are scoped to the caller** (#5393). `GET /api/v1/events/stream` forwarded every domain event to any signed-in user. An administrator still receives every event. Anyone else now receives only the events of repositories it may read, without the actor, and only for MCP server versions it is served. Events about users, groups, permissions, service accounts and quality gates are for administrators, except a `user.*` event about the caller itself. The `GET` routes under `/api/v1/peers` (a peer, its sync tasks, subscriptions, labels, connections, chunk availability and transfer sessions) now require an administrator, like their mutations. `GET /api/v1/peers` still answers a non-admin `200`, so a peer's heartbeat keeps working, but lists no peer. The OCI replication receive routes (`PUT /api/v1/repositories/{key}/oci/blobs/{digest}` and `/oci/manifests/{digest}`) now refuse a repository that is not an OCI repository, so a writer can no longer create OCI rows in an npm, Maven or MCP repository.
- **A Virtual MCP registry's hosted member now shadows a Remote member's server of the same name in any letter case** (#5394). The hosted-shadows-remote rule, which keeps an upstream server from taking over an internal name, compared names case-sensitively, so a Remote's `io.Acme/tool` was listed and served through the Virtual beside the hosted `io.acme/tool`. Ownership is now decided case-insensitively, as the deny lists and namespace allowlists already were.
- **An MCP registry read of a missing server version now does the same gate work as a read of a held one, narrowing the timing difference between them** (#5396). A held version and a missing one already answered the same `404` bytes on `/v0.1/servers/{name}/versions/{version}`, but a miss returned as soon as no row matched while a held version first evaluated its gates, so a caller could tell them apart by response time. A miss now runs the same repository-level gate reads before it answers (the refusal audit was already written off the response path). A residual remains and is documented: a held version also checks its pinned backing packages.
- **A search request no longer re-evaluates the MCP listing decision for every version of every MCP registry in scope** (#5397). `/api/v1/search/*` hides MCP server versions a caller is not served (#5253). For a caller who does not hold a registry, anonymous ones included, every request read all of that registry's artifact rows (unbounded) and ran the listing decision over up to 10,000 versions, with nothing reused between requests, so cheap repeated searches could load the server.
Whether the caller holds a registry is still decided on every request. The set of artifacts served to readers, which is the same for every caller who does not hold the registry, is now computed at most once per registry every 5 seconds on each replica and shared across requests; concurrent requests wait for one computation, and a holder's decision is never cached. A hold, release or status change therefore reaches search within 5 seconds (the `/v0.1` listings and the generic artifact routes still evaluate it on every request). Search now shows a reader of such a registry only the artifacts that set names, read with a bound of 10,000 versions and 20,000 rows per registry: rows past the bound, and rows stored since the set was computed, are hidden until it is recomputed (fail closed).
- **An MCP registry's package mapping, deny list and other MCP settings are shown only to its administrators** (#5398). `GET /api/v1/repositories/{key}` returned `mcp_package_repos` (the backing repository key per `registryType`, which can name a private repository the generic routes hide behind a `404`), `mcp_server_deny`, `mcp_namespace_allow` and the rest of the Phase 2/3 MCP settings to every caller who could read the registry, anonymous callers of a public one included. The repository responses (`POST`, `GET` and `PATCH`) now carry them only when the caller has the repository `admin` action or is an instance administrator (the same check as `mcp_caller_is_repo_admin`), and omit them for everyone else. `mcp_package_policy` is still echoed to every reader.
- **MCP registries: the remaining generic read routes no longer reveal server versions a reader is not served** (#5428, #5253). A caller who does not hold an MCP registry (an administrator, or `read` through a grant) could still tell a held, refused, resolving, unresolvable or soft-deleted server version from a missing one on several generic routes. The security scan listings (`GET /api/v1/repositories/{key}/security/scans`, `GET /api/v1/security/scans`, `/security/artifacts/{id}/scans`, `/security/scans/{id}` and its findings) now leave out such a version's scans, and their `total` counts only the rest. `GET /api/v1/packages` decides its MCP rows before paging, so `total` and every page count only the rows the caller is shown, and a row that shows a newer servable version in place of a held one carries that version's size, downloads and times. `POST /api/v1/repositories/{key}/artifacts-missing` reports such a path as `not_found`. The by-id sub-resources (`/artifacts/{id}/labels`, `/artifacts/{id}/package-analysis`, `/quarantine/{id}`, `/quality/health/artifacts/{id}`, `/quality/checks`, `/sbom/by-artifact/{id}`, `/sbom/cve/history/by-artifact/{id}` and `/sbom?artifact_id=`) answer it as a missing artifact. `GET /api/v1/repositories/{key}/storage` and `storage_used_bytes` of `GET /api/v1/repositories/{key}` count only the versions the caller is served. A package version list whose lookup fails is now empty instead of complete. Holders keep full access.
- **MCP registry listings no longer reveal held server versions through cursors, nested Virtual deny lists apply, and a gate outage or a long run of held versions no longer changes which version is served** (#5429, #5253, #5471). A held version (quarantined, refused, unresolvable, resolving, denied, deleted or awaiting Remote approval) is meant to be indistinguishable from a missing one for a reader, but several paths let it show through or let a lower-priority answer take its place.
`nextCursor` on `/servers` (hosted, Remote and Virtual) and on the holds listing used to be base64 of `name:version` (or `m:<member>:<cursor>` on a Virtual) of the last row read, which could be a held row, and any hand-made cursor was accepted. Cursors are now opaque: sealed with AES-256-GCM under a key derived from `jwt_secret` and bound to the issuing repository and listing. A cursor this instance did not issue, including one from before the upgrade, is `400 mcp_invalid_query` ("invalid cursor"); start the listing again without it.
A nested Virtual's `mcp_server_deny` was ignored when its parent Virtual was read; it now hides the name from every member reached through it (the union of the deny lists along the member's path). When a member's gate cannot be evaluated, Virtual listings used to list the same version from a lower-priority member and take `latest` from the next member; that version is now left out (a detail read still answers `503`). Hosted `version=latest` likewise no longer falls back to an older version when a newer one cannot be decided. `latest` now reads the newest 100 non-deleted versions on every surface, so `/versions/latest`, `isLatest` and `version=latest` agree for a server whose newest 100 versions are all held. The asynchronous package fill now recomputes its fill repository from the current mapping and fails the version (`mcp_package_not_found`, publish again) unless it is the Remote recorded at publish, instead of pinning the package under a mapping and a Remote from different