HHostingByteSoftware
Ctrl K
ComunidadGuías…

CLAUDE.md

Rama main · réplica de lectura del repositorio original.

Archivo original ↗Preguntar a KiBot
# Artifact Keeper Development Guidelines

Auto-generated from all feature plans. Last updated: 2026-01-14

## Active Technologies
- Rust 1.75+ (backend) + wasmtime 21.0+, wasmtime-wasi, wit-bindgen, git2, axum
- PostgreSQL (existing), filesystem for WASM binaries
- Rust 1.75+ + axum, sqlx, tokio, reqwest
- Rust 1.75+ + axum, serde, serde_json

## Project Structure

```text
src/
tests/
```

## Commands

### Fast CI (Tier 1) - Every Push/PR
```bash
# Backend lint and unit tests
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo nextest run --workspace --lib --bins --test-threads 8
```

`--bins` is required: `backend/src/main.rs` is a second compilation target
with its own tests, and plain `--lib` silently excludes them (#3494).

Tier 1 also runs every integration target whose tests are **not** `#[ignore]`d
(workflow contract tests, security regression pins, the streaming-invariant
ratchet, PyPI conformance): see the "Run workflow contract and pure
integration targets" step in `.github/workflows/ci.yml`. They cannot go in the
Tier 2 allowlist below: that step runs `--run-ignored ignored-only`, and
non-`#[ignore]`d tests would report `0 tests` and pass.

The unit-test job is also the coverage run: it builds once with `cargo llvm-cov`
instrumentation, runs the suite once, and uploads `lcov.info`; the
`📊 Code Coverage` job only evaluates the gates from that report.

### Integration Tests (Tier 2) - Pushes & Backend PRs

CI names an explicit list of test files and runs their `#[ignore]`d cases
serially. Every `backend/tests/*.rs` file MUST be either named in a `--test`
invocation in a workflow or carry a justified exemption in
`backend/src/ci_test_surface.rs` — the `ci_test_surface` gate (a `--lib` test,
so it runs on every PR) goes red otherwise (#3494). Exempt targets are the
ones that need live cloud credentials or a running HTTP backend.

```bash
# Backend integration tests (requires PostgreSQL)
cargo nextest run --workspace --run-ignored ignored-only --test <test_file_name>
```

They run one at a time: `.config/nextest.toml` puts every integration target
(`kind(test)`) in the single-threaded `db-serial` test group, because the
suites share schema state through global DELETEs.

**Validating locally: DB-backed tests SKIP silently without `DATABASE_URL`.**
They report PASS in ~0.0x seconds (0.04s / 0.01s) without executing anything.
A sub-0.1s "pass" on a DB suite means it did not run. Always export
`AK_TESTS_REQUIRE_DB=1` alongside `DATABASE_URL` when you need proof a test
ran — it turns a missing/unreachable database into a hard failure (#2924).
CI runs this suite in the `🧪 Backend Integration Tests` job on every push
**and** on every pull request that touches `backend/**`, `Cargo.toml`,
`Cargo.lock`, `.sqlx/**`, or `.github/workflows/ci.yml` (#3124). A failure
there fails the required `🧪 Backend Unit Tests` check.

### Full E2E Tests (Tier 3) - Release/Manual Only
```bash
# Run all E2E tests with default (smoke) profile
./scripts/run-e2e-tests.sh

# Run with specific profile
./scripts/run-e2e-tests.sh --profile all      # All native clients
./scripts/run-e2e-tests.sh --profile pypi     # PyPI only
./scripts/run-e2e-tests.sh --profile smoke    # Quick smoke tests (default)

# Include stress and failure tests
./scripts/run-e2e-tests.sh --stress --failure

# Run with test tag filter
./scripts/run-e2e-tests.sh --tag @smoke       # Only smoke-tagged tests
./scripts/run-e2e-tests.sh --tag @full        # Full test suite

# Cleanup after tests
./scripts/run-e2e-tests.sh --clean
```

### Native Client Tests
```bash
# Run individual native client tests
./scripts/native-tests/run-all.sh smoke   # PyPI, NPM, Cargo
./scripts/native-tests/run-all.sh all     # All 10 package formats
./scripts/native-tests/test-pypi.sh       # Individual test
```

### gRPC SBOM Tests
```bash
# Run SBOM JSON structure validation unit tests (no database required)
cargo test sbom_service::tests --lib

# Run gRPC integration tests (requires PostgreSQL at localhost:30432)
DATABASE_URL="postgresql://registry:registry@localhost:30432/artifact_registry" \
  cargo test --test grpc_sbom_tests -- --ignored

# Run gRPC E2E tests with grpcurl (requires backend running with gRPC on port 9090)
./scripts/native-tests/test-grpc-sbom.sh
```

### Dependency-Track Integration Tests
```bash
# Run Dependency-Track integration tests (requires docker compose up)
./scripts/native-tests/test-dependency-track.sh

# With API key for full tests
DEPENDENCY_TRACK_API_KEY=your-key ./scripts/native-tests/test-dependency-track.sh
```

### WASM Plugin E2E Tests
```bash
# Run all WASM plugin tests (requires backend running on port 8080)
./scripts/native-tests/test-wasm-plugins.sh

# Run individual test suites
./scripts/native-tests/test-wasm-plugins.sh git        # Git installation tests
./scripts/native-tests/test-wasm-plugins.sh lifecycle  # Enable/disable/uninstall
./scripts/native-tests/test-wasm-plugins.sh reload     # Hot-reload tests

# Run with custom API URL
API_URL=http://localhost:8080 ./scripts/native-tests/test-wasm-plugins.sh
```

### Stress and Failure Tests
```bash
# Stress tests (100 concurrent uploads)
./scripts/stress/run-concurrent-uploads.sh
./scripts/stress/validate-results.sh

# Failure injection tests
./scripts/failure/run-all.sh
./scripts/failure/test-server-crash.sh
./scripts/failure/test-db-disconnect.sh
./scripts/failure/test-storage-failure.sh
```

### GitHub Actions
```bash
# Manually trigger E2E workflow
gh workflow run e2e.yml -f profile=all -f include_stress=true
```

## Code Style

Rust 1.75+: Follow standard conventions

## Docker & Compose Files

This repo ships multiple Dockerfiles (`docker/Dockerfile.backend`,
`docker/Dockerfile.backend.dev`, `docker/Dockerfile.backend.alpine`, plus
scanner/mock images) and multiple compose files
(`docker-compose.yml`, `docker-compose.local-dev.yml`,
`docker-compose.*-e2e.yml`, `docker-compose.test.yml`, etc.) that each
independently encode assumptions about what the backend image provides (e.g.
whether it has a shell, which packages install which files). These have
drifted out of sync before: #2126 fixed production's `docker-compose.yml`
after the hardened runtime image (#2059) dropped `/bin/sh`, but
`docker-compose.local-dev.yml` carried an independent copy of the same
`dtrack-init` pattern that nobody updated, so it silently reintroduced the
exact bug #2126 had already fixed.

**When changing any Dockerfile or compose file, check every other one for the
same assumption.** Concretely:
- If a Dockerfile's runtime image gains or loses a shell, package manager, or
  any binary (e.g. `/bin/sh`, `curl`, `jq`, `protoc`), grep all other
  Dockerfiles and compose files for the same pattern before assuming only the
  one you're editing is affected.
- If a compose file's `dtrack-init` (or any init container) is fixed to stop
  reusing the hardened backend image through a shell, check every other
  compose file for a service doing the same thing.
- Prefer a regression test that scans *all* matching files (e.g. every
  `docker-compose*.yml` at the repo root) over one hardcoded to the single
  file you just fixed — see
  `shipped_compose_does_not_run_hardened_image_through_a_shell` in
  `backend/src/config.rs` for the pattern.

## Git & GitHub

### Branch Protection — NEVER push directly to main

All changes must go through pull requests — since 2026-09-20 this is enforced
by branch protection (`required_pull_request_reviews` on `main`), not just
convention, so a direct push is refused rather than merely discouraged:

1. **Create a feature branch** from main:
   ```bash
   git checkout main && git pull
   git checkout -b feat/short-description   # or fix/, chore/, docs/
   ```

2. **Make changes and commit** to the feature branch

3. **Push and create PR**:
   ```bash
   git push -u origin feat/short-description
   gh pr create --fill   # or with --title and --body
   ```

4. **Merge via GitHub** after CI passes (squash merge preferred)

### Merge Requirements (MANDATORY)

**NEVER merge a PR unless ALL of the following are true:**

1. **CI workflow fully green.** Every check must pass: Rust check (clippy), unit tests, code coverage gate, duplication gate, security audit, CodeQL. No exceptions.
2. **Code coverage >= 70%** on new/changed lines. The CI coverage gate enforces this. If it fails, add tests until it passes.
3. **Code duplication <= 3%** on changed files. The CI duplication gate (jscpd) enforces this. If it fails, refactor duplicated code into shared helpers.
4. **No `--admin` bypass.** Do not use `gh pr merge --admin` to skip failing checks — since 2026-09-20 `enforce_admins: true` on `main` means it is refused outright, not just against the rules. If a gate is genuinely wrong (not a code issue), fix the gate first, get that fix merged, then rebase the PR.

If a CI gate is blocking a PR due to a systemic issue (e.g., the gate itself has a bug), **ask the user before bypassing.** Document why the bypass was needed and create a follow-up issue to fix the gate. This rule exists because bypassing gates erodes trust in the CI pipeline.

Branch naming conventions:
- `feat/` — new features
- `fix/` — bug fixes
- `chore/` — maintenance, dependencies, CI
- `docs/` — documentation only

### Parallel Agent Work (shallow clones)

When dispatching multiple agents to work on separate features or fixes in parallel, use **shallow clones in `/tmp/`** instead of git worktrees. Worktrees share the `.git` directory and agents end up switching branches in the main worktree, corrupting each other's state.

**Pattern for each agent:**
```bash
WORK_DIR="/tmp/$(uuidgen)-artifact-keeper"
git clone --depth 50 --branch main git@github.com:artifact-keeper/artifact-keeper.git "$WORK_DIR"
cd "$WORK_DIR"
git checkout -b feat/issue-description
# ... make changes, run checks, commit, push, create PR ...
rm -rf "$WORK_DIR"
```

Each agent gets a fully isolated repo copy. No shared state, no branch conflicts, no rust-analyzer cross-contamination. The agent must do ALL work inside `$WORK_DIR` and never touch the primary working directory at `/Users/khan/ak/artifact-keeper`.

### Pre-push Quality Checklist

Every commit must pass these checks locally before pushing. Do NOT use "push and see if CI passes" as a strategy.

```bash
cargo fmt --check                                          # formatting
cargo clippy --workspace --all-targets -- -D warnings      # linting
cargo nextest run --workspace --lib --test-threads 8       # unit tests
```

**Use `cargo nextest`, not plain `cargo test`.** That is the runner the CI
unit-test job invokes (`.github/workflows/ci.yml`), and the difference is
load-bearing rather than cosmetic: nextest runs each test in its own process,
and a number of unit tests depend on that isolation because they touch
process-global state (upload semaphores, proxy environment variables) or are
serialized only by `.config/nextest.toml`'s `db-serial` test groups, which a
plain `cargo test` run does not honour. Run the suite under the single-process
runner instead and a clean tree fails tests that have nothing to do with your
change (#3479). Install it once with `cargo install cargo-nextest --locked`.

The unit tests need the same environment CI gives them, or the DB-backed ones
quietly no-op:

```bash
# A throwaway database — never point this at a live instance; the tests
# create, mutate and delete rows across the whole schema.
export DATABASE_URL=postgresql://registry:registry@localhost:5432/artifact_registry
export AK_TESTS_REQUIRE_DB=1  # a DB-backed test that cannot connect FAILS instead of skipping
export SQLX_OFFLINE=true      # compile against the committed .sqlx query cache
export JWT_SECRET=any-value-at-least-32-bytes-long-for-tests
sqlx migrate run --source backend/migrations
cargo nextest run --workspace --lib --test-threads 8
```

Without `AK_TESTS_REQUIRE_DB=1` a missing or unreachable database makes every
DB-backed test skip and report success — a green run that proved nothing. The
tell is a test that finishes in ~0.04s.

Additionally, before pushing:
- **Code coverage**: new/changed lines MUST have >= 70% test coverage. The CI gate measures only lines you added or modified (not the entire file). Extract testable logic into pure helper functions to make handler code coverable.
- **Code duplication**: changed files MUST have <= 3% duplication (measured by jscpd). Extract repeated patterns into shared helpers. Common offenders: cache read/write patterns, test setup blocks, filter construction.
- Check migration numbering: verify the migration number is not already taken (`ls backend/migrations/ | tail -5`)
- If coverage or duplication gates will fail, fix them BEFORE pushing. Do not push and hope CI passes.

### Maintenance Branches

Long-lived `release/X.Y.x` branches exist for shipping bug fixes to older release series:

- **`release/1.0.x`** — maintenance branch for the 1.0 series (created from `v1.0.0-rc.5`)
- **`release/1.1.x`** — maintenance branch for the 1.1 series (created from `v1.1.2`)
- **`main`** — continues with 1.2.x (and beyond) development

**Bug fix workflow for maintenance branches:**
1. Create a fix branch from the maintenance branch:
   ```bash
   git checkout release/1.1.x && git pull
   git checkout -b fix/short-description
   ```
2. Push and create a PR **targeting `release/1.1.x`** (not main):
   ```bash
   git push -u origin fix/short-description
   gh pr create --base release/1.1.x --fill
   ```
3. Tag releases from the maintenance branch:
   ```bash
   git checkout release/1.1.x && git pull
   git tag v1.1.3 && git push origin v1.1.3
   ```
4. Cherry-pick fixes between maintenance and `main` so both lines stay in sync. Bug fixes typically land on the maintenance branch first, then cherry-pick forward to main.

**Docker image tags** (set by `docker/metadata-action` in `docker-publish.yml`):
- Version tags **strip the `v` prefix**: git tag `v1.1.0-rc.2` → Docker tag `:1.1.0-rc.2`
- `:latest` is only set for stable releases (no `-rc`, `-beta`, etc.)
- `:1.0`, `:1.1` series tags are set automatically via semver parsing
- `:dev` is only set for `main` branch pushes
- `:sha-<commit>` is set for every build

### Releases

- Release notes for a **stable** `vX.Y.Z` are a curated body committed as `.github/release-notes/<version>.md` on the ref being released — a high-level paraphrase of that version's `## [X.Y.Z]` CHANGELOG section (see RELEASING.md "Release-notes style"). The file is required: `release.yml` refuses a stable tag without it (#3537). `generate_release_notes` is the fallback for **prereleases only** (`-rc.N`, `-beta.N`).
- **Do NOT hardcode static release notes** in the workflow itself. The curated body is a committed file that `release.yml` reads as `body_path`; no product descriptions, feature lists or format counts belong in the YAML.
- The release workflow is at `.github/workflows/release.yml`, triggered by `v*` tags.
- The release-gate (artifact-keeper-test) must pass for a release to be published. If gates fail, the release is created as a **draft** with binaries attached but not published.

### Changelog entries: one fragment file per PR

**Do not edit `CHANGELOG.md` in a feature or fix PR.** Add one new file,
`changes/unreleased/<issue-or-pr-number>-<slug>.md` (format and rules in
`changes/README.md`):

```markdown
---
section: Fixed            # Added | Changed | Deprecated | Removed | Fixed | Security
issues: [#4145, #4129]
---
- **Bold lead sentence saying what changed for the user** (#4145, #4129). The why and the what, exactly as the bullet would read in CHANGELOG.md; further paragraphs indented two spaces.
```

- Lead with the issue the PR closes: release preflight check 5 reconciles each entry by the first `#N` on its `- ` line.
- One fragment per user-facing change; CI-only / workflow-only changes need none.
- `python3 scripts/ci/changelog-fragments.py validate` checks every fragment; CI runs it in `check-changelog-unreleased.sh`.
- The release prep renders the fragments into `## [X.Y.Z] - <date>` with `scripts/release/assemble-changelog.sh` and deletes them (RELEASING.md step 3). A bullet added under `## [Unreleased]` still passes CI during the transition and is merged at the cut, but it conflicts with every other PR doing the same, which is why fragments replaced it.

### Changelog and Release Notes

Every CHANGELOG entry and GitHub Release must include recognition sections. This is required for every release, no exceptions.

**Process when writing a changelog entry or release notes:**
1. Look at all PRs/commits since the last tag
2. For each fix, check the linked issue(s) to find who reported it
3. Check `gh pr list --state merged` for external PR authors (not `brandonrc` or bots)
4. Run `scripts/release/sponsors.sh` (needs `gh` logged in as an org admin): it prints the `### Sponsors` block from the ACTIVE GitHub sponsorships, naming public sponsors and only counting private ones. `--level 2` prints the release-notes form; `--check FILE` fails if FILE omits a public sponsor. Do not copy the list from README.md or an older section; both have drifted before
5. Add a `### Thank You` section crediting external contributors who filed issues, reported bugs, or submitted PRs
6. Paste the `### Sponsors` block the script printed; never name a sponsor whose sponsorship is private (the script never prints them)
7. **Do NOT include maintainer `brandonrc`** in the thank you section, only external contributors
8. If no external contributors reported issues for this release, omit the Thank You section
9. The Sponsors section is always included if there are active sponsors

The recognition sections are added to the assembled `## [X.Y.Z]` section in the release prep PR, not to fragments.

**Example format in CHANGELOG.md:**
```markdown
## [1.x.x] - YYYY-MM-DD

### Sponsors

Thank you to our backers for supporting ongoing development:
- **Ash A.** ([@dragonpaw](https://github.com/dragonpaw))
- **Gabriel Rodriguez** ([@injectedfusion](https://github.com/injectedfusion))

[Become a sponsor](https://github.com/sponsors/artifact-keeper)

### Thank You
- @username for reporting the OCI auth issue behind reverse proxies (#123)
- @another-user for identifying the Maven SNAPSHOT re-upload bug (#456)
- @contributor for the SSO admin fix PR (#609)

### Added
...
```

**GitHub Release notes** follow the same structure. The release workflow auto-generates a changelog from merged PRs, but the Sponsors and Thank You sections must be prepended manually (or by the release script) before publishing.

### Other Git Rules

- **Do NOT add AI Co-Authored-By lines** (e.g., Claude, GPT) to commit messages — real human co-authors are fine
- **Do NOT include "Generated with Claude" or similar AI attribution** in PR descriptions
- **Always use `gh` CLI** for GitHub operations (PRs, issues, workflows, etc.)
  - Use `gh pr create` for pull requests
  - Use `gh issue` for issues
  - Use `gh workflow` for workflow operations
  - Do not use raw git commands for GitHub-specific features
- **Always use the PR template** when creating PRs with `gh pr create`. The template is at `.github/PULL_REQUEST_TEMPLATE.md`. Since `gh` does not auto-fill it, read the template and structure the `--body` to match its sections (Summary, Test Checklist, API Changes). Fill in checkboxes based on actual work done.

## Recent Changes
- 007-shared-dto: Added Rust 1.75+ + axum, serde, serde_json
- Frontend removed: Moved to separate repository (artifact-keeper-web)


<!-- MANUAL ADDITIONS START -->

## Infrastructure & Cost Rules

- **NEVER build Docker images or compile code on EC2/cloud instances.** Cloud compute costs money. All builds must happen locally on the developer's MacBook or via GitHub Actions CI.
- **Demo EC2 instance** (`i-0caaf8acac6f85d4d`, Elastic IP `3.222.57.187`): Only pull pre-built images from `ghcr.io`, never `docker compose build`.
- **SSH access**: `ssh ubuntu@3.222.57.187` (uses local SSH key)
- **Demo stack**: Managed via systemd service `artifact-keeper-demo` and Caddy reverse proxy for TLS. Compose file at `/opt/artifact-keeper/deploy/demo/docker-compose.demo.yml`.
- **Demo version pinning**: Set `ARTIFACT_KEEPER_VERSION` in `/opt/artifact-keeper/deploy/demo/.env` to pin a release (e.g., `ARTIFACT_KEEPER_VERSION=1.1.0-rc.2`). Omit the `v` prefix — Docker tags use semver without `v`. Default is `latest` if unset.
- **Demo update procedure**: `ssh ubuntu@3.222.57.187`, edit `.env` with the desired version, then `cd /opt/artifact-keeper/deploy/demo && docker compose -f docker-compose.demo.yml pull && docker compose -f docker-compose.demo.yml up -d`.
- **Docker images** are published to `ghcr.io/artifact-keeper/artifact-keeper-{backend,web,openscap}` by the Docker Publish CI workflow on every push to main and on release tags.
- **GitHub Pages site** (`/site/` directory): Combined landing page + Starlight docs, deployed to `artifactkeeper.com`.

<!-- MANUAL ADDITIONS END -->