Taskfile.yml
Rama main · réplica de lectura del repositorio original.
# Task runner (https://taskfile.dev). Self-documenting: `task --list` shows
# every public task. Prefer adding tasks here over adding ad-hoc shell
# scripts, so newcomers (human or AI) can discover what's runnable.
#
# Conventions:
# - Tasks are grouped by verb (build:*, test:*, lint:*, dev:*, db:*, qa:*).
# - Tasks that need a real DB or an MDM-signed artifact document the
# prerequisite in `desc:` rather than failing silently.
# - No task here should mutate production-ish state (the explicit carve-out
# is db:reset, which requires confirmation).
version: "3"
vars:
# The OpenSpec CLI version the release archive must run. Keep in step with the pin in .github/workflows/openspec-validate.yml,
# so the archive applies deltas with the same CLI that validated them.
OPENSPEC_VERSION: 1.14.0
SERVER_PKG: ./server/cmd/rawtrail-server
INGEST_PKG: ./server/cmd/rawtrail-ingest
AGENT_PKG: ./agent/cmd/rawtrail-agent
SERVER_BIN: server/tmp/rawtrail-server
INGEST_BIN: server/tmp/rawtrail-ingest
AGENT_BIN: agent/tmp/rawtrail-agent
tasks:
default:
desc: Show task list
cmds:
- task --list
# -------- build --------
build:
desc: Build server, ingest, agent, and UI
cmds:
- task: build:ui
- task: build:server
- task: build:ingest
- task: build:agent
build:server:
desc: Build the main rawtrail-server binary into server/tmp/
cmds:
- go build -o {{.SERVER_BIN}} {{.SERVER_PKG}}
build:ingest:
desc: Build the standalone ingest binary into server/tmp/
cmds:
- go build -o {{.INGEST_BIN}} {{.INGEST_PKG}}
build:agent:
desc: Build the macOS agent binary into agent/tmp/
cmds:
- go build -o {{.AGENT_BIN}} {{.AGENT_PKG}}
# Pin a STABLE ad-hoc code-signing identifier so the debug extension's XPC peer requirement keeps matching the
# locally-built agent across rebuilds (issue #623). `go build` on arm64 macOS ad-hoc-signs with the linker default
# identifier "a.out" and a content-derived cdhash, both of which change on every meaningful rebuild; the old pin was
# that cdhash, so it went stale constantly. Re-signing with an explicit identifier fixes it to "rawtrail-agent",
# the same designated identifier the notarized release carries (packaging/pkg/build.sh derives it from the binary
# name), so XPCEventServer.debug can pin `identifier "rawtrail-agent"` and never need re-pinning. Darwin-only:
# codesign exists only on macOS and the XPC peer gate only runs there.
- cmd: codesign --force --sign - --identifier rawtrail-agent {{.AGENT_BIN}}
platforms: [darwin]
build:agent:linux:
desc: |
Cross-compile the agent for linux/amd64 as a compile-only invariant check
(no binary is produced because the package pattern targets multiple
packages; `go build ./agent/...` reports compile errors but writes no
output). Exercises the build-tag split in agent/receiver (UAT plan M1):
the darwin-only CGo XPC bridge is excluded and the non-darwin stub is
selected so the agent module compiles cleanly on a non-macOS host.
CGO_ENABLED=0 keeps this check from accidentally depending on a local C
cross-toolchain when run on a native linux host. CI calls this from the
agent-test job to keep the platform-portability invariant green.
env:
GOOS: linux
GOARCH: amd64
CGO_ENABLED: 0
cmds:
- go build ./agent/...
build:agent:windows:
desc: |
Cross-compile the agent for windows on amd64 and arm64 as a compile-only invariant check (ADR-0018, Phase 0 of Windows support).
Like build:agent:linux this produces no binary (the package pattern targets multiple packages); it proves the build-tag split in
agent/commander (kill), agent/reconcile (liveness probe), agent/hostid (MachineGuid), and agent/config (ProgramData paths) compiles
cleanly for Windows. CGO_ENABLED=0 keeps the check independent of a local C cross-toolchain. `go vet` runs the same matrix so a
Windows-only type error surfaces here rather than on a Windows runner.
cmds:
- GOOS=windows GOARCH=amd64 CGO_ENABLED=0 go build ./agent/...
- GOOS=windows GOARCH=arm64 CGO_ENABLED=0 go build ./agent/...
- GOOS=windows GOARCH=amd64 CGO_ENABLED=0 go vet ./agent/...
- GOOS=windows GOARCH=arm64 CGO_ENABLED=0 go vet ./agent/...
build:agent:headless:
desc: |
Build the headless agent binary (UAT plan M2) for linux/amd64 into
agent/tmp/rawtrail-agent-headless. The headless binary wires the same
enrollment + queue + uploader pipeline as the production agent but uses
the non-darwin stub receiver and exposes a unix-socket control plane so
test scenarios can inject events directly. Used by the future L3
headless-agent + server integration job; today this target's primary
consumer is local development and the CI compile-invariant check.
env:
GOOS: linux
GOARCH: amd64
CGO_ENABLED: 0
cmds:
- go build -o agent/tmp/rawtrail-agent-headless ./agent/cmd/rawtrail-agent-headless
ui:setup:
desc: |
Install UI npm dependencies. Wired as a dep of build:ui and dev:ui so a fresh clone
or new git worktree builds without a separate manual `cd ui && npm ci` step. Skips
when `node_modules/.package-lock.json` (the snapshot npm writes on a successful
`npm ci`) is at least as new as `package-lock.json`, so a lockfile bump triggers a
fresh install but a no-op build does not pay the cost. A bare `test -d node_modules`
guard would leave stale dependencies in place after a branch switch that touches
the lockfile.
dir: ui
sources:
- package-lock.json
generates:
- node_modules/.package-lock.json
cmds:
- npm ci
build:ui:
desc: Build the React UI; output embedded in the server binary via server/ui/dist/
dir: ui
deps: [ui:setup]
cmds:
- npm run build
build:ui:e2e:
desc: |
Build the UI with production minification but source maps
ENABLED so the E2E coverage job (monocart-coverage-reports)
can walk the .map files and remap V8 coverage of the minified
bundle back to ui/src/**. Output lands at server/ui/dist/ same
as build:ui - replaces the production-shape bundle in-place,
so re-run plain build:ui afterwards if you need a clean prod
build. CI calls this from the e2e-test job.
dir: ui
deps: [ui:setup]
env:
UI_BUILD_SOURCEMAP: "1"
cmds:
- npm run build
build:extension:
desc: Build the macOS system + network extensions via xcodebuild
dir: extension/edr
cmds:
- xcodebuild -project edr.xcodeproj -scheme edr -configuration Debug -sdk macosx -derivedDataPath build
# -------- generate --------
docs:rules:
desc: Regenerate docs/detection-rules.md from the registered rule catalogue.
cmds:
- go run ./tools/gen-rule-docs
docs:rule-pack:
desc: Regenerate server/rules/internal/catalog/pack/*.yml, one declarative rule file per registered detection.
cmds:
- go run ./tools/gen-rule-pack
sync:openapi-embed:
desc: |
Copy docs/api/openapi.yaml into server/apidocs/embed/, which is the spec the
server actually serves at /api/openapi.yaml. The canonical file is the one
under docs/; this copy exists only because a go:embed pattern cannot reach
outside its own package directory.
Run this after editing the canonical spec. TestEmbeddedSpecMatchesCanonical
fails the build if you forget, so this is the fix rather than the reminder.
cmds:
- go generate ./server/apidocs/...
docs:attack-layer:
desc: Regenerate docs/attack-navigator-layer.json from the registered rule catalogue.
cmds:
- go run ./tools/gen-attack-layer
attack:table:
desc: |
Regenerate the vendored ATT&CK technique table (ui/src/components/attack-techniques.generated.ts) from MITRE's
published STIX bundle. Downloads the bundle to tmp/ first; the bundle itself is 53MB and is NOT committed, only the
extract is.
Pass the release to cut from. The default is READ FROM the generated table, not hardcoded, so running this bare
re-cuts the version already vendored and can never silently revert the catalog to an older release:
task attack:table ATTACK_VERSION=19.3
After regenerating, check navigatorATTACKVersion in server/rules/api/navigator.go still matches the major, and run
`task docs:attack-layer` if the layer's declared version changed. Two tests in server/rules/bootstrap enforce both the
version agreement and that every technique a rule maps has a table entry.
vars:
VENDORED_ATTACK_VERSION:
sh: sed -n 's/^export const ATTACK_VERSION = "\(.*\)";$/\1/p' ui/src/components/attack-techniques.generated.ts
ATTACK_VERSION: '{{.ATTACK_VERSION | default .VENDORED_ATTACK_VERSION}}'
cmds:
- mkdir -p tmp
- test -f tmp/enterprise-attack-{{.ATTACK_VERSION}}.json || curl -fsSL -o tmp/enterprise-attack-{{.ATTACK_VERSION}}.json
https://raw.githubusercontent.com/mitre-attack/attack-stix-data/master/enterprise-attack/enterprise-attack-{{.ATTACK_VERSION}}.json
- go run ./tools/attacktable -bundle tmp/enterprise-attack-{{.ATTACK_VERSION}}.json
attack:latest-check:
desc: |
Fail if the vendored ATT&CK table is not cut from MITRE's newest published Enterprise release. Release-checklist step;
needs the network, which is why it is a task rather than a unit test (a test reaching the internet fails on an
air-gapped runner and becomes noise). Regenerate with `task attack:table ATTACK_VERSION=<new>` when it fails.
cmds:
- go run ./tools/attacktable -check-latest
sigma:upstream-check:
desc: |
Fail if the vendored SigmaHQ macOS corpus (server/rules/internal/catalog/imported) differs from upstream: a new, changed or
withdrawn rule in any SigmaHQ rule tree. Needs the network, which is why it is a task rather than a unit test. Bring the
corpus up to date with `go run ./tools/sigma-sync -apply`, which copies rules verbatim, moves a recategorised rule (removing its
old copy), and regenerates the manifest, but never deletes a withdrawn rule or updates the pinned import counts.
cmds:
- 'GITHUB_TOKEN="${GITHUB_TOKEN:-${GH_TOKEN:-$(gh auth token 2>/dev/null)}}" go run ./tools/sigma-sync'
proto:
desc: |
Regenerate the gRPC control-channel Go code from internal/control/control.proto.
Requires protoc on PATH and the pinned Go plugins; the recipe installs the
plugins into GOBIN and invokes protoc with GOBIN on PATH. The generated
*.pb.go files are checked in (golangci-lint skips generated files).
cmds:
# Pin the plugin install dir to GOPATH/bin (overriding any custom GOBIN) so the protoc step below finds them on a known PATH.
- GOBIN="$(go env GOPATH)/bin" go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
- GOBIN="$(go env GOPATH)/bin" go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.5.1
- >-
PATH="$(go env GOPATH)/bin:$PATH" protoc
--go_out=. --go_opt=module=github.com/rawtrail/rawtrail
--go-grpc_out=. --go-grpc_opt=module=github.com/rawtrail/rawtrail
internal/control/control.proto
# -------- test --------
test:
desc: Run every Go + UI test (requires MySQL; see db:up)
deps: [test:go, test:ui]
test:go:
desc: |
Run Go tests with race detector (server + agent + internal). Uses
$EDR_TEST_DSN from the environment if set; otherwise defaults to the
docker-compose mysql_test service on port 33307 (see db:up). Pointing at
the dev DB on 33306 by accident is a common foot-gun because dev tests
truncate tables. CI sets its own DSN.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
cmds:
# ./tools/... is in scope because the repo's build-time tools carry real logic that nothing else exercises: the
# ATT&CK version comparison, for one, where a string compare would sort "9.0" after "19.2" and pass a stale
# table forever. Tests that no CI lane runs are not a gate (review caught exactly that on tools/attacktable).
- go test -race -count=1 -timeout=10m ./server/... ./agent/... ./internal/... ./tools/...
test:go:agent:
desc: |
Run Go tests with race detector for agent + shared code + the M3
fakeagent library. No DB required. Use from CI's agent-test job so the
server tests don't run on a runner that doesn't have MySQL.
Second invocation: the headless agent's tests (UAT plan M2) live behind
`//go:build !darwin || !cgo` and are skipped by the first invocation
under the default CGO_ENABLED=1. Run them with cgo disabled so they
execute on every PR.
cmds:
- go test -race -count=1 -timeout=10m ./agent/... ./internal/... ./test/fakeagent/...
- CGO_ENABLED=0 go test -count=1 -timeout=10m ./agent/cmd/rawtrail-agent-headless/...
test:go:agent:coverage:
desc: |
Agent + shared coverage flavor of test:go:agent. Emits coverage-agent.out
which CI uploads as an artifact for the Sonar scan job.
Two test runs: the default cgo + race run for the production agent, then
a CGO_ENABLED=0 run for the headless binary (UAT plan M2). The second run
narrows -coverpkg to ONLY the M2-new packages (the headless cmd directory
and agent/receiver) so its profile records files that the first run
cannot reach. The two profiles can therefore be merged by concatenation
without per-file overlap on queue/uploader/enrollment, which would
otherwise produce duplicate entries when Sonar parses the file.
common.go in agent/receiver IS instrumented by both runs (no build tag),
but its small surface (SetLogger + an atomic logger pointer) is already
well-covered by the first run; the second-run entries for it are
additive in -covermode=atomic, which Sonar resolves by taking the max.
cmds:
# See comment on test:go:server:coverage. Same -coverpkg fix so agent
# tests that exercise shared internal/ packages (observability, envparse)
# contribute to those packages' coverage on Sonar.
- go test -race -count=1 -timeout=10m -coverpkg=./agent/...,./internal/...,./test/fakeagent/... -coverprofile=coverage-agent.out -covermode=atomic ./agent/... ./internal/... ./test/fakeagent/...
# CGO_ENABLED=0 run for the headless package. -race is dropped because the race detector requires cgo. -coverpkg is narrowed to
# just the M2-new packages so the profile is essentially disjoint from the first run; see the desc above for the common.go
# caveat.
- CGO_ENABLED=0 go test -count=1 -timeout=10m -coverpkg=./agent/cmd/rawtrail-agent-headless/...,./agent/receiver -coverprofile=coverage-agent-headless.out -covermode=atomic ./agent/cmd/rawtrail-agent-headless/...
# Append the headless profile to coverage-agent.out, stripping its mode header.
- tail -n +2 coverage-agent-headless.out >> coverage-agent.out
- rm -f coverage-agent-headless.out
- go tool cover -func=coverage-agent.out | tail -1
test:go:server:
desc: |
Run Go tests with race detector for the server + shared code. Requires
$EDR_TEST_DSN set or the docker-compose mysql_test on port 33307.
Runs WITHOUT the integration build tag so the layer-2 + layer-3 tests
(`server/<X>/internal/tests/`, `test/integration/`) are excluded; for
those use test:go:server:integration.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
cmds:
- go test -race -count=1 -timeout=10m ./server/... ./internal/...
test:go:server:integration:
desc: |
Run the server's integration build-tag suite (layer-2 per-context tests
under server/<X>/internal/tests/ + layer-3 cross-context tests under
test/integration/). Requires a real MySQL via $EDR_TEST_DSN.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
cmds:
# -timeout=20m, not the 10m every other lane uses. This lane runs the DB-backed integration suites, and
# server/detection/internal/tests alone opens 109 isolated databases (full.Open applies all eight bounded contexts'
# schemas per test) under -race. Measured on CI: ~250s typical, against a 600s ceiling. That is only 2.4x headroom
# for a package whose cost is dominated by shared MySQL and ClickHouse containers, and it blew through twice in two
# days (#690, #697) purely on contention: both times the timeout panic listed tests that had been running only 5-26s,
# so the binary was still making progress rather than deadlocked. A ceiling that fires on a slow neighbour is not
# protecting anything, it just produces failures someone reruns. 20m is ~4.7x the measured median and still bounded,
# so a genuine hang fails in a reasonable time. Unit-only lanes keep 10m; a hung unit test should fail fast.
- go test -race -count=1 -timeout=20m -tags=integration ./server/... ./internal/... ./test/integration/...
test:integration:
desc: |
Shorthand for the integration-tagged Go suite. Same as
test:go:server:integration; kept as a top-level alias because the test
pyramid (unit / per-context / cross-context) is the user-facing concept.
cmds:
- task: test:go:server:integration
test:integration:agent-server:
desc: |
Run just the UAT plan L3 (M4) agent-server integration test:
test/integration/agentserver/. This is the headless agent + real server
+ 5-scenario starter-corpus pipeline check. Sets CGO_ENABLED=0 so the
test runs on darwin dev machines too (the test code imports the headless
package which is tagged `//go:build !darwin || !cgo`).
Requires a real MySQL via $EDR_TEST_DSN. CI does not call this directly;
the same tests are picked up automatically by test:go:server:integration
under the recursive ./test/integration/... glob on the ubuntu runner.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
CGO_ENABLED: 0
cmds:
- go test -count=1 -timeout=5m -tags=integration ./test/integration/agentserver/...
test:efficacy:
desc: |
Run the UAT plan L6 detection-efficacy harness at test/efficacy/.
Walks every test/efficacy/corpus/T*/ scenario through a real in-process
EDR stack (test/integration.Setup) via fakeagent's PostDirect, asserts
every rule_id in expected.yaml fires within the SLA, and walks the
ambient-noise lane to enforce the false-positive gate. Aggregate gate:
detection rate >= 95%, FP rate <= 1%.
Requires a real MySQL via $EDR_TEST_DSN. Wall clock is dominated by the
noise lane's per-scenario `within_seconds` settle (30s default each);
~60s on a 2-noise-scenario corpus, scaling linearly with noise count.
Nightly CI runs this via .github/workflows/efficacy.yml; this task is
for local iteration.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
CGO_ENABLED: 0
cmds:
- go test -count=1 -timeout=5m -tags=integration ./test/efficacy/...
uat:l5:
desc: |
Run the UAT plan L5 system-test driver against a local edr-qa VM (M11
milestone, local-execution flavour). Thin wrapper around
`scripts/uat/system-test.sh` that forwards every CLI arg through. The
operator picks the scenario explicitly because the harness will grow
past one scenario and a hardcoded default would silently mask the
growing corpus.
Required env (see scripts/uat/README.md "Auth flow"):
EDR_SERVER_URL e.g. https://edr.local:8088 (no trailing slash)
EDR_SESSION_COOKIE pre-minted edr_session cookie value from one browser
break-glass / OIDC login; reuse until it expires
VM_SSH_TARGET defaults to victor@192.168.64.7 (edr-qa)
Common forms:
task uat:l5 -- attack-runbook --dry-run # orchestration smoke
task uat:l5 -- attack-runbook --skip-install # already-enrolled VM
task uat:l5 -- attack-runbook \
--pkg-path=dist/rawtrail-v1.0.0.pkg
# full install path
Running L5 from a self-hosted GitHub Actions runner (so the cron lane
and release tags fire automatically) is tracked as a follow-up; this
task is the local-only flavour today.
cmds:
- scripts/uat/system-test.sh {{.CLI_ARGS}}
uat:scale:
desc: |
Run the UAT plan M12 scale-test harness at test/scale/scaledriver against
an externally-running EDR server (e.g. `task dev:server`). Fans out N
simulated hosts via fakeagent's PostDirect, records client-observed
ingest p50/p95/p99 latency, fails if the p99 budget (default 250ms) is
breached or any POST errors.
Defaults: 100 hosts, 80/20 quiet/active mix, 5 minute lane against
https://localhost:8088 with TLS verification enabled. Targeting
dev:server's mkcert cert requires either trusting the cert in the
system store (`task dev:certs`) or opt-in TLS skip via
`-- --insecure-tls=true`. The plan's 30 min baseline run is invoked via:
task uat:scale -- --duration=30m --output=test/scale/baselines/baseline.json
Required env:
EDR_ENROLL_SECRET shared secret the server reads from the same name
CLI flags forwarded via the task's -- separator (see `go run
./test/scale/scaledriver --help` for the full list). Common forms:
task uat:scale # 100 hosts x 5min
task uat:scale -- --hosts=10 --duration=30s # quick local check
task uat:scale -- --duration=30m \
--output=test/scale/baselines/baseline.json # baseline capture
cmds:
- go run ./test/scale/scaledriver {{.CLI_ARGS}}
uat:scale:gate:
desc: |
Run the per-RC scale gate (#203): a 200-host in-process lane (integration.Setup, production processor fan-out)
that asserts the event_queue backlog stays bounded. This is what .github/workflows/scale.yml runs on RC tags +
weekly; run it locally before promoting an RC. Needs a test MySQL (EDR_TEST_DSN); ~2 min wall.
cmds:
- go test -tags "integration scalegate" -v -count=1 -timeout=10m -run TestScaleGate_BacklogBounded ./test/scale/...
test:go:server:coverage:
desc: |
Server + shared coverage flavor of test:go:server. Emits coverage-server.out
which CI uploads as an artifact for the Sonar scan job. Runs with
-tags=integration so layer-2 + layer-3 tests contribute to the
cross-package coverage attribution; without that, the bootstrap.go and
service.go files (which the integration tests exercise) would report 0%.
env:
EDR_TEST_DSN: '{{.EDR_TEST_DSN | default "root@tcp(127.0.0.1:33307)/edr_test?parseTime=true"}}'
EDR_CLICKHOUSE_TEST_DSN: '{{.EDR_CLICKHOUSE_TEST_DSN | default "clickhouse://default:@127.0.0.1:19001/edr_test"}}'
cmds:
# -coverpkg=./server/...,./internal/... attributes coverage from one
# package's tests back to symbols in any other package they exercise.
# Without it, default Go coverage is per-test-package only, so e.g. the
# bootstrap.go and service.go files (constructed by integration tests
# in identity/internal/tests/) showed 0% coverage even though the
# tests fully exercise them. Per the modular-monolith pattern,
# cross-package exercise via the public api/ + bootstrap/ surface is
# the intended coverage path; coverpkg makes Sonar see it.
#
# ./test/scale/... is included so M12's per-PR smoke (5 hosts x 5s
# against integration.Setup) runs on every push. Wall-clock cost is
# ~5s, dominated by the smoke's own load lane.
#
# ./tools/... is included because no other CI lane runs those tests, and some are the only evidence for behaviour a spec
# names: the Sigma upstream sync workflow's step scripts are exercised only by tools/sigma-sync's tests. The tools' own code
# is not in -coverpkg, so it adds nothing to the denominator, but a tool test that calls server code counts toward its coverage.
# -timeout=20m, not the 10m every other lane uses. This lane runs the DB-backed integration suites, and
# server/detection/internal/tests alone opens 109 isolated databases (full.Open applies all eight bounded contexts'
# schemas per test) under -race. Measured on CI: ~250s typical, against a 600s ceiling. That is only 2.4x headroom
# for a package whose cost is dominated by shared MySQL and ClickHouse containers, and it blew through twice in two
# days (#690, #697) purely on contention: both times the timeout panic listed tests that had been running only 5-26s,
# so the binary was still making progress rather than deadlocked. A ceiling that fires on a slow neighbour is not
# protecting anything, it just produces failures someone reruns. 20m is ~4.7x the measured median and still bounded,
# so a genuine hang fails in a reasonable time. Unit-only lanes keep 10m; a hung unit test should fail fast.
- go test -race -count=1 -timeout=20m -tags=integration -coverpkg=./server/...,./internal/... -coverprofile=coverage-server.out -covermode=atomic ./server/... ./internal/... ./test/integration/... ./test/scale/... ./tools/...
- go tool cover -func=coverage-server.out | tail -1
test:ui:
desc: Run Vitest suite
dir: ui
cmds:
- npm test -- --run
test:e2e:install:
desc: |
Install Playwright + Chromium for the E2E suite at test/e2e/.
One-time setup; subsequent test:e2e invocations skip this. The
Playwright browser download is ~300 MB and lives outside the
repo, under ~/Library/Caches/ms-playwright on macOS.
dir: test/e2e
cmds:
- npm install
- npx playwright install --with-deps chromium
test:e2e:
desc: |
Run the Playwright E2E suite at test/e2e/. Covers the browser-
driven flows the Go integration suite can't: WebAuthn ceremonies
(via a virtual authenticator), OIDC sign-in (via the local dex
from `task qa:up`), UI affordances. The playwright.config.ts
auto-starts the dev server pointed at dex via
task:dev:server:qa-oidc when no listener is on the lane's port.
Pre-req: `task qa:up` (mysql + dex).
From the second worktree, pass the lane: E2E_PORT=8089
E2E_DB=edr2 (validated as a matching pair, since a crossed one
resets the other worktree's tables). The spawn is lane-aware and
creates that lane's MySQL schema and ClickHouse database if they
are absent, so a cold lane B boots its own server rather than
requiring one already running (issue #826).
Running `task dev:server:qa-oidc` by hand on a lane whose schemas
do not exist yet still fails on them; `scripts/ensure-lane-schemas.sh
<schema>` is what the spawn calls and can be run directly.
deps: [test:e2e:install]
dir: test/e2e
cmds:
- npm test
demo:credential-guard:
desc: >-
Demonstrate the AI agent credential guard in protect mode on an enrolled Mac (issue #1433): a fake ~/.aws/credentials, the
fleet's policy switched to protect, Claude Code asked about it, and the kernel's refusal checked; the policy and the file are
restored after. Needs EDR_SERVER and EDR_API_TOKEN; HOST is the Mac's ssh target (default edr-dev, victor@192.168.64.5),
INTERACTIVE=1 prints the prompt to type instead of running Claude Code non-interactively, and INJECTION=1 runs the poisoned
project scenario (a fake ~/.ssh/id_ed25519_deploy and a README that hides an instruction to print it).
cmds:
- >-
scripts/demo/credential-guard.sh {{.HOST | default "victor@192.168.64.5"}} {{if .INTERACTIVE}}--interactive{{end}}
{{if .INJECTION}}--injection{{end}}
test:e2e:demo:
desc: |
Run the Playwright demo smoke test (test/e2e/tests/demo/) against an
already-running demo stack. Unlike test:e2e it does NOT auto-start a
server or touch the qa dex: it drives the live, seeded demo at
https://localhost:8088 via playwright.demo.config.ts. Used by the
demo-nightly workflow. Pre-reqs: `docker compose -f docker-compose.demo.yml
up` (seeder finished) and a one-time `task test:e2e:install`.
dir: test/e2e
cmds:
- npm run demo
test:e2e:coverage:
desc: |
Run the Playwright default-env suite (auth + qa: C/D/F.4 + A.5)
against an instrumented Go server binary so the lines exercised
by the browser-driven flows land in coverage-server-e2e.out and
the running React bundle's V8 coverage lands in
test/e2e/coverage/lcov-e2e.info. Sonar + Codecov configs already
point at both files. Wall clock ~3 min.
Mechanics:
1. `go build -cover` the server into tmp/edr-server-e2e.
2. Start it with GOCOVERDIR=tmp/covdata-e2e so the instrumented
binary writes per-package coverage profiles on graceful exit.
3. Run the Playwright suites with E2E_COVERAGE=1 + E2E_REUSE_SERVER=1
so per-test V8 coverage drops to test/e2e/coverage-raw/.
4. SIGTERM the server so the cover runtime flushes profiles.
5. `go tool covdata textfmt` merges the per-package profiles
into coverage-server-e2e.out.
6. `npm run coverage:lcov` aggregates the V8 payloads into
test/e2e/coverage/lcov-e2e.info via monocart-coverage-reports
(which walks the embedded Vite source maps so paths map back
to ui/src/**).
Pre-req: `task qa:up` (mysql + dex). The orchestration script
lives at scripts/test-e2e-coverage.sh so the shell-glue logic
stays in bash (Taskfile's default mvdan/sh interpreter chokes
on the `VAR=value ./binary &` background pattern).
deps: [test:e2e:install, build:ui:e2e]
cmds:
- bash scripts/test-e2e-coverage.sh
test:uat:driver:
desc: |
Tests for the L5 driver's rule classification and the runbook's skip propagation (issues #965, #966). Needs no VM
and no server: the driver's --dry-run mode loads the scenario and short-circuits every remote call, which reaches
the part that decides pass / miss / skipped. The install path is not covered here and is exercised on edr-qa.
cmds:
- bash scripts/uat/system-test.test.sh
test:short:
desc: Fast Go tests only (no -race, no DB-dependent suites)
cmds:
- go test -short -count=1 ./server/... ./agent/... ./internal/...
test:ui:coverage:
desc: |
Run the UI test suite with coverage; emits ui/coverage/lcov.info in
the format SonarCloud expects via sonar.javascript.lcov.reportPaths.
With zero tests vitest's v8 provider skips the lcov write entirely,
so the task falls back to creating an empty file; Sonar treats empty
lcov as "no data" rather than an error, and CI's upload-artifact
step always has a file to read.
dir: ui
cmds:
- npm run test:coverage
- '[ -f coverage/lcov.info ] || { mkdir -p coverage && : > coverage/lcov.info; }'
# vitest records no line hit for a function's declaration line, so SonarCloud took the E2E report's zero for it; give each the
# function's own call count (scripts/lcov-declaration-lines.awk).
- awk -f ../scripts/lcov-declaration-lines.awk coverage/lcov.info > coverage/lcov.info.tmp && mv coverage/lcov.info.tmp coverage/lcov.info
# -------- lint --------
lint:
desc: |
Run every fast linter. Does NOT include lint:nilaway (inter-procedural
analysis, ~tens of seconds) or lint:vulncheck (network-bound; queries
the vulnerability DB); run those explicitly when needed.
deps: [lint:go, lint:ui, lint:swift, lint:actions, lint:md, lint:md:prose, lint:dashes, lint:sh]
lint:sh:
desc: |
shellcheck over the UAT / QA shell harness. Scoped to those scripts rather than every .sh in the tree: they are
release-gate infrastructure whose failures are silent (a driver that misclassifies a result still exits cleanly),
and the rest of the tree's scripts have not been brought to a clean baseline yet. The packaging scripts have their
own shellcheck step in .github/workflows/pkg-dryrun.yml and are deliberately not duplicated here.
Runs in CI inside the `L5 driver tests` job in test.yml, alongside the driver's own test suite, rather than as a
workflow of its own: the two guard the same harness and there is no reason to spend a second checkout on it.
cmds:
- shellcheck scripts/uat/system-test.sh scripts/uat/system-test.test.sh scripts/uat/lib/common.sh
scripts/uat/scenarios/*/attack.sh scripts/qa/attack-runbook.sh
lint:dashes:
desc: |
Two dash gates. (1) tools/lint-no-emdash.sh: no em dash (U+2014) or en dash (U+2013) in any tracked text file.
(2) tools/dash-lint: no spaced-hyphen em dash (a hyphen with a space on both sides) in Markdown prose or in code
comments / string literals. Both forbid the dash-as-aside; reword (prefer shorter sentences) or use a colon. CI gate
is .github/workflows/no-emdash.yml.
cmds:
- bash tools/lint-no-emdash.sh
- git ls-files -z | go run ./tools/dash-lint
lint:docs:issues:
desc: |
tools/doc-issue-lint: public-facing docs (operator docs, the OpenAPI spec, the rule pack files) may reference a GitHub
issue only while it is OPEN. A closed issue or any pull request is history, which belongs in CHANGELOG.md; describe the
current behavior instead. Needs GitHub API access (GITHUB_TOKEN, GH_TOKEN, or a logged-in gh CLI). CI gate is
.github/workflows/md-lint.yml, which also runs it daily because closing an issue can fail docs no PR touched.
cmds:
- 'GITHUB_TOKEN="${GITHUB_TOKEN:-${GH_TOKEN:-$(gh auth token)}}" sh -c "git ls-files -z | go run ./tools/doc-issue-lint"'
lint:go:
desc: |
golangci-lint across the whole Go module. Runs the CUSTOM golangci-lint binary
at tmp/golangci-lint-custom which has the repo-local commentwrap plugin compiled
in (see .custom-gcl.yml). The lint:install dep auto-builds it on first run and
short-circuits on subsequent runs via sources/generates. The upstream binary
cannot load the plugin, so `.golangci.yml` enables `commentwrap` under
`linters.enable` and the upstream binary would abort with "unknown linter"
before any real lint runs.
deps: [lint:install]
cmds:
- tmp/golangci-lint-custom run ./...
lint:go:fix:
desc: golangci-lint with --fix for auto-fixable findings (auto-builds the custom binary via lint:install)
deps: [lint:install]
cmds:
- tmp/golangci-lint-custom run --fix ./...
lint:go:newcode:
desc: |
NEW-CODE-ONLY strict lint: cognitive complexity (gocognit) + dead parameters
(unparam), scoped to lines changed versus origin/main via --new-from-merge-base.
Uses the separate .golangci-strict.yml so the whole-repo lint:go run is untouched.
Existing AI-generated complexity is grandfathered; new functions cannot add past
the gocognit threshold. CI fetches origin/main first; local runs need it present.
See docs/maintenance/tasks/consolidation-pass.md for paying down the existing stock.
deps: [lint:install]
cmds:
- tmp/golangci-lint-custom run --config .golangci-strict.yml --new-from-merge-base origin/main ./...
lint:install:
desc: |
Build the custom golangci-lint binary at tmp/golangci-lint-custom by running
`golangci-lint custom` against .custom-gcl.yml. The custom binary bakes in the
repo-local commentwrap plugin (issue #149) so `task lint:go` enforces it as a
standard linter. Re-run whenever .custom-gcl.yml or the plugin's source under
tools/comment-wrap-check/lint/ changes, or after a golangci-lint version bump.
sources:
- .custom-gcl.yml
- tools/comment-wrap-check/lint/*.go
- tools/comment-wrap-check/lint/go.mod
- tools/comment-wrap-check/lint/go.sum
- go.mod
- go.sum
generates:
- tmp/golangci-lint-custom
env:
# Build the custom binary with the same toolchain that go.mod targets. golangci-lint's own module still declares go 1.26,
# so whichever `go` happens to satisfy that first gets used, and a 1.26-built binary cannot read the export data a 1.27
# toolchain writes: it starts, then panics inside go/types partway through the run. Pinning this removes the dependence on
# PATH ordering. Move it in lockstep with the go directive in go.mod.
GOTOOLCHAIN: go1.27.3
cmds:
- mkdir -p tmp
- golangci-lint custom
lint:go:plugin:
desc: |
Run the commentwrap plugin's own analyzer tests (its sub-module is not picked up by
the parent module's `go test ./...`). Wired into CI's go-lint workflow as a safety
net: a regression in the linter itself would otherwise slip past the parent's test
sweep.
dir: tools/comment-wrap-check/lint
cmds:
- go test -race -count=1 ./...
lint:arch:
desc: |
arch-go architecture rules over the modular monolith bounded contexts.
Hard-fails on any cross-context import that isn't in arch-go.yml's
allow-list. Strict mode: no transitional exceptions.
cmds:
- go test ./test/arch/...
lint:ui:
desc: ESLint + tsc --noEmit across the React UI
dir: ui
cmds:
- npm run lint
- npx tsc --noEmit
lint:swift:
desc: swiftlint on the macOS extensions (requires `brew install swiftlint`)
cmds:
- swiftlint lint --strict
lint:c:
desc: clang-tidy + clang-format --dry-run on the CGo bridge (requires `brew install llvm`)
cmds:
- find agent/xpcbridge \( -name '*.c' -o -name '*.h' \) -exec "$(brew --prefix llvm)/bin/clang-tidy" {} +
- find agent \( -name '*.c' -o -name '*.h' \) -exec "$(brew --prefix llvm)/bin/clang-format" --dry-run --Werror {} +
lint:vulncheck:
desc: govulncheck against both binaries (installs govulncheck if missing)
deps: [lint:vulncheck:agent, lint:vulncheck:server]
lint:vulncheck:agent:
desc: govulncheck scoped to the agent binary's transitive deps
cmds:
- go tool govulncheck ./agent/... ./internal/...
lint:vulncheck:server:
desc: govulncheck scoped to the server binary's transitive deps
cmds:
- go tool govulncheck ./server/...
lint:actions:
desc: actionlint on GitHub Actions workflows (requires `brew install actionlint`)
cmds:
- actionlint
lint:nilaway:
desc: |
uber-go/nilaway inter-procedural nil-dereference analysis (slow; separate
from golangci-lint). Scoped to project dirs rather than ./... so the
gitignored tmp/ scratch area never gates lint runs.
cmds:
- go tool nilaway -include-pkgs="github.com/rawtrail/rawtrail" ./server/... ./agent/... ./internal/... ./test/...
lint:md:
desc: markdownlint-cli2 across all committed markdown (config in .markdownlint-cli2.yaml).
cmds:
- npx -y -p markdownlint-cli2@0.18.1 markdownlint-cli2
lint:md:fix:
desc: markdownlint-cli2 with --fix to auto-repair the rules it knows how to fix.
cmds:
- npx -y -p markdownlint-cli2@0.18.1 markdownlint-cli2 --fix
lint:md:prose:
desc: |
Prettier --check over committed Markdown: enforces the no-hard-wrap convention (proseWrap: never) and aligned table columns.
Config in .prettierrc.yaml, exclusions in .prettierignore. Fix locally with `task lint:md:prose:fix`. CI gate in
.github/workflows/md-lint.yml. Pairs with lint:md (markdownlint owns structure; Prettier owns wrapping + tables).
cmds:
- npx -y prettier@3.3.3 --check "**/*.md"
lint:md:prose:fix:
desc: Prettier --write over committed Markdown to reflow prose to single lines and align tables (the auto-fix for lint:md:prose).
cmds:
- npx -y prettier@3.3.3 --write "**/*.md"
# -------- dev loops --------
dev:certs:
desc: |
Generate a localhost TLS cert + key at tmp/dev.crt and tmp/dev.key so dev:server
can boot in the same TLS-required posture as production (issue #140, "dev mirrors
prod"). Prefers mkcert (https://github.com/FiloSottile/mkcert), which installs a
local CA into the system trust store and issues certs every browser accepts
without warnings; falls back to openssl self-signed when mkcert is absent (e.g.
a fresh CI runner) - browsers warn once and the operator clicks "Advanced ->
Proceed to localhost".
For warning-free dev, install mkcert once on this workstation:
brew install mkcert nss # nss is for Firefox trust on macOS
mkcert -install
Idempotent: the task short-circuits when tmp/dev.crt exists and the cert's
notAfter is more than 24h from now. Delete tmp/dev.crt (or both files) to force
regeneration. Cert SAN covers localhost + 127.0.0.1 + ::1; access via any other
hostname (e.g. a LAN IP from a UTM VM) shows a cert mismatch - add the extra
hostname to the mkcert invocation (or to the openssl -addext SAN list in the
fallback path) and delete tmp/dev.crt to force regeneration.
generates:
- tmp/dev.crt
- tmp/dev.key
status:
- test -f tmp/dev.crt && test -f tmp/dev.key && openssl x509 -checkend 86400 -noout -in tmp/dev.crt
cmds:
- mkdir -p tmp
- |
if command -v mkcert >/dev/null 2>&1; then
echo "generating dev cert via mkcert (browser-trusted)"
mkcert -cert-file tmp/dev.crt -key-file tmp/dev.key localhost 127.0.0.1 ::1
else
echo "mkcert not found; falling back to openssl self-signed (browser warns once)"
echo "install mkcert for warning-free dev: brew install mkcert nss && mkcert -install"
openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
-keyout tmp/dev.key -out tmp/dev.crt \
-subj "/CN=localhost" \
-addext "subjectAltName=DNS:localhost,IP:127.0.0.1,IP:::1"
fi
- chmod 0600 tmp/dev.key
dev:server:
desc: |
Run the server against local MySQL at https://localhost:8088 (issue #140, dev
mirrors prod). TLS uses the cert pair from `task dev:certs`. Binds 0.0.0.0:8088
so a UTM/QEMU VM agent on the bridge interface (typically 192.168.64.1 from inside
the VM) can reach it; the cert SAN covers localhost + 127.0.0.1 + ::1 only - a
remote host accessing via a different name sees a cert mismatch. On an untrusted
LAN, override EDR_LISTEN_ADDR back to 127.0.0.1:8088. The DSN uses an empty
password to match the docker-compose mysql service (MYSQL_ALLOW_EMPTY_PASSWORD=yes);
a `root:toor@...` DSN will fail with access-denied. OTel exporter targets the local
signoz-otel-collector on 4317; if no collector is running, exporter init logs
warnings and request handling continues.
For tooling that can't trust the dev cert, install mkcert + `mkcert -install` so
the local CA lands in the system trust store, OR use `curl -k` / equivalent
client-side overrides. The server binary has no plaintext-HTTP mode (issue #140);
reach for those client-side workarounds, not a non-existent HTTP variant.
EDR_UI_LIVE_DIR (issue #141) points the server at server/ui/dist on disk instead of
the compile-time embedded bundle, so `task build:ui` is picked up on the next
request without restarting the server. Production binaries leave the var unset and
serve the embedded copy.
deps: [dev:certs]
env:
EDR_DSN: "root:@tcp(127.0.0.1:33306)/edr?parseTime=true"
# ClickHouse event store (ADR-0015), required to boot: the dev clickhouse service on native port 19000 (see docker-compose.yml).
EDR_CLICKHOUSE_DSN: '{{.EDR_CLICKHOUSE_DSN | default "clickhouse://default:@127.0.0.1:19000/edr"}}'
EDR_ENROLL_SECRET: dev-enroll-secret
EDR_TLS_CERT_FILE: tmp/dev.crt
EDR_TLS_KEY_FILE: tmp/dev.key
EDR_LISTEN_ADDR: "0.0.0.0:8088"
EDR_LOG_FORMAT: text
EDR_UI_LIVE_DIR: server/ui/dist
# Dev-only deterministic root secret. Required on every boot: the host-token signing key
# and the pre-auth cookie signing key both derive from it (internal/keyring). Production
# deployments MUST supply a real random secret via EDR_SECRET_KEY_FILE (Docker secrets
# convention): never reuse this placeholder. Length: 49 bytes, well past the 32-byte
# minimum the config layer validates.
EDR_SECRET_KEY: dev-only-secret-key-do-not-use-in-production-xyz
# Default dev mode is break-glass-only (no OIDC IdP wired up); the server always boots and SSO is configured at runtime via the
# UI (issue #512). For the OIDC-against-dex flow used by Playwright e2e + the QA matrix, use `task dev:server:qa-oidc` instead.
# Break-glass RP_ORIGINS now uses https:// to match the server's TLS posture;
# RP_ID stays the registrable host (no scheme, no port). Mismatch between the
# browser-observed origin and any RP_ORIGINS entry rejects the WebAuthn ceremony
# with a generic "Setup failed" in the UI. CookieSecure flips on automatically
# via cfg.TLSEnabled() in cmd/main, so session + reauth cookies now carry the
# Secure attribute, same shape as prod.
EDR_BREAKGLASS_RP_ID: localhost
EDR_BREAKGLASS_RP_ORIGINS: https://localhost:8088
OTEL_EXPORTER_OTLP_ENDPOINT: http://127.0.0.1:4317
OTEL_EXPORTER_OTLP_PROTOCOL: grpc
OTEL_EXPORTER_OTLP_INSECURE: "true"
OTEL_SERVICE_NAME: rawtrail-server
cmds:
- go run {{.SERVER_PKG}}
dev:ui:
desc: Run the UI Vite dev server with HMR (proxies /api/v1 to the backend)
dir: ui
deps: [ui:setup]
cmds:
- npm run dev
dev:
desc: |
Run backend + UI dev servers in parallel for fast iteration. Backend lives at
http://127.0.0.1:8088 (Go server, serves `server/ui/dist/` via EDR_UI_LIVE_DIR --
a `task build:ui` in another terminal is picked up on the next request without
restarting the server, per issue #141); UI HMR lives at http://localhost:5173/ui/
(Vite dev server, proxies /api/* back to :8088). Edit .tsx/.scss -> Vite HMR
updates in <1s; edit .go -> Ctrl-C and re-run `task dev` (Go has no built-in HMR).
For QA against the embedded bundle (production shape), unset EDR_UI_LIVE_DIR.
deps:
- dev:server
- dev:ui
# -------- database --------
db:up:
desc: Start local MySQL + ClickHouse (dev + test) via docker-compose
cmds:
- docker compose up -d mysql mysql_test clickhouse clickhouse_test
db:down:
desc: Stop local MySQL + ClickHouse
cmds:
- docker compose down
install:multi-replica:up:
desc: |
Bring up the reference multi-replica HA stack (two server replicas + MySQL +
ClickHouse + NGINX) from packaging/docker-compose-multi-replica.yml for local
validation. Prerequisites: populate packaging/secrets/ (mysql_root, edr_dsn,
enroll_secret, secret_key, clickhouse_dsn) and packaging/tls/ (fullchain.pem,
privkey.pem) per docs/install-server.md, and either pull a released image or
build one. Then verify through the load balancer: curl -k https://localhost/readyz
(or https://localhost:$EDR_HTTPS_PORT/readyz if you override the default 443).
dir: packaging
cmds:
- docker compose -f docker-compose-multi-replica.yml up -d
install:multi-replica:down:
desc: Stop the multi-replica HA stack and remove its containers.
dir: packaging
cmds:
- docker compose -f docker-compose-multi-replica.yml down
db:reset:
desc: |
DESTRUCTIVE: drop the dev database and recreate empty. Requires explicit
confirmation (Taskfile's prompt:). Use only when the schema is out of
sync with current migrations; routine dev never needs this.
Credentials match the dev:server DSN (root, empty password, port 33306);
see the dev:server desc for why the password is empty (matches docker-compose
MYSQL_ALLOW_EMPTY_PASSWORD=yes; a `-ptoor` invocation produces access-denied).
prompt: About to DROP DATABASE edr. Continue?
cmds:
- mysql -uroot -h127.0.0.1 -P33306 -e "DROP DATABASE IF EXISTS edr; CREATE DATABASE edr CHARACTER SET utf8mb4;"
db:backup:
desc: |
Snapshot the dev database to a timestamped tmp/edr-dev-<ts>.sql.gz file.
Use before a destructive QA step (db:reset, manual schema surgery, walking
the break-glass redemption flow from scratch) so you can rewind if the run
goes sideways.
--single-transaction takes an InnoDB consistent snapshot without blocking
writes; --routines + --triggers pull in stored procedures the events table
doesn't have today but the schema may grow into. Output is gzipped because
audit_events + events tables compress 10x.
cmds:
- mkdir -p tmp
- |
OUT="tmp/edr-dev-$(date +%Y%m%d-%H%M%S).sql.gz"
mysqldump -uroot -h127.0.0.1 -P33306 \
--single-transaction --routines --triggers \
edr | gzip > "$OUT"
echo "wrote $OUT ($(du -h "$OUT" | cut -f1))"
db:restore:
desc: |
DESTRUCTIVE: drop the dev database and reload it from a .sql or .sql.gz
backup produced by db:backup (or a manual mysqldump). Requires
confirmation. Pass the file path via BACKUP_FILE:
task db:restore BACKUP_FILE=tmp/edr-dev-20260510-145322.sql.gz
The restore drops + recreates the edr schema first so it's idempotent
against the previous state; nothing in the current DB survives. Use
db:backup to snapshot before invoking this if you might want to walk
back the restore itself.
vars:
BACKUP_FILE: '{{.BACKUP_FILE}}'
prompt: 'About to DROP DATABASE edr and reload from {{.BACKUP_FILE}}. Continue?'
preconditions:
- sh: 'test -n "{{.BACKUP_FILE}}"'
msg: 'BACKUP_FILE is required. Usage: task db:restore BACKUP_FILE=tmp/edr-dev-<ts>.sql.gz'
- sh: 'test -f "{{.BACKUP_FILE}}"'
msg: 'BACKUP_FILE not found: {{.BACKUP_FILE}}'
cmds:
- mysql -uroot -h127.0.0.1 -P33306 -e "DROP DATABASE IF EXISTS edr; CREATE DATABASE edr CHARACTER SET utf8mb4;"
- |
# gzip -dcf works on both macOS and Linux: -d decompresses,
# -c streams to stdout, -f passes plaintext through
# transparently when the file isn't gzipped. POSIX-portable
# so the task runs the same on Debian/Ubuntu (where /bin/sh
# is dash and doesn't grok [[ ]]) as it does on macOS bash.
case "{{.BACKUP_FILE}}" in
*.gz) gzip -dcf "{{.BACKUP_FILE}}" | mysql -uroot -h127.0.0.1 -P33306 edr ;;
*) mysql -uroot -h127.0.0.1 -P33306 edr < "{{.BACKUP_FILE}}" ;;
esac
echo "restored edr from {{.BACKUP_FILE}}"
# -------- qa / local IdP --------
qa:up:
desc: |
Bring up the full QA stack: dev mysql, test mysql, and a local
dex (https://dexidp.io) OIDC provider on 127.0.0.1:5556. Used by
operators exercising the OIDC sign-in flow against a
local IdP instead of a real Okta tenant. The dex
static-passwords config lives at config/dex/dev-config.yaml; the
compose overlay is docker-compose.qa.yml. See `task
dev:server:qa-oidc` for the matching EDR-side env wiring.
cmds:
- docker compose -f docker-compose.yml -f docker-compose.qa.yml up -d
- |
echo "waiting for dex discovery endpoint to be ready..."
for _ in $(seq 1 30); do
if curl -fsS http://localhost:5556/dex/.well-known/openid-configuration > /dev/null 2>&1; then
echo "dex ready at http://localhost:5556/dex"
exit 0
fi
sleep 1
done
echo "dex did not become ready within 30s; check 'docker logs rawtrail-dex'"
exit 1
qa:down:
desc: |
Tear down the qa stack (mysql + dex). Use when switching between
OIDC and break-glass-only QA runs, or to free port 5556.
cmds:
- docker compose -f docker-compose.yml -f docker-compose.qa.yml down
dev:server:qa-oidc:
desc: |
Run the dev server pointed at the local dex (qa:up) instead of a
real Okta tenant. The server reads no EDR_OIDC_* env vars (issue
#512); SSO lives in the durable oidc_config store. This task applies
migrations, seeds the dex connection config into that store (via the
demo-seed --oidc-only path), then boots the server. The four
pre-provisioned dex users (analyst@qa.local, senior@qa.local,
auditor@qa.local, admin@qa.local) all share password
`qa-password-123`. First sign-in JIT-provisions the user as
`analyst`; promote via SQL per docs/authz.md to test the role matrix.
HTTPS-only since issue #140. Depends on dev:certs and seeds the
external URL as https://localhost:8088, so the derived redirect is
https://localhost:8088/api/auth/callback; config/dex/dev-config.yaml's
redirectURIs list mirrors that.
deps: [dev:certs]
env:
EDR_DSN: "root:@tcp(127.0.0.1:33306)/edr?parseTime=true"
# ClickHouse event store (ADR-0015), required to boot: the dev clickhouse service on native port 19000 (see docker-compose.yml).
EDR_CLICKHOUSE_DSN: '{{.EDR_CLICKHOUSE_DSN | default "clickhouse://default:@127.0.0.1:19000/edr"}}'
EDR_ENROLL_SECRET: dev-enroll-secret
EDR_TLS_CERT_FILE: tmp/dev.crt
EDR_TLS_KEY_FILE: tmp/dev.key
EDR_LISTEN_ADDR: "0.0.0.0:8088"
EDR_LOG_FORMAT: text
EDR_UI_LIVE_DIR: server/ui/dist
EDR_SECRET_KEY: dev-only-secret-key-do-not-use-in-production-xyz
EDR_BREAKGLASS_RP_ID: localhost
EDR_BREAKGLASS_RP_ORIGINS: https://localhost:8088
# Dex SSO connection config seeded into the durable store (not read by the server). Dex listens on plain HTTP :5556 (its own
# listener, not the EDR's); the OIDC client lib speaks HTTP to the issuer fine. The external URL drives the derived redirect, which
# MUST match config/dex/dev-config.yaml's redirectURIs entry (https://localhost:8088/api/auth/callback, issue #140).
EDR_DEMO_OIDC_ISSUER: http://localhost:5556/dex
EDR_DEMO_OIDC_CLIENT_ID: edr-qa
EDR_DEMO_OIDC_CLIENT_SECRET: edr-qa-client-secret-do-not-use-in-prod
EDR_DEMO_OIDC_EXTERNAL_URL: https://localhost:8088
OTEL_EXPORTER_OTLP_ENDPOINT: http://127.0.0.1:4317
OTEL_EXPORTER_OTLP_PROTOCOL: grpc
OTEL_EXPORTER_OTLP_INSECURE: "true"
OTEL_SERVICE_NAME: rawtrail-server
cmds:
# Apply migrations so oidc_config exists, seed the dex SSO config into the durable store, then boot. --oidc-force overwrites any
# existing row so a rerun always restores the QA dex issuer/client/external-URL (a plain seed is a no-op once a row exists).
- go run ./server/cmd/rawtrail-migrate
- go run ./server/cmd/rawtrail-demo-seed --oidc-only --oidc-force
- go run {{.SERVER_PKG}}
# -------- install tools / hooks --------
install:
desc: Install pinned tools (go, node, golangci-lint, lefthook) via mise (https://mise.jdx.dev) or asdf
cmds:
- mise install || asdf install
- lefthook install
hooks:
desc: Install the project's git hooks (runs lefthook install)
cmds:
- lefthook install
# -------- QA smoke --------
qa:
desc: |
Build every binary + UI. Does NOT run a runtime smoke test today; see
docs/testing.md (or start dev:server in another terminal and exercise
the login/enroll/ingest/logout path by hand). A scripted smoke harness
is tracked as future work.
deps: [build]
# -------- release --------
release:archive:
desc: |
Archive every pending OpenSpec change, in the order tools/spectrace computes.
Step 1 of docs/release-checklist.md; run it on the release-prep branch, after
capturing the archive-verify baseline.
The order is the hazard this removes from the human. `openspec archive` replaces
a MODIFIED requirement WHOLE, so when one pending change adds a requirement and
another modifies or retires it, the wrong order discards the later text with no
error and no downstream check notices (issue #901, and the losses #905 tracks).
`spectrace archive-order` computes a safe order; this applies it.
Hard stops, both of which archive nothing: archive-order exit 1 means no safe
order exists and a human must split one change or reconcile the requirements two
of them contend over, and exit 2 is a tool failure. There is no fallback to
alphabetical. A failed `openspec archive` stops the run where it is rather than
carrying on down the list.
Preview the sequence without touching the tree: `task release:archive -- --dry-run`.
Both dashes are required, and a bare word is refused rather than run as a real
archive, so a mistyped preview cannot become one.
It does NOT verify the archive was lossless; that is the separate before/after
archive-verify comparison in the checklist.
preconditions:
- sh: command -v openspec
msg: "openspec must be on PATH (npm i -g @fission-ai/openspec@{{.OPENSPEC_VERSION}})"
- sh: '[ "$(openspec --version)" = "{{.OPENSPEC_VERSION}}" ]'
msg: "openspec on PATH is not {{.OPENSPEC_VERSION}}, the version CI validates with (npm i -g @fission-ai/openspec@{{.OPENSPEC_VERSION}})"
cmds:
- go run ./tools/releasearchive {{.CLI_ARGS}}
# -------- packaging --------
pkg:dryrun:
desc: |
Build an unsigned .pkg locally using ad-hoc signing + the existing
Xcode Debug build output. Does not notarize. Useful for verifying
distribution.xml + script layout before a real release tag fires CI.
Expects extension/edr/build/Debug/edr.app to exist (run
build:extension once to seed).
vars:
TAG: '{{.TAG | default "v0.0.0-local"}}'
cmds:
- APPLE_TEAM_ID=FDG8Q7N4CC ./packaging/pkg/build.sh {{.TAG}} --dry-run
pkg:release:
desc: |
Build a signed + notarized .pkg locally. Requires Developer ID
Application + Installer certs installed in the login keychain and
the APPLE_* env vars set. For routine releases, push a v* tag
instead and let CI do this.
vars:
TAG: '{{.TAG}}'
# Separate preconditions per env var so each missing one reports itself.
# A concatenation like `test -n "${A}${B}${C}"` is true when any single
# var is set and would let the build proceed partway before failing.
preconditions:
- sh: test -n "{{.TAG}}"
msg: "TAG is required, e.g. `task pkg:release TAG=v0.5.0`"
- sh: test -n "${APPLE_NOTARY_APPLE_ID:-}"
msg: "APPLE_NOTARY_APPLE_ID must be exported"
- sh: test -n "${APPLE_NOTARY_APP_PASSWORD:-}"
msg: "APPLE_NOTARY_APP_PASSWORD must be exported"
- sh: test -n "${APPLE_TEAM_ID:-}"
msg: "APPLE_TEAM_ID must be exported"
cmds:
- ./packaging/pkg/build.sh {{.TAG}}
profiles:render:
desc: |
Render the three .mobileconfig templates with APPLE_TEAM_ID substituted.
Profiles ship unsigned by design; MDM channels sign them at delivery
(Fleet rejects pre-signed uploads).
cmds:
- APPLE_TEAM_ID={{.APPLE_TEAM_ID | default "FDG8Q7N4CC"}} ./packaging/profiles/render.sh
docker:server:
desc: |
Build the server container image locally as rawtrail-server:dev.
Uses the multi-stage Dockerfile; no push. For the prod-compose stack,
see docker-compose.prod.yml and docker-compose.prod.README.md.
cmds:
- docker build -f server/Dockerfile -t rawtrail-server:dev --build-arg VERSION=dev .
docker:prod:up:
desc: |
Boot the reference prod compose stack (MySQL + server) from the
checkout. Operator must populate ./secrets/ per docker-compose.prod.README.md
first. Idempotent: brings the stack up if down, leaves it alone if
already running.
cmds:
- docker compose -f docker-compose.prod.yml -f docker-compose.prod.build.yml --env-file .env up -d --build
docker:prod:down:
desc: Tear down the prod compose stack and remove its MySQL data volume.
cmds:
- docker compose -f docker-compose.prod.yml --env-file .env down -v