CHANGELOG.md
Rama main · réplica de lectura del repositorio original.
# Changelog
All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/), and the project aims to follow
[Semantic Versioning](https://semver.org/) from 1.0 onward.
## [Unreleased]
### Tutorial mode
- **Tours.** The console has a welcome tour and one tour for each page. The welcome tour
starts after the first sign-in, or on the first visit with authentication off. It shows the
sidebar, the page time filter, the search, the bell, the Ask soc-ai button and the help menu.
The tour of a page starts on the first visit to the page, after the welcome tour when both
apply. Two tours never run at the same time. A step dims the page and draws an outline around
one control, with a card beside it. A step whose control is not on the page is skipped. A tour
with no control on the page does not start.
- **The help menu.** The "?" button in the top bar opens a menu: Tour this page, Tour soc-ai from
the start, Keyboard shortcuts and Open the guide. The menu also turns the tours on again.
- **Turn off tours.** The last step of a tour and the Skip panel offer "Turn off tours". No tour
then starts by itself. The help menu still starts one.
- **Automated browsers.** No tour starts by itself when `navigator.webdriver` is true, when the
URL has `?tour=off`, or when localStorage `soc-ai.tours.off` is `1`. The help menu still starts
a tour, so an automated check can open one on purpose.
- **The tour state.** Migration 0065 adds the column `users.tours_state`. It holds the opt-out and
the version of each tour that the user saw, so the first sign-in holds across browsers. New
routes: `GET /api/v1/me/tours` and `PUT /api/v1/me/tours`. A bearer token reads and writes the
state of the user who created it. With no user behind the request, both routes answer 401
`no_session`, and the console keeps the state in the browser. A write with a bad tour id, a bad
version or more than 64 tours answers 400 `invalid_tours_state`. The body is capped at 8 KiB.
- All tour text is in `frontend/src/tours/content.ts`, for review in one place.
### Demo
- **Every page shows content.** The hosted demo and `docker-compose.demo.yml` hold one story on
an estate of 24 machines. An attack chain runs from a downloaded executable to lateral
movement and an upload to cloud storage, with benign noise around it. The Hosts page, the
host and entity pages, leads in every state, the analytic hits, a held analytic, a closed grid
gap with its review hits, the Status panels, the runbooks and a cancelled backtest now show
data.
- **One anchor for every timestamp.** The demo fixture file moves to version 2. Every timestamp
in it is written against one anchor, and the demo moves the anchor to the start instant. The
last day and the last week hold data on any day the demo starts.
- **The build.** `scripts/demo/build_story.py` writes `soc_ai/demo/fixtures.json` from
`scripts/demo/recorded_runs.json` and `scripts/demo/story.py`. Two leak gates run before the
write. `tests/test_demo_story.py` fails when the committed file is stale.
- **The mock grid.** `scripts/demo/mock_es.py` answers the host page, the same-flow reads, the
citation drawer and the alert pass. A time range now limits the alerts it lists.
- **The banner** reads "Demo. Some runs are recordings from a test grid. A script generated the
rest. Nothing here is live."
- **No background work on the demo.** The demo runs no lead hunt and no reaper loop. The seed
runs after the startup reaper, so the hunt in progress stays in progress.
### Event time
- **Event timeline.** The investigation page and the hunt page have a new section, Event timeline.
The section shows when the run is finished and two or more of its documents have an event time.
It lists the documents of the run in the order of their event time, in UTC. A new wave starts
after a gap of more than 30 minutes. The section states the gap between two waves. Each row gives
the time, the dataset, the two ends, the rule or the request, and a link to the document. soc-ai
reads the times from the documents that the run already holds. The page makes no new grid query.
The section counts the cited documents that have no stored time. A file hash, a CVE id or a rule
UUID is not a document, so the section does not count it.
- **The list of a finding investigation.** An investigation of one hunt finding lists the documents
of that finding and the documents that its verdict cites. Before this fix, the list held the
documents of every finding of the hunt. A prod run about web exploitation showed an SSH session of
another finding as a wave of its own. An older investigation that judged the whole hunt keeps its
full list.
- **Stories in time order.** The verdict summary, the hunt narrative and the Oracle summary keep
the bottom line first. Then they tell the events in time order, with UTC times. They name each
wave. They give each count its unit, and a window when the record states one. They never call an
earlier event a consequence of a later event. The investigation of a hunt finding reads the
documents of that finding in time order, with the waves named. When the finding cites no
document with an event time, the list gives the other documents of the hunt in time order with
no waves.
- **A time does not resolve a citation.** The writers give the time of each record that they
cite. The citation check ignores a date in a citation, because every alert holds its own date.
An id with a time after it is still checked against the documents that the run read.
- **The Oracle wire check.** On the tool path, the check of each Oracle request looks for a
learned host or user name outside the fixed system prompt and the tool definitions. The new time
rule put common words such as "exploit" and "hunt" in the prompt. A host with such a name
stopped each adjudication on the tool path.
- **Rule prevalence waves.** The rule prevalence tool reads the fire times of a burst that is
longer than 30 minutes. It names each wave with its start, its end and its fire count. A rule
that fired in three waves on one day no longer reads as one burst. The step on the page names the
waves too.
### Hunt findings
- **Each investigation judges its own finding.** An investigation of a hunt finding decides on the
claim of that finding. The rest of the hunt is context. On 2026-10-24 the investigation of an SSH
finding gave its verdict on the exploit. It named the SSH login only as context. A lead promotion
still judges the hunt as a whole.
- **A true positive needs an attack.** A login, a probe or a transfer that happened is not a true
positive by itself. When the evidence does not show if the activity is an attack or
administration, the verdict is needs more info.
- **The anchor is a document of the finding.** The run anchors on a document that the finding
cites. The documents of the other findings and of the lead anchor it only when the grid holds no
document that the finding cites.
- **Reuse needs the same subject.** An investigation of an alert no longer answers for a finding.
Two findings of one hunt never share an investigation. A finding that cites no document has no
subject fingerprint. A tool call is not a document. Such a finding gets its own investigation. On
2026-10-24 one hunt linked four different findings to one alert investigation.
- **A link between leads goes in the narrative.** The hunt prompt and the objective of a lead hunt
ask for it there. A threat finding whose title names leads becomes an observation. No
investigation opens for it. A lead whose hunt found only such a link does not close as clean. The
lead page states the reason "The hunt linked this lead to other leads."
- **The time pattern goes in the narrative.** The hunt prompt asks for the waves, their times and
the gaps in the narrative. It rules out a finding whose only claim is a wave, a time span or a
count. A scheduled hunt wrote "Exploit campaign held one wave on one host pair" as a threat
finding, and the finding got an investigation of its own. A finding about an attack in one wave
is still a finding.
- **The investigation page names the finding.** A new finding investigation reads "Subject: F2"
with the title of the finding. The hunt panel reads "Context: hunt". An older finding
investigation keeps its old labels, because it judged the whole hunt. The tip of the hunt chip on
the Investigations list now says that the subject is one finding of the hunt or the whole hunt.
The page still names the finding after an admin deletes the hunt.
- **The Oracle reads the claim.** The Oracle case of a finding investigation names the title and
the hosts of the finding. The Oracle judges that claim. The redaction learns each host of the
claim, so a host name reaches the Oracle as a label in the claim and in the other fields.
### Re-run of a finding investigation
- **Re-run.** The investigation page of a hunt finding shows Re-run investigation when the run is
not running. Re-run starts a new investigation of the same finding. Before this change, a
finished needs_more_info run of a finding could not run again. Request more info refused it,
and the hunt page linked only to the finished run.
- **The route.** `POST /api/v1/hunts/{hunt_id}/findings/{ordinal}/investigate` takes the optional
JSON body `{"rerun": true}`. The route then starts a new run over a run that ended. A running
run of the finding still answers with `existing: true`. Two presses at the same time start one
run. Without the body, the route keeps its idempotent answer.
- **The new run.** The new run judges the claim of the finding. It keeps the subject and the
synthetic-evaluation marker of the hunt. The hunt page links the finding to the new run. The
run writes nothing to Security Onion.
- **The old run.** The old run stays as a record. Its page names the new run. Its Re-run, Override
verdict and actions are off. The needs-you count and the bell leave it out. The hunt counts the
finding as investigated again when the new run ends.
- **Analyst verdict.** Re-run asks first when an analyst verdict is on the run. The verdict of the
new run then replaces the analyst verdict on the hunt page. The investigation of a finding has
no Deep re-run and no Request more info. On a run that the page calls stuck, Re-run states that
the server still runs the investigation.
- **Hints.** The 409 `hunt_kind_no_rerun` hint names Re-run on the investigation of the finding.
For a lead run, the hint names Resolve in chat. The bulk Re-investigate skips a lead run with
the new reason `lead_kind`, and the list names Resolve in chat.
### Host history gate
- **Hunt findings.** The host history gate reads every document that a finding cites. A malware
alert or an attack alert among them keeps the true positive. The documents of other findings and
of the lead do not count. Before this change, the gate read only one document of the hunt. That
document was often a Zeek record. When the grid holds no document that the finding cites, the
gate still reads only the anchor document. That anchor is a document of another finding or of the
lead.
- **Web attack evidence.** A true positive also stays when the summary or the citations name web
attack evidence. Examples are a web shell, a path traversal, `/etc/passwd`, a file inclusion, a
command injection, an SQL injection and remote code execution. Evidence of a type does not count
when a rule name in the alert history of the host names the same type. The word "exploit" and a
CVE number do not keep a true positive. The alert history of a host contains them too.
- **The summary.** When the gate changes a true positive to needs more info, the summary keeps the
text of the model. The summary starts with three sentences that state the change. The
investigation page shows the reason under Post-validator override. Before this change, the gate
replaced the summary with a fixed text.
- **The Oracle.** The gate does not run on an Oracle verdict. An Oracle true positive that changes
the local verdict must still cite evidence beyond the alert. The Oracle reads the change of the
gate at the start of the local summary. On 2026-10-29 the gate changed two Oracle true positives
on hunt findings back to needs more info.
### Host names
- **A connection document names one end.** An endpoint agent writes one document for each
connection and for each DNS lookup. Such a document has two ends. `t_host_summary` now reads the
`host.name` value of the document as the name of the agent's own end only. The
`network.direction` value tells that end: `egress` is the source and `ingress` is the
destination. On a DNS lookup document, the agent's end is the end that is not on port 53. When
the document does not tell the end, no end gets the name. Before this fix, a host got the name of
an agent that had a connection or a DNS lookup with it. This happened to hosts with an agent and
to hosts without one. The tool also reads an address in the form `::ffff:192.0.2.1` as the IPv4
address `192.0.2.1`.
- **The tool checks the name against the machines in the store.** `t_host_summary` now checks each
`host.name` value against the machines in the store. The tool does not use a name when the store
puts the address on one machine and the name on a different machine. The field
`evidence.hostname_withheld` then gives the address of the other machine. An address that the
store does not know keeps the name. A container address that two agents report is a machine of
its own in the store. Such an address now gets no agent name.
- **The host logs come from the right machine.** The `host_logs` block reads the documents of the
host by its address and by its resolved name. A wrong name gave the block the agent, the planes
and the counts of a different machine. On a live grid, the target of a test attack showed the
process events of the operator's own host. A hunt then read those events as the target's events.
### Earlier conclusions
- **Lead hunts.** A lead hunt reads the leads that closed as promoted in the last 7 days when the
new lead names their subject. It reads at most 3 leads. Each lead line names up to 3 entities.
The entities that the new lead also names come first. For each lead the hunt reads the title, the
verdict and the first summary sentence of up to 3 investigations. On a lab grid, four leads
formed on one host in 21 hours. Each hunt started with no knowledge of the conclusion before it.
- **Scheduled hunts.** A scheduled hunt reads the findings of the last complete run of its
objective. It also reads up to 2 older runs from the last 7 days. Each finding line gives the
type, the title, the hosts, up to 3 document ids and the verdict of its investigation. soc-ai
skips partial runs, runs that read nothing and runs of the synthetic evaluation. On a lab grid,
one run named the test workspace behind an exploit. A run two days later could not name the
driver.
- **Claims to check.** The hunt reads each earlier conclusion as a claim to check. The block names
no lead, hunt or investigation id, so the hunt cannot cite one. The citation check removes any
citation that the hunt did not read. The Objective step on the hunt page shows the block and its
source records. The objective does not
change, so a scheduled hunt keeps its comparison with the last run.
- **Redaction.** With redaction in fail-closed mode, soc-ai drops a block that holds a name the
redaction cannot remove. The hunt then starts without the block, and the Objective step states
this.
- **Investigations of the findings.** The investigation of a hunt finding reads the hunt
narrative, and the narrative can repeat an earlier claim. The prompt states that such a claim is
not evidence.
### Lead hunts
- **The tool name in the evidence block.** The evidence block of a lead hunt now names the tool
`t_get_event_raw`. Before this fix, it named `get_event_raw`. The hunt agent has no tool with
that name.
### Held leads
- **The lead page states the reason.** When a threat finding of a lead hunt has no verdict, the
lead waits for the analyst. The lead page now states "A threat finding of the hunt has no
verdict." The lead history records the hold once for each hunt. On a lab grid, one investigation
of a hunt finding stopped at its 900 s limit. Its lead then waited with no reason on its page.
Re-run the investigation of that finding on the hunt page, or decide on the lead.
- **Subject findings only.** The settle rule reads only the investigations of findings that are
subjects by the current rule. The hunt page already counted only these. Before this change, a
failed run on an older observation entry held the lead.
- **A reused investigation settles the lead.** An investigation of another hunt with the same
subject can answer for a finding. The hunt page now names that investigation on the finding, and
the lead settles on its verdict. Before this fix, the finding kept its failed run, so the lead
stayed held on each re-run.
### Auto-Investigate retries
- **Retry limit.** Auto-Investigate retries a failed alert two times after a short wait. After the
last short retry, the alert runs again once each 6 hours. A run fails when it ends in an error or
a pipeline fallback. The first retry waits 15 minutes, and the second waits 30 minutes. A verdict
on the alert resets its count. The setting `auto_triage_retry_limit` sets the number of short
retries, from 0 to 10. The skip reasons count the alerts that wait and the alerts at the limit.
The Status page counts the failed alerts that wait to retry. Bulk Investigate of selected alerts
and Re-run ignore the limit.
- **Alarm.** Five failed runs with the same error in two hours open one alarm in the bell. The
alarm names the error. Each hour, one of the failed alerts runs again as a test. A verdict on one
of the failed alerts closes the alarm. The other alerts with that error then get their retries
back.
- **Pause.** Five failed runs in a row with no verdict pause continuous Auto-Investigate for 15
minutes. Each failed run after that doubles the pause, up to one hour. A run that reaches a
verdict ends the pause. Bulk Investigate still runs during a pause. The Status page names the
pause and the error. After the pause ends, the Status page measures a missed run from the end of
the pause.
- **Fallbacks in the bell.** The bell shows a pipeline fallback as a failed triage. A dismissed
fallback leaves the bell. The bell then shows the older verdicts in its place.
- **Unexpected cancels.** A run that stops with an unexpected cancel fails only its own alert. The
batch goes on to the next alert.
- **The reason.** On 2026-10-27 the model gateway refused the triage schema for 11 hours. The
scheduler ran two alerts 109 times each. The bell showed each failed run as a needs_more_info
verdict, and no alarm named the fault.
### Dependencies
- `opentelemetry-api` 1.45.1. pip-audit found CVE-2026-114603 in 1.39.1. The lock drops
`importlib-metadata` and `zipp`, which the new version no longer needs.
## [1.6.0] - 2026-10-21
### Dependencies
- `aiohttp` 3.14.5 and `certifi` 2026.10.19. pip-audit found four CVEs in aiohttp 3.14.3 and
one advisory in certifi 2026.4.22.
- `virtualenv` 21.14.12, `packaging` 26.3 and `distlib` 0.4.4. pip-audit found five CVEs in
virtualenv 21.14.1. The newer virtualenv needs packaging 26.
- GitHub Actions: `docker/build-push-action` 7.4.0, `docker/login-action` 4.6.0,
`docker/setup-qemu-action` 4.4.0, `docker/metadata-action` 6.2.0,
`docker/setup-buildx-action` 4.4.1 and `astral-sh/setup-uv` 10.3.0. The docker actions move
to the Node 24 runtime. The workflows use no input that these versions removed.
### Operator workflow (2026-10-20)
- **Act on a waiting action from the list.** The Investigations list of runs with a waiting
action shows each action on its row, with the write named on its button and Dismiss. Both use
the routes of the action card. A write on a run that a newer run replaced answers 409. Select
rows and press Dismiss actions to dismiss their waiting actions with one confirm. The result
line names each row and offers Undo for the press.
- **One action rule.** The action card, the list row, the needs-you count and the execute route
read one rule. A write is off on a replay and on a superseded run. A write is off on a
promoted finding or lead, and on a document that a promotion uses as its anchor when the
document is not a Security Onion alert. A write is off on a run with no alert, on a comment
with no case, and in the demo. A write is also off when it disagrees with the analyst verdict
on the alert. Each case states its own reason. The execute route refuses a write against the
analyst verdict with 409 `analyst_verdict`. New fields on `GET /api/v1/investigations/{id}`:
`blockedReason` on each action, `writeBlockedReason`, `alertCases` and `alertCasesComplete`.
- **The anchor of a real alert.** The anchor rule refused every Security Onion write on a
document that a promotion uses as its anchor. On the range, lead 17 cites a real Suricata
alert, and the execute route refused "Escalate to case" on the investigation of that alert.
soc-ai now asks the grid whether the alert feed holds the anchor document, with the alert
filter of the Alerts page and no time window. A real alert takes the write, the ack and a new
run with writes on. Telemetry keeps the refusal, and its reason now reads "A promoted lead
uses this document as its anchor. The document is not a Security Onion alert." One search
answers a page of rows, and the answers stay in memory for 6 hours. When the grid does not
answer, the write is off with the new code `anchor_unread`. The execute route,
`POST /api/v1/alerts/ack-events`, `POST /api/v1/hunt` and request-more-info then answer 503.
Bulk re-hunt skips the run with the same reason. `POST /api/v1/hunt` over a telemetry anchor
of a lead now runs with Security Onion writes off, like `POST /investigate`.
- **The needs-you count holds actions that can run.** The count, the badges, the bell and the
pending filter leave out an action that no person can execute. The run page keeps it with its
reason. On the range, 8 of the 9 counted actions were on promoted findings.
- **The cases of an escalate.** The escalate card and the row confirm name and link each case
that the alert is already on. A press then opens no second case and records the action as
done. The scope of an escalate adds `cases` and `casesComplete`.
- **Same flow.** An Alerts group names the other groups on its network flow: an alert of each
group with the same community id within 1 hour of the other, or an alert of each group between
the same two hosts within 5 minutes of the other. A community id repeats across days for
fixed-port traffic such as NTP, so the community id link has a time bound too. The chip warns
when the verdicts are opposite. A report shows the same block, and Show in Alerts opens the
row of each sibling. The Alerts screen asks for it with
`GET /api/v1/alerts?links=true`. The read costs at most two searches and stays one minute for
each window. When a size limit cuts the read, the count reads "at least" and the list says
that it can be partial. New fields: `siblingsPartial` on each row, and `partial`,
`windowFrom` and `windowTo` on the new route `GET /api/v1/alerts/siblings`.
- **The case of an escalated alert.** The escalate confirm and the escalated marker on a row
link the Security Onion case that the alert is on.
- **Bulk decisions on leads.** The Hunts page selects leads. One press dismisses, reopens or
hunts the selection. The confirm states the count and the skips. A bulk hunt starts at most 3
hunts. Undo stays for 2 minutes. New routes: `POST /api/v1/hunts/leads/bulk-dismiss`,
`bulk-reopen`, `bulk-dismiss-again` and `bulk-hunt`, and `GET /api/v1/leads/bulk-hunt-plan`.
- **Cancel a backtest.** `POST /api/v1/backtest/{id}/cancel` stops the run after the alert in
flight. The run keeps the results it finished and reads "Cancelled after N of M alerts."
- **Check one stranded escalation.** "Check Security Onion now" on the Status page re-reads one
escalation claim and settles it when Security Onion confirms the case. It writes nothing to
Security Onion. The result links the case it found. New route:
`POST /api/v1/escalations/check`, with the field `case_url`.
- **Generated password banner.** Every page shows a banner while the account still uses the
generated password. The banner opens Change password.
### Dogfood fixes (2026-10-19)
A dogfood on the attack range and on a fresh install with authentication on found 141
defects. The range sensor was down from 2026-10-15 to 2026-10-19, and many defects came from
that real outage. Migration 0064 adds the `grid_gaps` table.
**Grid gaps**
- soc-ai records a grid gap when the grid stays unreadable for 3 minutes. It closes the gap
when the grid answers again. The ledger survives a restart.
- The sweep also records a gap when most hosts of a plane go dark in the same hours. One quiet
host among busy hosts still gets its departure.
- The sweep does not score the hours of a gap or the settle margin after it. The new setting
`grid_gap_settle_minutes` sets the margin. The default is 60.
- A profile stores a gap hour as missing. The baseline no longer learns the outage as zero.
- A hit inside a gap does not count toward the daily hit limit of its analytic.
- A closed gap stays in the bell for 24 hours. The Dashboard, Alerts, Hunts and the host page
show a strip for each gap in the window. The Alerts empty state no longer says "The grid is
quiet" over a gap.
- The general chat and a hunt get one fact line for each gap in their window.
- New route: `GET /api/v1/grid/gaps`.
- An operator can record a gap that soc-ai did not see: `soc-ai grid-gaps add --start --end`,
or `POST /api/v1/grid/gaps` as an admin. `soc-ai grid-gaps list` prints the ledger.
- "Review N hits in this gap" lists the hits that a gap wrote. soc-ai selects a hit when its
event window lies inside the gap and it is not a spike. The drawer states the reason under each
hit it does not select. The operator can change each selection. One confirm marks the selected
hits benign with the reason `grid_gap`. The marks have no undo.
- `POST /api/v1/grid/gaps/{id}/hits/benign` requires `ids`. A body without `ids` answers 422.
A selected id that left the gap list answers 409.
- The hold card of an analytic says how many of the hits that broke its limit fall inside a
gap. It offers **Return to live** when the gap hits alone broke the limit. A spike inside a
gap does not count toward the offer. The card still
finds the hold after the next sweep marks those hits shadow.
- The lead page leaves benign observations out of its weight.
**Verdicts and writes to Security Onion**
- Auto-acknowledge never acts on an alert that carries an analyst verdict other than false
positive. The guard covers a re-run and the cluster fan-out.
- A re-run reads the analyst verdict. Re-run, Deep re-run and Request more info ask first when
an analyst verdict stands on any run of the alert. The confirm names that verdict.
- The investigation page and the Alerts row name an analyst verdict that stands on another run.
- A new run from a backtest replay asks first. The confirm says that the new run becomes the
live verdict of the alert.
- Bulk Re-investigate skips an alert that carries an analyst verdict.
- An override on a superseded run gets 409. The console turns off Re-run and Override on a
locked record and says why.
- An Alerts row opens the newest complete run of its alert.
- The Acknowledge button and the escalate confirm state what one press writes, with the caps. A
second press names what is already done.
- The rule prevalence window ends at the alert time.
**Hunts, leads and analytics**
- The Analytics tab and the Status page show a failed or blind sweep. They no longer show a
green "live" over failed runs.
- An observation finding is no longer an investigation subject. The outcome counts threat
findings over all findings. A card names an investigation that it shares.
- A lead that soc-ai closed on the verdicts of its hunt says so.
- Every unread shadow hit counts and lists on every surface.
**Hosts and entities**
- The entity page reads every key of a machine. An address and a name of one machine show the
same counts.
- The host page lists the investigations of the machine. `GET /investigations` takes `entity`.
- The Users card reads Windows logons.
- The Connections rate divides by the window hours that hold records. It counts the part of
each edge bucket inside the window, and it leaves out the hours of a grid gap. The 7d rate
divided by 192 hours on a 168-hour window. `GET /dossiers/{ip}/activity` adds `window_end`.
- Every relative age reads one rule. The host header, the facts and the entity page read
"30h ago" where the lists read "1d ago". An age from 60 days reads in months.
- The host header reads "last grid event". The entity page reads "last soc-ai record".
- A stale traffic pattern shows its stored sentence. It read "a traffic pattern".
- The Role filter says that a guessed role also counts in "weak guess" or "old guess".
- The bulk note and its Undo show the role label. They showed the role identifier.
**Sign-in, Settings and Status**
- An admin cannot reset their own password in Settings. The new command
`soc-ai users reset-password <name>` recovers a locked-out admin. `soc-ai users list` prints
the accounts.
- The sign-in page states the real transport. With authentication off, it says so.
- Number fields no longer clip their values. A field now has room for its spinner.
- Data sources shows the grid in its first row, with the state of the health probe and the
event index pattern. The blocklist row names the feeds that the config loads.
- The Notification webhook section links to Settings > Notifications. It said "above".
- API keys turns off Set and states the reason when CONFIG_SECRET_KEY is not set.
`GET /config/api-keys` adds `can_store` and `store_hint` to each row.
- The blocklist remedy in Diagnostics names the feeds and shows its command as code.
- Each setting shows DEFAULT, ENV or DB. "Use the default" removes a database override.
- A model check that meets only transport errors reads "could not measure" and raises no alarm.
- The Status page shows the dependencies and each scheduled job.
- "Danger Zone" is now "Connections". It shows the current values and tests an edited value.
- An API token has a name. Revoke asks first.
- New doctor rows: `admin password` and `backups`. An empty alerts feed is WARN.
**Dashboard, shell and chat**
- The needs-you count leaves out actions on backtest replays.
- The Dashboard, Hunts, Investigations and host page keep their window and filters in the URL.
- A custom window means the same hours on every page.
- A chat turn that times out after its tools returned writes a partial answer.
- Chat answers link addresses, leads and investigations.
- Dialogs stay centred and fit the screen.
- `/` opens the Dashboard. An unknown page address opens the console not-found page.
**Text**
- The console, the backend operator strings, the docs and the in-app guide follow ASD-STE100.
- The timeline no longer shows prompt text or internal labels. It names the trigger of a
deeper investigation. Stored text that the timeline cannot read stays as it is.
- Toasts show at the bottom centre of every page.
### Swarm triage experiment (2026-10-08)
The experiment tests whether a team of agents triages an alert better than the one
investigator, at a cost the owner accepts. The design record is
`docs/dev/specs/2026-10-08-swarm-triage-experiment-design.md`. Production keeps the default.
- **One setting selects the investigator strategy.** `investigator_strategy` takes `single`,
`spend`, `ensemble`, `advocates` or `planes`. The default is `single`, and the loop runs as
before. The environment variable is `INVESTIGATOR_STRATEGY`. The override store accepts the
setting live. The Settings page does not show it, because it is an experiment setting.
- **The loop fills one result object.** The investigation loop hands the pipeline one
`LoopResult`. Each strategy fills the same object. The steps before and after the loop do not
change, and the gates do not change.
- **Two advocates and a judge.** The `advocates` strategy runs a threat advocate and a benign
advocate in parallel. Each advocate is the investigator of the loop, with a mandate and its own
limits. Each advocate writes a brief: a claim, the records that support it, the strongest
counterpoint, a confidence word and the gaps. A judge with no tools writes the report from the
alert context and the two briefs.
- **One worker for each plane.** The `planes` strategy runs one worker for each evidence plane of
the alert. A worker can use the tools of its plane only. A lead with no tools writes the report
from the briefs.
- **The judge cites only the briefs.** The pipeline removes from the report each document id that
no brief carries. The report counts the removed ids. The gates then check each citation that
is left.
- **Hard budgets.** Each agent has its own limits. The agents of one strategy share
`agent_tool_calls_limit`. An agent that reaches its limit writes its brief from the evidence it
has. An agent that fails returns an empty brief with the error, and the judge reads it.
- **Every agent is counted.** Each agent emits the tool, model and usage events of the loop. Two
new events, `swarm_agent_started` and `swarm_agent_finished`, record the role, the calls and the
tokens of each agent. The audit trail accepts them, and the timeline gives them a title. The
report names the strategy when it is not `single`.
- **The vote counts each sample.** The self-consistency vote emitted no usage event for its extra
samples, so a run with three samples read at the cost of one. Each sample now emits a usage
event.
- **A run cut at the budget votes.** When the vote re-runs the loop, a run that reaches its tool
budget votes with the partial verdict that one more call with no tools writes. The first run
votes the same way. Only a run that raises or ends with no verdict is dropped. Before, the vote
dropped each run cut at the budget, so an ensemble at 8 tool calls often ran as a single run.
The `self_consistency_vote` event states `requested`, `voted`, `partial` and `dropped`.
- **A backtest replays a fixed list.** `POST /api/v1/backtest` takes `alert_ids`, a list of 1 to
200 alert ids. The run replays those alerts in that order and reads the analyst label of each
from the grid. Each backtest status lists the ids it replays, so the next strategy can replay
the same alerts.
## [1.5.3] - 2026-10-15
1.5.3 is the detection ladder release. It holds the work from 2026-10-01 to 2026-10-14. The
release note is [docs/releases/1.5.3.md](docs/releases/1.5.3.md). The work started with a fleet
of twelve browser and API dogfood agents on 2026-10-01. The sections below hold what they found
and what changed after.
### Security (2026-10-15)
- **pydantic-ai-slim 1.107.7.** The lock moves `pydantic-ai-slim` and `pydantic-graph` from
1.107.0 to 1.107.7. The new version closes PYSEC-2026-4283 to PYSEC-2026-4291. The dependency
audit of the first 1.5.3 release run found them. The pin stays `>=1.0,<2`.
### Alert verdicts follow the time filter (2026-10-14)
- **Runs inside the page window decide the verdict of an alert group.** Everything on a page
follows the time filter of that page. The verdict counts of a group reached back seven days
past a 24-hour window. The alerts API now counts only the runs made inside the window. It
names the worst of them as the window verdict.
- **A group with no run in the window keeps its older verdict.** The Alerts row marks that
verdict "inherited" and states its age. The Dashboard outcome panel states how many groups
carry an inherited verdict.
- **The Dashboard and the Alerts list read one verdict.** The verdict filter, the verdict sort,
the "untriaged" count and the "Needs decision" chip read the verdict that the Dashboard
breakdown counts. A tile opens a list that holds the same groups. The row pill and the
confidence column show that verdict too.
- **A custom window reaches the server as a custom window.** The console sent the two bounds
without `range=custom`, and the verdict counts read the default 24 hours. The console now sends
the marker. The server also reads the two bounds alone as a custom window. A bound without an
offset reads in `so_timezone`. The grid query reads it in that zone too.
- **A backtest replay does not decide a window verdict.** The standing verdict already skipped a
replay. The verdict counts of a group now skip it too.
- **The alerts API adds four fields.** `GET /api/v1/alerts` adds `windowVerdict`, `windowConf`,
`windowInvId` and `verdictAt` to each group. `verdictCounts`, `worstVerdict`, `worstConf` and `worstInvId` now
read the runs of the window only. `verdict` keeps its meaning: the standing verdict of the rule.
### Host page window (2026-10-14)
The owner set a rule for every page. The time filter of a page filters everything on it. A
number that the filter does not change says its scope in words on the screen. A hover title is
not a label.
- **The host page shows every open lead on the host.** The page read the newest 50 leads of all
hosts and kept the leads on this host. A host lead older than those 50 did not show in the
first lines or in the leads list. The server now filters the leads by host before the row
limit. The filter reads every address and every name of the machine.
- **The lead counts read the whole set.** `GET /leads/count` gives the total for a filter. The
first lines and the leads heading on the host page and on the Hunts page use this total. The
heading said at most 50 before. When the list holds fewer leads than the total, a line under
the list says so.
- **Each count says its scope.** The leads heading says "all open" on the Needs decision and In
progress tabs, and "all time" on the Closed and All tabs. The first lines of the host page say
"all open leads" beside a lead and "last 7 days" beside a verdict count.
- **The sweep counts say their window.** The event count in the host header, the Events columns
of the Addresses table and the connection counts under Why? say "last 14 days". The number is
the "Dossier window (days)" setting. `GET /dossiers/{ip}` gives it as `lookback_days`.
- **`GET /leads` takes `entity` and the status `open_or_hunting`.** A blank `entity` gets the
answer 422 `entity_required`.
### Page windows (2026-10-14)
The owner set a rule for every page. The time filter of a page filters everything on it. A part
that the filter does not filter states its scope in visible words. A tooltip is not a label.
- **The Needs-you tile states the scope of each line.** Leads read "all open" and shadow hits
read "all unread". Alert groups read "last 7 days", and verdicts and actions read "last 30
days". The total says that it is the sum of the lines below. With no work, the tile names
every scope in one sentence.
- **The shadow-hit tile reads "all unread".** Its link reads "Read the last 7 days on Hunts",
because the hits block on Hunts lists 7 days.
- **The recent list on the Dashboard obeys the page window.** It listed the newest runs of any
age. An empty window now reads "No investigations in the last 15 minutes." View all opens the
same window. The running tile keeps its scope words while Auto-Investigate runs.
- **The tuning nudge states its two scopes.** Alert counts cover the last 7 days. Verdict counts
cover all investigations. The summary route sends the window as `volume_window`.
- **The Hunts header counts the whole window.** The header, the section badge and the type chip
counted one page of 100 rows, so a window of 140 hunts read "100 hunts". The new route
`GET /hunts/totals` counts every hunt in the window, per type. A line under the table states
how many hunts of the window the table shows. When the totals do not load and the page is
full, the counts read "100+".
- **The Needs-you strip on Hunts states its scope.** Shadow hits read "all unread" and leads
read "all open". The hit link reads "Read the last 7 days", because the hits block lists 7
days.
- **Bulk Investigate names its window.** The header sweep reads the last 24 hours, whatever the
list window is. The button, the progress line and the summary now say "last 24 hours". The
console sends the range to the server. A selection names no window.
### Notifications (2026-10-14)
- **The bell panel accounts for every unit of the badge.** In production the badge read 309 and
the panel listed 12 rows, all shadow hits. A "Needs you" block now sits above the events. It
states each part of the count with its scope, and links to its list. Examples are "300 shadow
hits are unread, all unread" and "2 verdicts need more info, last 30 days". The words match
the Needs-you tile on the Dashboard. The parts add up to the badge. A
part the server could not read says so.
- **The panel lists events only.** The block holds the leads and the shadow hits, so the panel and
the Notifications page do not show them again as rows. `GET /notifications` takes
`events_only=true` for this.
- **Show more loads more rows.** `GET /notifications` takes a `limit` from 1 to 200. The default
stays 12. The cap of each source grows with the limit, so a larger limit returns more rows.
The panel and the Notifications page load 20 more rows for each click.
- **Dismiss all replaces Clear all.** The confirm step states what happens. Dismiss all hides every
event up to the moment the panel loaded. This includes the events that the panel did not load.
It marks every unread shadow hit up to that moment as read on the server. Leads, alert groups
with no verdict, verdicts that need more info and recommended actions are decisions, and they
stay. An altered audit record stays too, because the server marks that notice undismissible.
- **Dismiss all marks only the hits it named.** The confirm step names a count of shadow hits,
and the call sends that count. A sweep can commit a hit after the count was read. When the
write would mark more hits than the count, the server marks none and answers 409
`shadow_hits_changed`. The panel then asks the analyst to check the new count.
- **A new route marks the shadow hits read.** `POST /hunts/shadow-hits/read-all` takes a `before`
time. It marks every unread shadow hit born at or before that time as read, in one write, and
writes a `shadow_hits_read` audit record. A hit born later stays unread.
- **Each notification row carries its time.** The `at` field holds the moment the row describes,
as ISO-8601 UTC. `GET /notifications` takes `after`, the time of the last Dismiss all, and
leaves out the dismissible rows up to it. Hidden rows then take no place in the limit.
- **The Notifications page header states the loaded count.** It said "the 10 newest". It now says
how many items loaded and whether more exist.
### Dashboard true positives (2026-10-13)
- **The outcome breakdown counts a group under its worst verdict.** A group took the verdict of
the newest run of its rule. In production, the rule "GPL SHELLCODE x86 setuid 0" held one true
positive and five later false positives on other sources. The breakdown read "True positive 0",
and the true-positive tile read 1. The breakdown now uses the worst verdict, which the Alerts
row already shows as a chip.
- **The Alerts list filters and sorts by the same verdict.** The "True positive" tile opens the
Alerts list with that filter. The list now holds the group that the tile counted.
- **A run in flight and a pipeline fallback do not hide a verdict.** A group with a running
investigation counts under the verdict it holds. A group whose newest run fell back still
names an older true positive.
### Owner's pass 2026-10-07
- **One chat on every page.** The chat bubble on the investigation, host and hunt pages and the
Dashboard chat panel are now one Ask soc-ai panel. The panel slides in from the right. It has
the same width and header on every page. The button at the right end of the top bar opens it.
The label names the object of the page, for example "Ask soc-ai about this host". The panel
keeps one thread for each scope, states the scope at the top and has a switch to the general
thread. The key `c` opens the panel, and Esc closes it. The Dashboard slot holds a card with
the three starter questions. A lead page has a chat now, on the new routes `GET`, `POST` and
`DELETE /api/v1/hunts/leads/{id}/chat`. The setting `general_chat_enabled` is now labelled
"General chat".
- **The peer diagram and the hunt map use a graph library.** Both diagrams now draw through one
`Graph` component on React Flow 12.12.0, with a d3-force layout. Each label shows whole, and no
label overlaps another. The host page puts the host in the centre and its peers on a ring. The
size of a peer and the width of its line show the connection events. A peer that an alert names
is red. The hunt map joins each finding to the hosts it names, and a threat finding draws red
lines. A node opens its page, and a hover card states its facts. "Reset view" fits the diagram
to the panel. The investigation graph uses the same component and loads on demand.
The owner asked two questions: "What is the difference between hunt and promote?" and "What is
the user flow?" This section holds what changed on the screen.
- **A lead offers one verb.** The lead button reads "Hunt this lead". The lead page states the
verb under its actions: "Hunt this lead. The hunt investigates what it finds and brings back the
verdicts." The hover states what a hunt returns. It also states that a hunt takes no action in
Security Onion.
- **"Open an investigation" is the manual control on a finding.** The hunt page shows it on a
finding with no investigation. "Promote" is gone from the lead page, the lead row and the lead
bar of the hunt page. The lead bar offers "Dismiss lead". The API routes keep their names.
- **A link to an investigation reads "View investigation".** "Open investigation" was one article
away from "Open an investigation".
- **The lead timeline names the act.** The store keeps the decision word `promoted`. The timeline
reads "dan opened an investigation on this lead", and the chip reads "investigation opened".
The refusal of a closed lead uses the same words.
- **A flow strip names where each object sits.** The alert drawer, the investigation page, the
lead page and the hunt page carry one line under the header. The line names each step of the
flow and its state: done, current, next or not yet. A done step links to its object. The next
step names the control that moves the flow, for example "Hunt this lead", "Acknowledge" or
"Re-run investigation". The line wraps on a narrow window.
- **The chain of each object.** An alert reads Alert, Investigation, Verdict, Action in Security
Onion. A lead reads Hit, Lead, Hunt, Investigations, Verdicts, Actions. A manual hunt starts at
its objective. The Investigations step counts the findings with a finished investigation, for
example "2 of 3 done". A failed run names "Re-run investigation" and never shows a verdict.
- **The strip reads the hunt investigation fields.** It reads `findingsInvestigated` and
`findingsTotal` on the hunt. It reads `investigationId`, `verdict`, `investigationStatus` and
`pendingActions` on each finding. An older server sends the `investigation` block of a finding,
and the strip reads that block.
- **The landing page is the Overview.** The page title reads "Overview". Under the Needs you tile
one sentence states the flow: "Alerts and sweeps come in. soc-ai triages, hunts, and
investigates what the hunts find. You decide in Needs you. Your actions go to Security Onion."
The "How this flows" link follows it. The tile shows no ellipsis while the count loads.
- **The chart draws both paths.** The "How this flows" chart draws the alert path beside the sweep
path. Both paths end in an investigation, a verdict and an action in Security Onion. The chart
uses the words of the controls: "Hunt this lead", "Open an investigation" and "hit". It defines
the two verbs. The drawer title is "How this flows", the name of the link.
`tests/test_hunting_flow_chart.py` keeps the console copy and the docs copy identical.
- **The guide states the flow.** `docs/WEBUI_GUIDE.md` opens with a section, "The flow". It names
the alert path, the sweep path, the one queue and the two verbs. `docs/HUNTING.md` states hunt
and open an investigation in two sentences in its lead section. Both state the rule: a hunt is
not finished until its findings are investigated.
- **The tags have their own column.** The chips of a row sat beside its title. Each title
started at a different place, and the list looked jagged. The chips now sit in one Tags column
at the end of the row. The column has one width and one order on the Investigations list, the
Alerts list, the Hunts table and the Dashboard recent list. The order is type, hunt subject,
hunt kind, depth, synthetic evaluation, backtest replay, handled counts, dismissed, superseded,
related runs and chat count. The title cell holds the title only, and the whole title wraps in
it. Each table fits a window of 1280 pixels. A narrower window scrolls the table sideways.
- **The console shows the whole text.** The owner banned the ellipsis. A title, a rule name, a
host name, an address and a notice no longer stop at "…". The text wraps in its cell. The
console has no `truncate` class, no line clamp and no middle cut of a rule name. A loading line
reads "Loading" or "Reading the leads" beside a spinner. A count in progress reads "counting". A
sentence names the whole investigation ID, and the About panel names the whole commit.
- **The backend sends the whole text.** Titles, reasons, errors, doctor lines, bell lines and
webhook bodies no longer stop at a character count with "…". A progress line such as
"Searching" has no dots. A text that goes to the model keeps its bound, and the cut ends in
"[truncated]". A database column or an external API keeps its bound too, with no ellipsis.
- **Two tests hold the ban.** `frontend/src/noEllipsis.test.ts` reads every console string. It
fails on "…", on "..." and on a class that draws an ellipsis. `tests/test_no_ellipsis.py` reads
every backend string. It fails on a string that ends in "..." or holds "…". Five model prompts
keep a placeholder, and the test names each one with the reason.
### Hunts investigate their findings (2026-10-07)
- **A hunt investigates its findings.** After the report lands, the hunt opens one investigation
for each finding that names an entity. A visibility gap, a failed query and a clean observation
are not investigated. The actor is `system:hunt`. The investigation runs on the document the
finding cites, else on the evidence of the lead. A finding with no document reads failed with
that reason. The first range re-hunt reported a threat finding with two hosts and no citation,
and a citation rule left it uninvestigated.
- **The lead pages read the hunt by its verdicts.** The lead list and the lead page label a hunt
with the counts the hunt page prints, "2 of 2 findings are threats". A lead the hunt closed on
those verdicts reads the hunt tail in its flow strip.
- **The hunt reads Investigating until the last one ends.** Two investigations run at a time, the
highest severity first. The findings section reads "Investigating 2 of 3 findings". A failed
investigation counts as ended. A cancel of the hunt stops its investigations.
- **The cap is a setting.** `hunt_findings_investigate_max` caps one hunt at 20 by default. A
finding past the cap reads "Over the cap, open by hand." `hunt_findings_investigate` turns the
phase off. Both apply live from the Config screen.
- **One investigation per subject.** A running or complete investigation of the same alert or the
same subject from the last 7 days is reused. A re-hunt of a lead opens no second investigation
for an unchanged finding. A failed run, a fallback and a backtest replay are never reused.
- **The outcome reads the verdicts.** A hunt reads "2 of 3 findings are threats" or "No threat
confirmed". The bell reads "Hunt finished: 2 of 3 findings are threats".
- **The lead closes itself.** When every finding of a lead hunt has a verdict, the lead closes as
promoted in the hand of `system:hunt`, with each verdict recorded. A finding with no verdict
leaves the lead to the analyst.
- **Each finding card shows its investigation.** The card shows the verdict, the confidence word,
the actions that wait and a link. The API carries `findingsInvestigated` and `findingsTotal` on
the hunt, and `investigationId`, `verdict`, `confidence`, `investigationStatus` and
`pendingActions` on each finding. Migration 0063 adds the columns.
- **The usage report counts them.** `soc-ai usage` lists the entry "Hunt finding investigation" and
prints `hunt findings investigated: N`.
### Dogfood 2026-10-06, the investigation page
- **The ack card states its scope.** "Acknowledge alert" acknowledged every unacknowledged alert of
the rule from the last 7 days on one click. The card now states the scope before the click, for
example "Acknowledge 9 alerts of this rule from the last 7 days, on 2 hosts." A confirm step
follows, with "Acknowledge this alert only". The receipt states the count. The group ack leaves
an alert with a true positive or a needs more info verdict open. The new route
`GET /investigations/{id}/actions/{index}/scope` returns the plan, and the execute route takes
`{"scope": "group" | "alert"}`.
- **Every write asks first.** Escalate and Dismiss on a recommended action, the settled bar and the
Alerts list Escalate have a confirm step. The escalate confirm says "Open one Security Onion
case for this alert". The Alerts list confirm names the count of cases.
- **A dismissal is stored.** Dismiss goes to the server with the actor and the time. A reload and a
second analyst see it. The card offers "Undo". The new routes are
`POST /investigations/{id}/actions/{index}/dismiss` and `.../restore`.
- **An executed action names who and when.** The card reads "Executed by auto-ack at 03:12" or
"Executed by <user> at <time>". When Security Onion shows the alert as not acknowledged, the
ack card does not say executed. It offers the ack again and names the earlier write.
- **A replaced run says so.** "Request more info" and "Re-run" start a new run. The older page now
shows "A newer run on this alert reached true positive (0.65). Open it." Its actions are off.
The detail carries `supersededRun`.
- **An override replaces the headline and the actions.** The headline reads "Verdict set by
<user>: <reason>". A true positive gets one escalate and no ack. The model paragraph folds
under "soc-ai said before the override". A toast confirms the save. The dialog lists the
verdicts as labels and preselects nothing.
- **Each entity is a link.** The ends of the alert, the Alert details, the Host context, the
Entity scope and the entities the rationale names link to their pages through `EntityLink`.
- **The confidence has a word.** Under the number the card reads High, Medium or Low confidence.
The tooltip names the thresholds, 0.80 and 0.70. The rule severity sits in its own row.
- **The open question comes from the rationale.** A run with no recorded question shows the
sentence of the rationale that names the gap.
- **A slow document read shows a spinner.** After 2 s the drawer reads "Fetching the raw event
from Security Onion".
- **A failed check folds.** A failed tool call reads "A follow-up check could not run" with the
error behind a toggle. The OQL whitelist adds `system.auth.ssh.event` and
`system.auth.ssh.method`. The parser maps the alert names the model sees, such as
`event_action`, to their grid fields.
- **The headline names evidence in words.** Field paths move to the cited evidence chips. A tool
id reads in words. The report prompt asks the model for a plain first rationale.
- **The chat acts.** "Resolve in chat" opens the chat and asks "What would resolve this
verdict?". A starter chip asks on click. A proposal equal to the current verdict offers no
Apply. A true positive proposal offers "Escalate to case" inline.
- **The Oracle row explains itself.** It reads, for example, "Oracle review: not needed.
Confidence 0.70 is above the review band." With the Oracle off it says so. The failed run page
no longer names `investigation_run_timeout_s`. The model reasoning is "Model scratch work",
folded, with a note that it is not evidence.
- **A promoted investigation names its finding first.** The page shows "Promoted finding: F2,
<title>" with its evidence and its hosts. The hunt prompt sits behind "Show objective".
- **The drawer moves to the next alert.** The drawer has "Next", and j and the down arrow open the
next report. The drawer shows Alert details: rule, source, destination and user.
- **A running run names its step.** The page reads "Now <step>" and states how long runs on this
grid take, from the median of the newest finished runs.
- **The page uses plain words.** The timeline groups read "Context gathered", "Intel look-ups", "Follow-up
queries", "Checks", "Verdict" and "Oracle review". The page title is the rule name, and Delete
moves to a More menu. An Alerts row that cannot count its handled alerts shows one "uncounted"
chip in place of two "?" chips.
- **A backtest replay offers no write.** The settled bar does not render on a replay, and the
closed actions of a replay read grey.
### Dogfood 2026-10-06, hosts and entities
- **Each entity is a link.** The new `EntityLink` component links a host, an address, a user or a
domain to the page for it. A machine opens its page. Any other value opens the entity page.
The investigation API sends an `entities` list with the type, the machine key and the link of
each value the run names.
- **The entity page says who the value is.** The page states the type of the value: an internal
address, an external address, a host name, a user account or a domain. It counts the open
leads and the true-positive verdicts that name the value. It gives the first and the last
sighting. A value with no record gets the sentence "soc-ai knows nothing about it yet." The
leads and the observations on the value show for every type.
- **An address with no machine opens the entity page.** The `/entity` route sent each private
address to the host page before it read anything. An address that no machine holds then
opened the "never seen" card. The entity page now asks which machine holds the value. The
"never seen" card links the entity page.
- **The palette finds users, domains, addresses and hunts.** The new route
`GET /search/entities` searches the accounts that leads and observations name, the domains of
the machines, the addresses that no machine holds, and the hunt ids and objectives. Each
palette result has a second line, so two results with the same name read apart. A result set
from the previous query no longer stays on screen after the query changes.
- **The host page opens on its open work.** The first lines state the open leads and their
state, for example "Lead 11 waits on a decision. Its hunt found threats." The alert verdicts of
the last 7 days follow, then the identity sentence. The sentence "Nothing else is known about
it yet" shows only when the host has no open lead, no verdict and no identity fact. The leads
and the observations sit above the peer graph.
- **The Alerts card names its two systems.** The card reads "Alerts raised · 7d". A second line
counts the alert verdicts on the host. The card says that it always counts 7 days. The Peers
panel no longer shows the verdict of the latest investigation.
- **The profile panel reads the profile setting.** With profiles on, a host with no profile reads
"This host has too few events for a profile." The dossier API sends `profiles_enabled`.
- **The agent banner agrees with the header.** The banner "No agent reports from this machine"
shows only when no agent ever reported. A late agent reads "The agent last reported 4 days
ago." The header and the primary row of the address table show one last-seen time.
- **The Hosts search finds an address by its last parts.** ".31" and "2.31" find 192.0.2.31. A
cut through the middle of a part still finds nothing.
- **Each summary card lists what it counts.** The Needs attention card reads "37 need
attention: 34 out of date, 3 never built", and each number links the machines it counts. The
list takes the filters `health=attention`, `health=stale`, `health=broken` and
`conflict=yes`. The summary API adds `out_of_date`. A card at 0 opens an empty filtered list.
The address view of broken builds is gone. A machine with no clean build states the sweep
error on its row.
- **The Hosts list names its populations.** The role bar reads "across all 37 machines, quiet
ones too" and groups the three buckets under "No confirmed role": weak guess, old guess and
no role. The Role menu says which list its counts cover. The panel title reads "Showing 18
active machines of 37. Show all." While a search is active, a line says that the cards count
every machine.
- **A click anywhere on a Hosts row opens the machine.** The name link covers the row. The stale
chip no longer stops the click.
- **A sweep failure reads as a sentence.** "192.0.2.46: TimeoutError:" now reads "The sweep could
not read 192.0.2.46: the grid timed out." on the host page, on the Hosts list and in the sweep
error lists.
- **The host page cards name their windows.** The Services card reads "Services · 30d" or
"Services · last sweep", and says that the 24h and 7d buttons do not change it.
- **The host page explains its words.** A legend under the profile heading defines measured,
learning, blind and stale. The evidence lines name records in words: "zeek.conn records"
reads "network connection records". The observations panel says "a sighting counts half as
much after 48 hours". The leads note defines a shadow lead. A lead card gives its weight at
formation beside the weight at which a lead forms. A withheld role states its confidence on
a scale of 1.00 with a word.
- **The facts counter and the identity sentence count inferred facts.** A guess that the
resolver withholds for age or weak evidence counts, and the sentence says that part of it
rests on old or weak evidence. A machine with no OS fact reads the OS its agent reports.
- **The Operator note form has one box.** The second Note box is gone from that form.
- **Small fixes on the host page.** A lead card says "alert" once. The role chip shows a mark
and a help pointer for its tooltip. Three near-equal connection rates read as one number,
and the tooltip keeps the three.
### Dogfood 2026-10-06, hunt page and analytics
- A hunt has one outcome on every surface. The API field `outcome` holds `threat_found`,
`no_threat`, `no_answer`, `could_not_run` or `running`. The field `outcome_label` holds the
words, and `outcome_headline` holds the reason or the gap. The hunt list, the hunt page, the
lead pill and the bell read the same field.
- A hunt that could not run reads "Could not run" with a grey chip. A hunt that could not answer
reads "No answer" and names the gap. A run that did not test its objective shows no confidence
ring, no severity chip and no Promote button.
- The bell reads "Analytic could not run: <title>. <reason>." with the warn tone. The catalog runs
of one sweep that share an outcome fold into one line. An agent hunt that ended in an error now
reaches the bell.
- A catalog run reads "<analytic>, last 24 h". A lead hunt reads "Lead 6 hunt: domainadmin on
SR-DC01" when the dossier names the host. The API field `title` holds the plain title.
- The hunt list holds the catalog runs. The row field `starter` reads `catalog`, and
`kind=catalog` filters to them.
- A lead hunt page shows "Lead 6 waits on you." with Promote and Dismiss. Delete moves into a menu
on every hunt page.
- The draft confirm box shows the title, the logic in one sentence, the YAML and the false
positives. It states how many documents of the source finding the logic matches. It warns when
the logic only repeats alert rule names. A second draft of one finding keeps the first id.
- The hunt page numbers the findings from F1. The version note of a drafted analytic uses the
same number. A draft offers Discard.
- The approve box states the shadow record and the end of the shadow week. An approval before the
end of the week asks a second confirm. The drawer names the end of the shadow week.
- A running hunt states its step, the usual run time on this grid and what it does now. Each search
names what it looked for. A tool error of a known shape reads as plain sentences.
- The finding button reads Promote. The help line names the detection control only while
detection authoring is on. The flow chart calls a hit a hit.
- The investigation API carries `promotedFinding`: the finding number, the title, the hosts, the
attacker and the victim.
- The alert-only severity note reads "Severity held at medium. The finding cites alerts only. An
investigation can raise it." An ATT&CK technique shows only beside a threat finding. The hosts
panel reads "Hosts named" when the hunt found no threat. A step for the operator moves to the
API field `operatorActions`.
- The Analytics tab explains "paused by soc-ai" and has a view for it. A candidate reads "Draft".
An analytic that the loop has not run yet waits for the next sweep. The filters read Source,
Method and Applies to. The Operate panel uses the same three method words.
- The origin chain summary reads as plain sentences. A tool citation on a finding reads in words.
### Added
- **The Dashboard panels move and resize.** The panels sit on a grid of 12 columns from
react-grid-layout. Click Edit layout, then drag a panel by its handle or resize it from its
bottom right corner. The arrow keys move a panel and Shift with an arrow key resizes it. Reset
layout puts back the shipped layout, which is the arrangement from before the grid. soc-ai keeps
one layout for each user account. With authentication off, the browser keeps the layout.
Migration 0061 adds the `dashboard_layout` table.
- `soc-ai spec-replay` replays the tier 2 detectors hour by hour against the grid. It writes into a
scratch store and reads the live store read-only. For each profile analytic it reports the hits
per 100 host-days and the Wilson 95 percent upper bound. It also gives a verdict against the
budget of one hit per 100 host-days. `--dry-run` prints the plan and sends no search. The profile build takes a
time anchor, so the replay can build a past day.
- **`soc-ai spec-replay` measures the learned detectors too.** The replay runs the `model`
evaluator beside the `profile` evaluator by default. `--evaluator profile` or
`--evaluator model` runs one of them. Each analytic runs at its status on the live install. A
detector in shadow writes shadow observations, and the replay counts a shadow hit as a hit. The
report gives the evaluator and the status of each analytic. It folds the seven detector states
into its state columns the way the analytic ledger does. The dry run names the analytics of
each evaluator and estimates the searches of the detectors.
- **A shipped analytic can ship in shadow.** A spec file can declare `ships_as: shadow`. On the
first catalog load, soc-ai writes a shadow status row and a version row that says the analytic
shipped in shadow. The analyst approves it to live on the Analytics tab. A file without the
field stays live. A later release that changes the field does not move a retired analytic or an
analytic in shadow. Before this change, a shipped analytic went live on its first deploy,
because nothing wrote a status row for it.
- **soc-ai can move a live analytic back to shadow.** The store function `demote_to_shadow`
writes a version row with the system actor, the reason and the evidence. The observations that
the analytic wrote stay as they are. A system actor can never approve an analytic to live and can
never retire one. The store refuses both. The Analytics tab marks a held analytic "held by
soc-ai". The drawer states the reason, and the version row shows a `soc-ai` chip with the
evidence.
- **A live analytic that breaches its own limits moves back to shadow.** A spec can declare
`fire_budget_per_day` and `precision_floor`. After each sweep, soc-ai reads the store and checks
each live analytic that declares a limit. A breach moves the analytic to shadow with the numbers
as evidence, and the bell shows the change. The precision compares two sets of hunted leads over
30 days. The first set reached a promoted finding or an investigation. A hunt closed the second
set clean. It counts only when 5 or more leads are decided. The window starts at the latest approval
to live, so one breach causes one demotion. The new setting `analytic_self_heal_enabled` is on by
default. An analytic with no limit never moves.
- **Learned detectors run in shadow.** An analytic gains a third evaluator, `model`, beside
`match` and `profile`. A `model` analytic names a detector and its parameters, and the profile
sweep runs it every hour. A hit is an observation with source `model`. It carries the
statistic, the baseline value, the documents, an OQL query and a reason sentence. A detector
states one of seven states for each entity it reads, and the sweep notes count them. A hit with
no document is dropped and counted. Every `model` observation is shadow until the analytic
lifecycle work lands. See `docs/HUNTING.md`, "Learned detectors".
- **`model-cross-plane-silence`.** One telemetry plane of a machine falls silent while another
plane of the same machine keeps its expected count. The planes are the six host coverage planes
and the sensor's network flows. Each plane is compared with the same hours of the week in the 4
earlier weeks. A hit cites the last document of the silent plane and a document of the live
plane. The detector is meant to replace `profile-connection-rate-collapsed` on machines that
ship two or more planes.
- **`model-logon-chain`.** A session lands on host B. Within 15 minutes, B makes its first logon
attempt to a third host outside its learned edge set. The edge set is learned over
30 days from Windows 4624 logons of type 3 and 10 and from sshd Accepted lines. A hit cites the
session document and the attempt document.
- `soc-ai validate-batch --local` runs a batch with no oracle grade and no cloud egress. The synth stratum still scores each verdict against the planted truth.
- `scripts/eval/run-arm.sh` runs one arm of the two-arm synthetic eval. It runs `synth-clean`, a local `validate-batch` with five repeats of every scenario, and `synth-clean` again. `SOC_AI_CLI` lets a second checkout run its own code.
- `scripts/eval/compare-arms.py` compares two batch directories. It prints the synth stratum recall and precision, the flip rate and the usage per run of each arm. It also prints the strict passes of each scenario and the change in input tokens.
- **The Oracle rule mode.** The setting `oracle_rule_mode` selects the rule that sends a verdict to
the Oracle: `classic`, `shadow` or `uncertainty`. The default is `shadow`. In shadow mode the
classic verdict-class rule decides. Each run also records an `oracle_shadow` event with the
reason that the uncertainty rule would give. No Oracle call comes from that event. Detection
tuning shows the tally of the last 7 days: what each rule sends, by reason, and the overlap. A
replay of 30 days of production found 38 cases a month that only the uncertainty rule sends. The
Oracle has never seen one of those cases. A shadow week shows the disagreement before the switch.
- **The Oracle route pauses after a usage limit.** When the gateway answers with a usage limit,
soc-ai makes no Oracle call until the reset time that the message or the `Retry-After` header
names. When neither names a time, the pause lasts one hour. Three server errors in a row also
start a pause of one hour. Each escalation in the pause writes an `oracle_skipped` row with the
reset time and keeps its local verdict. The doctor and the preflight gain an "oracle route" row.
The bell gets one row for each pause, and the webhook gets one message. The new setting
`notify_on_oracle_failure` is on by default. From 2026-09-10 to 2026-09-14, 69 calls went to a
route at its weekly limit.
- **The proxy path takes a private ACME CA.** `scripts/tls-proxy.sh enable <domain> acme
<directory-url> <root.pem>` puts Caddy on a certificate from an ACME CA on your network,
for example step-ca or a Caddy `acme_server`. Caddy obtains and renews the certificate. No
key leaves the CA. The script copies the CA root into `./certs/acme-ca-root.pem` and checks
that the served chain ends at it. A new `acme_ca` snippet in the `Caddyfile` carries the
directory URL and the root. Before this change, a fleet with its own CA had to copy a
certificate and a key by hand at each renewal.
- **soc-ai can keep the store in PostgreSQL.** Set `SOC_AI_DATABASE_URL` to a
`postgresql+asyncpg://` URL. SQLite stays the default. SQLite has one writer, and at
corporate scale the scheduler, the sweeps and the console contend for it. The same
migration chain builds both stores, and every timestamp stays naive UTC on both. The compose file
has an optional `postgres` service under the `postgres` profile, and `./setup.sh` asks on a
fresh install. `soc-ai store migrate --to <url>` copies every table of a store into an empty
store in foreign key order, in one transaction. `--dry-run` counts the rows and lists each
value that PostgreSQL refuses before the copy writes a row. On PostgreSQL, runbook search
uses the keyword ranker and chat memory uses PostgreSQL text search. `soc-ai backup` refuses
a PostgreSQL store and names `pg_dump`. Install the driver with the `postgres` extra. The
container image includes it. See `docs/DEPLOYMENT.md`, "PostgreSQL".
### Fixed
- The investigation page waits at most 0.75 s for the live acknowledged state of its alert. It waited 3 s on every open while a slow grid answered an id lookup across the whole index pattern.
- A backtest replay never teaches the rule prior and never shows on a host or entity timeline. The fence of the replay reaches both readers.
- multidict moves to 6.9.1 for CVE-2026-104874. The dependency audit of the public mirror found it.
- **The console shows the whole hunt title.** The hunt page header cut the title at 80
characters with an ellipsis. The header now wraps the title and never cuts it. The Hunts list,
the scheduled hunts and the re-hunt result wrap an objective to two lines. The bell clamps a
notice to three lines. A hover on each clamped title shows the whole text. The bell names a hunt
by the whole head line of its objective. The API cut the objective at 80 characters in the
middle of a word. The investigation page names its source hunt the same way. The API sent that
objective cut at 160 characters. The Investigations list wraps a name that came from a hunt.
- **The store keeps the whole finding title.** The catalog writer cut a finding title to 60
characters before the store. The development range held titles such as "Directory replication
requested by a non-machin…". The hunt agent cut a model title at 90 characters. Both writers now
keep the title whole under a safety cap of 500 characters. A finding from before this release
keeps its cut title.
- The doctor row `index pattern coverage` reads a count that passed the probe budget as grid latency, with the node load as the thing to read. It told the operator to fix connectivity while the reachability rows of the same run passed.
- The doctor `tls` row reads the listening sockets, as the `authentication` row does. It named the configured host, which keeps its loopback default under the systemd unit that binds every address.
- The replay writes its report and its scratch store into a private directory, and the report files read 0600.
- **A blind analytic states why.** The blind reason now shows beside the blind count. The sweep
notes, `soc-ai priors`, the journal and the analytic ledger carry it. The Analytics API carries
it in the coverage of a profile analytic. After the deploy of 2026-10-04, the two connection rate analytics were blind on
every host for 19 hours. The stored baselines held no hourly series, and no surface said so.
Migration 0060 adds `prior_spec_runs.blind_reason`.
- **A change of the profile shape makes a build due at once.** Each profile row records its shape
version. The code holds the current version as `PROFILE_SHAPE`. When no host row holds the
current shape, `shape_due` in `soc_ai/dossier/profile_job.py` gives the reason a build is due
now. The build reads each host whose rows hold an older shape, whatever their age, and its notes
and the journal say so. Migration 0060 adds `entity_profiles.shape_version`.
- **`soc-ai spec-replay` says within budget only on enough host-days.** Within budget needs the
Wilson 95 percent upper bound at or under the budget. A rate over the budget is over budget. The
new verdict `not enough host-days` covers the rest, and the command exits with 4. The notes name
the 381 host-days with no hit that a within-budget verdict needs. The range replay read 0 hits in
7 host-days as within budget, with an upper bound of 35 per 100.
- The replay report names the evaluators it ran and the analytics it left out, in its notes. It runs
the `profile` evaluator only. The two `model` analytics and the `match` analytics have no rate in
the report.
- **`soc-ai priors` counts an unmeasurable entity as blind.** The coverage line and each row have
the four columns of the trail. The CLI printed `unmeasurable=N` as a fifth column, and the journal
said fewer blind entities than the store. The per-state note keeps the count of each detector
state. Each row now states the status of its analytic from the effective catalog. A line
names each profile or model analytic that the sweep did not run.
- The standard class shows `t_query_detections` to the loop. The not-found hint of
`t_get_rule_content` names that tool, and each such case cost one `search_tools` call. 6 of the
8 `search_tools` calls on the range eval of 2026-10-05 loaded it.
- The standard class now includes the Windows plane in two cases. In the first, a rule names WMI, DCOM, remote execution, lateral movement, an admin share, pass the hash or Kerberoast. In the second, the session of the alert itself carried DCE-RPC, SMB or Kerberos. The stage 1 eval lost the WMI remote execution scenario because the loop ran without the lateral-movement section.
- A `model` observation is shadow when its analytic is. The evaluator wrote every hit as shadow whatever the status, so an approval to live changed nothing. Both shipped learned detectors declare `ships_as: shadow`, a fire budget and a precision floor.
- A repeated synth plant now carries its own scope key. The renderer kept the bare scenario id that every scenario file writes. The anchor guard refused repeats 1 and up, and repeat 0 read every copy. `--repeats` measured nothing before this fix.
- **A failed Oracle call says what failed.** The `oracle_adjudication_failed` row carries the HTTP
status, the error class and the gateway's message. soc-ai masks the gateway key and every
credential value in the message. The reason `no_parseable_verdict` now means a 200 answer with no
verdict in the body. Production recorded 23 weekly-limit HTTP 500 answers as unparseable.
- **An Oracle verdict changes the local verdict only with evidence.** The Oracle must cite an id
that the local run retrieved beyond the alert. With the tool loop on, a successful tool call of
its own also counts. Without evidence, the answer is an opinion on the run and the local verdict
stands. The auto-acknowledge skips that run. The single-shot path had no such gate. On
production, 6 answers changed needs_more_info to false_positive with no evidence, and soc-ai
acknowledged 2 of those alerts.
- **The egress guard learns a host name from free text only in a host shape.** A name under 3
characters, a path, a run of dots or a common English word stays where it is, as a label. It does
not join the learned set. A refusal names the rule and the field that a learned name came from.
With the code before this change, 4 of the 19 stored Oracle refusals still refused.
- **A hunt closes a lead only after it read its evidence.** A hunt can report a visibility
gap, a failed tool call, an error, a partial read or a budget fallback. Such a hunt keeps the
lead open under "Needs decision" with the reason. A clean hunt after a hunt that found a threat keeps the
lead open too. On the range a hunt that could not read the alert documents had closed a lead
with two critical findings.
- **Every lead decision stays in the history.** Dismiss, reopen, promote, hold and close by hunt
append to a decision list. Reopen and promote clear the dismissal. Migration 0053 backfills
the list and clears stale dismissals.
- **A promoted lead's verdict stays on its own host.** The alert observation that a promoted
lead's investigation writes back no longer lands on another host's lead.
- **The host page joins the hostname.** Observations, leads and baselines keyed on a strong
hostname now show on the host page of the address. A blind baseline row always states a
reason.
- **A dossier sweep keeps a role through a partial read.** A served-port read can end in a timeout, a
shard failure or a port set under half the previous build. Such a read keeps the previous role,
services and management plane. The host reads "Stale read" and the sweep report counts it.
- **Served ports carry their transport and drop ICMP.** ICMP type numbers no longer read as
TCP ports. A UDP service reads as udp.
- **The audit chain check reads the newest window first.** The console and the CLI verify the
last 7 days by default. They stream one page at a time off the event loop, and they keep the
newest records under the cap. Duplicate sequence numbers from concurrent writers are their own
condition with exit code 3. The doctor and the preflight gain an "audit chain" row.
- **Egress counts state why they are unknown.** Each row carries a reason when the audit index
cannot be aggregated. The doctor lists every egress destination.
- **The Analytics tab says when no sweep has run.** A green live dot no longer sits over an
analytic that never ran. The prior trail records the status each analytic ran under. The
catalog reads a profile analytic's runs from the prior trail, so a stale error from a past
sweep no longer shows.
- **Credential values never reach a stored report.** A secret scrub runs before a rationale,
summary, finding, open question or chat answer is stored and before an export. The key name
stays and the value reads "[redacted]".
- **An NMI verdict always offers Request more info.** The settled bar never says settled on a
needs-more-info verdict. The detail reads open questions from the transcript.
- **Cited event ids are links.** The headline, the summary, the validators step and chat
answers render each cited id as a chip that opens the document. Coverage reads as a percent.
- **Re-run is off while a run is in progress.**
- **The investigation page shows the recorded failure cause.** A permanent cause and a
superseded run drop the re-run advice. The bell lists only primary failed runs.
- **The investigation host context names each end of the alert.** The detail stores the
detection type the Alerts grid derives, so a Sigma detection no longer reads "suricata".
- **An unknown alert id gets a 404 before a row exists.** The investigate route and the CLI
triage command store nothing for an alert the grid does not hold.
- **The redaction preview runs the send path.** The preview tokenises user names the way the
Oracle client does.
- **The MCP cases tool reads Security Onion 3.3 case documents.**
- **Config Apply keeps the applied value in the field.** The form remounts after the refetch.
- **Slow calls carry their own budget.** Security Onion writes wait 90 s, run starts 60 s, the
chain check 150 s and the fitness check 90 s. A write that the console stops waiting for says
the change may still land, and the next poll that shows it clears the notice.
- **The escalate response names its cases.** The toast links each case. Escalate has a visible
control in the bulk bar and the drawer.
- **The Alerts filters live in the URL.** A reload keeps the window, the severity, the verdict
and the hide-acknowledged state. A bad filter stops polling and shows the hint.
- **The empty state names the active filter.**
- **The Dashboard verdict tiles open the Alerts list in the same window.**
- **The bell and the health pill close on Escape and on a click outside.** The bell names its
count. Tab order reaches the rows before Help.
- **With sign-in off the account menu offers no sign-out and no password change.** A status
write without a session is refused.
- **The shell reads the session before it polls.** An unauthenticated visit sends no protected
reads before the redirect.
- **The Notifications screen groups rows by their type and states its cap.** Clear all asks
once.
- **The API refuses an unknown filter value with a 422 and the accepted values.** A junk value
on a boolean, an enumerated or a bounded setting returns 400. Every error body under /api/v1
carries a reason and a hint. A bad filter hint names the column and the token.
- **A plain fitness GET never probes.** Only a forced check runs the gateway probe.
- **The health endpoint reports the live probe state.** A probe over its budget reads "slow".
A flip drops the preflight cache. The agent tools panel reads the live state. One probe
budget serves the pill, Test ES and the doctor.
- **A backtest refuses unknown keys and keeps a refusal apart from the stored run.** With no
escalated alert the agreement reads "not measurable". The finish time carries its zone.
- **The request body has a cap.** A body over 1 MiB gets 413.
- **The starters picker reads the Windows planes that exist.** A starter with missing telemetry
sits under "Not applicable". A failed inventory read is cached for 60 s.
- **Draft an analytic asks first.** The click previews the draft and a confirm step stores it.
A second draft for the same finding returns the existing analytic. The draft works from the
stored finding when the grid is slow.
- **The hunt timeline settles with the hunt.** The findings synthesis step reads done, stopped
or did not finish. A cancelled hunt says why once.
- **The Hosts screen filters withheld roles on the server.** Low confidence and stale are
separate buckets. The summary and the filter read one source. A stale sweep counts as needing
attention.
- **Service-discovery names and reverse names are not hostnames.** Broadcast addresses leave
the census.
### Backtest replays (2026-10-06)
- **A backtest replay writes no observation and forms no lead.** A replay is a measurement on an
alert that an analyst already closed. Before this change, the verdict of a replay wrote an alert
observation like a live run. On the development range, a lead formed 74 seconds after the
replay of the same rule. The store now writes no observation for a replay. At startup, soc-ai
marks each lead that formed from replays only with the decision `formed_from_backtest`. The
mark changes no status.
- **A backtest replay sends no notification and writes nothing to Security Onion.** The replay
runs with auto-acknowledge off. The webhook and the bell skip it. The page of a replay shows
each recommended action as not executable, and the execute route refuses it with
`backtest_replay`.
- **The verdict of a replay is not the verdict of its alert.** The Alerts screen, the Dashboard
true-positive tile and the auto-triage sweep read the live runs only. No other alert inherits
the verdict of a replay. Before this change, an inherited false positive from a replay could
acknowledge a live alert. A replay stands alone in the list and takes no primacy from a live
run of its alert.
- **The Investigations list hides backtest replays by default.** On the development range, 12
of the 14 true positives of one week were replays. The list and its header figures now count
live runs only, and the header names the replays it left out. Turn on **Show backtest
replays** to add them. A grey **Backtest replay** chip marks a replay on the list and on its
page. `GET /api/v1/investigations` takes `include_backtest=true`. Each row carries `ranBy`, and
the page carries `backtestReplays`.
### Dogfood 2026-10-06, hunts and leads
- **The leads come first on the Hunts page.** The page reads Needs you, Leads, Analytic hits,
then Hunts. With no unread hit, Analytic hits is one line: "12 hits in 7 days. None waits on
you. Show hits."
- **Show evidence lists the documents of a hit.** The hits route sends the document ids of each
hit, and each id opens the Document drawer. A hit that stores no id has no button. It reads
"Open lead 7 to read the documents."
- **A hit can be marked benign.** A live hit with no lead offers "Mark benign" beside "Hunt this
entity". The reason comes from the lead dismissal list. A benign observation counts toward no
lead, and a repeat keeps the mark. Migration 0062 adds the dismissal columns to
`entity_observations`. The route is `POST /api/v1/hunts/observations/{id}/benign`.
- **A hit with no lead says why.** The hover states the rule in numbers: "One type at weight
0.35. A lead needs 0.85 over 2 types, or 1.50 on one type." A removed analytic shows its title.
- **The lead page gives each hunt its answer.** Under "Hunts on this lead" each hunt states the
first sentence of its narrative and what it could not check. A hunt reads "Lead hunt · 17 steps
· 6 min · 0 threat findings". The id moves to the hover.
- **The lead pill states the worst hunt outcome on record.** A lead whose earlier hunt found a
threat reads "Hunted · An earlier hunt found a threat" in red. The lead page adds "3 of 4 hunts
found a threat. The newest hunt found none." The pill reads its words and colours from
`lib/labels.ts`.
- **A promoted finding's verdict comes back to its lead.** The lead page lists the findings
promoted from its hunts, with the verdict of each investigation. A strip asks to close the lead
as promoted. `POST /api/v1/hunts/leads/{id}/close-promoted` closes it, and the lead leaves
Needs you. `POST /api/v1/hunts/leads/{id}/keep-open` records the other answer.
- **The lead header says why this is a lead.** One sentence counts the observations by type, the
host and the span. The weights and the z scores sit behind a Scoring toggle. A rate departure
reads as a multiple: "That is 14 times the expected rate."
- **The lead title names the machine.** It reads "Lead 7 · `<name>` · `<address>` · `<role>`" from
the hosts resolve route. Each profile observation names the count it measures.
- **The lead timeline has one order and one clock.** Observations, decisions and promoted
findings read newest first. Each observation states its event window in UTC. A reopen names the
hand and the cause.
- **A stale hunt says so.** "The hunt ran 17 days ago. 2 observations are newer than the hunt.
Hunt again to read them."
- **The query of an observation has two acts.** "Copy query" copies it. "Hunt with this query"
opens a hunt whose objective names the host and the hours. It names no tool.
- **Promote states why it waits, under the button.** An observation whose analytic was retired
reads "analytic retired on 2026-10-06".
- **Dismiss opens with no reason chosen.** Four reasons join the list: "Authorized test or scan",
"Duplicate of another lead or investigation", "The analytic logic is wrong" and "Lab or test
host". After a dismissal the page states the store rule. The observations on the lead and their
repeats form no new lead. A new observation can form a new lead at once. No time window
applies.
- **A lead from a backtest replay carries a grey chip.** The timeline row reads the stored note.
- **New hunt states the range, the cap and the run time.** A time range control reads "The grid
answers up to 30 days back." The line "A hunt takes 5 to 8 minutes." reads the run time of the
last 50 finished hunts from `GET /api/v1/hunts/composer`. The starters name no tool. A builtin
keeps its tool hint in the agent's prompt. A starter that does not apply states its reason.
- **Small fixes.** A note that ends in a stop reads one stop. The lead header counts hosts and
accounts, not entities. The status column of the hunt list is 160 px wide.
### Dogfood 2026-10-06, labels, times and views
- **One map gives each verdict, hunt outcome and run status its label and colour.** The new
module `frontend/src/lib/labels.ts` holds one label, one tone, one colour and one chip class
for each stored word. needs_more_info reads "Needs more info" on every surface. A false
positive is grey, because green means a clean hunt. An error, a cancel and a restart read
"Failed". The tooltip names the cause.
- **The override select lists verdicts in words.** It printed the stored keys, such as
`true_positive`. The Detection tuning column and the Runbooks verdict mix spell out "needs
more info".
- **Confidence prints two decimals, with a word for the band.** `confidenceWord` returns High at
0.80 and above, Medium from 0.70 to 0.79 and Low under 0.70. The bands follow the server. At
0.80 the server keeps a malware verdict local after the investigation loop. Under 0.70 the
Oracle reviews a verdict, and no false positive is acknowledged on its own.
- **One time formatter.** A relative time reads in days from 24 hours on, "4d ago". An absolute
time reads "2026-10-01 08:18 EDT" in the browser zone with the zone named. A duration reads
"3m 19s". Runbooks, Backtest, the Investigations list and the list freshness marker use it.
- **Saved views say why they are off.** With authentication off, the list toolbar shows the Save
view control disabled. The line beside it reads "Saved views need a signed-in session. This
deployment has authentication off."
- **The Investigations list names its window.** The list opens on all time when the newest run
is older than 30 days, and a line says why the window is wider than 24 hours. A search with no
match names its window and offers "Search all time".
- **A host-log alert names its machine.** The flow cell reads "on `<host>` · host log". It read
an address, an arrow and a dash. The list API sends the machine as `hostName`.
- **A row with no verdict says why.** It reads "Running", "Failed. A later run replaced it.",
"Failed. An earlier run holds the verdict." or "Failed. No run reached a verdict."
- **The not-found pages name the absence.** An investigation that does not exist offers "Back
to Investigations". The page title and the top bar read "No such investigation" or "No such
lead" in place of the id.
- **Chips and tooltips use plain words.** The run class chip reads "Depth: quick", "Depth:
standard" or "Depth: deep", on every row or on none. The rule prior chip reads "rule history".
The tooltips say auto-hunt, daily hit limit, minimum precision, draft and paused by soc-ai. An
estate outlier reads "unusual across all hosts". The hunt map count reads "1 finding · 0
hosts". The severity chip names a rule severity or a finding severity.
- **The estate model help names an extra package.** The Config help and the Diagnostics row said
"the ml extra". They now say that the model needs an extra package that a deployment may not
have installed, and that Diagnostics shows whether it is present.
### Dogfood 2026-10-06, the work count and the navigation
- **One "Needs you" count.** `GET /api/v1/needs-you` states five parts and their total: leads
that wait on a decision, unread shadow hits, alert groups with no verdict, verdicts that need
more info and pending recommended actions. The pending-action count is new. A part that soc-ai
cannot read is null, and the total is then a floor.
- **The Needs-you tile leads the Dashboard.** It states the total with one link for each part. It
takes the place of "Awaiting investigation", which said "queue clear" while 11 leads waited.
- **The sidebar badges state the parts.** Alerts counts the alert groups with no verdict.
Investigations counts the verdicts that need more info and the pending actions. Hunts counts the
leads and the unread shadow hits.
- **The bell states the same total.** Its header reads "11 need you · 8 events". The bell lists
every open or hunting lead in the warn tone, with no 24 h window. Past five leads, one row
states how many more are open.
- **The bell names each hunt by its outcome.** A hunt row carries an `outcome` field and a chip
from `lib/labels.ts`. A run that could not run reads "Analytic could not run: <title>.
<reason>." in the warn tone. A hunt that stopped on an error now reaches the bell.
- **A failed run files its error under "Error".** The timeline put the error step under
Validators with a green check. The new group has a warning icon.
- **The Dashboard shows "…" for a window it has not read.** It showed "0" and "No alerts in the
last 30d" while the 30d read ran.
- **A slow grid reads amber.** The banner reads "Elasticsearch is slow. Counts can be late." when
the probe answered late. Each tile that reads the grid says "May be incomplete." Red stays for
a probe that did not answer.
- **The true positives tile counts investigations.** It reads the count of the Investigations
list query over the Dashboard window, and links to that list. It read 0 for 7 days while the
list said 15.
- **Status shows a failed analytic run.** A row whose last run failed reads "last run failed 7h
ago". The counters have legend lines. The panel states the analytic hits of the last 7 days.
- **The escalation list names the alert.** Each row names the rule and the host, and reads
"Escalation sent 5 days ago. Security Onion has not confirmed a case."
- **The navigation names.** The groups read Analyst and Operator. Config reads Settings, and
Operate reads Status. Notifications left the sidebar. The bell link "View all notifications"
opens the page.
- **Breadcrumbs and help.** A lead page reads "Hunts / Lead 18", and Hunts stays lit. The Status
page and an entity page have breadcrumbs. The help button opens the guide. The command palette
offers the keyboard shortcuts and the guide.
- **The top bar.** The workspace switcher shows only when more than one workspace exists. The
wordmark reads "soc-ai". `ShellContext.previousPage` holds the page the analyst left, for a back
link.
- **The Alerts empty state.** An empty list offers "Show the last 7 days". On a plain landing an
empty 24 h list reads 7 d and 30 d once and lands on the narrowest window that holds a group.
The quiet-grid hint no longer says "The configuration is sound."
- **An alert group row shows its worst verdict.** The row counts the verdicts of its alerts, for
example "1 true positive, 8 false positives". The worst verdict opens its investigation. Each
event chip names its verdict, for example "True positive 0.62, 4d ago".
- **An alert group row names who is involved.** It shows the flow of the latest alert, the count
of other sources and the machine or the sensor.
- **Settings, Diagnostics.** A "2 warnings" chip filters the doctor list to its WARN rows.
- **The layout toggle reads "Done" while editing.** A window under 1024 px says "Widen the window
to move or resize panels."
- **An unknown address says so.** It reads "No page at this address." with a link to the
Dashboard. It redirected to the Dashboard with no word.
### Dogfood 2026-10-06, the seams between the branches
- **The hunt list reads one outcome and one title.** A row reads the `title` the server writes. The
status dot and the findings cell read the outcome from `lib/labels.ts`. A run that could not run
is grey and states no finding count. A run that found a threat is red. The list compared the
old codes `failed` and `gap`, so a run that could not run wore the green dot of a clean hunt.
The status column is 220 px wide, so "No threat found, visibility gap" fits on one line.
- **The type chips include Catalog.** The analytic sweep records a run when an analytic cannot read
its data. The bell links a folded sweep line to the chip.
- **The lead surfaces read the outcome labels.** A lead that the settle rule closed reads "Closed.
No threat found." The pill and the hunt record read both outcome vocabularies through
`lib/labels.ts`. The type chip states the lead rule. Two or more types form a lead at 0.85. One
type forms a lead alone when its repeats reach 1.50. The weight hover reads "A sighting
counts half as much after 48 hours." on the lead page and the host page.
- **The investigation page goes back to the page the analyst left.** The back link reads
`previousPage` from the shell, with its search, for example "Back to Alerts" or "Back to Lead
18". With no page before it, the link reads "Back to Investigations".
- **The Dashboard reads the status words of `lib/labels.ts`.** A recent investigation that runs
reads "Running", and a run that stopped reads "Failed". The hover names the cause. Before this
change, the rows read "Investigating" and "Error" from a second map.
- **A demotion reason reads the console words.** It reads "The analytic wrote 5 hits in 24 hours.
Its daily hit limit is 3." and "Its minimum precision is 0.50." The spec fields keep their names,
`fire_budget_per_day` and `precision_floor`.
- **A dismissed recommendation leaves the needs-you count.** The pending-action part reads the
stored dismissal of the action card. An undo counts the action again. The console reads the count
again after a dismissal and after an undo.
### Range console dogfood (2026-10-05)
- **"Hunt with this" shows on a `match` analytic only.** The hunt tool runs a `match` analytic as
a grid query. It cannot run a `profile` or a `model` analytic, because the profile sweep runs
those every hour. The drawer and the Analytics tab rows now say so on a live analytic of those
types. Before this change, the control started a hunt whose first step failed.
- **A failed hunt step shows its reason.** The expanded step of a failed tool call starts with
the failure and the reason the tool gave. The step of a failed analytic run read
"result: live", which is the status of the analytic.
- **The investigation page reads the ends of an alert from the alert.** The endpoints chip
shows the source address and the destination address. A host that reported the alert shows as
"on" that host. It never sits in a direction slot. A grid node login failure with no
destination read "source <host> to destination <source address>".
- **Config, Diagnostics lists every doctor row.** The new Doctor list shows each row with its
status, its name, its detail and its fix line. It reads the newest stored result when the pane
opens. Refresh runs the checks again. The Setup health card on the Dashboard shows FAIL and
WARN only, so the INFO and PASS rows had no console surface.
- **The Analytic hits panel lists profile hits.** A departure that a profile analytic writes is
now a hit, with its receipts when it is a shadow hit. The panel read "No analytic hit in the last
7 days" while three live profile analytics had written ten observations. The Needs-you count,
the bell, the shadow band and the hits list now read one shadow clause. An observation that no
analytic wrote, for example an alert verdict, is not a hit on any of them.
- **A learned detector reads as a detector.** The drawer counts the runs of a `model` analytic as
"detector runs". Its tooltip says that the detector reads the grid and builds its own baseline.
The Analytics tab shows no baseline age for a `model` analytic, because the detector reads no
stored host profile. Operate lists the `model` analytics under their own label, "Learned
detectors".
- **The ship row of an analytic shows a neutral chip.** The version row that `system:catalog`
writes when an analytic ships in shadow now shows "shipped in shadow". It showed the amber
`soc-ai` chip of a demotion. The amber chip stays on a `system:self-heal` demotion.
- **The top bar names the machine.** On a machine page the top bar shows the machine name. The
machine key, for example `agent:<id>`, shows in the tooltip only. The top bar showed the raw key.
- **The bell names the findings it counts.** A finished hunt notice reads "3 threat findings".
It read "3 findings" beside a hunt page that counted 4 findings: 3 threat findings and 1
observation.
- **The Hosts search box shows its whole placeholder.** The box sizes itself to the text
"Search name, address, MAC, OS, role, agent". It cut the text at "OS".
- **The Hosts list says how old its rows are.** With automatic sweeps off, the line under the
cards now dates the list: "the list shows the state as of" the last sweep. A row whose agent
last reported more than 24 hours ago carries a "stale" marker. Its tooltip says that sweeps are
off and when the last sweep ran. The list showed a last seen time of 3 days while the machine
page showed live activity from 52 minutes before.
### Console verification (2026-10-05)
- **The Analytic hits list states its share and reads more.** The line under the list states
the share, for example "50 of 148 hits". "Show more" reads the next 50 hits. The hits route
takes an `offset`. Under All, the unread shadow hits now come first, then the live hits, then
the read shadow hits. Production listed 50 of 148 hits with no note. The one unread shadow hit
that Needs-you counted was not in those 50.
- **The blind reason uses plain words.** The coverage cell on the Analytics tab named the column
and the gate of the code. It read "no connection_rate profile has been built for this entity"
and "cannot apply role priors: role unknown or below the confidence gate (0.50 < 0.90)". It now
reads "no connection rate baseline exists for this host yet." and "the role of this host is not
known well enough. The confidence is 0.50 and the gate is 0.90." Every reason now opens with
its count, for example "35 of 56 blind hosts". A reason that every blind host shares had no
count.
- **The hold box names its controls.** The drawer of an analytic that soc-ai holds in shadow now
reads "Then select Approve or Reject". It said "approve it to live or retire it" above the
controls "Approve" and "Reject". The "held by soc-ai" chip tooltip uses the same words.
- **Operate shows the hold.** The Analytics card on Operate now shows the "held by soc-ai" chip
on an analytic that soc-ai holds in shadow. It showed the "shadow" chip of an analytic that an
analyst put in shadow. `GET /hunt-catalog` now holds the reason of the hold in
`held_by_system`.
- **A hit card says when its analytic is gone.** A hit whose analytic the catalog no longer lists
now shows the chip "analytic removed" and the analytic id as text. The card offers no Approve or
Reject for it. The card showed the raw id as a link and a "live" chip.
- **The Hosts "as of" note states its time zone.** The note now reads, for example, "the list
shows the state as of Oct 02, 2026, 08:10:28 AM EDT". The shared time formatter takes a `zone`
option for a line that dates a list. The note gave the browser time with no zone.
- **The Hosts role chip keeps a guessed role whole.** A withheld role now shows the guess in the
chip and the state words beside it, for example "security appliance" and "low confidence". The
pair wraps when the Role column is narrow. One chip cut "low confidence: security appliance" to
"security applianc".
### Tier 2 statistics (2026-10-04)
- **A rate departure is tested hour by hour.** The prior sweep reads the expected count for each
hour of the week from the hourly series of the host. The dispersion of an hour is the larger of
the pooled MAD and the square root of the expected count. A burst of one to four hours now moves
the test. The median of the recent hours could not move for it. An hour with an expected count
of zero and no spread is unmeasurable. A rate baseline from an older build is blind until the
next build.
- **Zero hours stay in the rate baseline.** The build fills the hours with no document between
the first hour and the last with zero. The three cells and the hourly series read them.
- **Novelty reads how many hosts in the estate know a member.** A new member that fewer than 3
hosts know is estate-rare, and its observation is born at 0.6. A new member that more than 20 %
of the profiled hosts know is estate-common, and it forms no observation. Two new settings hold
the bars: `profile_estate_rare_hosts` and `profile_estate_common_share`. Migration 0057 adds the
table `member_prevalence`.
- **One member new on 3 hosts in one sweep is one condition.** Each host gets a `scope_count`
observation that states the spread and how many hosts knew the member before.
- **A member is known after two days of sightings.** A member the baseline saw on one day is new
again. An investigation can confirm a window as a true positive. A member first seen inside that
window is not known, and those hours stay out of the rate baseline.
- **A set at the cap of 200 members reports no novelty.** The analytic is blind on that host, and
its note says "set full".
- **Peer groups read the role.** The refresh after each build writes the role on every profile row.
A new member that most peers in the same confident role know is a role trait, and it forms no
observation. The new profile test `rare_for_peers` writes a `rare_for_peers` observation for a
member its peers do not know. No shipped analytic uses it yet.
- **An observation carries its evidence.** Migration 0057 adds the statistic, its value, the
baseline value, up to 10 document ids and an OQL query to run again. The lead page shows them
under each observation, with a "Run this query" link to the console. The host page shows the
statistic. The hunt a lead starts reads each statistic and each query first.
- **The prior sweep takes a time anchor.** `run_prior_sweep(now=...)` reads, records and forms
leads at a past hour. A replay over seven days with planted departures runs in the test suite.
### Host identity (2026-10-02)
The Hosts list showed one row per IP address. On production one proxy had 14 rows, one
workstation had 10 rows with no name, and 282 of 337 rows had no hostname.
- **One machine per device.** The sweep groups the addresses it finds into machines by five
rules: agent, bridge and container, DHCP lease, unique short name, single address. A rule
never moves an address that a stronger rule placed. Migration 0054 adds the `host_machine`
table and the machine of each address. The per-address dossier stays.
- **The machine API.** `GET /api/v1/hosts` lists one row per machine. Search reads every name
from any source, every address, every MAC, the OS, the role and the agent name. An address
matches only exactly or by prefix. Each column sorts in both directions,
and the address sort is numeric. `GET /api/v1/hosts/summary` counts from the same rows, so each
card equals the list total of its filter. `GET /api/v1/hosts/resolve` and
`GET /api/v1/hosts/{key}` name and describe one machine. The `/api/v1/dossiers` routes stay.
- **Joins read the machine.** The observations, the profile and the entity page of a host read
every address and every name of its machine. The entity route returns `host_key`.
- **An agent name that is a public top-level domain stays.** Two agents on production carry a
gTLD as their name, and search could not find either machine. The agent's own name and a DHCP
lease hostname are no longer rejected as a bare TLD. A DNS name still is.
- **The sweep reads Security Onion 3.x DHCP leases.** SO 3.x writes `client.address`,
`dhcp.assigned_ip`, `host.hostname` and `host.mac`. The sweep read none of them, and no lease
named a host.
- **An address that only an agent reports counts the agent's documents.** It had 0 events, and
the default view hid it.
- **The agent address list holds 256 values.** One agent reported 111 addresses and the cap of 40
dropped real ones. A report that hits the cap writes a note on the run.
- **An address unseen for 30 days leaves the census.** `dossier_stale_address_days` sets the
window. An address with a declaration stays. A sweep whose census failed removes nothing.
### Scale (2026-10-04)
A synthetic estate of 20,000 hosts ran through the sweeps. Five parts held only 500 or
10,000 hosts, and one step was quadratic.
- **The profile build covers every host.** The build read each dimension with one aggregation
of 500 entities, and the other hosts got no profile. The build now reads the census hosts in
batches of 500. One search per dimension serves a batch. A last read finds the hosts outside
the census. A batch that the grid refuses for its bucket count splits down to one host.
- **The profile build skips a quiet host.** A quiet host keeps its rows. Its profile is younger
than the dossier refresh interval, and its last activity is older than the window of its
build. The run row states how many hosts the build skipped, read in batches and read alone.
- **`profile_build_workers` caps the batches in flight.** The default is 2. The setting applies
without a restart.
- **The census reads past 10,000 addresses.** One census search holds 10,000 addresses per
direction. When `dossier_max_hosts` allows more, the census reads the rest in pages. Before
this change, half of a 20,000-host estate had no dossier.
- **The sweep logs each stage.** One line per stage gives the wall time and the searches. The
refresh summary carries the same numbers. The profile build runs last, so it reads the census
of the same sweep.
- **The machine clustering is linear.** The merge with the previous sweep took 102 seconds at
20,000 hosts. It takes about 1 second now.
- **The census record and the role stamp write in batches.** The census record sent one SELECT
and one flush for each address. It now reads 500 addresses in one SELECT and flushes once for
each chunk. The role stamp sent one UPDATE for each changed row. It now sends one UPDATE for
each role and each group of 500 rows. At 20,000 hosts the census record fell from 26.5 s to
6.3 s, and the role stamp fell from 1.7 s to 0.7 s.
- **The prior sweep reads the whole estate.** The recent read of each dimension held the 500
busiest entities. It now reads pages of 1,000 entities, up to a ceiling of 20,000. A read at
the ceiling writes a note with the number, and the sweep scores no silent host for it. On the
synthetic estate of 20,000 hosts, the silent server and the novel served port are now
observations.
- **The agent inventory reads past 500 agents.** It reads pages of 1,000 agents, up to a
ceiling of 20,000. Before this change, a network of 600 agents lost 100 of them with no note.
At the ceiling the dossier run writes a note with the number.
- **The profile build reads every host outside the census.** The last read of the build held
the 500 busiest hosts per dimension. It now reads pages of 1,000 hosts, up to a ceiling of
20,000. A read at the ceiling writes a note on the run with the number. With an empty census
of 2,000 hosts, the build now writes 21,081 rows. Before this change it wrote 8,059.
- **A scale harness runs in CI.** `scripts/scale/run.py` builds a synthetic estate and runs the
sweep, the clustering, the profile build and the prior sweep against an in-process grid. A
CI job runs 2,000 hosts and fails when a stage passes its budget of time, searches or memory.
### Tier 3 estate model (2026-10-04)
Most hosts in a corporate estate have no declared role. The estate model learns the peer groups
and scores each host against the estate. It runs in shadow only, and it is off by default.
- **The `ml` extra.** scikit-learn 1.9.1 and numpy 2.5.3 come in the optional extra `ml`, with
five dependencies. The container image installs it, about 220 MB of site-packages. The
dependency audit in CI and in the publish script reads the extras the image ships. The app
imports the extra only when the estate model runs.
- **One behaviour vector per host.** The daily run reduces the stored profiles of each host to
numbers. Each set gives its members and its documents per day. The other numbers are the active
hours, the night share and the connection rate of each cell. One flag marks each plane, and one
flag marks each declared role.
- **Learned peer groups.** k-means groups the vectors of each key space. The silhouette score
picks k from 2 to 10. A group keeps its id from day to day. Migration 0059 adds
`estate_model_fits` and `estate_peer_groups`. A host with no confident role reads its learned
group as its peer group in the profile tests. This applies while the model is on and its newest
fit is measured.
- **An estate outlier score with a stated reason.** An isolation forest scores each host. A host
at 0.55 or more writes a shadow `estate_outlier` observation at 0.3 when three conditions hold.
A feature departs from its group by 3 or more. Fewer than 5 other hosts act the same way. The
grid returns a current document of the host. The summary names the top three features with the
value of the host and the median of its peers.
- **The model file is trusted by its hash.** The fit writes a JSON file under
`models/estate/` in the data directory. The store records its sha256, and the audit chain gets
the hash as `estate_model_fit`. The run refuses a file whose hash the store does not record.
- **States.** Each fit is measured, learning, drifted or held. Drifted means two or more features
have a population stability index above 0.25 against the previous fit. Held means more hosts
qualify than the fire budget. Only a measured fit writes observations. A new fit is a
challenger for 24 hours as a record only. The run scores once.
- **The doctor and the console.** The doctor row "estate model" states the state and the date of
the last fit. The setting `estate_model_enabled` sits under Behavioural profiles. A daily loop
fits under the single-flight slot of the dossier sweep.
- **Measured at scale.** At 20,000 hosts on the scale harness the fit takes 8.8 s and the run
15.3 s. See `docs/HUNTING.md`, "The estate model".
### Tier 1 ladder (stage 1, 2026-10-04)
Production spent 163.4 million tokens on 2,051 scheduled triage runs in 30 days, with one true
positive. These changes put the cheaper rungs first and measure each run.
`docs/ARCHITECTURE.md` describes the ladder under "Budget classes and the ladder".
- **Every run records what it cost.** A triage run, a hunt and a lead hunt store the model
requests, the input and output tokens and the tool calls. They also store the Elasticsearch
searches, the wall time and the budget class. Migration 0058 adds the columns. A hunt now records its model
usage. Before this change the store held none.
- **`soc-ai usage --days N` prints the cost per entry point.** The table gives the runs, the
median and the 90th percentile of each counter, and the outcome shares. An older run reads its
tokens and its tool calls from its stored events. A number the store does not hold prints as a
dash.
- **A run has a budget class.** Cheap is one synthesis request with no loop and no Oracle.
Standard is the loop. Deep is the loop with the whole prompt and every tool. The scheduler gets
cheap only where a dispositive template cleared the alert. A round-1 verdict that is not a
false positive, or that a verdict gate would change, escalates the run to standard. An
analyst's Investigate, a re-run and a bulk selection are standard and always run the loop. The
Deep re-run is deep. The Investigations list, the drawer and the page show the class as a
chip.
- **The rule prior rung, in shadow.** The rule prior covers a scheduled alert with the verdict
of its rule's latest model run when six safeguards hold. `rule_prior_mode` is `off`, `shadow`
or `live`, and the default is `shadow`. In shadow the model runs as before, and the new
`rule_prior_decisions` table records what the prior would decide beside the real verdict. In
live a covered alert gets a rule-prior run with no model call, except a 2% random sample. A
real run that disagrees suspends the prior for its rule. The Detection tuning panel shows a
Rule prior column with a Clear button. A rule-prior run never acknowledges in Security Onion
and never lends its verdict to a sibling alert.
- **The pipeline runs the web search the loop used to spend its turn on.** When an external
indicator has no enrichment answer, the pipeline searches it before the loop and gives the
loop the result. The loop spent its one tool turn on this search in 87% of production runs.
### Range dogfood (2026-10-05)
- **The doctor no longer calls an unmeasured model unfit.** A fitness leg that times out or
cannot reach the model now reads WARN "could not measure", with the cause. The row gives no
advice to replace the model. Only a measured capability failure reads FAIL. Each leg now carries a
`cause` field, `timeout` or `transport`. The console chip and the audit record keep the old
grade.
- **The auth-off start line states only what the app knows.** With `API_AUTH_REQUIRED=false`, the
line said "loopback bind 127.0.0.1" on the systemd path, where uvicorn bound 0.0.0.0. The line
read `SOC_AI_HOST`, and uvicorn holds the real bind. The line now names the bind only under
`soc-ai serve`. Otherwise it says "The bind is the server's".
- **The doctor has an "authentication" row.** It reads on or off. When authentication is off, it
lists the addresses that listen on `SOC_AI_PORT`, from `/proc/net/tcp` and `/proc/net/tcp6`. A
listener other than loopback reads WARN. A table that soc-ai cannot read also reads WARN.
- **The estate model row keeps the last fit when the setting is off.** The row now states the
date and the state of the last fit. Before, it read "off" with no trace of the fit.
- **`soc-ai estate-model show` and `soc-ai estate-model run`.** `show` prints the newest fit: the
time, the state, the hosts, the groups, the outliers, the model file, the hash and the drift
index. `run` fits once, now, with the guards of the daily loop. It runs when the setting is off,
and it says so. It refuses in a demo.
- **HUNTING.md states what a learning fit records.** The States table said a learning fit
records groups. Under 20 hosts the model does not fit, so the run records the fit row and no
group. The doc now has a row for each learning case. The code stays as it is.
- **The About section names the commit.** `GET /api/v1/about` has a `commit` field, and the
console About section shows it beside the version. The value comes from the `SOC_AI_COMMIT`
setting, else from the image stamp. A build that recorded no commit shows none.
`scripts/deploy-range.sh` now writes `SOC_AI_COMMIT` into the range `.env`, the way
`scripts/deploy.sh` does for production. It keeps every other line of the file.
- **The CLI no longer opens with the grid TLS warning.** With `ES_VERIFY_SSL=false`, every CLI
command printed the two-line elasticsearch `SecurityWarning` first. The CLI entry point now
hides that one warning. The library and the server keep it. The doctor states the setting once,
in a "grid tls" row under the elasticsearch row.
- **Only the service user can read the store.** The SQLite store held the password hashes at mode
0644 in a 0755 directory. soc-ai now makes the store file and its WAL files mode 0600 and the
data directory mode 0700. It tightens an older store when it opens it. `soc-ai restore` does the
same for the store it restores. A PostgreSQL store keeps its data directory mode.
`soc_ai/store/db.py` has helpers for the other files under the data directory:
`make_private_dir`, `restrict_file` and `restrict_sqlite_files`. The systemd unit sets
`UMask=0077`.
### Changed
- **The README is short, and the docs pages hold the detail.** The README went from
269 lines to 79. It keeps what soc-ai is, one screenshot, a quick start, one line for each
capability, the docs links and the roadmap chart. The rest moved into the quickstart, the docs
home page, the console guide and the roadmap. The docs nav sections are now Installation,
Operation, Design, Blog and Project.
- **The shipped prose follows the STE rules.** The docs pages, the blog posts, the Unreleased
changelog and the console strings lost their remaining dashes, asides and rhetorical
contrasts. The docs pages also split their long sentences and use plain-noun headings. The
lead quality note now reads "A threshold moves only on a week of data."
- The three profile analytics that fire most declare a fire budget of three hits a day and a precision floor of 0.25. The seven day production replay read 1.8 to 13.4 hits per 100 host-days against a budget of 1. The self-healing hold moves an analytic past its budget to shadow with the numbers as evidence, and an analyst approves it back to live.
- **The Oracle reviews an uncertain verdict.** A verdict goes to the Oracle in three cases. Its
confidence is from 0.4 to below 0.7. The decision template and the model disagree. A deep run
ends needs_more_info. A confident verdict that the template
agrees with stays local. The four Oracle settings narrow the rule and never add an
escalation. `oracle_escalate_below_confidence` now caps the band, and its default is 0.7. The
`oracle_escalation` audit row names the reason. A confident false positive on a malware rule no
longer escalates. It escalated 48 times in 30 days on production, and the Oracle confirmed
every audited one.
- **The standard loop gets the prompt and the tools the alert needs.** The prompt keeps the
sections that the planes of the alert make useful. These are a flow, an external endpoint, an
internal pair, host logs, a payload, a file hash, ICMP and a decoy. The other investigator tools stay registered and
load with one `search_tools` call. On a flow to the internet the request carries 15 of 27 tool
schemas and 34,948 of 40,018 prompt characters. The deep class keeps everything.
- **A drafted analytic describes a behaviour.** On the range a draft named one host and two
domains, and it could fire on that case only. The drafter now keeps exact values only for
stable discriminators. A check finds each clause that names the host, user, address or domain
of the finding. On a pin, the model rewrites the draft once. A draft that still pins shows the
pins in the confirm, in the drawer and as a "specific" marker on the Analytics tab. The dry
run states how many distinct entities matched.
- **Three role priors become match analytics that read their events.** Each prior tested a
different fact from the one its title stated. The new analytics read the Windows events they
name. soc-ai removes the three prior files. The table in `docs/HUNTING.md` maps each old id to
its new id.
- `identity-defender-detection` replaces `prior-defender-adjudication-on-server`. It reads
Defender events 1116 and 1117 on servers and domain controllers. The level follows the
Defender severity. The finding quotes the threat, the path and the action. The old prior read
new process names. On the development range it missed all 10 Defender detections and fired 73
times on Defender updater files.
- `identity-4719-audit-policy-change` replaces `prior-audit-policy-changed-on-dc`. It reads
event 4719 on domain controllers and excludes computer accounts. The finding quotes the
account, the subcategory and the change.
- `identity-privileged-group-change` replaces `prior-privileged-group-membership-changed`. It
reads events 4728, 4732 and 4756 on domain controllers when the group is privileged. An add
to any other group does not match. The finding quotes the account, the group and the member.
- **A match analytic can carry a role gate.** The gate reads the dossier role of the scope host.
A host in another role at confidence 0.9 or above is a non-match. A host with no confident role
is not dropped. The sweep reports its documents as a coverage gap that names the host. Declare
the role on the host page to decide it.
- **A match analytic can quote document fields in its finding and take its level from a field.**
The observation summary carries the quoted fields too.
- **A Sigma draft from a hunt finding reads the Windows event fields.** The draft evidence now
includes the event code, the provider, the channel and the fields the three analytics read.
## [1.5.2] - 2026-09-29
The TLS release. TLS has two paths. The proxy path is the production path. The direct path
validates the certificate and warns before it expires.
### Added
- **The proxy path.** `scripts/tls-proxy.sh enable <domain> [auto|internal|<cert.pem> <key.pem>]`
puts Caddy in front of soc-ai in one command. The `caddy` service lives in `docker-compose.yml`
under the `proxy` profile. `COMPOSE_PROFILES=proxy` in `.env` keeps it in every later
`docker compose up -d`. There is no separate compose project. Caddy terminates TLS and renews
the certificate. `SOC_AI_DOMAIN` names the site. `SOC_AI_CADDY_TLS` picks the certificate
source: empty for automatic HTTPS from Let's Encrypt, `tls internal` for Caddy's own CA, or
`tls /certs/proxy-cert.pem /certs/proxy-key.pem` for your own files. The Caddy container
receives those two variables and no soc-ai secret. The script writes the settings, reads the
compose network subnet into `PROXY_TRUSTED_IPS`, waits for the certificate and prints the trust
steps for the internal CA. `disable` restores the direct path. `setup.sh` asks for the HTTPS
domain.
- **The certificate inspector.** soc-ai reads the certificate and the key at start and states the
subject, the issuer, the names, the validity window, the chain order, the key match and the
SHA-256 fingerprint. It refuses a file over 256 KiB. It never stops the start: a bad file is
an error in the log, the doctor, the bell and the webhook.
- **`soc-ai doctor` has a `tls` check.** PASS for a valid chain. INFO for a self-signed
certificate, with the proxy path as the way to a trusted one. WARN 30 days before expiry, for a
broken chain, for plain HTTP on a non-loopback bind with no trusted proxy, and for a trusted
block that covers every address. FAIL for an expired or not yet valid certificate, a key that
does not match, or a file that cannot be read. INFO on the proxy path, with the trusted
addresses.
- **The Config screen has a TLS panel.** It shows the same record, the load time, and a
"Check again" button that reads the files on disk. When the files on disk differ from the
files loaded at start, it says so and names the restart command.
- **`GET /api/v1/config/tls`** returns the record for an admin, with `restart_required`.
- **A daily expiry check.** soc-ai compares the days left on the certificate it serves against
30, 14 and 7. When the band changes it fires the `tls_expiry` notification and puts a row on
the bell. A restart does not repeat the webhook: soc-ai keeps the last band it sent in
`tls-expiry-state.json` in the data directory. A swap on disk without a restart keeps the
alarm and notes the restart. `NOTIFY_ON_TLS_EXPIRY` turns the webhook off; the bell row stays.
### Changed
- **`PROXY_TRUSTED_IPS` accepts CIDR blocks.** A proxy in a container has no fixed address. The
operator lists the Docker address pool. An IPv4-mapped IPv6 peer matches its IPv4 block. An
entry that is not an address or a block trusts nothing.
- **One forwarded-header trust rule.** The HSTS header now follows the same trusted-proxy rule as
the Secure cookie flag. Before this release, any client could add the header with a forged
`X-Forwarded-Proto`. uvicorn's own proxy-header rewrite is off (`--no-proxy-headers` in the
image command and the systemd unit), so soc-ai holds the one rule.
- **The container healthcheck tries HTTPS and then HTTP.** It passes on both paths.
- **The CSRF guard accepts the https origin a trusted proxy reports.** Behind Caddy, soc-ai
receives plain HTTP while the browser's origin is `https://<host>`. The guard built the app's
own origin from the request scheme and rejected every cookie-authenticated write from the
console with `bad_origin`. A forged `X-Forwarded-Proto` from an untrusted peer is still
rejected.
- **`pyjwt` moves to 2.15.1, `urllib3` to 2.8.0 and `virtualenv` to 21.14.1.** They close
CVE-2026-101918, CVE-2026-102265 to CVE-2026-102272, CVE-2026-97688, CVE-2026-97689 and
PYSEC-2026-4011 to PYSEC-2026-4014.
- **The systemd unit passes the same cipher list as the image.**
- **The docs describe both paths.** DOCKER.md, TLS paths, holds the proxy recipe with the three
certificate sources, the network name rule, and the trust-block check. DEPLOYMENT.md holds the
host Caddy recipe for a source install.
This release has two parts. The first part is the set of fixes made after 1.5.0. The second part
is a review of the full repository: the code, the tests, the installer and the documents. Each
change below has a test that failed before the fix.
### Security
- A `resolution` marker that the model writes into a triage report no longer turns a valid verdict into a pipeline failure. Every synthesis path strips the privileged field. The paths are round-1, loop synthesis, investigator report, Phase-D, budget-cut partial and self-consistency samples.
- A wire-string pivot value now needs the same distinctiveness as GATE C: no stop-words, whole-word from 5 chars, substring from 8. The old code accepted a short, generic SMB file name, Kerberos SPN or DCE-RPC string on the alert's own flow. Example: a file named `name` or `alert`. Such a string made an alert-only citation count as "grounded in a pivot". Doc ids and JA3, hash and cipher values ground as before.
- The hunt citation gate no longer counts a tool short-circuit payload as gathered evidence. Such payloads are errors, duplicate-call stops and prefetch hints. A domain or IP that the model only asked about can no longer ground or corroborate a finding.
- Phase-D targeted dispatch of `t_web_search` / `t_crawl_page` now applies the same effective internal-identifier sets as the interactive tools. These sets are the env config plus the discovered hosts and suffixes. A discovered internal hostname can no longer reach a search engine or a crawler through that path.
- **A port no longer hides an internal name from the egress guard.** The guard keeps internal identifiers out of a web search and out of a crawled URL. The old code stripped the port only before the IP check. The internal-suffix check and the known-host check ran on the whole token. As a result, `dc01.corp.local:443`, `finance-pc:445`, `10.0.0.5_` and `10.0.0.5:80:1` reached SearXNG or the crawl4ai target.
- The Oracle sanitizer now tokenises `winlog.computer_name`, `ntlm.server_nb_computer_name` and Zeek `kerberos.client` principals from tool results before egress. Before this release, a bare NetBIOS name or a `user/REALM` principal in an OQL hit reached the Oracle verbatim on the default configuration.
- soc-ai now canonicalises `ORACLE_INTERNAL_SUFFIXES` entries given without a leading dot or in mixed case to lowercase with a leading dot. Example: `acme.example` or `.ACME.EXAMPLE`. This is the form the identifier store produces. The old code matched nothing for such a suffix in the Oracle sanitizer and its residue detector. That applied on the CLI, eval and DB-less fallback egress paths. Internal FQDNs and emails under the suffix left verbatim.
- Creating a user and changing a role now require an authenticated admin session, even with `API_AUTH_REQUIRED=false`. Reset-password and token minting already require it. An anonymous caller can no longer persist an admin login that survives turning auth on.
- Bulk re-hunt and request-more-info now refuse any investigation whose anchor document was promoted from a hunt finding or lead. The old code refused only the promoted row itself. A re-investigation opened over that anchor can no longer be relaunched with Security Onion writes enabled.
- The Security Onion write guard now recognises promoted leads as well as promoted findings. The guard refuses or disarms a re-investigation, a re-hunt, an ack or a write over a lead's anchor document. This is the same treatment as a hunt finding's anchor.
- `soc-ai backup` now writes the archive with mode 0600, whatever the umask. It no longer packs the one-shot `bootstrap-admin-password.txt`. Before this release, a backup taken before the admin's first password change held the plaintext admin password and the signing key in a world-readable file.
- The login throttle now buckets IPv6 clients by their /64 prefix. A single host can no longer avoid the failed-login lockout by rotating addresses inside its own prefix. Example: SLAAC privacy addresses or delegated prefixes.
- The analytic drafter's retry prompt no longer echoes internal hostnames, users or IPs to the cloud model. With analyst redaction on, the drafter validates the first draft in label space before the guard restores real values. The same residue check sweeps the retry prompt and the first prompt.
- The MCP server now writes every read-tool call from an MCP client to the tamper-evident audit index. The server runs as `python -m soc_ai.mcp_server`. Each call lands as a `tool_call` / `tool_result` pair under an `mcp-*` session id with `user=mcp`. Grid queries and IOC lookups made over stdio now show in `soc-ai audit verify` like the agent's own tool calls.
- The Oracle privacy gate now re-fangs defanged identifiers in runbook text and case comments before sanitization. Example: `10[.]0[.]0[.]5`, `dc01(dot)corp(dot)local` and `hxxp://`. An internal IP or hostname written in IOC notation is now tokenised like any other. The independent residue gate re-fangs too, and it refuses egress if one remains.
- Config → Egress policy now lists the MISP lookup client and the opt-in GitHub release check. The console no longer reports `zero_egress` while either is configured.
- `POST /api/v1/login` now requires a `Content-Length` header and answers 411 without one. This closes the chunked-transfer bypass of the 8 KiB login body cap.
- soc-ai now creates the sidecar file for the bootstrap admin password with mode 0600 from the start. It replaces a stale copy. If it cannot lock the file down, it removes the file. The old code left the file readable.
- **Acknowledge and Escalate on a filtered queue write only to the rows on screen.** The host page's Alerts KPI narrows the queue to one host with an OQL filter. The rows and the counts honoured that filter. Before this release, the two group writes did not send it. Acknowledging three events for one host acknowledged every event of that rule in the window, on every host. Escalate could open cases for hosts the analyst never looked at.
- **`setup.conf.example` works as shipped.** Before this release, a copy edited the way its header says leaked each ` # comment` into the value. `API_AUTH_REQUIRED=true` and `LITELLM_VERIFY_SSL=true` came out as `false` in `.env`. `ES_HOSTS`, `CONFIG_SECRET_KEY` and `BOOTSTRAP_ADMIN_PASSWORD` became the comment text.
### Fixed
- **A lead settles when its hunt finishes.** A hunt that finds no threat now closes the lead. The lead moves to Closed. It reads "Closed. The hunt found no threat." with the chip "closed by soc-ai". The lead quality report counts it under "Closed by hunt", apart from the analyst's dismissals. A hunt that finds a threat still waits on the analyst. A hunt that saw only a visibility gap also waits on the analyst. A gap the hunt notes beside what it observed does not hold the lead. A hunt that could not run returns the lead to open, and the loop tries once more. A second failure leaves the lead on Needs decision as "Hunted · Could not run". A lead the analyst dismissed and reopened waits on Needs decision, whatever its hunt found. Hunt again starts a second hunt. A repeat of a type the hunt cleared joins the closed lead as history while its observations are still live. A new type reopens the lead. A lead an earlier release left in hunting settles at the next start. Before this release, a lead stayed in hunting after its hunt had finished. A hunt that found no threat left the lead on Needs decision for the analyst to dismiss by hand. A hunt that could not run left the lead there with no second try.
- **A served port is a connection the host received.** A served port is now a port the host receives connections on. On an endpoint network event, only an accepted connection counts. A port also needs two different peers or two different days before it counts, in the baseline and at novelty alike. A port at 32768 or above needs both. An observation's occurrence count now moves only when the sweep cites a document it has not cited before. The row states the two counts apart: "6 documents in the last 24 h" and "seen on 3 sweeps". A lead's weight caps each type at 1.7, twice the lead threshold. One type that repeats cannot carry a lead alone. The lead page shows the weight of each type against that cap. Before this release, the served-ports dimension of a host's baseline counted a DNS lookup and an outbound flow as a port the host served. A client read as a server, and a first outbound call read as a new service.
- **The profile builder recovers from a refused query.** The active hours and the connection rate of a host's baseline now build through one query over the estate, split into partitions. When the grid refuses the query for too many buckets, the builder doubles the partitions and tries again, up to 32. When the grid still refuses, the builder records both dimensions as unmeasurable with the reason the grid gave. The host page row reads "not measured:" and the reason. The Analytics coverage line reads "baseline unmeasurable:" and the reason. A dimension whose telemetry plane the grid does not carry reads blind on every host, never measured. The profile sweep now owns the age of the baseline. It rebuilds the profiles when they are older than the dossier refresh interval, floored at twice the sweep interval. The dossier schedule setting does not change this. The coverage line reads "baseline N h old". Before this release, only the dossier sweep built profiles. A deployment with the dossier schedule off scored a days-old baseline and reported it as current. Migration 0051 adds the columns.
- With `verdict_consistency_samples` > 1 on the investigator-emits-report path, each vote sample now starts with a fresh tool-call dedup tracker. Its pivots now run. The old code returned `duplicate_call` stubs from the primary run.
- soc-ai now withholds every benign template on an attack-class alert, so the investigation loop always runs. Before this release, a protocol-housekeeping template could close an alert with an attack classtype as false_positive with no tool run. This happened when the rule name mentioned NTP, STUN, QUIC or a DNSSEC record type. Example classtypes: attempted-dos, attempted-recon, attempted-admin.
- soc-ai now shrinks an oversized plain-dict tool result to the per-tool budget by clipping long string fields. Example: `t_get_event_raw` on a Suricata alert with a large payload. The result lists the clipped paths under `__clipped_fields__`. The old code passed the result through whole with only a `__truncated__` flag.
- `t_origin_chain` now refuses a zero, negative or over-cap `lookback_minutes` with an error. The old code ran a query that was empty by construction and reported the host as self-originated.
- soc-ai now sends a chat reply that asserts SMB or file-share activity with no SMB evidence in the turn back for regrounding. If regrounding fails, soc-ai redacts the claim. Before this release, the claim shipped verbatim under the "unverifiable specifics were removed" line while nothing had been removed.
- Executing an approved write action while Security Onion is unreachable now returns the structured tool error. The same applies to a model-supplied `null` string argument. The old code returned an HTTP 500. An unattended auto-ack batch no longer aborts on the first transport failure.
- soc-ai now rejects an Oracle verdict whose confidence is NaN, infinite or outside 0.0-1.0 as unparseable and keeps the local verdict. Example: a percent-style `85`. The old code clamped the value to a fabricated 1.0 confidence that could clear the auto-ack threshold.
- `POST /api/v1/dossiers/{ip}/override` now refuses `role` and `criticality` sent as `value_json` with 400 `not_a_json_field`. This matches the bulk declare. Before this release, a grade or role written that way rendered on the host card. It was invisible to the importance sort, the role facet and the Hosts flags cell.
- A hunt whose SSE client disconnects mid-run now lands as `error` at once. The route is POST /api/v1/hunts/chat/stream. soc-ai now shields the terminal write from the request's cancellation, as the investigation stream already did. Before this release, the hunt stayed `running` until the orphan reaper. `hunt_in_progress` blocked every follow-up action on it.
- Opening an evidence chip while the grid is down or slow now answers the same retryable 503 `grid_unavailable` card as the other console reads. The route is `GET /api/v1/events/{id}`. `WEBUI_GRID_TIMEOUT_S` bounds the wait. A query the grid rejects is a 400 `bad_query`. The old code hung for the Elasticsearch retry budget and failed with a 500.
- A failure resolving the caller right after auto-triage planning no longer leaves the single-flight slot held. Example: a busy SQLite. Before this release, the held slot made every later Bulk Investigate answer "already running" and stopped the scheduled sweep until restart.
- A reinstated shipped analytic now runs in shadow. The sweep runs it, it collects dry-run receipts, and the analyst can approve it to live on evidence. Before this release, it read 'shadow' in the console while the sweep never ran it.
- **A blank connection setting can no longer stop the next restart.** Before this release, the Danger Zone API stored an empty Security Onion host, username or password as an override. The same applied to an empty Elasticsearch host list and an empty LiteLLM URL. The override applied at the next start. The Security Onion credentials went blank, or Elasticsearch had no host and the service never came up. No API removed the override again.
- **Hunt again starts another hunt.** The route now returns the running hunt while it runs and starts a second hunt once the first has landed. The first stays as history. Before this release, a lead whose hunt had finished answered every Hunt click with the old hunt. Start another hunt never started one.
- **A reopened lead can be promoted again.** Before this release, Promote read the investigation id the lead kept as history and answered with it. A reopened lead never produced a new investigation. A lead whose investigation was deleted reported a promotion that linked to nothing.
- **Deleting a hunt releases what pointed at it.** Before this release, a lead left hunting on a deleted hunt could never be hunted or promoted again. It sat in Needs decision with nothing to decide. A fired catalog condition that pointed at the deleted hunt stayed suppressed with no report left to read.
- **A hunt's chat recovers from an unclean restart.** Before this release, a follow-up question on a hunt that was in flight when the process was killed stayed pending for ever. The panel spun, and soc-ai refused every later question on that hunt as busy.
- **A schema upgrade can no longer empty a table's children while reporting success.** Before this release, the migration runner upgraded on a connection with SQLite foreign-key enforcement on. Alembic's batch mode rebuilds a table by dropping the original. SQLite treats that drop as a delete of every row. The first column change on `host_dossier`, `users`, `runbook` or `leads` would have cascaded at startup. It would have deleted every dossier field, saved view, runbook embedding and observation-to-lead link.
- **The pcap fetch reports a tcpdump failure on the sensor as an error.** Before this release, the pcap fetch treated tcpdump exit 1 as an empty match and discarded stderr. A sudo rule that needs a password, an unreadable pcap directory or a port outside 0-65535 came back to the model as an empty capture. That was evidence of absence that was never gathered.
- **The pcap window has a ceiling.** Before this release, `t_get_pcap` accepted any `window_minutes`. One call could sweep the whole Suricata ring buffer through `sudo tcpdump` over SSH, one file at a time. A value past 10^12 raised out of a tool that promises never to raise.
- **The pcap file search no longer depends on the sensor's timezone.** Before this release, the window start went to `find -newermt` as a wall-clock string with no zone. The sensor read it in its own local time, while the end bound was an epoch.
- `/health` now runs at most one pcap sensor probe at a time. Before this release, concurrent dashboard polls against a cold cache each opened their own ssh login to the sensor. That made the endpoint slowest when the sensor was already unreachable.
- The notification webhook now retries a transient transport error with the same bounded backoff as a 5xx response. Examples: a connection reset or a timeout. The retry policy already documented this. Before this release, a single connection blip dropped the notification for the hour.
- The nightly quality detector no longer raises a spurious `fallback_jump` alarm for a fallback-rate jump that lands on the 0.30 threshold. The rule now uses the same float tolerance as the agreement rules. The old code raised the alarm through float rounding.
- Role priors on the process and logon dimensions now find the host's role. These dimensions are `process_names`, `process_parents` and `logon_users`. Before this release, the sweep keyed roles on the dossier IP while those dimensions key entities on `host.name`. Every role-scoped prior on them was blind on every host. Declaring the role in the console changed nothing.
- `profile-connection-rate-collapsed` now fires for a host that went silent. The sweep now scores a silent host as zero in the cells the recent window covered. A silent host is one with a measured rate baseline that the recent read did not see. This happens only when the plane returned some other host. With nothing seen on any host, the shipper is down, and one finding per profiled host would say otherwise. A silent departure carries no sample ids and cites the baseline it fell from. Before this release, the recent read returned only hosts with at least one document. The sweep never evaluated a host whose agent was killed or whose shipper stopped.
- The prior sweep now reports a dimension that no plane on the grid carries as blind. It records one blind result per prior and a note that names the probe field and the datasets it tried. The profile lane already did this. Before this release, the sweep reported the dimension as "no recent activity". The recent readers collapsed "no candidate dataset carries the field" into an empty read. The trail row for every prior on that dimension was all zeros and read as an all-clear.
- The dossier now stores the logon-user baseline it builds for a host as a host profile. The domain-controller logon priors and the host page can read it. On an estate with CIDRs configured, soc-ai no longer drops or purges a host keyed on its agent name as out of scope. The coverage fill never overwrites a measured logon set with an empty one.
- Re-triaging an alert as a false positive now closes the lead that alert formed on its own. soc-ai dismisses the lead and records the reason. soc-ai reshapes a lead that still holds other observations. The old code left an empty lead open in the queue for ever.
- **A corrupt threat-intel feed file no longer stops soc-ai from starting.** Before this release, `BlocklistDB.from_dir` guarded only against a missing file. A truncated or empty `threatfox.json` or a ThreatFox row with a null `ioc_value` raised out of `build_local_enrichment_context`. So did a hand-edited `internal_seed.yaml` with a bare `- 1.2.3.4` line, and a feed with non-UTF-8 bytes. The app lifespan, the MCP server and the eval harness call that function unguarded.
- **A feed file that holds no indicators no longer reads as a clean check.** Before this release, a refresh that saved an error page as `urlhaus.csv` parsed without error. So did an empty body, a `{}` ThreatFox dump and a header-only CSV. The feed counted as loaded. Every lookup rendered as "reputation checked, no hits" while no intel was on disk.
- `soc-ai blocklists refresh` no longer replaces a good feed file with an empty HTTP 200 body. The same applies to an HTML sign-in or WAF page and a JSON error document. The command runs the download through the real loader first. It discards a body with no indicators, keeps the previous file, and reports the feed as `FAIL` with a non-zero exit.
- The cloud-prefix refresh now rejects a Cloudflare body with no CIDR line. It also rejects an AWS, GCP or Azure document without its top-level prefix list. The old code wrote such a body over the good file. The refresh now writes all four prefix files atomically, through a temp file and a rename. A crash mid-write cannot leave a truncated JSON behind.
- Several reads in flight on one expired Security Onion session now cost one Kratos login. The first 401 replaces the session, and the other reads retry with it. The old code made one login per read, which fed Security Onion's login throttling.
- Blank `SOC_AI_TLS_CERT=` / `SOC_AI_TLS_KEY=` lines now mean "no TLS". `.env.example` ships them blank. `soc-ai serve` starts on plain HTTP, and `healthz`/`triage` target `http://`. The old code read a blank line as `Path(".")` and failed with an SSL error.
- `LOG_LEVEL` now accepts the lowercase uvicorn-style spelling and rejects an unknown name at config load. Examples: `debug` and `warning`. Before this release, a lowercase value booted uvicorn and then crashed the app in startup with `ValueError: Unknown level`.
- Blank `GREYNOISE_API_KEY=` / `SHODAN_API_KEY=` / `MAXMIND_LICENSE_KEY=` / `ABUSE_CH_AUTH_KEY=` / `NOTIFY_WEBHOOK_URL=` / `CRAWL4AI_TOKEN=` lines now read as unset. The Data Sources page no longer shows GreyNoise, Shodan host or MaxMind as configured while every lookup refuses with `not_configured`. `soc-ai doctor` now gives the abuse.ch registration hint. The old code offered a refresh that kept skipping.
- `POST /hunts/chat/stream` now starts its hunt through the Hunt Console manager like every other start path. A streamed hunt counts against the shared concurrency ceiling and gets 503 `could_not_start` when it is full. The analyst can cancel it from the console with `POST /hunts/{id}/cancel`. The old code answered that cancel with 404 `no_live_hunt`. A streamed hunt keeps running to a persisted report if the streaming client disconnects.
- Config: clearing a stored API key or the notification webhook URL now asks for a confirmation click. The flow is Clear -> Confirm clear / Cancel. This matches every other destructive action. A mis-click beside Replace no longer wipes a write-only secret.
- **A chat turn survives one dropped poll.** Before this release, a single poll that timed out or hit a connection blip ended the turn on screen while the agent worked. The typing indicator stopped. A "Could not reach the server" bubble appeared. The reply that the backend finished moments later stayed in the database until the analyst reopened the page.
- **The analyst can open a notification in the bell from the keyboard.** Before this release, each row in the bell dropdown was a plain block with a click handler. Tab moved from one row's Dismiss to the next and never onto the row itself. The only action a keyboard user could take on a true-positive notice from the bell was to dismiss it.
- **One lead starts one hunt, even when the click and the loop's wake land together.** The console writes the hunt row and waits for the first event before it answers. The old code marked the lead only after that answer.
- **The auto-hunt loop no longer stalls behind a page of leads that cite nothing.** The loop skips a lead whose observations name no document, and the lead stays open. Such leads collect at the head of the queue. A false-positive verdict that empties a lead adds to them.
- **Run the full check starts the battery again.** Before this release, the request that starts the full fitness battery sent its JSON body without a JSON content type. The browser labelled it plain text, and the server refused it. The button did nothing and said nothing.
- **Sign out no longer hangs on a stalled API.** Before this release, the sign-out request had no time budget, unlike every other call. When the backend accepted the connection and never answered, the click did nothing for as long as the server did not answer. The session stayed alive.
- **The full check shows its result without a reload.** Before this release, starting the full fitness battery from Config → Models & Reasoning showed "Full check: … 1 of 4…" with both buttons disabled. The display never moved on, even after the server had finished. Nothing polled for the result after the start. A refused start looked the same.
- **The fitness chip no longer grades one model and shows it beside another.** The fitness check grades the analyst model the server is running. Before this release, picking a different model in the dropdown ran the check anyway. The chip beside the new, unapplied pick showed the old model's grade. A slow check could land on the chip after the pick had moved on.
- **A migration that fails half-way leaves the store as it found it.** Before this release, the startup migration meant to run as one transaction. The SQLite driver opened a transaction only ahead of a data statement. Every CREATE and ALTER committed the moment it ran, while the version stamp was rolled back.
- **`soc-ai doctor` notices a store that never got its FTS index.** Migrations 0017 and 0018 skip the FTS5 tables on a Python whose SQLite lacks the module, and they never retry. A store first migrated there and later run on the shipped image kept its head but had no `runbook_fts` or `chat_memory_fts`. Runbook search used the legacy ranker. Chat memory returned nothing. Before this release, doctor said BM25 retrieval was on.
- **"My queue" in the command palette no longer opens the whole alert list.** Before this release, the palette sent `?view=myqueue`, a view the Alerts screen does not have. An unknown `?view=` fell through to every row with no preset chip highlighted.
- **A failed acknowledge, assign, release or status change no longer shows a green check.** Before this release, every "Failed to …" notice from the Alerts screen went out through the same helper as its successes. The notice showed as a green check that vanished after six seconds.
- **"Acknowledge N events" reports a failure.** Before this release, the per-event acknowledge awaited its write with no catch. A grid error or a demo refusal produced an unhandled rejection in the browser console and nothing on screen. The button re-enabled.
- **The row's Investigate button reports a refusal.** Before this release, Investigate, Retry investigation and the o/Enter/i keys resolved the representative event and started the hunt. On any refusal, they only closed the "Starting investigation…" drawer. Example refusals: a hunt already running, or the grid down.
- **A group whose events could not be fetched no longer reads as "No events in window."** Before this release, a rejected per-group event page was cached as an empty one. A grid error looked like an empty group. Collapsing and re-expanding the group never asked again until a filter changed.
- **The Alerts screen drops an event page fetched under an old filter.** Changing the range, the query or "Hide acknowledged" clears the cached event pages. Before this release, a page already in flight landed afterwards and was kept. Acknowledged or out-of-window events were listed as matching.
- **Leaving the Alerts screen while Bulk Investigate starts no longer leaves a status poll running.** Before this release, a start request that resolved after the screen had unmounted created its interval after the cleanup had run. That interval polled the auto-triage status every two seconds for the rest of the sweep.
- **The demo container stops when its mock Elasticsearch dies.** Before this release, the supervisor polled `kill -0` on the mock from a detached subshell. Once the app was PID 1, the dead mock stayed a zombie that `kill -0` accepts. The loop never ended, and the demo served empty grids while reporting healthy.
- **Secrets keep their value through `.env`.** Before this release, `setup.sh` wrote every value bare. Compose's env_file parser expands `$name` and drops ` #…`. A Security Onion password like `pa$sw #x` reached the container as `pa` right after the installer had checked the full password against Elasticsearch.
- **`scripts/setup-audit-index.sh` handles a role that carries a query.** Before this release, the script spliced the role body into Python source as a string literal. A document-level-security query, or any escaped character, was unescaped before `json.loads` saw it. The grant died with a traceback after step 1.
- **`sudo ./setup.sh` no longer leaves a TLS key the container cannot read.** Before this release, the generated `certs/key.pem` was `chmod 640` and owned by the user who ran the installer. The container opens it as uid 1000. Unless uid 1000 was the installing user, uvicorn died on the key. The container restart-looped behind a "Liveness check timed out" message.
- **The installer polls and prints the port that `.env` publishes.** The liveness poll and the summary now use the port read back from `.env`, as the starter-pack step already did. Before this release, a re-run that kept an existing `.env` with a different `SOC_AI_PORT` polled 8443 for three minutes and printed an 8443 URL.
- **The installer names a malformed `setup.conf` line.** `setup.sh` now stops with the file name and the offending key. Before this release, `export KEY=value`, a hyphenated key or a key with a space made bash abort. Bash printed its own `invalid variable name` one-liner before the installer had said anything.
### Changed
- `t_get_pcap` now caps `window_minutes` at 60 on both the interactive and Phase-D paths. The cap is the half-width around the alert. A model-supplied window can no longer sweep the sensor's whole pcap ring buffer.
- **`suggest_rule_tuning` caps its window at 365 days.** Before this release, the tool rejected a zero or negative `lookback_days` but accepted any larger value. An agent turn could run the tuning aggregation over the grid's full retention. The import-count probe behind an empty result ran over the same range.
- The alerts-page and auto-triage inheritance lookup now reads only the columns its callers use. The old code loaded every completed investigation's report, summary and rationale to pick the newest row per detection pair.
- The config_overrides module and model docstrings now state that the table holds Fernet-encrypted secret overrides and plaintext connection identity. They say to treat the table as sensitive in dumps and backups. The old docstrings claimed the table never holds secrets.
- **Decoding a fetched pcap no longer stalls the app.** The SSH fetch already ran in a worker thread. Before this release, the packet decode ran on the event loop. A capture at the 50,000-packet cap froze every open investigation stream and health poll for about half a second.
- **The security-audit test harness no longer uses a deprecated test-client feature.** Each audit role now has its own client whose cookie jar carries its session. The shared client stays anonymous for the unauthenticated checks.
- **A test checks every model against the migration chain.** The test migrates a fresh store and compares all of it with the ORM: columns, types, indexes, uniques and foreign keys. A model change without a revision, or the reverse, now fails in CI. Before this release, such a change surfaced as a missing column or an autogenerate that proposed dropping an index.
- **The bell's completed-runs poll no longer sorts every finished run.** `GET /notifications` bounds and orders completed investigations and hunts on `finished_at`. Before this release, the only index was on `created_at`. Each 15-second poll from every open tab walked every completed row and sorted it.
- **Counting unread shadow hits no longer scans the observations table.** The bell counts `shadow AND unread AND not retired` observations every 15 seconds, and the sidebar badge does the same every 60 seconds. Before this release, the table had no index that clause could use.
- **The wheel and the image no longer carry an empty `soc-ai.db`.** A zero-byte store had been tracked next to the package since 1.1.0 and shipped in every build. This release removes it. The ignore file now lists `*.db`, `*.db-wal` and `*.db-shm`, so a store with triage history cannot be committed the same way.
- **`alembic -x db_url=...` targets the store it names.** Before this release, the migration environment advertised the option but ignored it. A manual `upgrade` or `downgrade` run from any directory other than the repo root created and migrated an empty `./data/soc-ai.db`. It left the named store alone.
- **Migration 0041's header names its own revision.** Before this release, the header read `Revision ID: 0040 / Revises: 0039` while the code set 0041 / 0040. Every other version file already matched. A test now keeps them that way.
- **The docs no longer say `blocklists refresh` downloads the MaxMind GeoLite2 databases.** No command does. DOCKER.md, BLOCKLISTS.md, `.env.example`, the compose file and the Data sources note now describe the manual download of `GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` into the MaxMind data directory. The troubleshooting entry for missing GeoIP no longer points back at the refresh.
- **The quickstart and the demo compose header open the console at `/app/alerts`.** The printed `/ui/alerts` URL has answered 404 since the retirement of the server-rendered console.
- **DOCKER.md's loopback recipe sets `SOC_AI_BIND`.** The old `SOC_AI_PORT=127.0.0.1:8443` form produced an invalid compose port mapping. The container reads the same `.env`, so that form also failed Settings validation before the app listened.
- **DOCKER.md offers `ES_CA_BUNDLE` for an Elasticsearch behind a private CA.** The old guide claimed Elasticsearch had no CA option and steered operators to `ES_VERIFY_SSL=false`.
- **DEPLOYMENT.md §7 says what a missing audit grant does under the shipped default.** Under `AUDIT_FAIL_CLOSED=true`, soc-ai aborts every acknowledge, escalate and comment until the `soc-ai-audit-*` grant lands. The guide no longer calls the failure "not fatal". The §10 symptom row says the write-back is aborted.
- **`.env.example` lists the `AUTO_TRIAGE_PER_TARGET_TIMEOUT_S` default of 1200.** A test now checks every commented default in the Timeouts & retries block against the code.
- **The Docker guide reads the bootstrap admin password from `bootstrap-admin-password.txt`** in the data volume. The old guide grepped a log that never receives the password. `.env.example` no longer states a wrong character count for the password.
- **HUNTING.md's command table names the required `--since` flag on `soc-ai spec-run`** and the `--until` / `--include-synth` options that shape the run.
- **The README no longer says no prebuilt image exists before the first release.** `./setup.sh --prebuilt` pulls `ghcr.io/nuk3s/soc-ai` pinned to the release version. `setup.sh` offers the source build if the registry refuses the pull.
- **The docs-site changelog page carries highlights for 1.2.7 through 1.5.0**, and a test keeps it at the shipped version. A test keeps the README badge the same way.
- **The pre-commit ruff hook is the ruff CI runs.** Before this release, the hook pin sat two minor versions behind uv.lock. Its `--fix` deleted the `# noqa: ASYNC240` comments the locked ruff still needs, and the pushed commit failed CI's ruff step.
- **Committing a change to `mkdocs.yml` no longer fails the pre-commit hooks.** The `check-yaml` hook is a safe YAML load. `mkdocs.yml` needs `!!python/name:` tags for superfences and the Material emoji index. Before this release, every nav edit needed `--no-verify`. That flag also skipped ruff and the private-key check.
- **`npm run lint` is gone from the frontend until an eslint setup exists.** The script called `eslint .`. eslint was not a dependency and had no configuration. The script failed on every checkout, or it ran the global eslint on the machine. It implied a lint gate that CI never ran.
- **CI audits the dependencies that ship, and Dependabot keeps every pin current.** A new `dependency audit` job runs pip-audit over the locked runtime Python set and `npm audit` over the SPA's production dependencies. A lockfile that carries a known advisory can no longer merge green.
- **CI builds the docs site strictly on every pull request.** Before this release, `mkdocs build --strict` ran only in the post-merge Pages deploy. A PR that renamed a page that the nav still listed merged green. It stopped the published site from updating.
- **CI holds `scripts/` to the same ruff gate as the package.** pyproject.toml already configured ruff for `scripts/**`, and the pre-commit hooks formatted it. Before this release, CI checked only `soc_ai/` and `tests/`. Two demo scripts were unformatted. The first commit that touched them carried unrelated formatting changes.
- **Every workflow job has a timeout.** Before this release, the docs build and deploy jobs and the GitHub Release job inherited GitHub's 360-minute default. One stalled install or `gh release` call could hold a runner for six hours. Through the `pages` concurrency group, it could queue every later docs deploy behind it.
- **The `mcp` dependency pin stays below 2.x.** mcp 2.x removes `mcp.server.fastmcp`, the one import the MCP server makes. FastMCP became `mcp.server.mcpserver.MCPServer`. A routine `uv lock --upgrade` could have taken the MCP server down with no `pyproject.toml` diff to review.
- **`pytest-vcr` is no longer a dev dependency.** Nothing in the suite records a cassette. respx and AsyncMock mock HTTP. Before this release, every `uv sync --dev` installed the plugin and every pytest run loaded it.
- **The test suite boots the app faster.** Before this release, every app boot ran the whole migration chain on an empty store and hashed the bootstrap admin's password at production bcrypt cost. That was about half a second of identical work per test, over a thousand times a run.
- **CI runs the backend suite in parallel.** Every test already owns its store. The suite chdirs each test into its own temp dir, and the data dir is relative. The `pytest` step now runs under pytest-xdist with `-n auto --dist loadfile`.
- The backend now depends on `pydantic-ai-slim[openai]`. The old dependency was the full `pydantic-ai` meta-package. That package pulled every provider SDK into the image and the lockfile. The lock drops from 211 to 132 packages. aiohttp, anyio, cryptography, mcp and the httpx2 pair move to their advisory-fixed releases. mcp stays below 2. `pip-audit` over the exported lock reports no known vulnerability.
- The frontend lockfile takes the non-breaking `npm audit` fixes: postcss, nanoid, browserslist and react-router 6.30.6. The rest needs a major upgrade to vite 8, vitest 5 and react-router 7. That upgrade waits for a release of its own.
- The ignore file, the image build context and the deploy scripts no longer name an editor tool's local scratch directory.
- `npm run typecheck` now runs `tsc -b`. The root tsconfig only holds project references. The previous `tsc --noEmit` therefore checked no file at all. CI's typecheck step passed on a tree the build rejected.
## [1.5.0] - 2026-09-23
1.5.0 is the hunting release. soc-ai now looks for the attacks that raise no alert. An analytic
finds a hit, hits on one entity form a lead, a lead starts its own hunt, and a hunt reaches a
verdict. The Hunts page holds that whole pipeline, and the Needs-you strip names the few things
that wait on you. The release also makes an investigation more accurate and restores every write
to a Security Onion 3.3 grid.
### The profile sweep runs in the app (2026-09-22)
- **The profile sweep needs no host timer.** The sweep compares each host with its own baseline,
records what departs, and forms a lead from what accumulates. It ran from a systemd timer on the
range. A container has no timer, so a Docker deployment recorded no profile observation and formed
no profile lead. Nothing failed and nothing said so. An in-process loop now runs the same code
path `soc-ai priors --record` runs, every 60 minutes, with a floor of 15 minutes. Two settings
control it in Config → Hunting: "Run the profile sweep", on by default, and "Minutes between
profile sweeps". Both apply live. The loop skips a demo, and it skips a sweep while Elasticsearch
is down, because a sweep that cannot measure records that nothing departed. It does not wait for
the dossier loop: a sweep with no baseline behind it reports what it could not measure as blind,
as the command does.
### What the sixth dogfood round found (2026-09-22)
The four decisions in this release were read on the range through the browser, the API, the
database and the service log.
- **A reopened lead with a finished hunt waits on the analyst.** A reopen keeps the hunt, so the
lead is open again with a hunt attached. That lead matched no tab: not new, because it names a
hunt, and not hunting, because the analyst reopened it. The loop leaves a reopened lead alone,
so nothing held it at all. The Needs-you count, the Needs decision tab and the sidebar badge now
hold it under both settings, and an open lead whose hunt runs reads In progress.
- **A hunt-subject investigation is not the anchor alert's investigation.** A promotion anchors
the pipeline's time windows on one cited document, so the row names that document. Every reader
that grouped by the column read the run as the alert's: the alert's own investigation arrived as
a superseded earlier run, and a false positive about the hunt's hypothesis would have read as
the alert's verdict on the Alerts page. One rule now answers whose run it is for a row, for a
grouping key and for a stored query, so no reader can answer it a different way. The two
prefetch events also say that the subject is the hunt.
- **The related rows carry the state of their own hunt.** The Related leads panel read the stored
status, which stays `hunting` after a hunt lands, so one lead read "In progress" in the panel and
"Hunted" on every other surface. Each row now carries `hunt_status` and `hunt_outcome_label`, in
one query for the page.
- **A lead that shares an entity is not a related lead.** Related leads answer one question: did
this move to another entity? A dismissed lead and the lead that forms next on the same host
share the analytic that formed them both, and the panel offered that successor as a relation.
- **The investigation timeline reads a hunt subject as one.** The context step said "Loaded alert
context + enrichments" over the hunt's findings, and the template step said no pattern matched
when the templates had not run at all. The steps now read "Loaded the hunt subject: N findings,
M documents" and "Templates do not run on a hunt subject".
- **The hunt objective says that its related list is a snapshot.** The objective is written once,
at the start of the hunt. The related list is computed on read. The line now reads "Related leads
at the start of this hunt:", so a reader hours later knows which list it is.
- **The lead auto-hunt help names all four leads the loop leaves alone.** It named the dismissed
lead only. The loop also leaves a reopened lead, a shadow lead and a lead that cites no document.
Four replaces the three that entries below this one name.
### Leads are hunted, hunts are investigated, leads relate (2026-09-22)
The screens for the four decisions in the design for the hunting layer.
- **A new lead says that its hunt is coming.** soc-ai starts the hunt on a new lead by itself. The
lead then waits on that loop and not on the analyst, and the strip still read "New", which is
the word for a lead nobody has touched. The pill on such a lead now reads "New · hunt queued".
The lead offers Dismiss and Promote. Hunt drops to a second act under the name "Hunt now",
because the hunt is already coming and the button only makes it come sooner. The lead page reads
the same words. The legend gains one sentence while the loop runs: "A new lead starts its own
hunt." The note that said soc-ai starts no hunt from a lead is replaced while the loop runs,
because it states the opposite of the truth. The strip reads the setting from the leads
themselves. A lead that carries a queued hunt is a lead the loop has taken.
- **An investigation of a hunt names the hunt.** The right panel of the investigation page named
the rule and the event. A hunt investigation has neither. The panel now reads "Subject: hunt"
for such a run. It carries the objective, the findings the run read, a link to the hunt, a link
to the lead when the hunt started from one, and the count of the documents the findings cite. A
backend that sends the finding ordinals without their titles gets the ordinals on screen. The
verdict block, the evidence and the citations are unchanged. The Investigations list carries a
type chip "hunt" beside the detection type for such a run.
- **Promote waits on a hunt.** An investigation of a lead reads the hunt's findings. Promote on a
lead with no finished hunt is now refused on the lead page, under the sentence "Hunt this lead
first. The investigation reads the hunt's findings." A refusal from the server carries its own
sentence, and the leads strip prints that sentence in place of its own "Try again."
- **A lead names the leads that relate to it.** One entity is one lead. A coordinated attack
across entities is several leads, and no single one of them reads as a campaign. The lead page
carries a Related leads panel. Each row names the entity, the lead, the reason the two are
related, and the state of the related lead. An empty panel states "No related lead in the last
7 days." The strip row carries a chip "+N related" that opens the lead page. A deployment whose
API computes no related leads shows no panel, because an empty panel there would read as an
answer nobody gave.
- **The Analytics tab counts what the lead rule produced.** The lead rule is a set of thresholds,
and a threshold nobody measures is a guess. A Lead quality block sits under the analytics table.
It states the rule and the noise floor rule in the server's own words, then counts per week the
leads formed, hunted, with a threat finding, promoted, and dismissed under each reason. A second
table counts the same per set of observation types. The reason columns come from the data, so a
reason nobody used costs no column. A failed read states the failure, because "no leads" over a
dead endpoint is a false all-clear.
### The screens after dogfood round 6 (2026-09-22)
The screen half of the sixth dogfood round.
- **A related lead wears the pill of its own hunt.** The Related leads panel built the pill from
the stored status alone. A lead whose hunt had finished read "In progress" there and "Hunted" on
the strip, on its own page and on the host page. The panel reads the whole row now, and the row
carries the hunt status and the outcome label.
- **The leads note names the leads the loop leaves.** The note claimed a hunt on every new lead.
The loop leaves a shadow lead, a reopened lead and a lead with no documents, so a shadow lead
read "New" under a note that promised a hunt that never came. The note states the three
exclusions, and such a lead carries the chip "left to you" beside its pill on the strip and on
the lead page.
- **An investigation of a hunt reads in hunt words.** The breadcrumb read "Alerts / Investigation"
over a run with no alert behind it, and it goes to Hunts now. The header carried the severity,
the host and the peer of the first cited document, which is one of many the run read. It reads
the first sentence of the objective and a link to the lead.
- **The related sentences name the alert rule.** The server joins two leads on four things. The
chip and the reason column named three, and the reason an analyst read on the range was the
fourth.
- **The lead page legend reads the setting.** The legend is a taxonomy of the deployment. The page
appended "A new lead starts its own hunt." only when this one lead was queued, so every hunted,
promoted and reopened lead read the legend without it.
- **Promote waits on a hunt on both surfaces.** The lead page disabled Promote on a lead with no
finished hunt and said why. The strip row kept the button live, and the click returned 409.
- **The New pill stops offering Promote.** The server refuses a promotion until a hunt has
finished. The pill names the hunt that starts on its own, the hunt by hand, and the dismissal.
- **The type chip states what a lead run is.** The chip on a lead run read "The detector that
raised this alert, as the grid names it." No detector raised it.
- **The Subject panel counts findings from one.** The list read "Finding 0" over the first finding
of the hunt. The stored ordinal is unchanged.
- **A promotion from the lead page says so.** The click left no mark, while the record already
held the promotion and the running investigation. The page states "Promoted. The investigation
is running.", links the investigation, and reads the lead again.
- **A lead title spaces the names of two entities.** The two names ran together under one dot.
- **The hunt page names a reopen.** The lead page ended a reopened dismissal with "Reopened." and
the hunt page ended it without, though both render one timeline component.
### A hunt is investigated as a whole (2026-09-22)
- **An investigation of a hunt has the hunt as its subject.** Promotion used to start the
ordinary alert triage on one cited document, so the scope collapsed from the hunt to one
event. The investigation now reads the hunt: its objective, its narrative, all of its findings
and every document those findings cite, up to forty, with the promoted finding's documents
first and the rest newest first. A lead hunt also carries the lead's observations and the
leads that relate to it. The verdict schema does not change. The prompt states what the three
verdicts mean for a hunt: a true positive means the hunt's hypothesis holds as an attack or a
compromise, a false positive means the findings have a benign explanation the evidence
supports, and needs more info names the gap a tool call can close. The summary answers the
hunt's objective in one paragraph.
- **The cited documents are the prefetched evidence.** They join the run's bundle before the
first model call, so a verdict that cites one of them resolves against a document the run
really holds. The evidence gate, the citation cap, the egress guard, the Oracle escalation and
the session gate all run. Three do not. The decision templates each match an alert rule class,
and a hunt has none. The unattended acknowledge has nothing to write to, because nothing in
Security Onion holds a hunt. A hunt has no session, so the same-session prior has nothing to
read.
- **Promoting a lead reads its hunt.** A lead with no finished hunt answers 409
`lead_not_hunted` and names the way forward. Promotion is idempotent as before.
- **The investigation page can say what the run was about.** `GET /investigations/{id}` carries
`subject` with the hunt id, the objective, the finding ordinals and titles, the lead id, the
document ids and the observation ids. The list rows carry `subjectType`, which reads `alert`
or `hunt`. The `kind` field is unchanged. Migration 0050 adds `investigations.subject_json`,
which is empty on every alert run.
### A lead starts its own hunt (2026-09-22)
- **A lead that has never had a hunt starts one when it forms.** A lead is already several
observations that crossed a threshold together, so the question "is this worth a look?" is
answered by the time the lead exists. Before this the lead waited for a click, and a lead that
formed overnight was still waiting in the morning. A loop wakes every 60 seconds and starts the
hunt for each open lead that has no hunt, oldest lead first. The hunt is the lead hunt the Hunt
button starts: the same objective, the same evidence block, the same type. The route and the
loop now call one function, so the two cannot start different hunts. The loop starts the hunt,
and not the sweep that forms the lead, because leads also form in the timer process that runs
the priors, and that process cannot run an agent.
- **The loop leaves three leads alone.** A dismissed lead is never hunted. A reopened lead is not
hunted again, because the analyst reopened it to decide again and Hunt again is their button. A
lead whose observations cite no documents is skipped. Its hunt would have no evidence to read
first and would search the grid from scratch. The skip is logged once an hour for each lead. The
lead keeps its Hunt button in every one of these cases.
- **Two settings, both live.** "A lead starts its own hunt" (`lead_auto_hunt`) is on by default.
Off restores the older behaviour, where a lead waits for an analyst to start the hunt. "Lead
hunts at once" (`lead_auto_hunt_concurrency`) is 2 by default. It counts the hunts the loop
started, so a hunt an analyst started by hand never blocks the loop.
- **The Needs-you strip counts what an analyst must act on.** With the loop on, a new lead waits
on soc-ai and not on the analyst, so it leaves the count and the Needs decision tab. A lead
whose hunt finished still waits, and so does a reopened lead. With the loop off the older rule
holds. Each lead carries `hunt_queued`, which the New pill reads as "New · hunt queued".
### Related leads, and the lead rule instrumented (2026-09-22)
- **A lead names the open leads it may share a story with.** A lead spans the entities that one
hit names. A coordinated attack does not, so an analyst who read one lead could not see the
rest of the attack. The lead page now carries a Related leads list, and the lead row carries
the count. A lead is related when it is open, when it formed in the last 7 days, and when it
shares one of three things: the same analytic on an observation within 24 hours, the same
external address or network in an observation, or the same ATT&CK technique. Each entry states
the share in one phrase, for example "same analytic within 24 h", "same external address
203.0.113.0/24" or "same technique T1558.003". soc-ai computes the list on read and stores
nothing. A stored relation goes stale as soon as an analyst answers either lead. IPv4 compares
on the /24, because two hosts that talk to neighbouring addresses usually talk to one place.
- **The objective of a lead hunt names the related leads.** The hunt reads them with the lead's
own observations, and the objective asks it to state whether the leads are one campaign, with
the evidence.
- **Every threshold in the lead rule states its derivation.** The lead threshold (0.85) and the
single-type threshold (1.5) were derived from arithmetic, and each docstring gives the
derivation. The hub limit (8) was set on the range on 2026-09-17 with no derivation on record,
and the docstring says that. None of the three is yet validated against a miss.
- **`GET /api/v1/leads/quality` reports what the rule produced.** Per ISO week: leads formed,
hunted, with a threat finding, promoted, and dismissed by reason. Per observation-type pair:
formed, dismissed, and with a threat finding. A threat finding is the outcome the Hunts page
paints as "Threat findings", read from the same function, so the two surfaces cannot disagree.
A week with no leads is still a row, because a quiet week is a measurement. The report states
the rule and the noise floor: a threshold moves on a week of data, never on a day.
- **`soc-ai leads --report` prints the same table.** It takes `--weeks N` (the default is 4). It
reads the local store. It calls no model and queries no grid.
### The Security Onion login (2026-09-21)
- **The login uses the Kratos browser flow, which SO 3.3 requires.** Security Onion 3.3 stopped
accepting a Kratos API-flow session token in an `X-Session-Token` header. The login still
succeeded and SOC then refused the session on every call, with HTTP 401 and the server-side
reason "Missing or invalid authorization header for bearer token". Every write to Security Onion
failed that way: acknowledge, escalate, case creation and case comments. Reads were unaffected,
because they read Elasticsearch with a different credential. soc-ai now logs in the way the
Security Onion web interface does. It reads the login flow document, submits the credentials
with the flow's CSRF token, and lets the cookie jar carry the `ory_kratos_session` cookie on
every later request. The srv-token handling that writes depend on is unchanged, and the Kratos
path prefix setting is unchanged.
- **One build works on every Security Onion release.** The setting "Security Onion login flow"
(`so_login_flow`) takes `auto`, `browser` or `api`. `auto` is the default. It runs the browser
flow, and falls back to the older API flow when the browser flow cannot complete: the browser
endpoint is absent, the flow document holds no CSRF token, the login answers a 4xx that is not
a credential rejection, or SOC refuses the cookie session on `/api/info` while it accepts the
same account's API-flow session. A wrong password never causes the fallback, because the same
password fails on both flows. soc-ai keeps the flow that worked for the life of the process, so
a fallback costs one extra login and not one for every request. `browser` and `api` force one
flow with no fallback. The setting applies at the next restart. One log line names the flow at
the first login that Security Onion accepts.
- **A refused session backs off instead of logging in again.** A 401 made soc-ai drop the session
and log in once more, on every request, with no ceiling. Over three days that made 32,420
throttled logins, 75,800 warnings in Security Onion, and an audit index sixteen times its normal
size. When SOC refuses a session that soc-ai has just established, soc-ai now holds the login.
The hold starts at 30 seconds and doubles to a ceiling of ten minutes. One log line names the
cause, and the next accepted call clears the hold.
- **A throttled login reads as a throttled login.** Security Onion sheds repeated logins by
redirecting them to its own login page. soc-ai read that page as JSON and reported "Kratos login
flow init failed: Expecting value: line 1 column 1 (char 0)". That message filled the log and
described a login that never started, which hid the session refusal behind it. The message now
says that SO throttled the login, and gives the status code.
- **The setup check names the cause.** On a 401 the check said the Security Onion API was unhappy
and sent the operator to the user's role grants. The grants were correct. The check now says
that SOC refused the session, and to check the SO version and the login flow. Its PASS line names
the login flow in use. The header status indicator says the same.
### The Hunts page spine (2026-09-19)
- **One list holds every analytic hit.** The band on the Hunts page showed shadow hits only, so
nobody could see what a real hit looks like. `GET /api/v1/hunts/hits` lists both halves from
the last seven days: the live hits first, newest first, then the shadow hits, unread first.
Each hit carries the analytic, its tier, the entity, the document count, the lead that holds
it and the lead's status. A live hit has no read flag and carries `read` null, so nothing
draws an unread dot on a hit nobody can read. The counts behind the filter chips are read over
the whole window, not over the returned page. `GET /api/v1/hunts/needs-you` answers the one
number the sidebar badge and the Needs-you strip show: the unread shadow hits plus the leads
that wait on a decision. `GET /api/v1/leads` takes two filters named by what the analyst must
do, `needs_decision` and `in_progress`. The existing shadow-hit routes are unchanged.
- **The catalog sweep writes the observation only.** A catalog hit made three records: a hunt
row of kind `triggered` with a report, a finding inside that report, and the observation. The
hunt list showed those rows beside the runs an agent made and the Hunt Console subtitle
counted them as hunts. The sweep now writes the observation, and the setting
"Record a hunt row for each analytic hit" restores the old write for one release. It is off by
default and applies live. The catalog hunt rows described in the older entries here are the rows
that setting restores. Rows written before this release stay in the database: the hunt list
leaves them out by default and `?kind=triggered` still lists them. A visibility-gap row is not
a hit and is unaffected, because a gap says the analytic could not see and silence there is a
false all-clear.
- **An approval keeps the hit that earned it.** Two rules met and cancelled a hit. A hit was
written shadow at birth and kept the flag for the life of the row. The hits surface then hid a
shadow hit whose analytic had left shadow. Approving an analytic from its own hit card took the
card off every hits surface, and no sweep could bring it back as a live hit. The flag on the
observation now records the status the analytic had at the latest sighting. An approval changes
no row, so the analyst keeps the shadow hit with its read state. The next sweep refreshes the
row as a live observation and the card reads as a live hit. Retirement is the one status that
hides a hit. A lead reads its shadow mark back from its observations, so a lead stops reading
shadow once its analytics are live.
- **The status chip reads the analytic's current status.** The hits payload took `analytic_status`
from the shadow flag on the observation row. That flag records the status the analytic had at
the latest sighting, and it holds until the analytic fires again. A hit recorded in shadow
therefore still read as a shadow hit after an analyst approved its analytic: the card wore the
shadow chip and offered Approve analytic on an analytic that was already live. `analytic_status`
is now the status the catalog holds, and a new `recorded_in_shadow` field carries the flag on
the row. The card takes the status chip and the two analytic decisions from `analytic_status`,
and the half, the border, the weight and the read state from `recorded_in_shadow`. A hit
recorded in shadow whose analytic is live carries a dim "recorded in shadow" chip. The halves,
the filters and the counts split on `recorded_in_shadow` as before, so the hit stays under Shadow
until the next sweep refreshes it. The one flag the entry above describes now answers only where
the hit sits, and no longer answers what the analytic is.
- **The bell counts what the page counts.** The bell wrote its own query for unread shadow hits.
It counted a hit from a retired analytic that the hits list and the Needs-you count both hide,
and it sent the analyst to a block that did not hold the card. One clause now answers "what is
an unread shadow hit" for the hits surface, the Needs-you count and the bell. The bell row and
the Notifications row link to `/hunts?hits=unread`, which sets the filter that holds the hit.
- **A lead says whether its investigation is still there.** `Lead.investigation_id` carries no
foreign key, so a deleted investigation leaves the id on the lead. A promoted lead then offered
Open investigation on a link that answered 404. `GET /api/v1/leads` and
`GET /api/v1/hunts/leads/{id}` carry `investigation_exists`. The list answers it in one query
for the page. The lead keeps its status, because an analyst promoted it and that decision holds.
- **An observation says whether its spec id names an analytic.** An alert verdict and a promoted
hunt finding are recorded under a spec id that names the adapter. The lead timeline linked every
observation to the Analytics tab, and those two opened a drawer that could not be read. The hits
payload, the observations payload and the lead timeline row carry `analytic_exists`. It is true
only when the id is in the catalog, so a renamed or removed analytic reads the same way.
- **The page says what each thing is, draws the flow, and folds.** The rebuilt page named a hit,
a lead, a hunt, a schedule and an analytic, and it said what none of them was. One dim line
under each section header now states it, and the lead page, the hunt page and the analytic
drawer carry the same sentence under their own titles. Each line ends in a link, "How this
flows", which opens the chart of the pipeline in a drawer: what an analytic finds, what forms a
lead, what starts a hunt, what ends in a verdict, and what waits on the analyst. The address
`?flow=1` opens the same drawer, and a tab change closes it. The four sections of the Hunts tab
fold from a chevron in their header. A folded section keeps its title, its count and its
definition, so it still says what it holds and how much. The fold is kept on that browser, and
a Needs-you link opens the block it jumps to before it scrolls there.
### Investigation turns (2026-09-19)
- **A setting lets the investigator write the report itself.** The investigation loop gathers the
evidence, then a second model call re-reads the transcript and writes the verdict. A turn audit
of 41 runs found the second call added no new evidence in 40 of them, cited nothing in 78% of
recent production runs, and reversed two correct verdicts on the range. With
`investigator_emits_report` on, the loop writes the report and the second call does not run. It
is on by default: on eleven range alerts with ground truth this path landed 31 of 31 correct
verdicts against 30 of 31, 42 s faster. Every gate still runs on the report either way: the
citation validation and cap, the confidence floor, the deterministic downgrades, the evidence
gate, the Oracle escalation and the session gate. The stored `triage_report` event names the
call that wrote it, `investigator` or `synth_round2`, so the two can be compared on real alerts.
- **The first-pass synthesis runs only when it can close the alert.** soc-ai ran a first-pass
verdict on most alerts, then ran the investigation loop and replaced that verdict. On the
production deployment the first pass ran 2,009 times and its answer was discarded 1,661 times.
It now runs only when a dispositive decision template has already cleared the alert. Every
other alert goes straight to the loop, which is the only path to a verdict on those alerts.
This removes about 18 seconds and 8,000 tokens from each affected run. The verdict is
unchanged: the loop never read the first-pass report. Turn on "Always run the first-pass
synthesis" to get the old behaviour back.
- **A run that stops early reads the evidence it gathered.** With the first pass skipped there is
no first-pass verdict to fall back on. A tool-call budget cut, and a concluding synthesis that
fails, now write the verdict from the tool results that did land. A run that gathered nothing
lands the honest failure report and stays retryable.
### Analytics, observations, leads and hunts (2026-09-18)
The Hunts page and the hunt catalog read as two products. One word did three jobs. This part
gives each noun one home.
- **Analytic.** One detection logic. The catalog reads two tiers: shipped analytics are YAML
files, local analytics are rows. A local analytic starts as a candidate. An analyst moves it
to shadow, approves it to live, or retires it. Every transition writes a version row with the
spec text before and after. A shipped analytic can only be retired.
- **Observation.** One thing one analytic observed about one entity, with a weight that decays.
Every source writes to one table: profile departures, catalog hits, triaged alerts (true
positive 1.0, needs more info 0.5, false positive not recorded), promoted hunt findings.
- **Lead.** Observations on one entity that are worth examination together. A lead forms at a
live weight of 0.85 across two or more types, or on a finding with no benign baseline, or on
one type that repeats. A lead spans the entities a hit names and merges through them, never
through a hub. A lead has a status: open, hunting, dismissed with a reason, promoted.
- **Hunt.** An analyst, a schedule or a lead starts a hunt. A lead hunt reads the documents
that formed the lead before it queries. The catalog sweep no longer writes hunt rows.
- **Shadow hits.** A shadow analytic's hit brings receipts: the matched documents, a 30-day dry
run, the overlap with live analytics, and the baseline for a profile analytic. A hit with
incomplete receipts reads "could not run" and names the missing part. Shadow hits are visible
in five places: a band on Hunts, the bell, the sidebar, a Dashboard card, and the hit list.
- **Screens.** Hunts has two tabs, Hunts and Analytics. A lead has a page with its timeline.
An analytic opens to its outcome ledger and its versions. The host page lists its
observations from every source. Operate's panel reads Analytics.
- **Agent.** The hunt agent runs a catalog analytic as a step with `t_run_analytic`. A starter
can name analytics to run first. A threat finding offers "draft an analytic", which writes a
candidate after a 30-day dry run.
- **Writing.** Every string a person reads follows Simplified Technical English. The model prompts
ask for the same style.
- **Vocabulary.** The word type replaces kind in every label, legend and hint.
- **Fixed on the way.** The production Oracle refused every escalation because the credential
redacter learned "n" and a shellcode dump as usernames. A hunt that timed out read
"Complete · 1 finding". A hunt row for a lead ignored the lead's own documents.
### The release, as first written
Proactive hunting. Everything up to now waited for a person: an alert arrived, or you typed an
objective. This release is soc-ai deciding what to go looking for.
### The case for it, measured rather than argued
On the Security Onion grid soc-ai is developed against, measured on 2026-09-04, the grid's own
sensors had produced about 3.4 million documents. **3,940 of them are tagged as an alert**: about
a tenth of one percent. Exactly **two** datasets out of roughly seventy ever produce one. In the
48 hours before the measurement, live telemetry produced 17.
That is not everything on the grid, and this entry as first published counted everything: 56,158
alert-tagged documents out of 23,055,409, a quarter of one percent. That denominator was wrong for
the argument. About 19.6 million of those documents carry an import marker, and 18.8 million of
them are a single Windows event-log import: something somebody loaded, not something a sensor saw.
The numerator was mostly imports too. The gap between 56,158 and 3,940 is alert documents that
carry the same marker, because a replayed packet capture raises Suricata alerts like any other
traffic. A census of what the sensors saw has to leave imports out, and the figures above do. Live
counts grow every day, so the share is the stable number and the totals are not.
So an analyst working the alert queue is working a tenth of one percent of what the grid saw, and
hunting that starts from that queue inherits the loss. A full credential-abuse chain of DCSync,
Kerberoasting and AS-REP roasting was run against a real domain controller. It produced no alert
anyone would have seen at the time. The Sigma rule for it had been enabled for months; it did not
fire because the rule engine tails the stream and its cursor had already passed those documents. A
query has no cursor.
### Added
- **A hunt can now be a document rather than a conversation.** A hunt spec is YAML that compiles to
one Elasticsearch query and runs with no model call at all. That is the whole point: hunting you
can afford to leave running, instead of hunting you pay for each time you ask. Four ship in the
box, covering DCSync, Kerberoasting, AS-REP roasting and decoy interaction.
- **Findings from a spec cannot be invented.** Nothing generative runs on that path. The title and
the explanation come from the spec's own text, written by a person and reviewed before it shipped.
- **The catalog can run on a schedule** (`hunt_spec_sweeps_enabled`, off by default), and a
condition it has already shown you does not come back. A finding fires once, not once per sweep,
which on a one-analyst SOC is the difference between a useful signal and another thing to ignore.
- **Run it in shadow first.** `soc-ai spec-sweep --shadow` reports what every spec *would* have
surfaced without recording anything and without spending the fire-once budget, so a week of
watching does not leave a spec silent on the day you switch it on.
- **soc-ai can tell "nothing happened" from "I could not see."** Every spec declares what telemetry
it needs. When that telemetry is absent the result is a coverage gap in your findings, never an
all-clear, and a spec whose grid read came back partial reports that it could not see rather than
reporting nothing found.
- **The Windows Security fields these detections turn on are now readable**, and classified for
redaction in the same change, so adding a field is one reviewed decision rather than two that can
disagree.
### Fixed
- **soc-ai could not see the busiest sensor on the network.** 632,523 documents carried their
identity in a field it did not read, so an entire family of network telemetry went missing from
the list of data soc-ai believes exists: the flow, DNS, TLS and HTTP records from the sensor
watching the live segments. Any evidence found there was discarded as unidentifiable. It now reads
the data-stream name as well.
- **A network sensor's hostname is no longer attributed to the machines it watches.** Two hosts with
no agent of their own were each reported as the router observing them, because a network document
names the box that shipped it rather than either end of the connection. Hosts with no agent were
the worst case and the reason it went unnoticed: for them the sensor's name was the only candidate.
- **The investigator is no longer told its answers change something they do not.** The prompt
promised that confidence would be capped if it reported its coverage honestly. Nothing had applied
that cap for some time. Both the promise and the field it asked for are gone.
- **A field the query language refuses is no longer readable another way.** One tool would return
the top values of a field the query language would not let you filter on.
- **The evaluation can now describe an attack that never became an alert.** Every scenario was
required to open with one, so the case this release exists for could not be written down. Four
scenarios now carry no alert at all, and the two types of scenario are scored separately, because
a scenario the triage harness cannot run should never have counted against it.
- **Model-fitness battery runs are audited again.** Every one of them had been failing to write its
audit record. The event kind was used at the call site and never declared, so construction raised
and a `try` that exists to stop an audit failure killing a completed battery swallowed it. The
battery ran; its trail did not.
- **An investigation kept working after it had the answer.** The loop stated its verdict on the
first turn and then spent two or three more turns confirming it. Those turns were 79% of the
time an alert took, and not one of them changed a verdict. The loop now stops as soon as every
evidence item carries a citation and one fact supports the verdict beside one fact that tried to
contradict it. A signature that claims malware still needs a tool result behind its verdict.
Tools that cannot answer are no longer offered: the playbook lookup is registered only on a
deployment that keeps playbooks, and the rule-text lookup only when the alert does not already
carry the rule.
### Known gaps, stated plainly
- **There is no health surface for the hunt catalog.** A CI gate proves each spec still matches its
own fixture; nothing yet says whether a spec still fires against a real grid, because no sweep
result is persisted. Tracked as issue #56, along with a related and worse one: on a grid
with no alerts to sample, the nightly quality evaluation writes nothing at all, so its trend
quietly stops growing and no surface says why.
- **Nothing yet knows what normal looks like.** The analytics measure cadence, entropy and rarity.
There is no peer-group or seasonal baseline anywhere, which is the next release's subject.
### Then: the catalog's own health
1.5.0 shipped hunting that runs without you and said, under its known gaps, that nothing could
tell you whether it was still running. This release closes that gap.
### Added
- **Every sweep leaves a trail.** Each spec writes one row per sweep, clean sweeps included, so
"ran and saw nothing" is a fact on disk rather than an absence in the log. Shadow and backfill
sweeps are marked as such and never count as a firing. Rows are kept per spec, so a catalog that
grows does not shorten the history of the specs already in it.
- **The Operate hub shows the catalog.** A panel lists every installed spec with its level, when it
last swept, when it last fired, and its 24-hour counts of fired, fresh and already-handled
conditions. A spec that is blind is marked amber; one that errored is marked red with the reason
on hover. The status line says whether sweeps are on, how often they run, how far back they look
and when the last one landed. Sweeps that are switched on but have not landed a row in two
intervals show amber, not green: the toggle being on is not the loop running. Backed by
`GET /api/v1/hunt-catalog`.
- **Catalog hunts are marked on the Hunts list.** A type badge (manual, scheduled, catalog) sits
beside each objective, and the list's presets filter by kind on the server
(`GET /api/v1/hunts?kind=`), so a catalog hunt that fell off the first page is still found.
- **The demo carries a week of catalog sweeps.** Four specs, one blind by construction, one that
fired two days ago into a catalog hunt, one transient error in the history, so the panel and the
Hunts list's Catalog preset show the feature working rather than four empty rows.
### Fixed
- **The quality trend could not say that it had stopped.** On a grid with no alerts to sample the
nightly evaluation wrote nothing, which was right, and then said nothing, which was not: the only
record of the attempt lived in process memory, the card showed the last point as if it were
current, and a month-old point looked like last night's. The trend now carries a freshness block:
when the newest point landed, whether the nightly is scheduled at all, whether the trend is stale
by two runs, and what the last attempt did and why. The card dates its newest point, names the
last attempt, and marks a stale trend amber. A run that wrote nothing is also logged as a warning.
The other half of issue #56, and it closes.
- **The CI leak gate had never run.** 1.5.0 said the check that keeps private identifiers out of
the publishable tree was in CI. It was, and the job image had no `git`, so it raised before it
scanned anything and every pipeline since had been red for that reason alone. The image now has
git, and the gate fails with a message that says what is missing rather than a traceback.
- **A failed Refresh from the dashboard's stale line changed nothing on screen.** The stale notice
was checked before the failed-refresh notice, so a Refresh that failed re-rendered the same line.
### Known gaps, stated plainly
- **The catalog trail is a table and a panel, not an alarm.** Nothing yet notices that a spec which
used to fire has stopped, the way the nightly triage evaluation notices a regression. That needs
weeks of rows to be meaningful and follows once they exist.
- **Nothing yet knows what normal looks like.** Unchanged from 1.5.0, and still the next release's
subject.
### Then: what a day against the range found
1.5.1 was tested by its own unit suite and a seeded loopback stack. This is what happened when it
met a live Security Onion grid and a browser.
The catalog itself held up. It fired on a genuine honeypot interaction the grid had not alerted
on, reported the same condition once rather than on every sweep, and every number its read-model
publishes recomputes exactly from the rows underneath. What follows is everything that did not
hold up.
### Fixed
- **The 1.5.0 notes measured the alert queue over the wrong denominator.** They said 56,158 of
23,055,409 documents on the grid were alerts, a quarter of one percent. About 19.6 million of
those documents were imports, 18.8 million of them a single Windows event-log import, and most of
the 56,158 alerts carried an import marker too. Over what the sensors actually produced, the share
is about a tenth of one percent. The 1.5.0 entry, the README and both roadmaps now say so.
- **The DCSync spec could not see Get-Changes-All.** `identity-4662-dcsync-nonmachine` matched one
replication right, DS-Replication-Get-Changes. A domain controller writes one 4662 per right it
checks, and a later reading of the range found an attacking-account document carrying only
DS-Replication-Get-Changes-All, the right that releases secrets, which that clause could not match.
The spec now matches any of the three replication rights (Get-Changes, Get-Changes-All,
Get-Changes-In-Filtered-Set); the machine-account exclusion, which carries all of the precision,
is unchanged. The fixture plants each right alone, and a new CI gate requires every document from
the expected actor to match on its own, because the journey score cannot see one missed document
while a sibling carries the candidate. The range counts quoted in the spec were measured with the
narrower clause and are now dated as such.
- **The panel said "Sweeps off" while the catalog was sweeping and firing.** The cadence, the
look-back and the time of the last sweep were built only in the branch that renders when the
scheduler is on, so an operator running sweeps by hand, which is what the panel's own hint tells
them to do, saw a line that read as nothing running while the counts moved beside it.
- **A row could not say when its spec was last swept.** The time was on the wire and the row used
it as a yes or no. The critical DCSync row read the same whether it was checked ninety seconds
ago or last week, which is the one question this release exists to answer.
- **A shadow sweep made a spec look broken.** Shadow runs count toward fresh and never toward
fired, by design, and the panel tells a new operator to run shadow first. So an ordinary new
install read "fresh 2, fired 0" with nothing to explain it. Rows now say how many of their
sweeps were shadow, and the counts name the window they cover on screen, not in a tooltip.
- **A catalog hunt showed 0.00 confidence in large type.** These hunts make no model call, so the
absent value was being coerced to zero on the way out and rendered as a verdict. The detail
endpoint now preserves the absence the way the list endpoint always did, and the page shows no
dial at all. A model that genuinely scores 0.00 still shows it.
- **A finding mixed what matched now with prose written months ago.** Counts measured when a spec
was authored sat in the same paragraph as this run's result, so they read as fresh measurement
and would quietly go false as a grid grows. They are now labelled as the spec's own rationale.
- **The type filter on the hunt list was invisible to the address bar.** It reset on reload, could
not be linked to, and the Operate panel's own link therefore landed on the unfiltered list.
- **`soc-ai spec-sweep --shadow` did not run.** The panel and the config console both print that
exact command and it exited with an argument error, because the window was required and neither
hint mentioned it. It now defaults to the configured look-back.
- **The scheduler and the command line disagreed about what one sweep covers.** Both widen a
window narrower than the interval, but only the scheduler applied a floor to the interval and
said so in the log. One function now computes it for the scheduler, the command line and the
API, and the API reports the window a sweep will actually use rather than the raw setting.
- **No shipped command could see a planted evaluation scenario.** The four no-alert fixtures exist
to prove the catalog finds attacks that raise no alert, and until now only a test could use
them: `spec-run` and `spec-sweep` never passed the flag that reads the synthetic index. Both
take `--include-synth` now, off by default, and a hunt recorded under it is marked as one.
- **Planting a scenario with nothing to triage wrote every document and then refused.** The check
ran after the loop. It now runs first, and says nothing was planted.
- **Two empty states named the wrong thing**, one of them pointing at a control that would not
have helped, and a catalog hunt's row showed its spec id where its title belonged.
### Known gaps, stated plainly
- **Two of the four shipped specs are no longer detecting what nothing else sees.** Security
Onion's own rules for DCSync and Kerberoasting now fire on the same grid. The catalog's unique
contribution today rests on AS-REP roasting and the decoy. Endpoint antivirus verdicts look like
the strongest addition: on the range they are the only telemetry that caught a credential-dumping
tool running on the domain controller, and like the decoy they need no threshold.
- **A grid census that counts documents does not ask what a document is.** On the range, one
imported file split into millions of lines dominated every count, and soc-ai's inventory reads
those indices the same way the census did. Not fixed in this release.
### Then: a fortnight of living with it
Nothing above had been run against a real deployment for longer than an afternoon. Between
2026-09-05 and 2026-09-14 it was: the home deployment against its own Security Onion, and the
attack range against its exploit chain, with every read and write exercised the way an analyst
would. That found more than the three iterations before it. The largest were a denominator and a
gate. Every prevalence, novelty and baseline figure had been computed over everything on disk, and
on the development grid 85% of that was imported packet captures, a baseline of somebody else's
network. Every population statistic now counts live telemetry only and says so. And the catalog,
which is where a detection's doctrine lives, could not be read by a triage: the same real DCSync
alert closed false positive one day and escalated true positive the next, both runs grounded,
because triage never learned that directory replication by a non-machine account has no benign
population to compare against. A spec can now declare that, and a triage that closes such a
detection benign on a volume argument is refused.
The rest is below, as it was written at the time. A recurring theme: a number that said one
thing and counted another, a surface that rendered the same whether the thing behind it was
healthy or dead, and a warning that could not be cleared. The home deployment's "pipeline errors
that never heal" and its recurring quality alarm turned out to be four distinct defects, each
fixed.
#### Fixed
- **A count nobody could take was reported as a count of zero.** The alert queue's "N
acknowledged" chip comes from a filter aggregation on `event.acknowledged`. Elastic Defend
writes its alerts to an index Elastic's own package maps `dynamic: false`, without that
field or `event.escalated` beside it, so Security Onion's flags land in `_source` where no
query reaches them and the aggregation answers 0 whatever the truth is. Measured on a live
SO 3.2.0 grid on 2026-09-06 against a 16-event endpoint group: all 16 acknowledged, SO
answered 200 each time, and a query for the flag still matched none of them. The row said
"untouched" about a group an analyst had cleared, every single time.
The write path already refuses to trust that query and reads the flag off each hit. A
per-group count has no such option: a group can hold thousands of events and the console
will not page them to draw one chip. It does the other honest thing instead. The aggregation
now resolves each group's datasets alongside its counts, and a group holding documents on
an index that cannot store the flag reports the count as unknown rather than as zero. So
does a group whose dataset list came back truncated, because past the cap a blind dataset
cannot be ruled out. A mixed group counts as unknown too: the number the aggregation
returns there is a real undercount of unknown size, and an undercount reads exactly like a
complete count.
The row renders that as a faint chip carrying a question mark and the reason, next to where
the green check would have been, and points at the expanded group. Each event carries the
flag in its own document, which is the one place the answer still exists. A grid that
answers is unaffected, including the ordinary case of a group with nothing acknowledged:
zero is a real answer on an index that maps the field.
- **`/healthz` said `ok` about things it had never looked at.** The container healthcheck
polls it and Docker prints the result beside the container as the word *healthy*. That word
was read as a verdict on the product by people whose grid was unreachable, whose model
gateway was down, or who had no analyst model configured at all. The endpoint had probed
none of that. It never should: a liveness probe that failed on a dependency outage would
have the orchestrator restart a container whose dependencies are merely down, turning
somebody else's outage into a restart loop and taking away the surface that could have
explained it.
So the logic is unchanged and the wording is not. The response is a `LivenessResponse`, its
status reads `alive` rather than `ok`, and it carries a `checks` field saying in plain words
that nothing was probed and naming where the verdict does live (`GET /api/v1/health`, or
`soc-ai doctor`). That sentence is on the wire and not only in a docstring because the two
places this body is read are a paste of `curl` and `docker inspect`'s health log. The
Dockerfile and compose healthchecks, `setup.sh`, `soc-ai healthz --help` and the deployment
and Docker guides all say liveness where they used to say health, and each points at the
doctor for the rest.
- **The verdict-quality alarm had no way to reach a human.** Its two channels were an audit
record, which nobody reads unprompted, and the notification webhook, which is off by
default and stays off on any install that has not opted into egress. So on a stock
deployment the Quality card on the dashboard was the only surface where a regression in the
verdicts this product exists to produce could be learned about. It is a card you have to
already suspect something to go and look at.
It is now a standing bell entry, on the same terms as every other standing condition here
(a dependency down, a broken audit chain, a dossier conflict): read from the newest
snapshot only, one indexed row, fail-soft, and never a probe on an endpoint polled every
fifteen seconds. An eval that crashed is not reported as a verdict regression. "The grader
could not run" and "the verdicts got worse" need different responses, and the bell separates
them the way the card already did.
The entry's id is the identity of the finding: the alarm's rule codes and the moment the
condition started. It is not the id of the run that noticed it. The writer holds that start
steady for as long as the condition persists, so one dismissal covers every night that
re-observes the same condition, while a different code, or the same code raised again after a
clean night, arrives undismissed. It is the id the Quality card was already minting for its own
dismiss control, so clearing the alarm in either place clears it in both.
- **Ten nullable JSON columns stored an absence as the text `null`.** SQLAlchemy's `JSON`
type serialises a Python `None` through `json.dumps`, so it lands as the two-byte string
`null`. The column holds a JSON value where SQL NULL belongs. It reads back as `None`, which
is why it survived: every assertion in the suite is spelled `row.col is None` and every one of
them passed. SQL says the opposite of the truth. `WHERE col IS NOT NULL` matched every row in
ten tables; `WHERE col IS NULL` matched none.
Nothing was visibly broken, because every reader coalesces with `or []`. That is what made
it worth fixing: the columns were a loaded trap for the first query that asked whether a
lane held anything, and one subsystem had already stepped in it. A cleared dossier override
stored that way would have read as still held forever, which is why the dossier store grew a
per-statement workaround for three columns and never covered the two beside them. The fix
now lives on the column type, where it cannot be skipped by a statement someone forgot to
route through a helper.
Existing rows are normalised by the migration rather than left alone, because a half-normalised
table is a worse trap than a uniformly wrong one: the query that motivated the fix would work on
this month's rows and silently skip everything older. The repair cannot take a legitimate value
with it. A stored string `"null"` serialises with its quotes, so only the defect matches. The
downgrade does not put the old representation back: a rewritten NULL and an original NULL are
the same value, and restoring one would fabricate the other.
- **A quality snapshot could not say what was running when it was measured.** The nightly
trend recorded agreement, fallback, errors and latency, and nothing at all about the
instrument, so a bend in the line had nowhere to point. The published image is tagged
`latest` and every build of one release carries the same version string, which means the
version alone cannot tell two builds apart either.
Each snapshot now carries three things. `app_version` and `code_commit` say which build ran;
the commit is baked into the image at build time, because `.dockerignore` excludes `.git`
and a container has no other way to know. `analyst_model` says which route the verdicts came
out of. The failure this whole subsystem exists to catch is an inference-engine swap or a
model bump, and neither of those changes a line of code. All three are visible on the
Quality card and on `GET /api/v1/quality/trend`, and `/healthz` reports the commit too, so a
bug report from a container install can name its build.
Unknown stays unknown: an image nobody stamped records no commit rather than a guess, and
rows written before this change are left as they are. A wrong commit is worse than a missing
one when the entire point is attributing a regression to a change.
- **An empty threat-intel feed read as a clean answer.** The blocklist feeds have never been
refreshed on either deployment and the data directory does not exist, so `BlocklistDB`
loads nothing and every lookup misses. The only signal was a warning in the process log:
the enrichment result carried no field saying so, which made "we checked and found
nothing" and "we could not check" the same object. The timeline rendered that object as
"no blocklist/MISP match" and three triages leaned on the phrasing to support a false
positive.
`IndicatorEnrichment` now carries `blocklist_sources`, the feeds that actually answered,
and reads it back as `blocklist_checked` with three states rather than two, because there
are three: unrecorded, recorded-and-nothing-loaded, and recorded-with-feeds. Collapsing the
first two would have relabelled every archived enrichment as unchecked, which is its own
false statement, so only an explicit "nothing loaded" changes any behaviour. The lookup
also appends a line to `errors`, the same way the cloud tagger already reports that IPv6 is
uncovered, so the model reads the distinction in prose as well as in a field.
Every renderer of the empty case reads it. The timeline outcome says the blocklist was not
loaded instead of reporting a miss, on an internal address too, because the curated
internal-seed feed is the one that names known-bad internal hosts. The entity graph note
says its unflagged nodes were never checked. The prefetch line stops implying that
"enriched N indicators" means N reputation checks. The synthesizer gets a citable coverage
bullet in the same shape as the endpoint-coverage gap, worded as neither exoneration nor
guilt. The `clean_internal_traffic` template keeps its two locality grounds but no longer
writes down a reputation check that did not happen, and `informational_external_unknown_asn`
stands down entirely: its only ground beyond informational-and-allowed IS the reputation of
an external address, so with no feed it has none. The synthesizer rubric no longer names an
empty blocklist as positive evidence without qualifying it.
- **The field that answers the first question in Windows triage was not queryable.** A hunt
asking whether an authentication was Kerberos or NTLM failed four queries with
unknown-or-forbidden-field errors. `winlog.event_data.AuthenticationPackageName` is the
field the range's own operating documentation names for separating management traffic from
real users, and it is populated: counted over fourteen days on the range, Negotiate 20614,
Kerberos 19984, NTLM 526, all on `system.security`. It is now queryable, along with
`LogonProcessName`, `LmPackageName` and `winlog.logon.type`, the readable alias for the
numeric logon type that was already admitted. Each is enumerated individually, never as a
`winlog` prefix, for the same reason the existing entries are: a prefix admits several
hundred leaves in one line, many of them carrying identifiers no redaction route
classifies. Their values are public Windows constants but word-shaped, so unlike the GUIDs
and hex masks already admitted they do not pass the egress value check on their own; each
is allowlisted by path in the Oracle backstop so a Kerberos-versus-NTLM finding is not
adjudicated with the package itself redacted.
- **A contains query on a keyword field matched nothing and said so quietly.** `field:~value`
compiled to an Elasticsearch `match`, which is right on analyzed text and useless on a
`keyword` field: the keyword analyzer emits the whole value as one token, so the query
degenerates into an exact full-value comparison. `process.command_line` is keyword-mapped.
Measured on the range on 2026-09-07 over seven days, all four counts in one aggregation:
a `match` for a known command matched 0 documents, the analyzed `.text` sibling matched
405, and the leading wildcard the validator refuses as too expensive also matched 405.
A bare zero on the process command line is an absence a model reads as innocence.
The substring search is not expensive here, so refusing it would have been the wrong kind
of honesty: Elastic's Windows and endpoint integrations map an analyzed `.text` sibling
alongside the keyword field, and the answer was one field name away. `:~` now asks both
names in a single `multi_match`, phrase-typed when the value was quoted. The named field
is never dropped, so a grid that maps the field as analyzed text with no sibling behaves
exactly as before, and an unmapped sibling scores nothing rather than erroring. The set of
names that can reach Elasticsearch stays the whitelist plus one fixed suffix per entry,
because validation still runs on the field the caller wrote. The leading-wildcard refusal
stands and now names the form that works.
- **Every "in the last N hours" question was answered over N divided by two.** The event
query tool centres its window on the alert, so a 1440-minute request became twelve hours
before the anchor and twelve after, and for a live alert the forward half holds nothing.
Measured on the range on 2026-09-07, anchored on a real alert: a grid-wide 24-hour count
came back 504 where the grid held 1061, and a one-host 48-hour count came back 81 where
the grid held 113. Counting the grid over the effective half-windows reproduces 504 and
81 exactly, so the window was the cause rather than a coincidence. The tool's description
did say the window was centred, but for a live alert that is never what the caller means,
and nothing checked the arithmetic afterwards: the citation validator resolves document
identifiers, not numbers.
These are two questions, so they now take two forms rather than one parameter that has to
mean both. `window_mode="around"` is the default and is unchanged: the window stays
centred, which is the right shape for "what happened around this alert", where the minutes
before it are the setup and the minutes after are what followed. `window_mode="before"`
puts the whole requested span behind the anchor, which is what a how-often question means.
With no anchor at all there is nothing to centre on and both modes look back from now, as
they already did. The result also carries a `window` block naming the mode, the two bounds
and how far the window reached either side of the anchor, on the same reasoning as
`counted`: a number handed over without its span gets the span the reader assumed. The
Zeek pivot keeps the centred window with no mode at all, because it follows one flow and
answers no prevalence question.
- **Every agent in a container ran without the query-language reference, and nothing said
so.** The system prompts for triage, hunting and chat are assembled from
`docs/OQL_PRIMER.md`, resolved as parent-of-package. The Dockerfile copied `soc_ai/`,
`pyproject.toml`, `runbooks/`, two demo scripts and the built SPA, and never copied
`docs/`. Measured inside the running container on 2026-09-07: the primer path did not
exist and the prompt block was 971 bytes, all of it the fallback stub, whose text is
"Primer file missing on disk; OQL is unavailable." So every investigator, hunt and chat
prompt shipped a sentence telling the model the query language was unavailable in place
of the field reference and the worked examples. The hunt flavor lost its examples twice
over: the splice keys on markers the stub does not contain, so it never even read the
hunt file, which was equally absent.
Nothing surfaced it in any direction. The `FileNotFoundError` branch was annotated as a
development-only safeguard and logged nothing. The test that checks the primer's splice
markers reads the repository, which always has the file. No doctor check looked at a
prompt asset, which is how the doctor reported fifteen passed and zero failures while it
was true. The development range runs from source under systemd and has the file, so a
week of dogfooding against the range could not have caught it and did not.
`docs/` is now copied into the image WHOLESALE, the same way `soc_ai/` and `runbooks/`
are, because naming the two files individually is the mistake that produced the defect:
the next prompt asset added under `docs/` would silently not ship. `docs/img` and one
internal subtree are excluded in `.dockerignore` instead, about 10 MB of screenshots and
vendored guides that nothing reads at runtime.
An audit of every path the application resolves outside its own package found exactly
these two files missing; `runbooks/`, `frontend/dist` and `pyproject.toml` were already
copied, and everything else a prompt or a tool reads lives inside `soc_ai/`.
The absence is now loud in three places, because it was silent in all three. The app
refuses to start when a declared prompt asset is missing, before the store, any client
or any request: the standing rule that a false all-clear outranks a crash is about
upstreams, where an honest degraded answer exists and can be handed to the analyst, and
here there is none. The verdicts keep arriving and keep looking like verdicts. A missing
file is knowable with two stat calls at boot, where an operator is already watching,
rather than at the moment an analyst is trusting the answer. `soc-ai doctor` grows a
`prompt assets` row that names which assets are present, which are missing, and what the
absence costs; it FAILs rather than WARNs, since the WARN band is for things that
degrade gracefully. And the fallback stub, which stays so the CLI and the doctor can
still run inside a broken deployment to report it, now logs at ERROR every time it is
produced.
The regression gate replays the Dockerfile's own COPY directives into a scratch tree and
imports the package from there in a subprocess, so it grades the deployed layout rather
than the checkout. Real copies, never symlinks, since `Path.resolve()` follows a symlink
back to the repository and would hand the checkout's answer back under a container's
name. Its negative control drops the one COPY that carries `docs/` and asserts the
defect reappears.
Not a claim about accuracy, but the reason to look: on the deployed instance
`t_query_events_oql` has run 3,870 times and 2,132 of those calls (55%) returned zero
hits, with 173 more rejected outright. Every one of those calls was made by a prompt
built from the stub, so the recorded history contains no primer-present period to
compare against and cannot, on its own, attribute the rate to the missing file.
- **`rule_prevalence` printed a rate over the observed span under the words "while active".**
Measured against a live grid: a rule with 438 fires on 17 of the 30 days it spanned reported
`fires_per_day=14.612, rate_basis='observed_span'` and a summary reading "about 14.612/day
while active". 14.612 is 438 divided by the span. While active it is 25.765. A second rule
reported 8.555/day "while active" where the figure on its active days was 13.9. The number had
been moved off the lookback window and onto the observed span, which was the right denominator,
but the sentence kept its old adverb and so named a denominator that had not been used.
Both quantities are real and they answer different questions, so both are now reported, each
under its own name. `fires_per_day` is unchanged: fires over the span, with `rate_basis` still
saying so. `fires_per_active_day` is the new field, fires over the days the rule actually fired
on, and the summary prints the span figure as "across that span" followed by the active-day
figure as "on the N days it fired". A rule that fired on every day it spans has one rate, not
two, so the second clause is dropped when the numbers agree rather than printing the same
figure twice.
- **`burst_fires_per_minute` divided the burst by the window that held it.** 335 fires clumped
onto 4 days of a 25.4-day span came out as 0.01 fires per minute, and went into the JSON the
model reads under a name that says burst. It is the same window-over-burst error that had just
been taken out of the daily rate, alive one field along: when the fires sit on a handful of
days inside a long span, most of that span is silence, and dividing by it describes the gaps
rather than the episodes.
The shared burstiness test now reports `fires_fill_span`, and the per-minute figure is emitted
only when the observed span is the episode. A day histogram has no sub-day timing in it, so
there is nothing honest to put in its place and nothing is offered; `fires_per_active_day`
carries the magnitude instead, and the summary says both that the span is mostly silence and
what the rule did on the days it fired. The one-episode burst is unchanged: 1531 fires in 59
seconds still reports 1560.2 a minute.
- **`rule_prevalence` called a rule it had never queried "first-seen".** The query was pinned to
`suricata.alert`, so every Sigma rule, Zeek notice and endpoint rule came back
`observed: false, total_fires: 0, noisiness: "first-seen"` regardless of how often it had
fired. On a live grid the Sigma rule "Security Onion - Grid Node Login Failure (SSH)" had
fired 48 times over seven days and reported as never seen; a Zeek capture-loss notice with 841
firings reported the same. The narrow scope was deliberate and the module docstring said so.
The output did not: "first-seen" is a claim about the network, and the reading a model takes
from it is the exact opposite of the truth.
The query now covers every dataset that carries a detection (Suricata alerts, Sigma alerts,
Zeek notices, endpoint alerts) and resolves a rule name across `notice.note` as well as the
ECS and legacy fields. It stays an allowlist rather than dropping the dataset filter, because
`windows.sysmon_operational` populates `rule.name` with Sysmon's operator-supplied config tag,
which is not a detection. Comparability is handled by naming the source instead of narrowing
the search: a rule name belongs to one engine in practice, `fires_by_dataset` breaks the total
down whenever it does not, and the summary then says outright that the total pools sources
whose firing rates are not comparable. The noisiness thresholds were set against IDS volume, so
the summary names the source the count came from. When nothing matches, `searched_datasets` and
the summary both say where the tool looked and that it did not look anywhere else.
Widening surfaced the same defect shape one field over. Sigma and Zeek docs keep their
addresses outside `source.ip` and `destination.ip`, so the cardinality aggregations returned 0
and the summary read "0 source hosts / 0 dest hosts" for 841 real detections. Zero distinct
hosts over a non-empty match is a missing field, not a measurement, so those counts are now
`None` and the sentence says the docs carry no address. The `noisy` bucket needs host spread to
be measured; where it cannot be, the bucket is withheld rather than awarded on an untested
condition, and the summary says which test could not be applied. Where spread is measured
nothing changes.
- **The host address on an alert row was the shipper's, not the host's.** On a host-shaped
detection Security Onion nests the originating document under `event_data`, and the endpoint's
own address sits at `event_data.host.ip` while the address of whatever forwarded the log to the
grid sits under the beats input metadata. The resolver read only the beats path. On the range's
DCSync detections that meant the row named `sr-dc01` and showed the log-forwarding host's
address, and an investigation pulled a host dossier for the forwarder believing it had the
domain controller. The neighbouring host *name* resolver already preferred the nested field,
which is how the name and the address on one row came to describe two different machines.
The nested endpoint address is now tried first and the beats path kept as a fallback, since on
an agent that ships its own logs the two are the same machine. That field is an array of every
address the host claims, in interface-enumeration order, and it lands on the row as a live pivot
to `/entity/<ip>`, so the first entry is no longer taken blindly: loopback and link-local are
skipped when anything else is on offer. One endpoint on the measured grid reports a routable v4,
a link-local v6 and two container-bridge addresses. Nothing beyond those two is ranked, because a
bridge address is still an address that host answers on.
- **A planted evaluation fixture reached the live alert queue as a critical detection, and the
guard that let it in then hid the evidence needed to recognise it.** The synthetic kill switch
excluded documents carrying a scenario marker at the top level of the document. Security Onion's
Sigma pipeline re-nests the whole originating document under an `event_data` envelope, so the
marker lands one level down and the exclusion missed it. Measured on the range on 2026-09-06:
56,162 alert-tagged documents survived the exclusion and 2 of them carried the marker in the
nested position. Those two were a planted DCSync fixture, and they presented as a critical Sigma
alert in the queue.
The chain compounds from there, and the second half is the worse one. The underlying event does
carry the marker at the top level, so it is excluded from every query the agent can run. A triage
of that alert made fifteen tool calls, got zero results from every corroborating query, exhausted
its budget, stated that the account's purpose could not be independently verified, and issued
false positive at 0.60 anyway. The absence was the guard's, not the network's, and nothing in the
run could tell the two apart. Because the group verdict applies to the whole group, three genuine
DCSync events on the real domain controller then read "inherited, same detection" under a
false-positive badge.
Two changes. The exclusion now names every position the marker is known to occupy, top level and
under a detection pipeline's envelope, and is built in one module that every read path calls; the
alerts queue, the dataset inventory and the two host-dossier passes each carried their own copy
naming the top level only, and now do not. Adding a pipeline is one line. A wildcard field
expansion would need no list at all and finds the same two documents, but it costs 219ms against
9ms on the same query and returns no clauses and no error when it matches no mapped field, which
is the same silent nothing this guard exists to prevent.
And an investigation whose own subject the guard hides now refuses instead of investigating. The
anchor is fetched by document id, so it is the one read in a run that does not pass the exclusion
clauses, while every pivot after it does; that combination cannot produce evidence, only zeros.
The refusal happens before the first pivot, names the scenario, and yields no verdict at all
rather than a low-confidence one, because a disposition on a fixture is still a disposition and
still counts. The same check catches a batch-eval run handed a sibling scenario's alert id.
Negative control, measured on the same grid: the three genuine DCSync detections on the real
domain controller carry no marker in either position, survive the widened exclusion, and are
investigated unchanged. The `suricata.alert` class did not move (56,107 before and after); the
`sigma.alert` class went from 55 to 53.
- **A shellcode rule was being acknowledged unattended, and the classification table already
knew why.** Measured on the deployed instance: 156 unattended acknowledgements across two
`GPL SHELLCODE` rules, three of them after the classtype description fix went live and the
most recent today. All four arms of the high-stakes guard missed the alert. Its Suricata
category, "A system call was detected", normalizes correctly to `system-call-detect`, and
neither the routing map in the classifier nor the attack-classtype set the escalation guard
uses had an entry for it; "shellcode" is not one of the malware signal tokens; and the
severity, medium at score 2, is under the severity arm.
The hole was not a missing token, it was a table with a silent default. The classification
table enumerates all 43 of Suricata's classifications and the routing map had an opinion about
20 of them; the other 23 fell through to "unknown", which reads as no opinion and spends as
safe. The map now covers the same ground as the table: every classification is either routed
to a class or named in an explicit unrouted set, and a test fails if a new one arrives with
nobody having decided what it means. What makes a rule high-stakes is therefore the
classification its author declared, read through the one table the codebase has, rather than a
substring found in the rule's name. The token list stays where it earns its keep, on alerts
that carry no classification at all, and is deliberately not where new signals get added: it
also drives benign-template routing across the whole decision layer, so a token added to close
an auto-ack hole would change verdicts for every rule whose name contains it and would still
leave the next unrouted classification open.
Negative control: the six classtype values actually measured on the production grid, in the
description form the sensor writes, are still eligible for unattended acknowledgement. A
widening that switched the feature off would be a different defect, not a fix.
- **One call site kept comparing a raw classtype, so an exemption could never fire.** The gate
that downgrades a true positive resting on nothing but the host's alert history has an
exemption for the case where the focus alert is itself attack-class. It lower-cased the raw
classtype and tested it against a set of shortnames, and the field carries Suricata's
classification description, so on live data the comparison never matched. The consequence is
the inverse of the auto-ack hole above and comes from the same root cause: the exemption never
fired, and genuine attack-class true positives were downgraded to needs-more-info. The
comparison now runs through the same normalizer every other classtype comparison uses, and a
sweep found no other site reading the raw field for anything but display. Two assertions in
the quality-spine suite that compared the raw value on synthetic fixtures now normalize too:
the scenarios render shortnames, so those assertions passed while standing for code that
could not match a sensor.
- **The daily audit-chain alarm reported one break out of many, and could not be cleared.** The
scheduled verification ran and correctly found a break. It named a single sequence number. The
window it scanned held 41 distinct duplicated sequence numbers across 51 extra records, some
positions claimed by four writers. An operator could not tell a single collision from a
widespread one, and could not see the fact that mattered most: all 41 fall before the fix that
stopped the forking, so the honest reading is that the damage is historical and bounded.
Every channel now carries the blast radius as well as the first break: how many positions are
claimed more than once, how many records that is beyond the first at each, the most writers at
any one position, how many records no longer match their own hash, and the oldest and newest
timestamps among the records involved. That last one is what says whether anything has broken
since. Counted per epoch, because sequence numbers restart at zero on every process incarnation,
and only over the epochs that actually failed, since an intact one censuses to zero by
construction. The same sentence goes to the bell, the notification webhook and
`soc-ai audit verify`, so the three cannot disagree.
The alarm also could not be dismissed. It lived only in application state and the bell entry's id
embedded the detection timestamp, so every run minted a new identity: an undismissable danger
notification every day until the forked stretch aged out of the seven-day window, which was
several days away. The id is now the identity of the FINDING, in the same shape every other
standing alarm here uses. What goes into it is the safety argument: the break types present, so
dismissing a known historical fork can never suppress a record being edited; the newest record
involved in any break, so anything that breaks afterwards re-raises; and the count of records
that no longer match their own hash, so a further alteration re-raises even when the type is
already showing. The duplicate counts are deliberately excluded, because a rolling window sheds
old records every day and keying on them would re-raise the same scar every morning.
A finding that includes an altered record is not dismissible at all: it carries no dismiss
control and "Clear all" steps over it. A duplicated position is what a second writer leaves
behind and an operator can reasonably acknowledge a bounded historical scar; a record whose
content no longer matches its own hash is someone changing the record of a decision, and one
click should not be able to sweep that away. None of this touches the reporting: the audit record
and the webhook still fire on every run for as long as the chain does not verify.
- **Sigma and host detections reached the agent with every pivot field empty, and one benign
verdict then silenced the rule everywhere.** Security Onion's Sigma pipeline does not merge
the document a rule matched into the alert it writes: the detection's identity goes at the top
level and the whole originating document is nested under `event_data`. `SoAlert.from_es_hit`
read only the top level. Measured on 55 `sigma.alert` documents from a live grid, not one
carried a top-level source address, destination address, host name, host address, user name,
process id, file hash, community id, event action, event category or message, while 48 carried
a nested source address and 53 a nested host name. The alert queue already knew about the
nesting and worked around it for the host column alone.
The model and the queue now read through one shared envelope reader. Precedence is
top-level-wins, decided on the three fields that are present at both levels: `event.module`,
`event.dataset` and `tags` disagree on every one of those documents, because the top level
names the Sigma detection and the envelope names the log it fired on. Letting the envelope win
would relabel every Sigma alert as the dataset it matched, so the fields that name the
detection are not unwrapped at all and only the fields that describe what happened are.
The worse half was the clustering. Auto-triage keyed each cluster on rule plus both addresses,
a missing address degraded to the empty string, and the result was that every address-free
detection of a rule shared one key, for all time, on every machine. On the same grid 100 of 208
investigations carried that key, and one Sigma rule held a false positive and a true positive
under it at once, so whichever landed last answered for the other everywhere. The key now
carries the host, but only when both addresses are empty: a flow seen by two sensors carries
two host names and must not split, while a detection with no flow has no other subject. When
even the host is unknown the key names nothing at all, and a verdict is no longer handed along
it. Every legacy row keys there, so none of them inherits onto anything any more; the
in-flight duplicate guard is untouched, because a coarse stop is safe where a coarse verdict
is not.
- **One tile, three denominators.** "True positives · 24h" was labelled for a 24-hour window,
counted detection groups, and carried a subtext counting investigation runs over thirty days.
Read in two tabs at the same minute on 2026-09-07 the tile said one and the investigations
list said three, and nothing on the tile said the two were counting different things: one
detection group with a standing true-positive verdict against three true-positive runs.
Every number in the tile now names what it counts. The headline figure reads "1" over "of 3
detection groups", which is the denominator the neighbouring Events tile already uses, so a
reader can see immediately that it is not a count of runs. The needs-info clause sits beside
it because it counts the same groups over the same window. The pipeline-error clause moves to
its own line carrying its own unit and window, "runs, last 30d", because it is neither.
- **A deployment with authentication off gave the same caller three different identities.**
`GET /api/v1/me` invented "analyst" so the sidebar had a name. Every write recorded
"anonymous". Saved views refused outright. So assigning an alert group to yourself stored an
owner the identity endpoint did not recognise: the row grew an avatar and an owned chip, and
the Mine filter stayed at zero and always would. Both names render "AN" as initials, so
nothing on screen gave it away. The feature did nothing while appearing to succeed.
The two halves get different answers, because they are different problems. Ownership works
once the caller has one identity: `/me` now reports the same name every write on that
deployment is recorded under, so claiming a group and filtering for it agree. Saved views
genuinely need a user row, which an unauthenticated deployment has none of, so they still
refuse. What changed there is that the refusal is now visible. `/me` carries `signed_in`,
false for a token caller and for an open deployment alike, and the four list screens render a
disabled "Save view" control with the reason beside it rather than nothing at all. Rendering
nothing is what taught an analyst the feature did not exist.
A deployment WITH authentication behaves exactly as before: the signed-in user is reported as
themselves, ownership records their username, and saved views work.
- **The pipeline-error tile promised a count the list it opens could not reproduce.** The tile
counted runs after excluding the ones an operator had dismissed and the ones a later run had
already superseded. The list behind its link applied neither exclusion and offered no filter
or marker for them. Watched live on 2026-09-07 the tile read nine, then eight, then seven
while the list sat at twenty, and a run dismissed seconds earlier rendered exactly like a
counted one. Twelve of those twenty were superseded and three were dismissed, so the gap was
most of the list.
`GET /api/v1/investigations` takes an `error_state` of `live` or `handled`, and the deep link
carries `errors=live`, so both surfaces read the same number off the same query. One query
cannot make the split, because "superseded" is a fact about an alert's whole run group and not a
column on the row, so the server decides it per row over one capped read and counts the
partition it returns rather than the wider filter set. When that read is not the whole set the
response says `partial`, and the tile renders its count as a floor instead of reading as exact.
The list grows a three-way control naming which half is on screen, defaulting to the half the
tile counted and one click from all of them, and every dismissed or superseded row is marked
as such wherever it appears. An unknown `error_state`, like an unknown verdict, is dropped
rather than refused, so a mangled link widens the query instead of wedging the screen.
- **The header connection pill stayed green through a hung Security Onion.** The pill covered
Elasticsearch and the model gateway only. It never covered the Security Onion web API, which
is the path every acknowledge, escalate and case write travels, so on 2026-09-07 it read
"connected" in the same frame as a page reporting the grid could not be read, and again beside
a setup-health card saying the Security Onion check had timed out. This is the product's one
always-on trust indicator, which makes it the worst place in the app for an optimistic answer.
`/api/v1/health` now carries a third component, `so`, probed on the same short TTL and the
same single-flight lock as the other two, so the pill's colour cannot disagree with its own
dropdown. The probe issues `GET /api/info` through the shared auth client the write endpoints
already hold, which means it exercises the live session rather than logging in afresh every
fifteen seconds, and it classifies what went wrong the way the Elasticsearch probe does:
refused, timed out, shedding load, or credentials rejected. An API that answers with a 401,
403 or 5xx counts as down, because that is exactly what the next acknowledge will get. The
dropdown lists Security Onion as its own row, the dashboard's connection banner names it, and
a down API becomes a standing bell entry like any other dependency. A payload with no `so`
field, which is what an older server sends, still reads as "not reported" rather than "down".
- **The alerts-filter check compared totals, so it passed while a whole alert class was
invisible.** Measured on the deployed instance on 2026-09-06: the configured
`event.dataset:suricata.alert` matched 1,892 documents in 24 hours and the broadest
alternative, `tags:alert`, matched 1,896. Every ratio and margin the check applies reads that
as a rounding difference, and it PASSed. The four documents between them were Security Onion's
Sigma engine, whose host-behavioural detections the configured filter matches none of and
never has: 397 of them over thirty days, none of which reached the triage queue, the alerts
console, or any hunt that reads the alert plane. Zero coverage of a class is a different
failure from a filter that is merely narrower, and no ratio between two totals can express it.
Each label probe now also breaks its matches down by `event.dataset`, a terms aggregation on
the size=0 count the check already ran, so no extra round trip. The check now WARNs whatever
the totals say when an alternative finds a class the configured filter finds none of. The
message names the class rather than the shortfall, because "your filter is narrower" is not
something an operator can act on and "the queue has never seen a sigma.alert" is. The remedy
stays a widening: the recommended value is the configured filter ORed with the label that
covers the most invisible documents, never that label on its own. A class list the grid cut
short (`sum_other_doc_count` above zero) proves nothing, so the comparison stands down rather
than raising a false alarm on a healthy grid.
- **`soc-ai doctor` graded the environment file, not the configuration the app runs.** Saved
console settings live in `config_overrides` and are applied over the environment-loaded
settings at startup, so the two diverge in ordinary use. On the deployed instance
`ORACLE_MODEL` is one model in the file and another in the database; the running app uses the
second, while the doctor probed the gateway for the first. It also meant the alerts-filter
row would go on reporting a filter the operator had already fixed in the console, which is the
one place its own hint sends them. The CLI path now applies those overrides before running the
checks, in the same order the app uses, and says on the config row how many it applied. Reading
them is fail-soft in every direction: a fresh install has no store yet, and `check_store` is
the row that reports a missing or broken one. A caller that passes its own settings (the in-app
preflight) already holds the live singleton and is untouched.
The filter hint now also says which of the two places wins, because it sends the operator to the
console while the value on that instance lives in a file. A console save applies immediately
and outranks the file; an edit to the file while an override exists changes nothing, and a
sentence offering two equal-looking options gave no way to know that.
- **A triage that died was announced nowhere, and could not be cleared.** On the deployed
instance 188 investigations of real alerts ended in an error status with no verdict, no
rationale, no summary and no report, over ten weeks. Not one had been acknowledged, and no
surface in the product had ever mentioned any of them: the Dashboard's pipeline-error count
matched only the E1.2 fallback marker, which is stamped from a report these runs never wrote;
the notification bell listed in-flight runs and completions and nothing else; and the dismiss
endpoint refused anything that was not fallback-marked, so even a count that had included them
could never have been worked down. A monitoring product that fails loudly is fine. This one
failed silently, and the alerts behind those runs sat unacknowledged and unqueued with nobody
able to see it.
One rule now stands behind all three surfaces: the run displays as an error and carries no
verdict. The `pipeline_error` verdict filter matches those runs as well as fallbacks, so the
Dashboard tile and the list its deep link opens both reach them; the bell announces each one as
"Triage failed, no verdict", capped at three so a gateway outage cannot become the whole panel;
and the terminal-failure panel in the drawer now carries the same Dismiss control the fallback
panel has always had. A dismissed run stays dismissed: the bell reads the acknowledgement stamp
in its own query, and the tile applies the exclusion it already applied to fallbacks.
Cancelled and interrupted runs are deliberately outside all of it. An operator asked for the
cancel, and a restart orphan stays re-huntable so auto-triage picks it up again; counting
either would be a nag about work that needs none. An errored run that did reach a verdict is
outside it too, being reachable under that verdict. The bell's status filter now grades a row
by its DISPLAY status, so a complete run with a blank verdict is reported as the failure it is
rather than as "Verdict untriaged".
- **The investigator gathered the evidence and the handoff threw it away.** The investigator queries
the grid and writes what it found into a transcript, one fact per line with the tool, path or
document id that supports it. The synthesizer reads that transcript and writes the verdict, and
its citation list comes out empty. On the deployed instance 1,187 of the 3,379 runs that produced
both a transcript and a report lost between six and thirteen evidence strings that way, a median
of eight, and 713 of those reports were then acknowledged in Security Onion as verdicts resting on
nothing. The evidence was never missing. It failed to reach the report, and every surface
downstream read the absence as an absence of grounds: the citation gate, the drawer, the analyst.
The transcript's evidence is now carried into a report that cited nothing, next to the
template-grounds adoption and ahead of it: where a dispositive template and a real investigation
are both available, the run's own retrieval is the better answer to what the verdict rests on.
Three properties keep this a carry rather than a manufacture. It only fires when the report cited
nothing, so the model's own citations are never overwritten or padded. Every bullet goes to the
resolver unfiltered, so one that does not resolve lowers coverage exactly as a fabricated citation
would, and nothing is dropped to flatter the number. It also requires the loop's real message
history, because without it the tool-citation resolver falls back to a substring match against the
transcript's own text, and a bullet carried out of that transcript would resolve itself. That last
one is the way this change could have been worse than the bug, so it has a test of its own.
Measured on 400 of the affected production runs, the carried bullets resolve at a mean coverage of
0.996: 3,075 strict resolutions against 15 that fail. The grounds were real and structurally
checkable the whole time. Practically, that means the citation gate added alongside this stops
refusing the acknowledgements it should not have been refusing, and the timeline says which
findings the verdict was recorded against.
- **Automatic acknowledgement wrote back verdicts that cited nothing.** The gate on the unattended
write asked four questions: is the toggle on, is confidence above the threshold, did the run
retrieve anything, is the alert high-stakes? None of them asked whether the report supported
itself. Nothing downstream asked either: the confidence cap deliberately stands down when there
are no citations to measure, because a ratio over an empty set is undefined rather than bad, so an
uncited verdict keeps its full confidence and clears the threshold. On the deployed instance 1,315
of 3,696 completed runs shipped a report that cited nothing and 834 of them were acknowledged in
Security Onion, at a mean confidence of 0.77.
The citation state is now a refusal in its own right, alongside the others. An uncited report is
not acknowledged, and neither is one whose citations were all measured and none of them resolved.
That second case is the worse of the two, because the report named its grounds and not one of them
could be tied to anything the run retrieved. Partial coverage still acknowledges: the question is
whether the verdict rests on something, not on everything, and the confidence cap already prices
the shortfall. Measured against the recorded history, 834 of the 2,768 acknowledgements would have
been refused on this bar alone, and 714 of those had retrieved something, so the retrieval bar
above would have let almost all of them through. The control that matters holds: 1,776 of the
2,768 both retrieved and cited, and every one of them still fires. The verdict itself is
unaffected, and the drawer now says why the ack is waiting for a person instead of leaving two
identical-looking cards to behave differently.
- **The second acknowledge path was forty times larger than the one that was repaired, and nothing
could see it.** Auto-triage inheritance hands one investigation's false positive to every sibling
alert on the same rule and address pair and acknowledges them in Security Onion. On the deployed
instance's own audit index that path had written 110,693 acknowledgements since 2026-07-07, every
one recorded `ok`, against 2,768 from the direct auto-acknowledge that the evidence bar was added
to. It creates no investigation row and emitted no acknowledge event of its own, so it appeared on
no surface in the product and the bar did not reach it.
It now applies the same bar, closing the inheritance gap an older entry here records as open.
The bar reads the recorded run rather than a live message history: a verdict is only lent
to a sibling alert when the investigation behind it made a successful tool call, dispatched a
Phase-D tool that returned discriminating data, or the Oracle called a tool in its own loop. The
content test is one function now, shared with the live reading, because two implementations of
"did this run retrieve anything" is how the two paths came to disagree. It fails closed: if the
store cannot answer, nothing is written. Measured against the recorded history, a 24,000-write
sample across July, August and September says 16.4 percent of the acknowledgements inherited a
verdict from an investigation that had retrieved nothing, so roughly 18,100 of the 110,693 would
have been refused. The verdicts are untouched and still read as confident false positives. They
stop authorizing unattended writes.
On the amplification: the inheritance key is `(rule, source ip, destination ip)`, and the sample
says the key is not the problem. 24,000 writes fall into 393 clusters and none of them is a
both-endpoints-empty catch-all, so no cluster is collecting unrelated traffic. The fan-out is a
time integral rather than a batch: one verdict covers every future alert on that exact pair for
the whole inherit window, which is a median of 11 acknowledgements per investigation, 146 at the
90th percentile and 945 at the largest. The 27 sources that had retrieved nothing were among the
loudest, averaging 145 writes each and reaching 790. The key is not being changed.
Two records come out of every write now. An `auto_ack_inherited` audit record carries
`inherited_from`, so an acknowledgement can be walked back to the reasoning that authorized it.
The `ack_alert` records already written name the alert and the user string and nothing else. An
`inherited_ack` row lands on the SOURCE investigation, which puts the running total somewhere
durable and answers the question in the direction an analyst asks it: what has this false positive
I closed three weeks ago been acknowledging on my grid since? One row per source, updated in
place. The sweep runs every few minutes and the largest fan-out on record is 945, so a row per
sweep would bury the investigation's own timeline under its aftermath. The auto-triage status
reports the all-time total alongside the per-sweep count, and says why acknowledgements were held
back.
- **A tool divided a 59-second burst by 30 days and handed the model the answer as a background
rate.** A lateral-movement alert asked `rule_prevalence` for its signature's base rate. The tool
returned 1531 fires, 51.033 per day, "occasional", with a `first_seen` and a `last_seen` 59
seconds apart in the same dictionary. Checked against the grid, all 1531 documents fall in one day
bucket, from one source host to one destination host, on a single source port, inside one TCP
session. It is one burst, and the tool's docstring told the agent the per-day rate was the number
to weigh. The run closed the only real intrusion in that window as a false positive at 0.85 and
recommended acknowledging it.
The error scales with how bursty the detection is: the reported rate is wrong by the ratio of the
lookback to the burst, here about 43000 to 1. On the deployed instance, of 3798 recorded
`t_rule_prevalence` results that claimed a rate, 270 divided by a window the data occupied less
than 5% of, and 227 of those by a window it occupied less than 1% of. The worst was four fires at
one instant divided by 90 days. Most of the tail understated rather than overstated, which pushes
a bursty rule toward "rare" and toward escalation, so the fabricated number has been distorting
triage in both directions rather than only closing cases.
The rate is now computed over the span the fires actually occupy, and no per-day rate is emitted
at all when that span covers under 5% of the window or the fires are clumped onto under a quarter
of the days they span. `fires_per_day` is null there and `rate_basis` names the denominator when
there is one. Both conditions are needed: 700 fires on day 1 and 800 on day 29 span 93% of a
30-day window and are still two episodes. Offering a substitute number would invite the same
mistake, so none is offered.
Burstiness becomes the signal instead of the thing that gets erased. A new `burst` noisiness
bucket, plus `is_burst`, `observed_span_seconds`, `active_days`, `span_fraction_of_window` and
`burst_fires_per_minute`; source-port cardinality is now queried too, because one port is one
session. Against the same alert the tool now returns: "fired 1531x in 59s on 2026-09-01 - one
burst, 1 active day of the 30d window, from 1 source host / 1 dest host / 1 source port, and
nothing before or after it. That is a single episode, not a background rate, so no per-day rate is
reported."
Guarding the inverse error mattered as much. A span-based rate alone would call 12 fires in six
hours 48 a day and label a small clump background noise, so `noisy` and `occasional` now require
the window average to agree with the span rate before either is claimed. The lookback-normalised
figure survives only as that floor and is never reported. A rule that genuinely fires steadily
across the window reports the same steady rate it always did, which is the control the change is
pinned against.
- **A burst also cleared the bar for recommending the signature be muted.** `suggest_rule_tuning`
reads the same alert count and its volume floors carry the same assumption, that the alerts
recurred. The 1531 fires clear the mute bar by 15x, so with three of them acknowledged the tool
would have recommended muting the signature that fired on the intrusion. It now asks for the span
and the day histogram as well, and a burst can be surfaced but never muted. A separate defect in
the same function: the heuristic counted `fp + tp + nmi` as its data points, and on the
Elasticsearch path `nmi` is every alert nobody has touched, which made the floor satisfiable by
volume alone and produced the line "investigated 1531x" about a rule no analyst had opened. The
count is now the dispositioned alerts only.
- **A packet flood was reported as the most perfectly periodic beacon on the wire.** When every
packet in a flow shares a timestamp the mean inter-arrival gap is zero. `_compute_inter_arrival`
guarded that division by returning a coefficient of variation of 0.0, which its own docstring
tells the reader means a perfectly periodic beacon. `cv` is `None` there now: unmeasurable rather
than perfect. `beacon_profile` already carried a local guard for this case; `decode_pcap` did not.
- **`origin_chain` looked after the activity it said it looked before.** Every sentence the tool
emits reads "in the N minutes before the activity", and its empty branch is the load-bearing claim
"nothing was observed driving this host, so its behavior appears self-originated". It borrowed
`query_events`' anchored time filter, which centers the window on the anchor, so at the default 30
it read 15 minutes before and 15 after. A driving session 20 minutes earlier fell outside the
window while sessions that happened after the alert were counted, named as peers, and offered as
the closest preceding one. Re-running the lateral-movement alert, a verdict cited the tool for
"inbound SSH sessions from admin before/after ... consistent with an admin-managed update push";
the only session was six minutes after the alert. The tool now builds its own
`[anchor - lookback, anchor]` window and drops any post-anchor hit alongside the existing
direction guard. Centering stays in `query_events`, where the caller did ask for context around an
alert.
- **`host_summary`'s DNS counts were a sample presented as a window.** `top_peers` and `top_ports`
are aggregations over every matching document; `top_dns` is a tally over the 200-document sample,
and all three sat side by side in the identical `{value, count}` shape next to a full-window
`event_count`. On a busy host that reads short by three orders of magnitude. The key is now
`count_in_sample` and `top_dns_sample_size` travels with it.
- **Every Elastic Defend document failed validation, so no hunt finding anchored on endpoint
telemetry could be promoted.** `SoAlert.event_action` was typed `str | None`. ECS says
`event.action` may be an array and Elastic Defend writes one: on the measured range every
`endpoint.events.*` document carries `["start"]`, `["end"]` or `["start", "end"]`, which is
175,812 of 919,839 documents in a 24h window, 19%. On the deployed grid the same planes are
3,074,734 of 5,952,745 documents, 52%. Promoting such a finding produced an investigation that
errored at its second timeline entry with zero tool calls and zero pivots: `1 validation error
for SoAlert / event_action / Input should be a valid string / input_value=['end']`.
Pydantic enforces the annotation by refusing the whole document, so one array field cost the
reader every other field on it. `event_action` now reads through the same `_first` helper the
neighbouring `event.category` has always used, and so does every other scalar attribute on
`SoAlert`, `SoCase`, `SoDetection` and `SoPlaybook`. Elasticsearch has no array type and
guarantees nothing about arity on any field, so enumerating the fields that "can" be arrays only
postpones the next outage; the guard is a test that wraps every leaf of a realistic document in
a list and asserts the model still parses it, which fails the moment a new scalar attribute is
read without narrowing. Narrowing does drop the tail of a genuinely multi-valued field, and that
is the smaller cost: the untouched document is still on `alert.raw`, whereas a refused one is
unreadable in full.
Three things fell out of the sweep. `zeek_ssh_auth_success` was computed as `bool(value)` on the
raw read, so a list-wrapped `False` reported a FAILED SSH authentication as a successful one, and
the same held for `zeek_ssl_established` and `zeek_dns_rejected`. `tags` had the asymmetry
inverted: annotated `list[str]` and built with `list(...)`, so a grid writing the scalar
`"alert"` produced the five tags `a`, `l`, `e`, `r`, `t`, silently and with no error. And the
regression test guarding the behavioural-summary pivot used a list-where-a-scalar-is-expected
document as its needle, which this fix makes valid, so the needle moved to a shape that still
fails.
The failure looks different to each caller, and the difference matters. On the promotion path it
is LOUD: the prefetch retry wrapper deliberately does not retry a validation error, the
orchestrator has no fallback for a failed prefetch, and the investigation lands `status=error`
with the pydantic message in its timeline. On the best-effort pivots it is an ABSENCE: the
behavioural-summary pivot catches every exception and returns `[]`, and auto-triage's
inherited-acknowledgement pass logs a warning and skips the alert. Those read as "no such
evidence" and "nothing to acknowledge" rather than as a parse failure.
- **The ambient inventory listed datasets and told the model to query them a way that matches none
of their documents.** The "data available on this grid" block is rendered into the model's prompt
as ground truth, and it said to ask for any listed plane with `event.dataset:<name>`. On the
measured grid 374,688 of 919,845 documents in a 24h window, 41%, carry no `event.dataset` at all.
Five `network_traffic.*` planes there are named only by `data_stream.dataset`, and one of them,
the flow records, is the largest plane on the grid. The census was taught to find those planes
earlier, so they appeared in the list. The filter the block implied still matched none of their
documents. The model was handed a plane and a way of asking about it that returns nothing.
This is the asymmetry that hid an entire network-metadata plane before, resurfacing one layer up:
fixed at the census, still wrong at the prompt.
The census is the only part of the program that knows which of its two aggregations produced a
row, and it was discarding that when it merged them into one ranked list. `DatasetInfo` now
carries the `identity_field` it was found under and exposes a `predicate`, and the block renders
that predicate at the head of every line: `event.dataset:zeek.conn` for a plane that has one,
`data_stream.dataset:network_traffic.flow` for a plane that does not. Nothing is inferred at
render time, so a mixed grid gets the right filter on every row instead of one guess applied to
all of them.
A model that learns the right field from the block still meets tools that assume the other one.
`t_describe_dataset`, `t_field_values` and `t_first_seen` now match a dataset name under either
field, through one shared filter builder. Before this,
`t_describe_dataset("network_traffic.flow")` answered "no documents", which reads exactly like
the plane not existing.
- **Two alerts from one TCP session reached opposite verdicts, and nothing in the product could see
they were the same session.** Twenty-eight minutes apart on the range: one true positive
recommending escalation, one false positive recommending acknowledgement. Same source, same
destination, same port, same community id, same fifty-nine second window. The second run's own
prefetch matched a benign template and closed it with no tools. An analyst working the queue top
down meets the false positive first, acknowledges on the product's recommendation, and never
reaches the row saying the same session was lateral movement.
The finest key an investigation row carried was `(rule_name, src_ip, dest_ip)`: no ports, no
protocol, and a match only when the rule name matched too. So two sessions between one pair looked
identical, and two alerts on ONE session looked unrelated. Migration 0038 adds the community id,
which is the hashed five-tuple, stamped off the enriched alert exactly as the endpoints already
are.
Before synthesis the pipeline now asks whether a completed investigation already covered this
session, and puts the answer into the prompt as a constraint rather than as context. It is
resolved ahead of the decision-template fast path, because a template can close a case with no
tools and a constraint arriving after that has arrived too late. A true positive on the session
also forces the investigation loop, and deliberately not behind `memory_enabled`: that flag
governs whether the model is shown resemblances it may weigh, and this is a contradiction inside
one conversation.
A deterministic gate has the last word, downstream of the Oracle and of the auto-acknowledge. It
does not adopt the earlier verdict, because a gate that can promote a verdict is a gate that can
invent one. It refuses the close: `false_positive` becomes `needs_more_info`, the acknowledge
recommendation goes, and the note names the prior investigation. Both timelines carry the relation
whether or not the verdicts end up disagreeing, so the analyst is told the two alerts are the same
session either way. A true positive on a different session between the same two hosts changes
nothing, and neither does an alert with no session behind it.
- **The Alerts footer and the preset chips counted rows the analyst had already filtered away.** A
list showing one row was captioned "59 detections", and every preset chip carried the same 59.
All of it was measured against the array the grid returned, before the severity facet, the verdict
facet and the hide-acknowledged toggle were applied, so the caption described a list nobody was
looking at. A count under a list is a claim about that list.
The header and footer now count the rows on screen, presets included. The chips are counted after
the facets and before the preset, which is what a chip badge means: how many rows clicking it
would produce. Each chip runs the same match the list itself uses rather than a second copy that
can drift, so the All chip and the footer can no longer disagree about the same list.
- **A configuration override that failed validation was written onto the live settings anyway, and
bricked every later change.** `validate_assignment` runs the model validators after the field has
been set, and pydantic does not put the old value back when one of them raises. So the assignment
"failed", the key was correctly reported as not applied, and the rejected value was sitting on the
settings object all the same. From then on every `setattr` re-ran the same validator and failed
too, which meant every subsequent hot apply on that instance was refused with a message naming the
setting that was stuck rather than the one the operator was changing. One bad value took
configuration down and misdirected whoever tried to work out why.
`POST /config/setting` and the Danger Zone save already prove an assignment on a copy before
committing it. The shared apply path does the same now: each override is assigned to a throwaway
copy first, and only a copy that validates gets the assignment for real, so a rejected value
leaves the previous one in place. This covers the startup replay of stored overrides too, which
is where a single bad row used to poison every override after it.
- **An escalate reported success for an alert it put on no case, and wrote that claim into the
ledger.** Pressing escalate on a range investigation answered `{"status": "executed", "detail":
"Case created: ..."}`. What Security Onion actually did was create the case, accept the attach,
and attach nothing: the alert was not on that grid at all. The attach is query-shaped, so
Security Onion rebuilds a search from the fields it is given and answers 200 with `{"count": 0}`
when the search matches nothing, and soc-ai read the status code instead of the count.
Two things went wrong at once and the second is the worse one. An empty case is now sitting in
the queue wearing a title that reads like a real incident. And the escalation ledger, which
exists so a second press cannot open a duplicate case, recorded the alert as escalated to that
case, so the machinery would have refused to open the case actually needed. A duplicate-prevention
record is only worth having if it is true.
`escalate_to_case` now keys `alert_linked` on the count Security Onion reported rather than on its
status code, and a 2xx whose body carries no count at all is treated the same way: an attach
nobody observed is not a link. Nothing is stamped escalated over an attach that did not happen.
The execute-action route reports the three outcomes as three different sentences (attached and
stamped, attached but not stamped so the alert still reads untriaged in Security Onion's own list,
or not attached at all) and only counts an escalate when the alert was genuinely linked. An
unattached escalate takes a bare ledger claim instead of a resolved row, which reads as "soc-ai
tried, outcome unknown" and is reconciled against the grid's case links on the next group press.
Both the single-alert response and the group escalate name the empty case id, because closing or
reusing that case is something only the operator can do.
Reachable from the shipped product, not just from a test: the demo investigation ships with a
pressable escalate card, and the same path runs on any live alert whose document rolled over or
was deleted between a group scan and the write.
- **A pipeline error raised while the LLM gateway was unreachable told the operator to go and check
Elasticsearch.** The fallback that records a failed run worked out which dependency had failed by
reading the exception string. A refused gateway connection arrives as `ModelAPIError:
"Connection error."` with nothing in it naming LiteLLM, so the ambiguous connect arm fell through
to the grid branch and both the error event and the stored `pipeline_fallback` report carried
"elasticsearch / Security Onion unreachable. Verify the SO grid is online and ES_HOSTS in
soc-ai's .env points at the right node." Reproduced with a healthy grid and the gateway pointed
at a closed port: the run landed a `needs_more_info` fallback whose only piece of guidance sent
the analyst to the one dependency that was working.
The phase already answers the question without guessing. The synthesizer runs, round 1, round 2,
the loop synth and its partial-report variant, have no tools registered on them, and the
investigation loop's tools each catch their own grid failure and hand the model a structured
result, so no Elasticsearch exception can leave any of those phases. Phase-A prefetch is the
mirror image and only ever reads the grid. Both surfaces now attribute by call site, and only a
phase that could genuinely be either falls back to the string matching. The same outage now reads
"the LLM gateway did not answer: soc-ai could not open a connection to it. This phase makes no
Elasticsearch call, so the grid is not the problem."
A gateway that never accepted the connection is a different fault from one that answered and
could not reach its own backend, so the two now have separate remedies; the LiteLLM markers in
the error string still separate them.
The same misattribution ran in the other direction and is fixed with it: an Elasticsearch connect
timeout inside prefetch renders as "Connection timed out", matched the generic timeout arm, and
was reported as "LiteLLM gateway slow or unreachable", which prefetch never calls. A genuine grid
outage still reads exactly as it did, verified byte for byte against the unfixed build.
- **The opt-out for partial Elasticsearch results was also switching off the report that the read
was partial.** `es_fail_on_partial_results=false` is a reasonable answer to a chronically red
shard: keep working off the shards that still answer rather than failing every query on the ones
that do not. It reached the health probe through the same `ElasticClient.search` path, so an
operator who set it had also, without being asked and without being told, turned off the one
surface whose job is to say the grid is half-read. Reproduced against a grid answering HTTP 200
in 7ms off 2 of 4 shards: `/api/v1/health` returned `es.ok: true` with the detail "readable", the
topbar stayed connected, and the bell, which derives its dependency-down entry from that same
probe result, had nothing to say. The operator who accepted short answers had bought silence
about the cause along with them.
The probe now reads with `require_complete=True`, the existing per-call opt-out from the opt-out,
so it reports a partial read whichever way the setting is left. What a query does about a partial
read stays the operator's call; whether anyone is told is not. The same grid now answers
`es.ok: false`, `kind: "partial"`, naming the two failed shards and the shard reason, and the
bell carries "reading only part of the grid".
Two more consumers had the same conflation, and both are health surfaces rather than queries.
`soc-ai doctor`'s `elasticsearch` check took the half-read grid's zero count at face value and
reported "the events pattern matched no documents", with the remedy "check
EVENTS_INDEX_PATTERN": a config diagnosis for a shard fault. `check_index_pattern_coverage` did
the same, reporting "matches no suricata/auth/syslog events" with "wrong pattern or an idle
grid", and it already had a `GridPartialResultsError` arm written for exactly this case, plus a
docstring claiming it inherited the guard. The opt-out had made both the arm and the claim dead.
Both checks now require a complete read, the coverage arm is reachable again, and the
`elasticsearch` check gained a partial-read result of its own so a grid that answered in
milliseconds is not filed as "unreachable" with a connectivity remedy. The coverage arm's hint
moved from "fix Elasticsearch connectivity first" to shard health, for the same reason.
The config console's help text for the setting now states the scope, so an operator can see what
they are and are not turning off before they turn it off.
A census of the remaining `ElasticClient.search` call sites that omit `require_complete=True`
found the same exposure across a long tail of surfaces whose answer is a claim about absence or
coverage rather than ordinary query data: grid inventory and dataset discovery, the alerts
console's "quiet network" reason, the dossier's identity-retraction pass, auto-triage's degraded
mark, and several agent tools whose own docstrings promise to distinguish "could not look" from
"nothing there". None of those are changed here. Each needs its own answer to what the caller
should do with a short read, and a blanket `require_complete=True` would hand an error page to
the operator who deliberately asked for the opposite.
- **Triage was closing exploitation attempts as false positives without looking anything up, and
acknowledging them in Security Onion.** On the deployed instance, 13 of the last 60 recorded runs
reached a verdict with zero tool calls. All 13 matched `clean_internal_traffic`, all 13 settled
false positive at 0.85 to 0.90, and all 13 were auto-acknowledged. Nine were the same alert: "ET
HUNTING Potential Forced OGNL Evaluation - HTTP Body", an exploitation-attempt signature against
an internal HTTP service, recurring daily since 2026-08-22 and closed unread every time. Three
independent things had to be wrong for that to happen and all three were.
**A decision template was seeding the verdict on network locality alone.** The template's grounds
were that both endpoints were private and no blocklist named either of them. Blocklists never
name an RFC1918 address, so the second ground is vacuous, and the first is true of every
east-west flow on a flat network including an exploit landing on an internal service. Templates
exist to make routine traffic cheap and turning them off would be a real cost, so the fix is a
distinction rather than a deletion: a candidate is now `dispositive` or `provisional`. A
dispositive template reads what the rule DETECTED, which is why STUN/QUIC keepalives, DNSSEC
record queries and NTP keep the zero-tool fast path: the protocol's ordinary operation is the
entire content of the alert and no retrieval would change the rea