CHANGELOG.md
Rama main · réplica de lectura del repositorio original.
# Changelog
All notable changes to **agami** are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
The `version` in `.claude-plugin/marketplace.json` and `plugins/agami/.claude-plugin/plugin.json`
is the source of truth a host installs against — bumping it is what invalidates a
user's plugin cache (see [CONTRIBUTING.md](CONTRIBUTING.md)). Each released section
below corresponds to one such version.
## [Unreleased]
## [0.9.22] — 2026-11-05
### Changed
- **The activity log records only the arguments core declares on its own tools (#625).** For
`list_datasources`, `get_datasource_schema`, `get_prompt_examples` and `execute_sql`,
`tool_calls.arguments` now holds only the properties core's own input schema declares for that
tool, less those with a column of their own as before. A property a consumer adds to one of these
tools, whether by serving an extended copy through `extra_tools` or by editing the shared `TOOLS`
schema, is no longer recorded there: core doesn't know what it holds or which of the consumer's
rules apply to it, so recording it is the consumer's job. The declared names are taken when `tools`
is imported, before a consumer can extend the schema. The HTTP transport and a direct
`record_tool_call` behave the same, and a tool a consumer registers itself is recorded as it was
sent, as before.
- **Editing an approved metric, entity or join sends it back to review (#537).** An edit through
`sm curate` or the explorer that changed an approved item's binding, calculation, unit, source
tables, an entity's mappings, or a join's columns, `on:` or cardinality kept `review_state:
approved` with the old signer and time, so a receipt reported a definition nobody approved as
signed off. Such an edit now sets the item to `stale`: it is back in the review queue and a receipt
reports it as not signed off until someone re-approves it, which records the new signer and time.
Only `description`, `other_names`, a metric's `business_question`, a metric's or entity's `caveats`, an entity's
`plural`, and a join's `for_questions_about` keep the approval; every other field,
including any added later, counts as a change, and an edit that sets a field to the value it
already has changes nothing. Edits to an unreviewed, stale or rejected item leave its state as it
was. The old `signed_off_*` stay on a stale item as the record of the approval it lost, and an
approve with no signer can't reuse them. An edit can no longer set a sign-off field
(`review_state`, `signed_off_by`, `signed_off_at`, `signed_off_role`, `confidence`,
`migrated_from`) on any entry, so a stale item can't be re-approved under its old stamp: the op is
skipped and points to approve / reject / include. In one batch an approve after the edit signs off the
edited version (the explorer now sends approvals after edits) and an edit after the approve leaves
the item stale; `sm curate` lists the items an edit sent back to review under `stale`. The Review
tab, `sm review-queue` and `sm approve-queue` show what changed on a stale item since its approval,
read from `curation_log.jsonl` (e.g. "binding (sqlite) changed from SUM(amount) to SUM(amount) -
SUM(discount)"), or "changes not recorded" when the log doesn't reach back to it ("some changes not
recorded" after the changes it could read, when a line since the edit can't be read). This
supersedes #559's re-stamp of a person's structural join edit as their approval (#511): the edited
join goes stale instead, and a refresh still leaves it alone, since it re-labels only the joins
introspection approved itself, and never merges an edited split join into a database's composite
foreign key (`composite foreign key not applied: … edited since its approval`); a refresh that
lowers a stale join's confidence because a column's type changed keeps it `stale`, and a stale
`on:` join keeps its confidence. A table move (#513) keeps a stale join, metric or entity stale
with its old signer, and the review surfaces still show what changed; when a move finds two copies
of one join, a stale copy counts as a person's sign-off, so it is kept over an engine-approved one,
and a move where it disagrees with a copy a person signed off is refused. A move's log entry lists
the metrics and entities it carried (`carried`), and an item's changes follow it back only through
a move that carried it (a move logged without that list gives "changes not recorded"). The log entry
also lists each join the move placed again (`joins`: where it was and is, its identity before and
after with its schemas, and, when it met a copy already there, that the copies were `merged` and
which was `kept`); a join's changes follow back only the copy kept, say the copies were merged and
the other copy's changes are not recorded, and ignore a move of a same-named table in another
schema. An item's
changes come only from edits of that item: a cross-area metric or entity reads only edits of the
datasource-level lists, and an op that named no area (or another one) logs the area it resolved
to (`resolved_area`). `stale` names a moved item where it now is, and an approve after the move in
the same batch clears it.
### Added
- **Introspection finds placeholder values and asks what each one means (#445).** Warehouses carry
values that look like data: an Unknown dimension row keyed `-99999` and dated `2099-12-31`, a
`9999-12-31` valid-to on the current version of a row, `'-'` filler text. Answers counted them as
real, so a "by fiscal year" breakdown grew a year-2099 bar and a "latest date" came back as
2099-12-31. Introspection now reads one bounded query per key, date, year and code column of each
small table, and of each dimension (the one side of a join, or the referenced side of a one-to-one
join) under a million rows, on the existing
value-read budget, and flags a date before 1901 or more than 50 years ahead, a negative key among
positive ones, a `999999`-style value and filler text on a large share of rows. A big fact table
is not read, nor is a table whose sample could not be read and whose row count is unknown, nor
a column flagged sensitive or whose values look personal (on a refresh, by this run's reading
too). Each finding is
listed in the introspection report and in the `/agami-model` Review tab (and `sm review-queue`
under `placeholders`); agami-connect asks about each in plain words. The person's answer is
stored on the column (`placeholders`: `unknown`, `current` or `not_set`, signed off by them) with
a caveat in words, through a `kind: placeholder` curate op; `reject` says the value is real data.
`get_datasource_schema` and the table context serve each answered value with its meaning and how
to write the SQL for it, and the query rules say how to use each. Nothing filters a query, and
`sm approve-queue` never answers one. A refresh keeps every answer and doesn't ask again, and its
report lists a value found or no longer found under `changed <table>: placeholders`.
**Reading a model:** a column that sets `placeholders` (a finding waiting for an answer, or an
answer) is refused by a release older than this one, which does not know the key. Columns with
none stay readable, because the key is left out of the file when the list is empty, so a model
where nothing was found is written exactly as before. Upgrade every server that reads the model
(including deploy images) before building or refreshing one with this release.
- **A join can be added through `sm curate` and the model explorer, and introspection reports the
joins it tested but didn't write (#512).** The curate op `{"op":"add","kind":"relationship",
"from_table","from_column","to_table","to_column","relationship"}` (or an `on` condition instead
of the columns, for a key over several columns or a filtered join; `from_schema`, `to_schema`,
`join_type` and `description` optional) writes the join `unreviewed`, in its tables' area or in
the cross-area joins when they are in two areas. It is validated, logged in the curation history
and listed in the review queue; an add never approves. The same join already in the model is
skipped with a reason (an `on` that differs only in a quoted value's case, `'Y'` against `'y'`, is
another join, with the engine's own string forms read as quoted values too, such as PostgreSQL's
`$$Y$$`, `E'..'` and `N'..'` and MySQL's backslash escapes, and a quoted name keeping its case
where the engine compares names by case; a refresh, a model spec and approve, reject, edit,
exclude and move compare `on` conditions the same way, so each such join is curated on its own),
and a table or column the model doesn't serve fails the batch and undoes it, including every column
an `on` names (beside an explicit column, filters and casts too), each matched to its end by schema
and name. The join is placed by the areas that serve its tables, never by an excluded or stale copy,
and keeps the model's spelling of its tables. A moved table's joins are placed the same way. The explorer's Joins tab
has a **+ Add a join** form that queues the op. Introspection now tests each column no join covers
against the other tables' one-column keys when the name or the type matches (`orders.cust_ref`
against `customers.id`; a column with a value list, or one column of a key over several, when its
name points at the table), skipping excluded or stale columns and keys, also beside declared
foreign keys, and lists what it found in the report with each match rate, and in
`.introspect/join_candidates.json` as a ready `add` op; `sm curate <model> --ops-file <that file>
--candidate <id>` adds one. These candidates are never written to the model, so a refresh writes
the same model as before. Each test is a read-only query over at most 1,000 sampled values, at
most 300 per run, skipping tables over 3,000,000 rows; the report says when the cap stopped the
scan. A column flagged sensitive, or whose sampled values look personal, is never tested at either
end, and the report and the file carry only names and counts, never a sampled value.
- **An example can be tagged with several metrics, and is found under each (#546).** A prompt
example's tag held one metric name (`metric: revenue`), so an example computing two metrics could
be linked to only one of them. It is now a list, `metrics: [revenue, order count]`; a single
`metric:` and a `metrics:` given as one string still read. `sm add-example` and `sm seed-examples`
write one metric as `metric:`, as before, and several as `metrics:`; the examples a call doesn't
name are written back as they were, so existing files don't change. Each tagged metric must be one
the model declares, by name: an example naming another is refused and listed in `skipped` (in
`rejected` for `sm seed-examples`). `get_prompt_examples` takes `metric_names` and `sm examples`
takes `--metric` (repeatable): the examples tagged with more of those metrics rank first, then
by the question as before. Without them the ranking is unchanged. A metric two areas both declare
is named by the selector `get_datasource_schema` gives it (`revenue (finance)`); that selector
matches the `revenue` tag on that area's examples and on cross-area ones, not on the other
area's, on a local and a served deployment alike. Only the first 100 distinct names are used, so
an oversized `metric_names` can't tie up the server.
**Reading a model:** a release older than this one serves a `metrics:` list unchanged but cannot
rank by it, so upgrade the server before or with a model whose examples carry several metrics.
Examples with one metric are written as `metric:`, which every release reads.
- **`seed-examples`, `seed-validate`, `remove-example` and the examples dashboard take several subject areas in one call (#544).**
Each command took one `--area`, so writing or checking the examples of a database with 30–50
areas took dozens of calls. Repeat `--area`, or leave it out for every area in the model, and the
command prints one result: the single-area fields, each area's own result labelled with its
`area`, and `totals`. One `--area` prints exactly what it printed before. `seed-examples` then
writes each seed to the area it names in an `area` field. `seed-validate` runs each area's seeds
through `execute_sql` scoped to that area, with the same per-statement limits. `remove-example`
takes a question as `<area>/<question>`, looked up in that area only, or matches it in whichever
area has it; a question that more than one area has is refused, with the areas, and nothing is
rejected. The dashboard renderer reads the combined `seed-validate` output, takes `--items-file`
more than once (a single area's file labelled `AREA=PATH`; an unlabelled one, or the same file
twice, is refused), and shows each card's area. An area the model doesn't have is refused with the list of valid ones, where one
`--area` used to run against an empty area (and `seed-examples` created its folder).
- **`execute_sql --build-profile` for profiling while you build a model (#523).** Builders turned the
safety check off (`--no-safety`) to read `information_schema` or expand a JSON column. The new flag
keeps every check and allows only those: reads of `information_schema`'s structure views on the
engines that have one (not Oracle or SQLite), plus `pg_catalog`'s structure tables and SQLite's
`sqlite_master`, each matched as the engine resolves the name (a quoted `"INFORMATION_SCHEMA"` on
Postgres is an ordinary schema), and table functions such as
`jsonb_each`, `json_each` or `unnest` over a declared column. Catalog views that hold values or
query text (MySQL's `COLUMN_STATISTICS` and `PROCESSLIST`, BigQuery's `JOBS`) stay refused. A
column the model leaves out is still refused, `SELECT *` is still refused, a `USING` or `NATURAL`
join is refused under the flag (write an explicit `ON`), and the 50,000-character cap is
unchanged. Beside a table function over a model table every column is qualified, except a name the table function returns,
such as BigQuery's `UNNEST(o.tags) AS tag` value or `WITH OFFSET AS i` position, or on Postgres the
function's own outputs (`key` and `value` for `jsonb_each`) and its alias column list. A bare alias
over `jsonb_each` names the row, not a column, and a column inside a table function's argument is
always read from the tables before it, so it must be qualified. A column qualified by an alias two
sources share apart from case (`c` and `"C"`) is refused unless every one of them declares it, with
or without the flag, and so is one whose alias an inner SELECT spells differently in case
(`"O"` inside, `o` outside), since the engine may read the outer table. Under the flag a statement
cannot read a catalog view and a model table together: SQLite, MySQL and DuckDB read a column the view
lacks from an enclosing table of that alias (`(SELECT o.x FROM sqlite_master o) … FROM orders o`). Also
under the flag, beside a model table a dotted name no source of the query supplies (`info.x`, `o.info.x`)
and DuckDB's `COLUMNS()` are refused. Agami's own catalog query that calls `pg_get_constraintdef` runs under it too, matched by its exact shape, so a statement an
agent writes cannot use that function. The flag is refused on a hosted server and the MCP tool
cannot set it, so apart from those alias rules what the tool accepts and refuses is unchanged.
- **`execute_sql` shows the database's error on your own machine, and `--batch` always writes its
manifest (#523).** A failing statement told a builder only a fixed sentence, often "an unexpected
error" (exit 6). Run from your own command line, that sentence is now followed by
`Database error: <the database's first line>` (and the SQLSTATE where the driver gives one), with
credentials (any `password`, `secret`, `token` or key field, escaped or quoted, and the login named in
`for user 'sa'` however short), hosts and IPv4/IPv6 addresses redacted and HINT, DETAIL and
CONTEXT lines dropped; only the line's first 1,200 characters are read, so a very long line
costs no more. Exit codes are unchanged. A hosted server and the MCP tool still return the fixed sentence alone. In
`--batch`, a malformed item is recorded as failed and the rest still run, an item whose output
cannot be written stays in the manifest with its error, its CSV is emptied so no earlier rows are
left behind, run files and the manifest are written whole or not at all, and the manifest is
written even when the run stops part way. A plan that is JSON `null` is refused like any other
plan that is not a list. agami-connect's sampling step no longer says
`SELECT * … LIMIT 5`, which was refused: it names the columns and runs with `--build-profile`.
### Fixed
- **A refusal's echo drops Unicode control characters, and a self-reported question is capped
(#415).** The expression a receipt echoes back stripped only ASCII controls, so C1 controls, the
Unicode line and paragraph separators, text-direction characters and lone surrogates reached the
calling model intact. It now strips the same set the activity log does. `user_question` and
`agent_query` on a tool-call row were stored at any length; each is now cut at 8,000 characters,
and a value that is not a string, or is empty, is stored as NULL.
- **A description edit without `source` keeps whose words the description is (#549).** A `sm curate`
edit to a table or column description that passed no `source` stamped it `description_source:
human`, so fixing a typo in a description from the customer's data dictionary silently dropped
its `metadata` origin. Such an edit now keeps the provenance it finds: `metadata` stays
`metadata`, and a description agami wrote stays `ai_unvalidated` and still comes up for a check.
A first description, or one written for a column agami couldn't read (`ai_unknown`), is `human`
as before. Changing provenance takes an explicit `source` (`human`, `metadata` or `ai`; any other
value is skipped), and `sm curate`, `sm describe-file` and `sm enrich-metadata` report each
change as `provenance_changed: [{item, from, to}]`. The curation log records
`description_source_before` and `description_source_after` on every description edit. A
correction saved through agami-save-correction now passes `source: human`.
- **Numbers in the examples preview and in query answers are formatted by their column's type and
unit (#547).** A year shows as `2011`, not `2,011`; a ZIP code read as text keeps its leading
zero (`07094`); `-719` no longer shows as `-,719`; a NUMERIC value drops its trailing zeros
(`0.12500000000000000000` shows `0.125`), while a money column keeps its symbol, grouping and
two decimals. A column the model types as text, a key, a `dimension` or a choice field, and a
year taken out of a date, is shown as written; without the model, a column named like a year or
a postal code is shown as written only when every value has that shape. A column now gets a unit
only when that column has one: a CASE takes it from the values it returns, not from a money
column in its condition, a bare column is read from its own table (and schema), and the
difference of two dates is not a date, so a duration in days no longer shows as dollars. A `0`
or `NULL` branch, a window's `PARTITION BY` and a `FILTER`'s condition take no unit away, and
`GREATEST`, `LEAST`, `COALESCE` and `NULLIF` keep the unit, date encoding or label of what they
return, whatever constant or `NULL` sits beside it. A table inside `WHERE EXISTS (…)` lends no
column to the outer SELECT, and a quoted column name matches as the engine matches it. A CTE's
or derived table's column (`t.amount`) is whatever its body computed under that name, a scalar
subquery in the select list is its own projection read in its own FROM (a correlated column is
the outer row's), and a UNION's column has a unit only when every arm gives it the same one.
`ARG_MAX`, `FIRST(… ORDER BY …)`, `LAG`, `NVL2`, `DECODE` and the like keep the unit of the value
they return, whatever other column they take.
Two output columns of one name each keep their own unit and label, by position, and an `EXCEPT`
takes its column's unit from its left arm. A constant branch of a CASE (`ELSE 100`, `ELSE 'n/a'`)
takes no unit away. A text value with a huge exponent (`12e345678901`) is shown as written rather
than expanded, and a statement too deep to trace loses its units instead of failing `sm prepare`.
`sm prepare` returns these label columns as `labels`, and `sm format-table --labels` takes them.
Big integers are no longer rounded through a float. Reports built by `csv_to_sections.py` follow
the same rules.
- **One rule decides whether a metric binding nests aggregates, in `sm add`, the validator and at
query time (#519).** `sm add` accepted `AVG(SUM(amount))`, which the database then refused on
every query that used it, while the check for derived metrics refused a legal aggregate over a
subquery and missed a `SUM` inside a percentile's `WITHIN GROUP`. The rule is now judged per
SELECT: an aggregate directly inside another, or in its `WITHIN GROUP` or `FILTER` clause, is
refused, and so is a window function inside an aggregate (`AVG(SUM(amount) OVER ())`); an
aggregate over a derived table or a scalar subquery is not, nor is a window over an aggregate.
An aggregate in a scalar subquery that reads only the outer query's columns belongs to the outer
query, as on PostgreSQL and SQLite, so `AVG((SELECT SUM(orders.amount)))` is refused, while one
that sums the subquery's own rows is not. A column's table is matched by schema and name as the
engine compares them (on PostgreSQL, Snowflake and Oracle a quoted alias keeps its case, so
`FROM orders AS "ORDERS"` does not hide an outer `orders`), and a source can be an aliased
`VALUES` list or a parenthesized table or join. A window function inside another window
function's argument or `OVER` clause (`SUM((SUM(amount) OVER ())) OVER ()`) is refused too. On
SQL Server, which refuses any subquery inside an aggregate, `AVG((SELECT SUM(amount) FROM
orders))` is refused with the derived-table rewrite; on a derived metric that is a warning. The
one exception is Oracle, which runs two levels of aggregate in the argument of a SELECT with its
own GROUP BY: an Oracle binding `(SELECT AVG(SUM(amount)) FROM orders GROUP BY customer_id)` is
accepted, while a bare `AVG(SUM(amount))`, a third level, or that binding on any other engine is
still refused. `sm add` (a subject area or `--area _cross`) and a `curate` edit of a metric's
`bindings` or `inner_grain` refuse such a binding with the reason, and write nothing for it. A metric already in a model gets a
`metric_binding_nested_aggregate` warning, never an error, so no model that validated before stops
validating; a derived metric nested in its argument is still the `derived_metric` error it was,
also when its base only nests in a `WITHIN GROUP`. An `sm add` batch reads the model once, not
once per derived metric. `resolve_metrics` names why a matched metric will not run, and a
database's own "aggregate function calls cannot be nested" (or its equivalent on SQLite,
Snowflake, BigQuery, Databricks, SQL Server and MySQL) is answered with a sentence naming the
rewrite, with the same failure kind and exit code as before. `metric_null_filled` no longer warns
on an average of the values present, `SUM(COALESCE(score, 0)) * 1.0 / NULLIF(COUNT(score), 0)`: a
numerator that only zero-fills columns inside a `SUM`, over a denominator that only counts the
present values of those same columns (a qualifier included, so `a.score` is not `b.score`, and on
PostgreSQL, Snowflake and Oracle a quoted `"Score"` is not `score`), counted with `COUNT(col)` or
a searched `CASE WHEN col IS NOT NULL`. Any other fallback, another aggregate or a term such as
`COUNT(*)` keeps the warning. The metric guidance shows that form for several columns, with the `*
1.0` that stops integer division. An `sm add` batch judges a derived metric with the batch's other
metrics known, whatever their order, and a `SUM(...) FILTER (...)` inside `AVG(...)` is nesting in
the argument, not in a clause.
- **`sm` resolves a relative path from the folder you run it in (#542).** The launcher runs the CLI
from `/` so nothing in your folder can shadow the installed package, and every relative path
argument resolved there too: `sm curate <root> --ops-file ops.json` looked for `/ops.json`, and a
relative model root or output path failed the same way. The CLI now moves back to the caller's
folder once it has loaded. `sm curate` also takes the ops file as a second plain argument
(`sm curate <root> ops.json`, before or after the other options, or after `--`, which also lets
a file name start with `-`); `--ops-file` still works, and giving both is a usage error.
- **The `measured_figure` check also finds ranges, counts in words and figures tied to now
(#541).** Model text such as "values range 49–142 here", "covers twelve of forty sites", "about
1,200 sites", "40 sites today", "1,204 orders carry no region" or "hundreds of rows have no region"
passed the check. It now finds numeric ranges stated as seen in the data, counts written in words
(up to the thousands, plus "a dozen" and "hundreds of"), hedged figures, figures with a unit or
next to "here", "today", "currently" or "as of", and counts inside a caveat sentence. Statements
about how a formula behaves ("returns 0 because no row meets the threshold"), its literals ("ELSE
0", "divide by 100"), scales ("ranges 0–100"), bounds ("more than 5 users", "top 10 customers"),
years used as dates ("since 2024"), versions and labels ("status 9 rows") are not flagged, and "2
rows per order" is no longer read as a count. It stays a warning.
- **agami-core requires sqlglot below 30.26.** sqlglot 30.26.0 (released 2026-10-29) parses a
`VALUES` arm of a `UNION` as one more `SELECT`, so the receipt counted an extra arm and could
label a set operation's output columns against the wrong position. The `model` extra and the
semantic model's `requirements.txt` now pin `sqlglot>=20,<30.26` until the receipt reads the new
parse; an installed 30.26 is replaced by the latest earlier release on the next install.
## [0.9.21] — 2026-10-29
### Fixed
- **`sm validate` and the query runner work when started from inside the plugin's `lib/` folder (#539).**
The folder held a partial copy of the model package (`semantic_model/` with only `__init__.py` and
`units.py`). With `lib/` as the working folder, or first on the import path, Python loaded that
copy instead of the installed package: `python -m semantic_model.cli` failed with `No module named
semantic_model.cli`, and the query runner refused every statement with `model_unavailable`. The
folder now ships `units.py` as `_semantic_model_units.py`, so nothing there can stand in for the
installed package. Runs from anywhere else are unchanged.
- **`sm remove-example` names each example in full and says it keeps them (#550).** It cut each
question to about 50 characters, so two that opened the same way couldn't be told apart; its
`rejected` labels and the curation log now carry the whole question. The output also adds an
`examples` list (`area`, `question`, `id` when the example has one, `status`) and a `note` saying
they were marked rejected (kept for audit), not deleted; the existing keys are unchanged. Its
help says the same. `sm curate --help` no longer says `--signer`/`--role` are for approve ops only:
they are recorded in the curation log on every op, and an approve (or an edit of an approved
join's tables, columns or cardinality) also stamps them on the item as its sign-off.
- **Sensitive-column detection reads whole words, checks sampled values and looks inside JSON
(#516).** A new build split column names into words, so `phoneme_error_rate`, `mobile_sessions`,
`headphones_sold`, `adobe_campaign_id` and `ip_address` are no longer flagged (they stay in the
suspected list), while `customer_name`, `contact_name`, `first_name`, `customer_ssn`, `zip_code`,
`pan_number` and `e_mail` now are. Every other name the old rule flagged is still flagged. The
table's one sample (now taken for any table with text or JSON columns) also flags a column whose
values are mostly email addresses, lists phone-number-shaped and birth-date-shaped columns as
suspected (with a caveat on the column, shown by `sm sensitive` and the explorer's PII tab), and
flags a JSON column whose keys hold personal data, with a caveat naming the key paths
(`Contains personal data at: email, contact.phone`); a key that may itself be data (a person's
name used as a key) is shown as `*`. Postgres JSON read through the CLI is recognised too. A column
holding emails or phone numbers (digit-only ones included), or flagged or suspected as personal,
gets no value list, whether read from the data or declared by an enum type or CHECK; a list
with even one key that is a phone number is dropped, even when the sample is empty or unreadable. No sampled
value is written to the model or the report. A birth date stored as a timestamp at midnight is
recognised with or without a time zone (`Z`, `+05:30`), and digit-only values in an identifier
column (`customerId`, `trackingCode`, `id_number`) are not read as phone numbers. `ipv4_address`/`IPv4Address` stay suspected-only. An existing
model keeps its flags on refresh; no model field is added.
- **`metric_binding_unknown_column` resolves schema-qualified source tables (#500).** The check
looked a metric's `source_tables` up by bare table name, so `s.orders` matched nothing and a typo'd
column in its binding was never reported, and `sales.orders` and `archive.orders` shared one column
set, so a column on the other schema's table hid a typo. A qualified source now finds that schema's
table (or a schema-less definition of the name), a bare one every table of that name, and a table
whose name embeds its schema (`name: public.orders`) is found either way. It stays a warning.
### Changed
- **A wide database's proposed subject areas follow its joins, and keep the plural (#513).** A
single-schema database of more than 24 tables used to get one area per table-name prefix, with
any name starting with a shorter one's prefix pulled in (`pro_x` with `products`) and names
singularised by dropping an "s" (`supplie`, `statu`). Tables are now grouped where they join each
other most densely (greedy modularity over the join graph), with no target number of areas, so a
hub table that half the database joins doesn't pull everything into one area; no group passes the
area size limit, and ties go to the tables sharing more words in their names and descriptions.
Tables with no joins go with tables sharing their whole first name word. Area names are the
tables' own words, and the grouping no longer depends on the order the tables arrive in. Only
proposals change: a curated model is never regrouped, while a model whose areas are still exactly
the engine's (a batched first build, or one built by an earlier release and never edited) is
regrouped by `introspect --append`. A join whose two tables end up in one area is written as that
area's join, and a refresh that would write a file the model can't load back is refused, with the
model left as it was.
- **`sm suggest-metrics` no longer proposes a share-of-true rate for every boolean column (#530).**
On a large onboarding these `<table>_<column>_rate` metrics were 698 of the 706 it suggested and
about half of all metrics in the models, while the business's own definitions named one. They are
now left out by default; the output carries one `flag_rates` summary instead (how many boolean
columns could have one, up to 20 of them by name, and how to ask). `--flag-rates
<table.column>[,...]` proposes the named ones (matched case-insensitively, `schema.table.column`
when the table name is in more than one schema; a name that matches no boolean column is reported
in `errors`, and one that matches more than one table is refused before anything is written, with
the matches listed), and `--flag-rates all` proposes one for every flag column
as before. `--cite "<document>"` records where a requested rate is defined: its description reads
`Defined in: <document>.`. Requested rates go through the same dry run, `--write`, `--names` and
`--profile` test run as every other proposal. Rate metrics already in a model are not changed; the
explorer's Review tab can reject them in bulk. The agami-connect skill requests a rate only when the
customer's definitions name it.
### Fixed
- **`sm add` refuses a subject area the model doesn't list, and takes `--area _cross` (#513).**
The loader reads only the areas `datasource.yaml` lists, so a metric added under any other name
was reported applied and validated and then never served. The error now lists the model's areas.
`--area _cross` writes the datasource's cross-area list (`cross_subject_area_metrics.yaml`, or
`_entities.yaml`), validated like any other add, and those items can be signed off through
`sm curate`, where `"area": "_cross"` reads only the cross-area lists and otherwise acts as an op
with no area does (a missing name, or one in both places, fails the batch). Re-adding an item that `datasource.yaml` holds inline replaces it there, so one name is
never defined twice.
- **A foreign key over several columns is now one join on all of them, and a query joining on only
some of them is flagged (#510).** Introspection wrote each column of a composite key as its own
approved many-to-one join, and either one used alone repeats rows (981 became 2,502 in one
database). Every dialect's foreign-key query now returns the constraint name and column position,
and the generic `information_schema` query (Redshift, Snowflake, DuckDB, Databricks) and BigQuery's
match the referenced column by position, which they used to pair with every column of the key.
Where a catalog can't run the generic one (an older Redshift may lack
`position_in_unique_constraint`), introspection falls back to the query it used before, keeps its
single-column keys as before, and leaves out a composite key with a note rather than guess how its
columns pair; the report says when the fallback was used. A
key is written as one `on:` join, its cardinality read from the whole key; single-column joins are
unchanged. Where a catalog gives no constraint name, the columns are still kept apart but left for
review. A refresh replaces the split joins introspection approved for such a key with the one
join and lists that in the preview; if a person signed off any of them, nothing changes for that
key and the preview says so. On the receipt, a join written on part of a declared composite key
makes an aggregate on either side `multiplied`, with a `fan_trap` naming the missing columns; the
query still runs. Code that read only `from_column`/`to_column` now reads every column of an
`on:` join too (adding reference joins no longer raises on one), and the validator warns when an
`on:` names a column its table doesn't define. A join that is a key equality plus a fixed value
(`... AND customers_curr.current_flag = 'Y'`) is not a composite key: its receipt and refresh are
as before, its key column counts as a referenced column, and hiding that column now hides the
join. BigQuery and Oracle keys are kept apart per dataset/owner, a BigQuery key into another
dataset keeps its single-column join, and an `on:` now also parses in the engine's own grammar
(`` `OrderId` ``). A refresh keeps a single-column FK the catalog declares beside a composite
one, and only an `on:` join introspection signed goes back to review when a column is retyped.
The partial-key check reads each join on its own (an aliased self-join and a `USING` join too),
skips SEMI/ANTI joins and a join another declaration names whole, and counts only the equalities
between the join's two tables. A column an earlier `USING` or `NATURAL` join merged counts for
every table it merged, and one whose table can't be told leaves the aggregate `undetermined`. A
key column or table named after a reserved word (`order`) is quoted in the `on:`, and a refresh
matches a split join by its schema even when the schema is written in the table name
(`crm.shipments`), and keeps such a join in its own schema's area only.
A refresh's relabelling
from the database's keys (#511) labels a composite join from all its columns, and a split join the
refresh replaces is not also listed as relabelled.
### Fixed
- **Column scope checks an unqualified column that sits beside a CTE or subquery (#339).** When a
query read a declared table and a CTE or derived table in the same `FROM`, an unqualified column was
skipped, since the CTE might have supplied it. A column the model excludes was then returned:
`WITH c AS (SELECT 1 AS n) SELECT secret_col FROM orders, c` ran, while the same statement without
`c` was refused. The gate now reads what each CTE, derived table, `LATERAL` subquery, `VALUES` list
and table function names in its projection or alias. A column one of them names passes. Otherwise it
is refused when that scope reads a declared table, and a scope with no source supplying the name
defers to the enclosing query as a correlated reference does, so `SELECT (SELECT secret_col) FROM
orders` is refused too. An output with no readable name (`SELECT count(*)` with no alias) does not
count, and the refusal says to name it with `AS` or to qualify the column with the source's alias.
Alias column lists rename by position, a quoted output name keeps its case where the engine does,
and a CTE inside a scalar subquery still sees the outer query. A key a `USING` join compares must
be declared on the side of that join it comes from, and a name a `NATURAL` join compares from a
CTE or subquery must be declared on the tables of that join. An output alias stands for itself
only as a whole `ORDER BY` or `GROUP BY` item, or as a window-function alias in `QUALIFY` (DuckDB,
Snowflake, BigQuery, Databricks), where engines read the alias. Anywhere else, including
`HAVING`, expressions and `FILTER`, the engine reads the input column of that name, so `SELECT
secret_col AS secret_col FROM orders` is refused, and so is an alias referenced in `HAVING`; that
refusal says to write the expression out, and the SQL rules and server instructions now tell the
model to. A parenthesised join is read like the `FROM` around it: the sources inside it name their
columns, its `ON`, `USING` and `NATURAL` are checked, and its alias (`(...) AS w`) reaches only
the columns its tables declare, so `w.secret_col` and `ON secret_col = ...` inside one are
refused where they ran before.
### Added
- **A table can move to another subject area, keeping its joins, metrics and entities (#513).**
The curate op `{"op":"move","kind":"table","name":"<table>","area":"<from>","to_area":"<to>"}`
(`"schema"` too when two schemas share the name) moves the table and its area entry, and creates
the target area when the model doesn't have it. A join that now spans two areas becomes a
cross-area join, and a cross-area join whose two tables now share an area moves into it. Metrics
and entities on that table move with it when every table they read is now in the new area; one
that also reads a table outside it stays, and the result's `notes` names it; one left in another
area moves too once all its tables have reached the new area by separate moves. A join that names
its table by a bare name shared by two schemas is resolved by the area it sits in and gets the
schema written; a move with a join on the table whose other end matches tables in two schemas is
refused, with nothing written. A join to a table no area defines stays where it is, and `notes`
says so. A join the target area already holds is kept once: a copy a person signed off over one
the engine approved, and that over an unreviewed one, taking a description only the other copy
has; `notes` says which was kept. A move is refused when two copies a person signed off disagree.
A composite-key `on:` join moves intact beside another join between the same tables. The batch is
validated and undone on failure (folders it created included, and no `notes` reported), logged
with both areas, and committed when the model is its own git repository. The model explorer has a
**Move to area** control on every table that queues the op.
- **A metric can carry `caveats` (#524).** Tables and columns could, metrics could not, so a limit
of a metric's formula ("gives no value when the start date is missing") went into its description
or calculation, where it reads as part of the definition. A metric now takes an optional
`caveats` list, validated like a column's (no empty entries) and editable through `sm curate`
and the model explorer. It is served wherever the metric is: the metric's entry in
`get_datasource_schema`, metric search, the admin model page, the explorer's metric card and the
review queue. The validator reads it too: a metric caveat saying a missing value counts as 0
silences `metric_null_filled` the way the calculation saying so already did.
**Reading a model:** a metric file that sets `caveats` is refused by a release older than this
one, which does not know the key; files with no caveats stay readable, because the key is left
out of a metric file when the list is empty. Upgrade every server that reads the model (including
deploy images) before writing metric caveats.
### Added
- **A customer's data dictionary is applied as `metadata` in one command (#522).** `sm describe-file`
stamped every description `source: ai`, so a dictionary the customer wrote could only land as their
own text through a script that built `curate` ops, which agami-connect forbids. It now takes
`--source metadata` and reads a CSV (`table`, `column`, `description`, optional `area` / `schema`
header; quoted text may hold commas and newlines) or JSON (a list of those objects, or an object
mapping `"table.column"` to a description) as well as the TSV it already read, by file extension or
`--format`. Each row is matched to a model column, exactly first and then ignoring case, and lands
on the model's spelling. A `SCHEMA.TABLE` in the table column is read as that schema's table when
the model has the schema, and listed under `schema_prefix_stripped`; a schema the model doesn't
have is reported as such. A row that can't be applied comes back under `unmatched` with a reason
(`no_table`, `no_column`, `ambiguous_table`, `ambiguous_column`, `malformed`, `empty_description`),
where a TSV's bad lines used to be skipped without a word; a CSV row with more fields than the
header is `malformed`, and broken quoting fails the whole file (exit 2) rather than merging rows. A
folder with no semantic model is refused (exit 2). A dictionary row never replaces a description a
person wrote; those are listed under `kept_human`. A `metadata` description is taken as written
from the customer's dictionary; it is not checked against the data (see #564). Without the flag
the command stamps `ai` as before. agami-connect applies a dictionary file this way and reports
the unmatched rows.
- **A column's list of allowed values is the whole list, or none (#517).** Introspection built each
`choice_field` from the first 500 rows the engine returned, so on a table of a few thousand rows a
status with 14 values could list 6, and the list read as complete: a question about a missing value
was written against the list, and the reconcile probe graded a correct literal as a query defect.
Values the catalog declares now come first, on a table of any size and with no table scan: a
Postgres enum type, a MySQL/MariaDB `ENUM(...)` column, and a CHECK constraint that only lists one
column's values (`col IN ('a','b')`, `col = 'a' OR col = 'b'`), read on Postgres, MySQL 8, SQL
Server, Oracle, SQLite and DuckDB; a declared list is kept as declared, up to 200 values, integer
columns included. Only a CHECK the engine enforces on every stored row counts (not Postgres `NOT
VALID` or `NO INHERIT`, MySQL `NOT ENFORCED`, a disabled or untrusted SQL Server CHECK, or an
Oracle one that is disabled or `NOVALIDATE`), a CHECK whose name another table also uses is skipped,
and one that casts the column to a non-string type, or casts a listed number in a way that can change
it (`CAST(1.9 AS INTEGER)`), declares nothing. Two columns whose names differ only by case (Postgres
`status` and `"STATUS"`) get no declared list. A declared list that can't be kept (over 200
values, two CHECKs that disagree, or a value the column's type can't hold) gives the column no list
at all. On SQL Server and a SQLite `COLLATE NOCASE` column a string list also gets a caveat that the
database compares values case-insensitively; on MySQL/MariaDB, case- and accent-insensitively. Postgres shows a CHECK's text only to the table's owner (the function that shows
it to any role is refused by the recon gate), so a SELECT-only role reads enums and no CHECKs. When nothing is declared, a small table (the same size rule as before) reads the
list with one bounded `SELECT DISTINCT` per column, at most 1,000 in a run; more than 25 values
gives no list rather than part of one. Keys (`*_key`, `*_sk`, `*_fk`), measures and years get no
list from the data. Values are listed exactly as stored: 'Open ' is no longer trimmed into 'Open',
and a column whose values differ only by spacing or case gets a caveat naming them; loading a model
no longer trims a `choice_field` key either. A refresh leaves an existing column's list alone, so
labels a person filled in are kept; new columns get the new lists.
**Reading a model:** a release older than this one trims `choice_field` keys as it loads them, so it
reads a list holding both 'Open' and 'Open ' as one 'Open'.
### Fixed
- **A table is identified by its schema as well as its name (#258, #337).**
- **Storage.** The served model stored tables keyed by subject area and bare name, so
`sales_data.orders` and `staging.orders` in one area landed on one key and the deploy failed.
Migration `028_model_table_identity` adds the schema to `model_table`'s key. Rows
already stored copy across unchanged and still load, and the next deploy writes them in full.
One area defining the same table twice is now the validator error `duplicate_table_identity`,
which includes `public.orders` written next to `orders` with `schema: public`.
- **`dataset_names`.** It used to strip the schema, so `public.orders` could return
`archive.orders`. It now returns the table in the schema it names. A name declared under two
schemas returns both tables, each keyed `schema.table`. A schema the name isn't declared in is
answered with the spellings that do resolve.
- **Routing map.** On `mode="index"` it publishes `schema.table` where a name is ambiguous, and
passing that name back returns that table.
- **Name carrying its schema.** A table defined as `name: public.orders` can now be selected as
either `orders` or `public.orders`, and the scope gates match it.
- **Joins.** A join in `joins_to_other_tables` whose schema the returned table can't confirm now
stays whole, so it no longer reads `from: orders` with the schema dropped.
- **Scope gates.**
- A model's tables live in the connection's own database. A reference qualified with another
database or project is refused once the credentials say which one the connection is in. When
the credentials don't name one, the qualifier is refused. On Databricks the connection is
always in the workspace's default catalog, so a catalog prefix is refused there; write
`schema.table`. A Snowflake database in the credentials is compared as Snowflake reads it,
upper-cased unless quoted, so `database: analytics` accepts `"ANALYTICS".SALES.ORDERS`.
- A four-part name (`server.database.schema.table`) never matches a declared table.
- `database..table` is refused, saying to write the schema: it reads the connecting user's
default schema, which on SQL Server can be `dbo.orders` beside a declared `sales.orders`.
- On Postgres, Snowflake and Oracle, a quoted name has to be the declared spelling exactly, so
`"Orders"` no longer passes for `orders`, nor `"ORDERS"` for a declared `orders` on Snowflake.
Unquoted names still match without regard to case there. Where case alone tells two declared
tables apart, column scope reads the columns of the one the statement names.
- A dot inside a table name is part of the name when the model gives the schema separately, so
Postgres `public."orders.v2"` is one table.
- On MySQL a table or database name has to match the declared case exactly, quoted or not:
on Linux, its default, `sales.Orders` and `sales.orders` are two different tables.
- The other engines match names without regard to case, as they resolve them.
- **A join on a unique key is labelled one-to-one, so a metric over it no longer carries a false
fan-trap or chasm-trap finding (#511).** Introspection labelled a join from the primary keys
alone, so a join onto a UNIQUE column, such as `passports.person_id → users.id`, came out
many-to-one. Unique keys are now read from the catalog on PostgreSQL (unique constraints and
unique indexes), MySQL, SQLite, SQL Server, Oracle and DuckDB, and kept on the table as
`unique_keys`. Snowflake and Redshift declare unique keys without enforcing them, so one is kept
only when the data bears it out; BigQuery, Databricks and Trino have none. A join is one-to-one
when its columns are the whole of a key on both sides, and the same rule now labels a join a
query writes that the model doesn't declare, a join to a grouped CTE, and the references
enrichment adds. When no key settles a join but its many side turns out unique in the data, the
label is kept and the join waits for review with a note saying so, since data that is unique
today is not a promise. A refresh relabels each join introspection approved itself from the
database's keys and lists it under `joins corrected from the database's constraints`; a join a
person signed off is never changed, and a person's edit of a join's cardinality, columns, `on` or
tables in the explorer makes the sign-off theirs (with no signer recorded, the engine's sign-off
is cleared and the join waits for review), so a refresh leaves the edit alone. Only enabled,
valid keys count: an invalid PostgreSQL index, a disabled SQL Server index and a disabled or
unvalidated Oracle constraint are not read, a SQLite unique index over an expression is not
shortened to its plain columns, and a SQLite unique index written with its own `COLLATE` is not
trusted. Labelling reads each table's own keys, so two tables of one name in two schemas no longer
lend each other a key, at introspection or at query time. At query time a join the model doesn't
declare counts as one-to-one by its keys only when its `ON` is on its own table, that table is read
once in the query, and every other join beside it is one-to-one too; otherwise its numbers stay
`undetermined`. A refresh that can't read a table's unique keys (a failed catalog read or data
check) keeps the stored ones and leaves that table's joins alone, with a note
`unique keys not re-read for <table>`.
**Reading a model:** a table that sets `unique_keys` is refused by a release older than this
one, which does not know the key. Upgrade every server that reads the model (including deploy
images) before re-introspecting with this release. A model with no unique keys stays readable by
older releases. CI that validates the model with a newer release won't catch this.
### Fixed
### Fixed
- **`sm suggest-metrics` prints its proposals before writing, and its bindings run and mean what
they say (#514).** It used to write straight into the model, and every proposal kept from it on a
large onboarding had to be fixed by hand. It is now a dry run by default: it prints each metric's
binding, calculation, source table and flags, and `--write` saves them (`--names` keeps only some,
or a flagged one a person chose). `--profile <name>` test-runs every binding first, one bounded
read per table (opt-in, as a pay-per-scan warehouse bills it): a binding that errors is never
written, and a constant, all-zero or NULL result is flagged `degenerate` and left out unless named.
Proposals on a table under 50 rows are flagged `low_confidence` (from the test run's own row
count when there is one). A table defined by SQL is test-run over that SQL; when that read fails
its proposals are flagged `not_test_run` and each proposal says whether it was test-run. The SQL itself: a mixed-case
column is quoted where the engine folds case (PostgreSQL, Redshift, Snowflake, Oracle), and a
column named after a word the engine reserves (`index` on SQLite, `key` on MySQL) is quoted; a pair of
DATE columns subtracts to days on PostgreSQL and DuckDB, where `EXTRACT(EPOCH ...)` of the integer
failed, and uses `DATE_DIFF` on BigQuery; BigQuery and Oracle timestamp pairs no longer fail; a
flag rate leaves NULL rows out instead of counting them false, and a T-SQL `BIT` flag is compared
rather than used as a predicate. A duration leaves out rows where the end is before the start and
its calculation says so (compared directly on engines whose day difference rounds a reversed
same-day pair to 0), pairs a planned start only with a planned end (and actual with actual),
uses an audit column such as `sys_created_on` only when nothing else fits, and offers an ambiguous
pairing flagged rather than picking one (its reason names up to three of the others). A storage type with no SQL dialect now fails clearly
instead of being written as PostgreSQL.
- **A curate op on a join, entity or metric that doesn't name an area is resolved, or fails the batch (#538).**
An op on a join between two subject areas was skipped unless it named one side's area, so the batch
reported success with that change missing. A metric or entity defined across areas could not be
curated at all, so `sm approve-queue` skipped every cross-area metric. An op with no area is now
searched for in every area's files and in the cross-area joins, entities and metrics. Exactly one
match is applied; an entity or metric whose declared name equals the op's name exactly wins over
others that only share its file slug (`revenue ($)` and `revenue (%)`). No match, or more than one (two joins between the same tables), fails the whole
batch with an error naming the op and the candidates, and nothing in the batch is written;
`from_column` / `to_column` (or `area`) picks one. A join op whose named area doesn't hold the join
also finds it in another area. An op that names an area behaves as before otherwise: a missing
target is reported under `skipped` and the rest of the batch applies.
### Fixed
- **A second-order metric can sit on a base metric that spans several tables, and its table names
are quoted for the engine (#525).** `OUTERAGG({base})` with `inner_grain` used to be refused when
the base metric read more than one table, and the synthesized statement wrote table and column
names bare, so a mixed-case table failed on PostgreSQL. The synthesized inner query now joins the
table the base metric measures to each other source table by the one relationship declared
directly between them. That table is the base's `primary_table` (matched to its source table
with or without the schema), else the one table that declares every column the base's
aggregates read; a `primary_table` that is not a source table, or a base whose measured columns
span tables or can't be placed, is refused and asks for `primary_table`. It is also refused,
with the reason, when there is no such edge, more than one, or the join would fan out the base
aggregate's rows. A bare `inner_grain` column must be declared by exactly one of the tables; a
`table.column` entry (`orders.order date`, `line items.id`) must name a source table and a
column it declares. Identifiers are quoted only where the engine needs it (including words one
engine reserves, such as `index` on SQLite or `key` on MySQL), so a plain lower-case name is
emitted as before; a reserved lower-case name on Snowflake or Oracle is quoted in upper case
(`"LIMIT"`), the name those engines store it under.
Tables with the same name from different schemas in one base are refused. An `inner_grain` entry
naming a declared column (`order date`) is that column, quoted; the base metric's own subject
area is read first for the table's columns.
### Fixed
- **The receipt names the table behind a subquery or CTE alias, and reads a `FILTER` clause (#521).**
`COUNT(*) FILTER (WHERE s.x > 0)` over a subquery `s` listed the reference as `s.x`, and the
aggregate said it names no column. A column qualified by a derived table or a CTE is now followed
through a projection that passes one column through unchanged (bare or renamed) to its table, so
the reference reads `public.t.x`. A computed projection (`x / id AS r`) keeps the alias it was
written with. A qualifier is resolved in the query it was written in, so an alias reused in a
nested query no longer borrows the other query's table. An aggregate's `FILTER (WHERE …)` columns
count as columns it reads, the same as a `CASE` inside it, so a single-table `COUNT(*) FILTER (…)`
is resolved and one over a join can report a fan. A CTE name binds as the scope gate binds it (a
schema-qualified table, a later sibling or a non-recursive self-reference is the table), and a
column list (`WITH s(y, x)`, `AS s(y, x)`) renames outputs by position. A subquery's columns are
not credited to the outer query's aliases: a `FILTER` holding a subquery, an aggregate holding
one and an aggregate computed inside a nested SELECT report `undetermined` (a fan proven on the
outer columns still reports `multiplied`). A body with a star keeps the written alias.
What the scope gate refuses is unchanged, and a value read from a subquery is still not matched to
a metric.
### Added
- **The model explorer can reject in bulk, beside every bulk approve (#531).** Each "✓ Approve all N
proposed" and "✓ Approve N" now has a "✕ Reject all N proposed" / "✕ Reject N" next to it. It acts
on exactly the items shown (the sub-tab, kind group, status filter and search), skipping anything
already approved or rejected, confirms with the count and kind ("Reject 104 metrics?") and asks for
one reason, which is required. Each queued reject op carries that `reason`, which reaches
`curation_log.jsonl` with the op; the model itself does not store it. Like approvals, the rejects
are queued and applied only when the feedback block is submitted. A single reject still carries no
reason.
### Fixed
- **"Approve all" on the Metrics tab names the kind on each op (#531).** The Metrics tab's bulk
approve queued ops with no `kind`, so the Queued tab showed "approve undefined" and the curation
engine skipped them as "op needs kind + name". They now say `metric`, as the Review tab's do.
### Fixed
- **The interpreter resolver no longer picks a Python that can never be brought up to date (#529).**
`connect_resolve.py` checks each candidate for pip and for the agami-core version it carries, read
from installed metadata. A candidate with no pip and an agami-core that is missing or older than the
plugin is skipped, with a `reason` in `candidates_scored`; a pip-less one whose library is current is
still chosen, and `$AGAMI_PYTHON` is never skipped but gets a warning. The output says
`chosen_because`. `sm` and `py` apply the same rule to the Pythons on PATH, so an activated
pip-less venv no longer shadows a working one; a configured interpreter that fails it gives way to
a PATH one that passes, and is kept (so `sm` can say what is wrong) when none does. When the library a Python runs is older than the plugin, `connect_resolve`
(as a `library_older_than_plugin` anomaly), `py` and the scripts' library loader warn on stderr,
naming both versions, the interpreter and the fix (`sm install`, or `AGAMI_PYTHON`). `sm` says
"<python> has no pip" instead of a bare "couldn't install". The plugin version is read from
`plugin.json` by one shared helper, with the cache folder's name as the fallback.
### Fixed
- **Validator: fewer false positives, entity mappings checked everywhere, collision messages name
the table (#520).** `metric_binding_unknown_column` read the aliases of a binding's own subquery
as missing columns: `SELECT AVG(s.r) FROM (SELECT …, SUM(pts) / SUM(maxpts) AS r …) s` warned
about `r`. It now leaves out a column a derived table or CTE the SELECT reads projects (through
a `SELECT *` too, when the star is over another derived table) and a select-list alias used in
the same SELECT's HAVING or ORDER BY; a typo inside the subquery, one read through a star over a
base table, and one an unused CTE happens to project still warn. The loader's rule
for hiding a metric that reads an excluded or stale column is unchanged. An entity mapping to a
table outside its own area was never checked, nor were `cross_subject_area_entities`; both now
resolve against the whole model, with new warnings `entity_mapping_unknown_table` (no area
defines the table) and `entity_mapping_column_not_found` (the table has no such column). They are
warnings so a model that validated before still writes. `cross_area_entity_collision` printed
`?.?.<table>` for a table the area only listed, and so warned when two areas mapped the same
table; it now resolves the table across the model (on the connection the area lists it on, when
two connections define it), compares the resolved table, and prints
`schema.table` with each area's name (the connection too when there is more than one).
### Fixed
- **A subject area named where a table was wanted also names that area's tables like it (#567).**
`get_datasource_schema(dataset_names=["orders"])` answered "'orders' is a subject area, not a
table: pass it as `area`" and nothing else, so in a model whose tables carry a suffix (`orders_v`)
the agent spent a second call finding the table. The hint now ends "or did you mean 'orders_v'?"
with the same names in `did_you_mean`, matched against that area's own tables only, so a
like-named table elsewhere is never guessed and never crowds the area's own out. The activity
log's `missed` entry for a name answered with any wrong-kind hint (an area or a column) now
carries `hinted: true`, so an empty suggestion list no longer reads as "the caller was given
nothing".
- **The forked transport no longer relays a child's whole stderr on an `other` failure (#583).**
Every sanitized failure is rebuilt by the parent from the exit code, because stderr is shared with
anything a library logs during the run. `other` (exit 6) had no fixed sentence, so it fell through
to the relay meant for the authored configuration errors (exit 2 and 3), and a library's line,
values or paths included, reached `failure.message`. It now gets the generic sentence, or a
SQLSTATE sentence (#561) relayed only as an exact match; a stream holding neither goes to the
server log.
- **A datasource.yaml key this agami-core does not read is reported, not silently dropped (#575).**
The loader reads a fixed list of top-level keys, so a model written for a newer agami-core that
declared a field the running one did not know lost it without a word, and a deploy fed by that
load stored the model without it. The model still loads and the field is still ignored, but `sm
validate` lists the keys as an `unknown_datasource_key` warning and the deploy prints them, saying
the model may need a newer agami-core. The load itself stays silent, since the executor loads the
model in a child process whose stderr is its output.
- **On Redshift, discovery finds the tables of a read-only user (#558).** `sm discover`, and an
introspection run without a table list, listed schemas from `information_schema.schemata`, which on
Redshift shows only the schemas the user owns. A read-only role owns none, so discovery reported
"no tables discovered" for a catalog it could read in full; passing `--tables` worked, which hid
it. Redshift now lists the schemas the user has USAGE on (`pg_namespace` with
`has_schema_privilege`), leaving out the system schemas. Other engines are unchanged.
## [0.9.20] — 2026-10-26
### Changed
- **Units can carry a scale and a rate, and a guessed currency is never applied without review
(#515).** A `unit` now reads `<ISO code>[_<scale>][/<period>]`: `USD_millions` shows 12.5 as
`$12.5M` (scales `thousands`, `millions`, `billions`, `lakhs`, `crores`), `USD/day` shows 120 as
`$120.00/day`, and `USD_millions/year` combines both, in answers and charts alike. There is no new
field: a release older than this one reads these strings as a plain label, so the model still loads
everywhere. `suggest-units` now splits a name into whole words, so `total_minutes`, `credit_hours`,
`points_balance` and `total_weight` are no longer taken for money, while `gross_usd`, `value_usd`
and `fx_dollar` now are; each row gains a `suggested_unit` read from the name
(`revenue_usd_millions` → `USD_millions`). `set-units` without `--columns` prints its plan and
writes nothing until `--yes`, and gives a column whose name states a currency that unit, with
`--columns` too (so a subset of the plan is written as planned). A bare
`units` column (`reading_units`) is no longer classed additive at introspection; `units_sold` and
the like still are, and classes already in a model are left as they are.
### Fixed
- **A model outside git keeps a reviewable history of its edits (#528).** agami commits a curation
only when the profile folder is its own git repository, and it never creates one, so on a fresh
artifacts folder an edit left nothing a reviewer could compare: `curation_log.jsonl` listed the ops
with no time and no prior value, and the datasource description and glossary writes weren't logged
at all. Every model write now logs each change with its UTC time (`at`), the model version it
produced (`version`) and, for an edit or a review-state change, the value it replaced (`before`)
beside the new one. The description and glossary writes are logged too, and a curate batch logs
only the ops that took, not the ones it skipped. A new `sm history <root> [--limit N]` prints the
log newest first, and a curate result outside git points to it. Older log lines without the new
keys read as before. The docs no longer say onboarding runs `git init` or that a failed edit is
reverted through git; it is restored from backups.
### Fixed
- **A column's `value_transform` and a join's `on:` parse in the datasource's own SQL dialect, and a
text column of ISO dates gets a date format whatever it is called (#518).** Both fragments were
parsed with sqlglot's generic grammar, so a valid Postgres transform such as
`CASE WHEN code ~ '^[0-9]+$' THEN code::int END` failed validation. A fragment is now accepted when
it parses in the engine's dialect or in the generic grammar, so nothing that validated before
starts failing; a datasource whose engine can't be resolved keeps the generic parse, and the
incremental validation cache includes the dialect. A field inside a JSON column is reached by a
transform on the JSON column that extracts the key itself (`(profile->>'signup_date')::date` on
Postgres, `json_extract(profile, '$.signup_date')` on SQLite); agami-connect now says so.
Introspection set `iso8601` only on columns with a date-like name; it now also sets it on a text
column with any name when at least 20 sampled values are present and every one is a real ISO date
or timestamp. Such a column no longer gets a `choice_field` skeleton.
- **The model explorer counts subject areas as "subject areas", not "schemas" (#527).** The totals
line at the top of the page and the line the renderer prints said "3 schemas" for three subject
areas, which reads as database schemas. Labels that really are about database schemas, such as a
cross-schema join, are unchanged.
### Fixed
- **A metric built on a rejected metric is no longer served (#571).** The loader left out a derived
metric whose base was dropped only when a table or column was also hidden, so with just a metric
rejected, a metric built on it stayed in the served model and validation reported "references
unknown base metric". It now runs on every load, per area and cross-area, and repeats down a
chain of derived metrics. Only a base that was rejected or hidden drops a metric: a base the model
never had is a mistake in the model, and stays for the validator to report. An ambiguity naming a
dropped metric loses that metric, as for any metric not served. The curation view is unchanged.
- **A statement the database rejected says so, by its SQLSTATE (#561).** An execution error none of
the classifier's patterns read came back as `other` with "an unexpected error on the server", so a
client could not tell a statement it should rewrite from a broken server, and guessed. The
database's own text still never travels, since it can carry a value from a row. Its SQLSTATE now
does: five characters, no values, and a class that means the same on every engine that reports
one, so one table covers them all without a rule per database. Class `42` (the statement broke a
rule), `0A` (unsupported feature) and `22` (a value did not fit an operation) come back as `syntax`
with the code and "rewrite it"; `53`/`54` (too heavy) and `40` (a conflict, retry) with the code
and what to do; `42501` as `permission`, `08` as `network` and `28` as `auth` with their usual
sentences; any other class as the code beside the usual sentence. The
code is read from the driver's exception (`sqlstate`, `pgcode`, or `redshift_connector`'s `C`
field), never from its message, and an injected executor can pass it as
`ExecutorError(sqlstate=…)`. An engine that reports no code is unchanged, and a dialect rule (#566)
still answers first. On the forked transport a `syntax` sentence is relayed only as an exact match;
the others travel as every `other` failure does today.
- **`sm enrich-metadata` exits 0 when no metadata preset applies (#526).** Most databases carry no
in-database dictionary, and the command reported that as a failure (exit 1), which stopped a
script running it. It now says no preset applies and exits 0. Naming a preset with `--preset` whose
source tables are not in the model is still an error.
### Fixed
- **A SUM with a zero fallback on the many side no longer reads as a fan trap (#579).** The
receipt flagged `SUM(CASE WHEN orders.status = 'shipped' THEN COALESCE(order_items.quantity, 0)
END)` over `orders JOIN order_items` as multiplied: the `0` belongs to no table, so the check that
decides which table the summed value comes from found none, and the `orders` column in the
condition made the join look like it repeated the value. A `0` or `NULL` adds nothing to a sum
however often its row repeats, so directly under a `SUM` (through `CASE`, `COALESCE`, `IF`,
parentheses and casts) it is now left out. `COUNT`, `AVG` and arithmetic are unchanged, and
`SUM(CASE WHEN … THEN 1 ELSE 0 END)`, which counts the joined rows, is still reported.
- **An entity is unreviewed until a person approves it (#508).** The entity model defaulted to
`review_state: approved`, so an entity written during enrichment without the field came out
signed off, with no signer, and was trusted at query time although no one had looked at it. The
default is now `unreviewed`: such an entity waits in the review queue, and approving it in
`/agami-model` or with `sm approve-queue` records who signed it off and when. The shape reference
agami-connect follows showed `review_state: approved` in its entity example; it now shows
`unreviewed`.
**Existing models:** an entity file that states `review_state: approved` stays approved. Only one
that leaves the field out now loads as unreviewed and appears in the review queue.
### Fixed
- **A join only the metadata preset assumes is not engine-approved (#509).** `sm enrich-metadata`
adds reference joins from two sources: the database's own dictionary, and the preset's list of
standard joins for that platform. A standard join whose values overlapped in the data was written
confirmed and approved by the engine, although this database never declared it. Such a join is
now written inferred and unreviewed, says it is the preset's standard join and not declared, and
waits in the review queue. A reference the database's dictionary declares is still
engine-approved once its values overlap, as a declared foreign key is during introspection.
Introspection already wrote every join it inferred from names and values unreviewed. New tests
now pin that for a database with no declared foreign keys.
## [0.9.19] — 2026-10-25
### Fixed
- **A local `get_prompt_examples` ranks and caps the library, as a served one does (#573).** It returned every
area's examples file verbatim whatever the question (tens of thousands of characters a call on a
large library, rejected examples included), in a different shape from a served deployment. It now
ranks by word overlap with `query`, returns at most `top_k` (default 10) within the same character
budget, drops rejected examples, and replies in the served shape with each example's `id`. A
served deployment is unchanged. Measured on a large local library: about 52,000 characters a call
became about 18,000.
- **A refresh rewrites only the files whose content changed (#472).** `sm introspect --append --tables
<one>` changed one table and re-dumped every file in the model, with keys reordered, empty defaults
added and lines re-wrapped, so a one-table refresh was a whole-model diff. `write_tree` now compares
what it would write with what is on disk, as parsed data (a null value or an empty list counts as
absent), and leaves an equal file alone, keeping its line endings and formatting. The introspection report
says how many files were already up to date. On a real 87-file model, rewriting it unchanged now
touches no files.
- **Renaming a column group carries the ambiguity anchors that name it (#487).** An anchor such as
`orders.dates` used to fail validation after `dates` was renamed, rolling the whole edit back. A
rename (a new group with exactly the old group's columns) now moves the anchor; a split or merged
group is still left for validation to reject, naming the ambiguity.
- **A join rejected at sign-off between two subject areas is no longer served (#543).** Rejected joins
inside one area were already dropped on load, but the cross-area loaders never applied the filter,
so a rejected cross-area join (often the single-column piece of a composite key, which repeats
rows) reached `sm context`, the schema reply and a deploy with no review status, and looked usable.
Rejected cross-area entities and metrics had the same gap. The curation views still see them. A
deployment keeps serving them until its model is redeployed.
- **A statement the engine rejects for a known dialect gap comes back with the rule that fixes it
(#566).** The engine rules (Redshift: no `STRING_AGG`, unit-first `DATEADD`, `LISTAGG` beside a
`DISTINCT` aggregate, boolean casts, …) reached a client only through `list_datasources`, and a client
that already knew its datasource skipped that call and wrote the PostgreSQL the rules warn against.
The database's error is still never relayed; the classifier now recognises the constructs a rule
covers and answers with that rule as a `syntax` failure. Before, `function string_agg(...) does not
exist` read as a missing column (telling the client to re-introspect) or as an unclassified failure. The forked transport relays the
rule only as an exact match against the messages built from the rules themselves.
### Security
- **The server's CA bundle no longer trusts the compromised eMudhra (emSign) intermediates (#504).**
Their private keys were weak enough to forge a certificate for any host, which opens the server's
outbound HTTPS (OIDC discovery and token exchange) to a man-in-the-middle. The `server` extra now
requires `certifi>=2026.10.19`, so every install resolves the fixed bundle rather than whichever one
it happened to get. The image upgrades pip to 26.2.2 from a hash-pinned requirement before
installing, because the pip bundled with `python:3.12-slim` vendors an affected certifi of its own.
## [0.9.18] — 2026-10-21
### Added
- **A missing value is not filled with 0 by default (#478).** Enrichment wrote metric bindings like
`COALESCE(staff_count, 0)` on columns whose description says NULL means "not recorded", which gives
a row a result it should not have and pulls averages and rates toward zero. A new
`metric_null_filled` warning fires when a binding fills a column with a numeric constant through
`COALESCE`, `IFNULL`, `NVL` or `ISNULL` and nothing gives a reason: neither the column's
description and caveats saying a missing value means zero, nor the metric's calculation saying the
definition it follows counts a missing value as 0 and naming that definition ("the KB formula counts
a missing percentage as 0"). A bare "(null as 0)" names no source and still warns, and so does a
statement that denies the zero ("NULL is not zero", "the definition never treats it as 0"). Source
tables written with their schema (`s.operations`) resolve to that table only. A total over no rows (`COALESCE(SUM(x), 0)`), a fallback column and a
text label are left alone. Every model's derived context now carries the row-keeping convention:
a row whose value cannot be calculated stays in a list, ranking or grouping, shown empty and last,
and counts in neither the numerator nor the denominator of an average, total or rate.
agami-connect now says to let NULL propagate, and to tell a recorded value that is missing (left
empty) from no related rows at all (often a real zero, decided per metric).
- **A metric about today measures to today (#482).** Enrichment sometimes bound "days since start"
or "age" to the latest date found in the data, or to a fixed date, so the figure was reproducible.
When the data is not loaded up to today that silently changes the answer and which rows qualify.
A new `metric_today_not_current_date` warning fires when a metric's calculation or description
says today, the current date, the run date or "until now", but its binding has no current date in it: it uses
a fixed date, a `MAX` over the data in a subquery, or neither. Every engine's form of today is
recognised (`CURRENT_DATE`, `NOW()`, `GETDATE()`, `SYSDATE`, `CURDATE()`, `today()`, SQLite's
`'now'` as a date function's argument, not as a compared label), and a date literal beside it,
such as a placeholder end date, is not flagged. agami-connect now says to bind `CURRENT_DATE` and
to keep how far the data lags today out of model text, since the receipt carries it.
- **The validator warns on an average of per-row ratios of two amounts that add up (#480).** A metric
bound as `AVG(online_sales / total_sales)` weighs a store with 100 sales the same as one with 1,000,
so for a group it is not the share of the total and disagrees with the totals beside it. A new
`metric_avg_of_ratio` warning fires when both sides of the ratio are `additive` columns, and
suggests `SUM(numerator) / NULLIF(SUM(denominator), 0)`. A ratio of aggregates, a ratio computed per
group in a subquery, a division used only as a condition (`CASE WHEN a / b > 0.5`), columns of
unknown class, and columns whose copies in different subject areas disagree on their class are left
alone; source tables written with their schema resolve to that table, and a metric whose calculation
says it is an average of each row's ratio is not flagged. agami-connect now says to bind a share of
a total as a ratio of totals — also when the definition gives the formula for one row, since a
binding with no aggregate leaves the query to average the rows — and to mark amount columns
`additive` so the check can see them.
- **The validator warns on figures measured on the data in model text (#479).** Enrichment wrote
build-time counts into metric calculations and descriptions and into table, column and entity
caveats ("covers only 57 of the 231 operations", "427 of 670 accounts have a value"). After the
next load these are wrong, but they read as part of the definition, and sessions repeat them in
answers. A new `measured_figure` warning, one for the whole model, names each such text and the
phrase found: `<n> of <m>` (with a denominator of 10 or more), "covers only <n>", "<n>% of
rows" or "<n>% null", "<n> rows", and "NULL on <n>". Thresholds, weights and unit factors are
not flagged. agami-connect now says to state the lasting fact, not its current count.
### Changed
- **agami-connect binds a value as its label and unit say, even when the data looks implausible
(#481).** Enrichment sometimes decided a stated meaning was wrong because the values did not look
as expected — reading a field stored as `'0.73%'` as a 0–1 ratio because every value was below 1,
or swapping a documented column for a neighbour whose values looked more plausible — and stated the
guess as a rule that every answer then inherited. The skill now says to keep the stated meaning,
record the oddity as a caveat, and ask the user which reading is right before changing a binding.
### Fixed
- **`get_prompt_examples` serves the same examples on every read (#203).** Examples were read in
no set order, so when more of them tied on word overlap than the top-10 cap admits, which ones
were served could change from one process to the next. Ties are now broken by example id, the
order the store's primary-key index already returned in practice, on SQLite and Postgres alike.
- **Introspection escapes a backslash in a string literal where the engine reads it as an escape
(#439).** MySQL, Snowflake, BigQuery and Databricks treat `\` inside a string literal as an escape
by default, and introspection only doubled single quotes, so a schema or table name ending in a
backslash could end the literal early. Those four now double the backslash too. The other
engines are unchanged, since doubling it there would change the name being looked up.
## [0.9.17] — 2026-10-18
### Changed
- **A schema reply is one JSON document (#477).** `get_datasource_schema` used to append its domain
context (the `datasource.md` narrative, the derived model summary and `USER_MEMORY.md`) as text
after the JSON object, so a client that saved a large reply to a file and parsed it as JSON failed
with "Extra data". The same text now rides inside, as `domain_context`. **Clients:** read
`domain_context` instead of the text after the object; the golden-eval runner reads either shape.
- **A table reply sends joins to tables you didn't name compactly (#477).** A `dataset_names` reply
keeps every join between the tables it names in full, and lists each join to another table under
`joins_to_other_tables`: one entry per pair of tables, with its keys as `"named_col=other_col"`
(or its own condition, verbatim), its cardinality and outer-join direction from the named table's side (a `LEFT` join read from
its other end is `RIGHT`), and its schema or join type only when they differ from the usual. Those joins were the largest part of a table
reply, and a statement almost always uses only the tables it named. Replaying a 51-call question
run on a 48-table model: 13% fewer schema characters, with no join dropped.
- **Ambiguities are disclosed progressively (#477).** An entry can be tied to `columns`
(`table.column`), `column_groups` (`table.group`) or `metrics` as well as `applies_to` tables.
The full entry arrives with the reply that opens one of them; any other reply that reaches it
carries it under `ambiguity_signposts` as `{name, when, open}`, where `open` is
`{datasource, ambiguity_names: [name]}`. The new `ambiguity_names` argument returns the named
entries in full; alone, the reply is just those entries, so following a signpost never depends
on how a table, group or area would resolve. The overview no longer carries table-tied entries in full. An entry whose
anchors are all hidden, excluded, stale or rejected is dropped rather than widened. The new fields are left out of a model's
dump when empty, so a model that doesn't use them still loads on older servers. A model that does
use them needs this version: older servers refuse the unknown fields, so upgrade the server before
adding them to a model.
## [0.9.16] — 2026-10-15
### Added
- **Declared ambiguities (#473).** `datasource.yaml` can list `ambiguities`: kinds of question the
model owner knows can mean more than one thing. Each entry has:
- `when`, the kind of question it applies to;
- `options`, each with a `label` and the SQL `how`, with at most one `default`;
- `behaviour`, either `ask` (ask the user before running SQL) or `answer_and_offer` (answer with
the default, say so, offer the others);
- `applies_to`, the tables it concerns.
`get_datasource_schema` returns the entries relevant to its scope, with an `ambiguities_note`
beside them saying how to act on them. That note takes precedence over a rule or example that
picks one option, and it still arrives when a client trims the server instructions. Validation checks the options and that every table exists,
and a refresh keeps the list. Models without it are unchanged. Guidance, not enforcement.
## [0.9.15] — 2026-10-14
### Changed
- **A wide table's schema reply sends its column groups, not every column in full (#449).** For a
table with column groups, a `dataset_names` reply of `get_datasource_schema` lists each group (its
description and column names) and sends full detail only for the grain, the columns a metric in
the reply binds, and columns with a caveat, a value transform or a sensitive flag. The new
`column_groups` argument (`"group"` or `"table.group"`) brings whole groups in full; unknown names
come back in `unknown_column_groups`. Every column stays named, so nothing becomes unreachable.
In the same reply, lean joins (to tables not requested) between the same two tables are merged
into one entry with `column_pairs`; `get_table_context` and the CLI context commands are
unchanged. On a 150-column fact table, replies were about 15% smaller in a live check, with
the agent picking the right groups from their descriptions alone.
- **`/agami-connect` describes the column groups it creates, on every build route (#465).** A wide
table's groups are regrouped by meaning, but kept the engine's descriptions ("Columns named exp*
(grouped by prefix)"), and a model built from a spec workbook skipped the regroup step entirely.
The step now writes a one-line description per group, saying what it holds and which questions
need it, and runs after a spec too. A group still carrying the engine's description is the sign
that a table hasn't been regrouped.
- **Regrouping a table shown by several subject areas works.** A curate `column_groups` edit
updated only the owning area's `expose_column_groups`; another area showing the table kept the
old group names, so validation failed and the batch rolled back. Every area is now updated, by
columns: it shows the new groups that hold any column it showed before. A new group that spans
the old boundary adds its other columns, instead of the area widening to every column.
### Fixed
- **A regroup no longer rewrites another area's own copy of the table.** The curate `column_groups`
edit from #467 updated `expose_column_groups` in every subject area that listed the table,
including an area that defines its own copy, whose groups are independent. That could widen the
copy to every column. Such areas are now left alone; areas that only reference the table are still
updated.
- **Receipts flag a sensitive column whatever case the statement writes it in (#456).** With
`orders.ssn` declared sensitive, `SELECT ORDERS.SSN FROM ORDERS` or `SELECT O.ssn FROM orders o`
reported no sensitive column projected, because names were compared exactly. Table, alias and
column names now match ignoring case, in the check and in the receipt's column list. Nothing is
refused; this changes only what the receipt reports.
## [0.9.14] — 2026-10-13
### Added
- **The model build asks which date a large table is filtered on (#451).** Introspection suggests
every date column of a large table as its `recommended_filters`, and on a warehouse many of those
are load or audit timestamps. The build report now lists each large table with its row estimate
and date columns, `/agami-connect` asks which date a question would mean, and the new
`recommended_filters` curate edit changes the list without touching the row count beside it.
- **Receipts flag a large table pulled in from the outer side of a join, and a timeout points at the
join (#451).** Each table entry on the receipt carries `scan_risk`, a sentence or null. It is set
when the table is large (about a million rows or more, by the model's row estimate) and the
statement reads it from the outer side of a join from tables the model sizes as small
(`small LEFT JOIN large`, which keeps every small row and stops the large table being narrowed
first). When a statement with an outer join hits the time limit, the refusal now describes that
join in the statement's own table names and suggests the inner-join shape, ahead of the generic
advice. It names no model fact, as no refusal may. Advisory only; nothing is refused for it.
### Fixed
- **Views and never-analysed tables get a row count, so large ones are flagged (#451).** The model
sizes each table from the database's statistics, and a view has none: Redshift's `svv_table_info`
lists only tables, and Postgres reports a view or an unanalysed table as `0` or `-1`, depending on the version. A fact
view with millions of rows therefore never reached `large_tables` or got its date columns as
`recommended_filters`, so nothing warned about a query scanning all of it. When the statistics
give no usable figure, the model build now counts the rows instead, with a read-only query under
the normal statement time limit. If the count fails or times out, the size is unknown (a
Postgres `-1` is no longer stored as a size). MySQL and BigQuery are left out until introspection reads carry their own byte and
server-side time limits (#459). The counts share a 10-minute budget per build; tables past it stay
unsized and the build report says how many. Refresh a model built from views to pick the counts up.
- **Receipts find a fan-out when a column is qualified in a different case from its table (#453).**
`SUM(ORDERS.total) FROM orders JOIN order_items …` reported the inflated total as
`not_multiplied`, because the qualifier matched no table in the statement. The fan and chasm
checks now match a qualifier to its table case-insensitively.
- **The "summing a rate" and "summing a balance over time" warnings no longer depend on case.** On a
model whose names are stored in capitals, `SUM(o.rate) FROM orders o` passed without the warning,
and so did `SUM(O.rate) FROM orders o` on any model. Tables, aliases and columns in those checks
now match case-insensitively. The receipt's sensitive-projection fact is unchanged.
- **Fan and chasm findings name tables as the model declares them** (`ORDERS`, not `orders`),
undoing the lowercase naming 0.9.13 introduced.
- **A connection string with no `://` is never shown as a database type.** The local listing could
echo the whole string, password included; it now reports the type as unknown.
## [0.9.13] — 2026-10-12
### Fixed
- **`list_datasources` reports the database type the model declares (#420).** `database_type` was
read from the connection string, which went wrong three ways: it never looked for a tenant's own
connection setting, so on a multi-tenant server it was usually empty; a Redshift warehouse
reached over a `postgresql://` connection read as `postgres`, steering the agent to SQL Redshift
rejects; and a per-user warehouse login has no connection string to read. On a served deployment
it now always matches `engine`, and is empty only when the model declares no single engine. A
local profile still falls back to its credentials when no model declares a single engine.
- **Receipts no longer clear a fan-out written in capitals (#201).** `FROM ORDERS JOIN ORDER_ITEMS`
against a model that says `orders` and `order_items` matched no declared join, and the receipt
reported the inflated total as `not_multiplied`. The fan and chasm checks now compare table
names case-insensitively on both sides, as the table scope check already did, so a model whose
names are stored in capitals is covered too. Their findings name the tables in lowercase.
- **Receipts say when a fan-out cannot change the number (#450).** Each aggregate now carries
`result_affected`. A `MIN`, `MAX`, boolean fold or `DISTINCT` aggregate over a join that repeats
rows keeps `status: multiplied`, because the rows really were repeated, and now says
`result_affected: false`, so neither the agent nor the receipt panel flags a correct number. It
is `true` whenever a fan or chasm trap is found (a `MAX` under a chasm included), and `null`
when undetermined, including an immune aggregate that also reads a subquery or CTE.
`/agami-reconcile`'s part ledger now follows it, so that `null` case stays unresolved there
instead of being confirmed.
- **A chasm-trap finding names the same shared table on every run (#202).** When two tables
qualified, which one was named depended on Python's set order and could differ between processes.
- **The skills' own commands work on Windows (#362).** Claude follows the skill instructions to the
letter, and three of their habits broke under Git Bash, which is what Claude Code uses on Windows:
- **Bare `python3`,** often the Microsoft Store stub there, which runs nothing. Every plugin
script and Python check now runs through the new `scripts/py`, which finds the interpreter
agami-connect configured exactly as `sm` does. The two now share one resolver (`_python.sh`).
It runs in UTF-8 mode, and the `$PY` bootstrap no longer needs a Python to find the right one.
- **Scratch files in `/tmp`,** which Claude's file tools and Git Bash place in different
folders on Windows. They now go in `<artifacts_dir>/local/tmp/`, git-ignored with the rest of
`local/`, which `sm` and `py` create.
- **The `open || xdg-open || start` chain,** which could never reach Windows' `start`. Charts,
the model explorer and links now open through `scripts/open_in_browser`.
- The credentials permission check is skipped on Windows, where NTFS has no Unix modes and the
check refused every install. `tests/test_skills_cross_platform.py` keeps all three habits out.
## [0.9.12] — 2026-10-10
### Fixed
- **Refreshing a model after the database changed works, and keeps what people curated.** Asking
agami-connect to pick up new columns failed three ways:
- **It couldn't read the database's column list once a model existed (#428).** The reads that
build the model went through the check that confines a query to the model's own tables, so
every table in a second batch or a refresh came back with "no readable columns". Those reads now
skip that check; the always-on read-only gate still applies, and seed examples and enrichment
stay confined. That check also now finds the model in a folder chosen through the saved pointer,
where before it found none and stayed off.
- **Re-reading a table threw away its curation (#437).** It was rebuilt from scratch, and the
subject areas were re-proposed. Now an existing table keeps every description, sensitivity
flag, caveat and sign-off. A new column is added; one the database dropped is kept on disk,
marked `stale` and no longer offered to the agent (a query naming it would fail at the
database), and restored if the database has it again; a changed type is updated. Subject areas,
entities, metrics and existing joins are left as they are once anyone has curated them; a new
table joins the area named for its schema, never one already holding a table of the same name.
A bare `--tables customers` refreshes the stored `main.customers` instead of adding it twice. A
merge that doesn't validate isn't written, and the report says so first. `--dry-run` previews
the change as `changed <table>: added …; dropped …; retyped …; key …`, and agami-connect shows
that before writing. A first build in `--append` batches still ends with the same subject areas
as a one-shot build.
- **What a refresh does with each kind of database change.** A changed primary key moves the key
flags, and the grain when the engine had set it (a grain a person or a model spec stated is
kept). A changed type re-derives its date encoding, timezone and (engine-set) aggregation,
resets a value list that no longer fits, and sends confirmed joins on that column back to
review. A table that crosses the deep-table size gets column groups. A table the database
dropped is kept, marked `stale`, and no longer served; it is restored if it comes back. Row
counts are refreshed. A new table never pushes an area past 30 tables, never shares an area
with a same-named table, and carries the model's own connection. A SQL-defined table is never
merged with a database table of the same name; the refresh stops and says why.
- **What the agent sees leaves out anything naming a column or table that isn't served** —
excluded by a curator or stale after a refresh: joins, grain columns, table references, default
filters, entity mappings and metrics on it. They stay on disk. This also closes the same gap
for columns and tables curators excluded before.
- **Curation never noticed a model kept inside a team's git repository (#436).** It looked for
git only in the model's own folder, so it said `committed: false` with no reason. It still
commits when that folder is its own repository. Inside a larger one it never commits (that
would stage unrelated files on whatever branch is checked out), and every result now carries a
`commit_note` saying why nothing was committed and which folder changed. A folder that
repository ignores counts as not in git, and an exported `GIT_DIR` can't send the commit elsewhere. Rollback never depended
on git, and agami-model's skill no longer says it did.
## [0.9.11] — 2026-10-10
### Fixed
- **On Windows, a query's result no longer arrives with a blank row after every real one.** The query
step writes CSV, whose rows end `\r\n`; a piped stdout on Windows turned the `\n` in that into
`\r\n` again, so the local MCP server and `sm seed-validate` read an empty row between each pair —
a 10-row answer reported 21 rows, and a preview showed half its rows. The stream now writes UTF-8
and leaves the rows as csv wrote them, and every caller decodes it as UTF-8 (each Python child runs with `PYTHONUTF8=1` and
`PYTHONIOENCODING=utf-8`). The CSV itself is
byte-for-byte unchanged on every platform (#362).
- **Windows: text is read and written as UTF-8 everywhere, and a test keeps it that way.** 54 reads
and writes, the six credentials-file reads among them, used the ANSI code page: a template with an
arrow stopped three dashboards rendering, and a non-ASCII password failed to authenticate.
`tests/test_windows_text_io.py` now fails on any text read, write or child process that names no
encoding, across the package, the plugin's scripts and the vendored `lib/`. `sm`'s CLI writes UTF-8
too, so `sm --help` and an amount in rupees print instead of raising, and the query step's error
text reaches its caller intact (#362).
- **`sm` finds the interpreter agami-connect configured, even when `python3` is the Microsoft Store
stub.** It read `.config` by running `python3` — the stub, on the machines that need the configured
one most — and then picked the stub. It now reads the path in plain shell, and never takes a
`WindowsApps` stub from PATH (#362).
- **Windows:** the model explorer and prune pages render (`%-d` is glibc-only); a credentials
template saved by Notepad or PowerShell with a byte-order mark is promoted as UTF-8 rather than
crashing; a credentials file is no longer reported world-readable, which NTFS reports for every file;
re-running agami-connect no longer fails moving onto an earlier legacy backup; and a schema named
`con`, `aux` or another reserved device name gets a usable area folder — on every platform, so a
model moves between machines; an existing area with such a name becomes `aux_` on the next
introspection (#362).
- **Claude Desktop from the Microsoft Store gets its MCP entry.** `setup_desktop_mcp.py` wrote to
`%APPDATA%\Claude`, which the Store build never reads; it now writes to the package's own folder
when that exists, sets `PYTHONUTF8=1` for the server, prints this platform's restart steps and log
path, says so when it repoints an entry at another profile, and leaves a config that is not UTF-8
untouched instead of calling it invalid JSON (#362).
- **`sm seed-validate`'s preview shows a year as `2022`, not `2,022`.** It formatted each cell alone
and skipped the label rule answers use; both now share `units.format_rows` (#362).
- **agami-connect no longer needs `TodoWrite`** to show its progress checklist (#362).
## [0.9.10] — 2026-10-07
### Added
- **A model spec workbook builds the model exactly as its owners describe it.** A warehouse whose
joins are not declared — a star schema of views, keys named for their role (`ship_to_customer_key`
joins the customer table), dimensions shared by several facts — introspected as disconnected tables
grouped by name prefix, and a person had no way to say otherwise short of editing YAML. Now they fill
in [`shared/model-spec-template.xlsx`](plugins/agami/shared/model-spec-template.xlsx): subject areas,
which area owns each table and which others list it, the joins (a plain column pair, or a full
condition such as a `current_flag = 'Y'` filter, each with a role), metrics, sensitive columns and
rules. agami-connect recognises the workbook, introspects exactly its tables, and applies it with
the new `sm apply-spec` as one validated step: every problem is reported at once on a dry run,
nothing is written unless the whole model validates, and a failure restores the previous model.
Each join is placed where questions find it (in the area owning both tables, else between the two
owning areas), and what the spec marks approved is signed off as the person running it, after
asking. An optional Columns sheet carries the person's own column descriptions (a data dictionary
pastes straight in), and a Metrics row that is a plain aggregate of one column is flagged on the
dry run: its name belongs in that column's description, the rule `suggest-metrics` already follows.
An optional Grain column on the Tables sheet states the column(s) that make a row unique — what the
engine can't probe on a large table and the fan-out check relies on. The workbook is checked before
anything is built: a row missing a required value is reported by its row number (an empty Tables
sheet would otherwise have meant "introspect everything"), and join types, metric names and source
tables, the fiscal month, and table names two schemas share (write `schema.table`) are validated;
two such tables can't share one area, a join gives either a column pair or a condition (not both),
and two metrics in one area can't share a name once punctuation is ignored. Re-applying keeps the
metrics the spec doesn't redefine, matched by area and name, and the glossary.
agami-connect introspects the spec's tables in one call, then still proposes entities. `model_spec_workbook.py` counts what it read, so a run reports the workbook's numbers
instead of estimating them. Format: [`shared/model-spec-format.md`](plugins/agami/shared/model-spec-format.md).
- **The model explorer's PII tab clears or marks a whole list at once.** A model with dozens of flagged
columns had to be cleared one click at a time. Each list now has "Clear all N shown" / "Mark all N
shown", which acts on whatever the search has narrowed it to (search `email`, clear those). Like
every explorer change it only queues edits until the footer submits; clearing asks first, with the count.
### Security
- **agami-connect's environment check no longer prints the database password.** It echoed the whole
credentials profile into its output, so a password, token or a connection URL's inline password
landed in the agent's transcript on every run. Secret fields now show only that they are set, and
a URL keeps its host and user with the password masked. Nothing read the values; the skill only
checks which fields are present. If you ran agami-connect before this release, consider rotating
the database password it used.
### Fixed
- **A join declared with a fixed-value condition matches a statement that wrote it.** A versioned
dimension joined to its current row (`on: f.key = d.key AND d.current_flag = 'Y'`) could never be
matched: the receipt treated any declared `on:` with a literal conjunct as unreadable, so every
current-version join came back `undetermined` however exactly it was written. The fixed-value
conjunct now counts toward the match, and a statement must write it too — one that leaves the filter
out (or uses another value) is still not credited with the declared join, which is what the old
refusal guarded, and a filter written on another alias of the same table doesn't count. Such a join
stays `undetermined` rather than `undeclared`, since the filter may sit in a WHERE; the join-probe
grader reads it the same way instead of calling it a wrong key.
- **A total behind a join no declaration settles is no longer called clean.** The fan-out check
only weighs joins it can match to a declaration, so a total behind an `undetermined` or `undeclared`
join was reported `not_multiplied` — a positive claim made without the check. It is now
`undetermined`, and its reason names the joins, in the receipt and in `sm prepare`'s pre-flight
alike. Only joins that can feed the number count: its own branch or a CTE, not another UNION arm or a
`WHERE … IN` subquery (a join past the receipt's cap, which is never read, downgrades every number).
- **Rewriting `datasource.yaml` keeps the glossary and the cross-area entities and metrics stored in
it.** `write_tree` wrote only the fields it built, so a re-introspect or a spec re-apply dropped
terminology added with `sm set-terminology`. They are carried over like the description.
- **Signing off an example no longer strips it.** `sm add-example` replaced an example whose question
already existed, so re-saving it with a signer, as agami-connect does to record a validation, dropped
its `confirmed` status and scope tags. A re-save with the same SQL now keeps the fields it doesn't
restate; different SQL is still a correction and replaces the example.
- **Approving a join approves that join, not the first one between its two tables.** Two tables are
often joined several ways (one per date or person role), but the explorer keyed those joins by the
table pair — every date role's join shared one key, so approving one approved, and overwrote, them all —
and `curate` signed off the first match. Each join now has its own key in the explorer, its approval
and edits carry the join's columns or condition (and both tables' schemas, so `sales.orders` and
`archive.orders` stay apart), and a `curate` op that still matches more than one
join is refused with a reason instead of guessed. `approve-queue` names each join the same way, and
a cross-area join is found wherever it is stored (`add_relationships` writes them to their own file,
which approvals never looked in).
- **The validator warns about a join that won't be served.** An area's joins are served only among
the tables it defines, so a join to a table the area merely lists validated cleanly and then vanished
when a question was answered. It now warns, naming the table and the fix (a cross-area join). An
endpoint with a schema is compared by schema too, so owning `sales.orders` doesn't cover `archive.orders`.
- **Metric and entity files are named the same way by every writer.** `write_tree` used the display
name and `sm add` / `curate` a slug, so rewriting a model after `sm add` duplicated items and broke
lookup by name.
- **A missing or unreadable input file is a JSON error naming the resolved path**, for `curate
--ops-file`, `add`, `add-example`, `set-terminology`, `seed-examples`, `describe-file` and `apply-spec` — not a
traceback a skill read as "nothing to apply".
- **Warehouse key columns aren't offered a currency.** `invoice_key` and `payment_key`
matched the money words in their names; `key`, `sk`, `fk` and `nbr` now rule a column out.
- **The plugin run from a checkout uses that checkout's agami-core.** `sm` compares versions only in a
released plugin folder, so from a checkout (`claude --plugin-dir`) a released agami-core installed
earlier satisfied it and the checkout's own code never ran. It now reinstalls from the checkout when
the importable package isn't it.
- **`execute_sql --batch` reports where each result landed.** A relative `out` resolves from the current
directory (now documented), and the manifest records the absolute path.
- **agami-connect's documented commands work as written**: the examples-validation page needs `--title`
and `--profile`, and the pre-seed recount is `review-items --scope preseed`.
- **agami-serve uses the Python agami-connect already proved, and won't write a Desktop entry that
can't connect.** It searched the usual install locations and could pick an interpreter without the
database driver: Claude Desktop then started the server and every query failed. `AGAMI_PYTHON` and
then the interpreter recorded in `local/.config` now go first, as in `sm`, and after installing, the
chosen one must import both the driver and agami-core or nothing is written.
- **A year, period or identifier in a result isn't printed as a quantity.** The result formatter
grouped every unit-less number, so a fiscal year came out as `2,024` and an id as `1,234,567`. A
column named like a label (`year`, `fiscal_year`, `quarter`, `month`, `period`, `*_id`, `*_key`,
`*_code`, `*_number`, `zip`) now shows the value as written; a column with a unit is unaffected.
- **A value list is built only from rows that represent the column.** The sample is the first rows a
table returns, so `current_flag: N` could be recorded for a column that is mostly `Y`, and the SQL
writer would filter on it. A list now needs at least two distinct values, and is built only when the
sample is the whole table or the catalog says the table is small; on a large table or a view with no
estimate, no list is better than a wrong one.
- **A generated description no longer replaces one a person wrote or a data dictionary supplied.**
Enrichment's `source: ai` edits overwrote any column description, including `human` and `metadata`
ones; such an edit is now skipped with its reason. A person can still change either.
- **Redshift late-binding views introspect with their columns.** A view created `WITH NO SCHEMA
BINDING` is listed in `information_schema.tables` but its columns are not in
`information_schema.columns`, so a schema of such views came out as tables with no columns (and,
without an allowlist, nothing at all). Redshift now reads columns from `svv_columns`, which lists
them for tables, views and late-binding views alike. A column read that fails (a dropped connection)
is retried once instead of the table being dropped as having no columns.
- **`add_relationships` accepts a join written as an `on:` condition, and a failure part-way restores
every file.** Labelling the result read `from_column`, which an `on:` join doesn't have, so the call
crashed — after earlier areas' files were already written, leaving the model half-updated. The label
now names the tables, and any error during the batch restores every file it touched.
## [0.9.9] — 2026-10-07
### Changed
- **Below the overview, a schema reply's domain context is cut to the tables in scope (#419).** The
`## Domain context` section rode every `get_datasource_schema` reply whole, and a question makes
several schema calls: in a run of real-world questions it was about a third of everything the
agent received. The overview (no `area`, no `dataset_names`) still carries all of it. An `area` or
`dataset_names` reply now keeps:
- the narrative (`datasource.md`) in full, because its rules change answers, and a conversation
can reach SQL without ever making the overview call;
- the datasource glossary terms that name one of the tables in scope, and those tables'
coded-value legends (a hosted deployment's company glossary stays whole: its terms are
business vocabulary, not table names);
- a count of the terms left out, plus a line saying the overview has the full glossary.
It drops the subject-area listing. What is cut depends only on the call's own arguments, never on
a guess about what the client already holds. On a 76-table model, a one-table reply's context
drops from about 28,000 characters to 12,000–16,000. Re-running 20 real-world questions, a
table-scoped schema reply averaged 40,120 characters instead of 50,851, and the questions
received 20% fewer characters in all. Every question still answered, and the ones whose answer
depends on the date-window rules wrote the same filters.
- **Tool replies are compact JSON (#418).** Every tool reply was serialised with `indent=2`. Its
reader is a model, which gets nothing from indentation and pays for every character of it. Measured
on a 76-table model: an overview reply drops from 59,168 to 50,941 characters, a one-table
`get_datasource_schema` reply from 73,558 to 62,148, and a 100-row `execute_sql` result from 18,010
to 14,030 — about 10% of everything an agent receives across a run of real-world questions. No field
changes, so nothing a client parses changes; only whitespace.
The hosted HTTP server re-serialises every reply to stamp `caller_identity`, so it uses the same
separators — otherwise it would have put the whitespace back on every hosted reply. Files written
to disk (snapshot and batch manifests, the discovery inventory) and the `sm` CLI's own output stay
pretty-printed: people read those. A test fails if `indent=` reappears in either module that builds
replies.
- **`metric_index` lists only metrics with a description; the rest are counted (#406).** A metric
with no `description`, or one that only restates its name, was listed with its own name as its
description: characters on every `get_datasource_schema` call that told the agent nothing the key
did not. It is now counted in a new `metrics_without_description` field instead, and still comes
back in full when its table is opened. On a 76-table model with 22 of 175 metrics described, the
index drops from 12,257 to 2,220 characters on every schema reply. A `query` now also matches an
undescribed metric's `calculation`, through the capped word-overlap ranking only, so a question
can still find it. The validator warns once per model (`metric_undescribed`) with the count and
the first few names.
**Clients:** "every metric appears in `metric_index`" no longer holds. A client that looked a
metric up there by name should also check the full `metrics` block of a table-scoped reply.
- **`sm suggest-metrics` no longer proposes plain `COUNT(*)` / `SUM(col)` / `AVG(col)` metrics
(#406).** #404 kept one when it carried a column `unit`, a caveat, or (for a row count) a grain
that is not one primary key. All of those already reach the agent on the table it opens, and the
formatter carries a column's unit through `SUM`/`AVG` by itself, so the metric added only a name.
The generator now proposes flag rates and start-to-end durations, all left for review, and its
output drops `auto_approved`. The agami-connect skill now writes a one-line `description` for
every metric beside its full `calculation`, including metrics imported from LookML, dbt or a
metrics file, whose source description used to go into `calculation` alone.
### Fixed
- **`sm suggest-metrics` no longer undoes a rejection (#406).** It checked proposals against the
model as loaded for serving, which leaves rejected metrics out, and then wrote each proposal over
any file of the same name. So a re-run reset a metric a person had rejected to `unreviewed`. It
now checks against every metric file on disk, rejected ones included. It still proposes nothing
on a rejected table or column.
## [0.9.8] — 2026-09-30
### Security
- **A `column_scope` refusal lists what the tables it read DO declare (#386).** It said a column
was not declared and not what the agent could use, so the agent guessed again: refused,
guessed, refused. `remediation` now leads with the declared columns of the tables the refused
statement reads — the same columns `get_datasource_schema` shows for them, up to 40 per table
and 5 tables — and the closest listed name for each refused column, as a typo hint. This is a
recorded amendment of the rule that a refusal never lists the declared surface; see SECURITY.md,
"What a refusal may name", for the boundary and the residual.
### Changed
- **`get_datasource_schema` answers a wrong name with the right ones.** A datasource, area,
table or metric name the model does not have now comes back with `did_you_mean` (the closest
real names, or nothing when nothing is close), and a `hint` when a real name was sent in the
wrong parameter: an area as a table, a column as a table, a table as an area. A table scope
naming only unknown tables is refused rather than answered as an empty model; unknown
`metric_names` are reported in `unknown_metric_names` instead of dropped. Values of the wrong
shape are repaired where the intent is clear (`"orders"` for `["orders"]`, `["sales"]` for
`"sales"`) and otherwise refused as `invalid_argument`; an unknown `mode` is refused rather
than silently becoming `summary`. The tool description now says where each name comes from.
- **`get_prompt_examples` refuses an unknown `area`** with the same suggestions, instead of
answering as if it were a real area with no examples. Omitting `area` still returns the top
examples across every area.
- **The activity log records more of what each tool call did.** Each tool call now
records the parameters it was sent, the names the model did not have and what was offered
instead, how many names missed, and the size of the reply. A call that answered with one name
missing is still a success, with a miss count above zero. Migration `027_tool_calls_facts.sql`
adds the five columns; no tool response changes.
### Fixed
- **Introspecting Postgres as the read-only role now finds its keys (#175).** Postgres shows
constraints in `information_schema` only to a table's owner or a role with a privilege other than
SELECT, so the role `readonly-grants.md` creates saw no primary or foreign keys: every grain was
guessed and every join inferred from column names, which can bind the wrong parent. The PostgreSQL
dialect now reads `pg_constraint`, which any role can read. Re-run `/agami-connect` to pick the
keys up; the model gains `confirmed` joins and catalog grains where it had guesses. Redshift keeps
the `information_schema` queries, since it lacks the `LATERAL` and `unnest ... WITH ORDINALITY`
the new ones use. When no declared foreign keys are visible at all, the introspection report now
says the joins were inferred instead of staying silent. A declared foreign key to a table outside
the model (pruned, or in a schema not introspected) is skipped, and the report counts it, rather
than becoming a join to a table the model does not have.
- **A failed `execute_sql` on the HTTP server records its real kind.** The activity log
said `failed` for every failure. It now says which: `syntax`, `timeout`, `auth`, and so on.
- **The read-only guard now speaks every dialect it serves (#395).** `execute_sql` advertises the
MCP `readOnlyHint`, but the dangerous-function deny-list held Postgres primitives only, so a
single `SELECT` could still carry a side effect on every other engine: `SELECT SLEEP(30)` and
`SELECT GET_LOCK('x', 10)` on MySQL, `SELECT SYSTEM$ABORT_SESSION(…)` on Snowflake,
`SELECT reflect('java.lang.Runtime', …)` on Databricks, `SELECT … FROM OPENROWSET(BULK
'/etc/passwd', …)` on SQL Server, `SELECT UTL_HTTP.REQUEST('http://…')` on Oracle,
`SELECT EXTERNAL_QUERY(…)` on BigQuery, `SELECT load_extension(…)` on SQLite / DuckDB. The
deny-list is now organized by engine and covers all of them, and the whole Snowflake `SYSTEM$…`
namespace is refused rather than a member list that would go stale.
- **Three Postgres holes in families the list already claimed.** `pg_notify` — the function
spelling of the denied `NOTIFY` keyword, which `\bNOTIFY\b` cannot see because `_` is a word
character. The advisory-lock family beyond the four names listed (`pg_try_advisory_lock` and the
`_shared` variants — eleven functions, matched by one prefix now). And the siblings of the
already-denied `pg_drop_replication_slot`: `pg_promote`, WAL-replay pause/resume, backup
start/stop, slot create/copy, `pg_logical_emit_message`, and `pg_logical_slot_get_changes`, which
is a destructive read — it advances the slot, so a downstream consumer loses those rows.
- The deny-list names SQL Server's `OPEN…` functions individually rather than matching `OPEN\w+`,
so the ordinary `OPENJSON` still runs, and the false-positive corpus was extended with the names
most able to collide (`sleep_minutes`, `AVG(sleep)`, `lock_count`, `reflection_score`).
- **One deliberate over-refusal, since these are the first deny-list entries that are ordinary
English words.** `WORD(` is also the column-list form of a CTE or derived table, so
`WITH benchmark (region, target) AS (…)` and `JOIN (…) AS benchmark (region)` are now refused,
naming a function the statement does not contain. A lookaround narrow enough to admit them would
let a real call through in some position, and this gate fails closed, so the behaviour is pinned
as `REJECT_CTE_NAME_COLLISION` instead of worked around. If the trade is judged the wrong way
round, the fix is to drop `sleep` and `benchmark` — both are pure time-wasters already bounded by
the executor's per-statement timeout.
- `docs/mcp-server.md` carried a third copy of the Postgres-only function list, beside `SECURITY.md`
and `plugins/agami/shared/sql-generation-rules.md`; all three now say "every engine it dispatches
to". `load_extension` is labelled SQLite-only — DuckDB loads extensions with `INSTALL` / `LOAD`,
statements the opening-keyword step already refuses — and Trino likewise adds no entry.
## [0.9.7] — 2026-09-26
### Changed
- **`get_datasource_schema` stops repeating the join graph at both tiers.** Two wire-shape
changes to the same response, each measured on a wide model (22 subject areas, 76 tables).
`mode="index"` sent `cross_area_relationships` as one entry per declared cross-area edge. The
projection drops the join columns on purpose — mechanics belong on the `dataset_names` tier —
but the column is the only thing telling apart several edges that reach the same pair of
tables, so those serialized identically and every copy was sent: 283 entries for 181 distinct
facts, 32,596 chars. It is now an adjacency map, `{table: [table, …]}`, at 5,112 chars. Names
are bare, which is the only form `dataset_names` can honour: it strips every qualifier and
resolves first-match, so a qualified node would look precise and select a different table.
Where one name is declared under two schemas both edges land on one node — a merge the next
call cannot avoid either, and #258 is its fix. The key is omitted entirely when there is
nothing to route — an empty map is truthy where the old empty list was not.
`dataset_names` sent every relationship *touching* a requested table in full. On a hub table —
a user or group dimension half the warehouse references — that is the whole graph: a two-table
request returned 136 edges, of which 4 joined the two tables asked for, and the block ran
98,724 of the response's 129,858 chars. Every edge still ships, because the scope gate admits
any table the model declares and a correlated `EXISTS` needs only the join key. What changes is
the detail: an edge between two requested tables is unchanged, and every other edge carries its
join — endpoints, columns or the `on:` expression, schemas, cardinality, join type — without
the sign-off block, review state, confidence, subject-area labels or the generated description.
The trust block costs nothing to omit: the receipt recomputes review state and sign-off for
each join the statement actually wrote. `executable` survives whenever it is not `same_engine`,
since a `split` edge written as a `JOIN` cannot run.
Both tiers now also drop `for_questions_about` (deprecated, never written) and `executable` at
its default, so one response no longer omits the common value on one edge and states it on the
next.
- **The SQL dialect rules ride `list_datasources`, not every schema response (#405).** They
describe the ENGINE, so they never varied with the scope being asked about — the same ~1,750
chars were re-sent on all four `get_datasource_schema` tiers and on every call of a multi-call
question. They now sit on the call that answers "which datasource, on what engine", which is
made once: a listing entry gains `engine` (the `storage_type` the model declares) and
`dialect_rules`. A schema response carries `dialect: {engine, rules_from: "list_datasources"}`
instead — it still names the engine, and a client that arrived without listing datasources is
told where to get the rules. On a wide model the `mode="index"` call goes from 50,178 to 48,470
chars, and the saving repeats per call.
All three fields are conditional, and absent rather than null when they do not apply: `engine`
and `dialect` when the model declares no single engine (no connection, or two that disagree —
"which dialect" has no answer then, and guessing is worse than saying nothing), `dialect_rules`
when the engine has no known gaps. `contracts.DatasourceInfo` and `DatasourceSchemaResult`
declare them accordingly.
`engine` is emitted beside `database_type`, not instead of it: that field is derived from the
DSN and reports the connection's scheme, which is `postgres` for a Redshift warehouse reached
the usual way. The two can disagree, so the one the MODEL declares is the one the rules are
chosen by, and it is named.
- **The metric generator stops proposing metrics that restate their own name (#404).** A plain
aggregate over a column whose `aggregation` class already licenses it is not a metric:
`_check_aggregation_semantics` enforces that class on every statement with or without a named
metric, so `orders_total_amount = SUM(amount)` adds a name and nothing else. Proposed per table
and per column, these were most of a wide model's catalogue — 137 of 175 served metrics had a
bare `COUNT(*)` / `SUM(col)` / `AVG(col)` as their whole binding.
`suggest_metrics` now proposes a plain aggregate only when the proposal carries something the
class does not, and only something it actually writes onto the metric: a `unit` (the formatter
needs it), a caveat (appended to the `calculation` prose, so the caveat that justified the
metric arrives with it), or — for `COUNT(*)` — a grain that is not exactly one primary key, which
means it may not be counting things. A table's `default_filters` deliberately does **not**
license one: `execute_sql` does not apply default filters and a bare `COUNT(*)` binding does not
embed them, so a filtered table's plain count says exactly as little as any other. Flag rates and
duration pairs are unaffected — those encode a choice. Re-run against the same wide model, the
generator's plain-aggregate proposals fall from 203 to 58. A curator can still add any plain
metric by hand; this is only what the generator proposes unprompted.
Two consequences worth stating. **A caveat-justified metric no longer auto-approves.** A trivial
binding used to skip the review queue with a system sign-off, and that claimed only that
`SUM(col)` is `SUM(col)`. Such a metric now survives precisely because someone attached a caveat,
the caveat is its whole justification, and it rides as prose nothing verified — so it waits for a
person, for the same reason a flag rate does. A metric carrying only a `unit` is still
judgment-free and still auto-approves. And **`--max-per-table 0` now returns no metrics** rather
than one: the cap was floored at 1 to protect a row count that was unconditional, and it is not
unconditional any more.
## [0.9.6] — 2026-09-23
### Changed
- **Read-only tools now say so on the wire.** Every core tool (`list_datasources`,
`get_datasource_schema`, `get_prompt_examples`, `execute_sql`) is listed with the MCP
`readOnlyHint` annotation, `destructiveHint` off, on both transports (HTTP and the stdio harness
`/agami-serve` wires into Claude Desktop). A client's confirmation policy keys off these hints:
Gemini Enterprise treated each un-annotated tool as potentially destructive and asked the person
before every call, once per distinct argument set, so each query cost a prompt. The hint is opt-in
per registry entry (`read_only: True`, or `tools.register(..., read_only=True)`) and only by the
literal `True`; `create_app` refuses a non-bool value. A consumer tool that writes keeps the
client's default caution. After deploying, reload the connector's actions so the hints are
re-imported.
- **`mcp` floor raised to 1.7**, the first release whose `mcp.types` carries `ToolAnnotations`. An
older SDK would fail every `tools/list`.
- **The `SELECT *` ban is now stated up front, not only inside the refusal (#387).** A star is
refused wherever it appears — the outer query, a subquery, a CTE body, `t.*` — and the served
instructions never said so, so a client met the strictest rule on this surface for the first time
as a refusal. This is the same gap #360 closed for column scope, in the same place, for the same
reason. The gate is unchanged: every projected star still refuses, and `COUNT(*)` and other
aggregates over a star are unaffected — there the star is inside the call, not the
projection, and the instruction says so rather than reading as a wider ban than the gate.
- **And the refusal says what is true of a star** rather than what we happen not to know. The old
sentence — "every column must be named so it can be checked" — read alongside the reasoning that
a star's columns "live in the catalog" led a caller whose star sat over a CTE naming its own
columns two lines up to conclude the refusal was mistaken about their query. It was not: a star
returns columns the statement never names, whatever can be inferred about which ones they are.
The remediation now names the CTE case explicitly, because that is where a caller is most likely
to believe the rule cannot mean them.
## [0.9.5] — 2026-09-19
### Fixed
- **A caller's `datasource` is bounded before it reaches the audit row (#370).** The name is
caller-written text and arrives at `_record_execution` before anything establishes that it names a
datasource we serve, so an arbitrarily long value was stored once per call. The statement beside
it has always been capped; this column was missed. It now shares `LOG_DATASOURCE_MAX_CHARS` with
the refusal log line, so the line and the row cannot disagree about what was sent. Every real name
is far inside the limit — a value that is cut was never one.
### Internal
- **The wheel's contents are asserted (#122).** Every test runs against an editable install, where
`migrations/` and `static/` resolve into the source tree whatever the packaging config says — so
that config was the one part of the repo the suite could not see, and it has shipped broken once
(a missing `migrations/` globbed to nothing and the server booted on an empty schema with no
error). A new test builds a wheel and looks inside it; `build` is declared in the test
dependencies so the guard cannot quietly skip.
- **The test suite no longer reads the developer's own artifacts directory (#293).** `artifacts_dir()`
resolves a pointer at `~/.config/agami/path` and then `~/agami-artifacts`, one level earlier than
the fixtures that isolate files inside that directory — so `organization.yaml`'s real minted
`org_id` leaked into tests asserting `local`. 41 tests failed on a maintainer's machine and none
in CI. Both are now redirected per test; `AGAMI_ARTIFACTS_DIR` is left alone, because a test that
sets it is saying which directory it wants.
## [0.9.4] — 2026-09-17
### Security
- **A statement a person hands to reconcile now reaches the database only through the guard.**
Phase 1.5 used to have the session run the person's statement, its zero-row check and every probe
by hand, on whatever tier the profile queries on. On psql, mysql, snowsql, sqlite3 or DuckDB
nothing checked that the statement was read-only, in scope or bounded, so a pasted statement that
writes met the database with only the database role in its way. The new
`plugins/agami/scripts/check_statement.py` does those steps for one row directory, in code: the
guard's own read-only and recon gates first (a refusal is written to `run.json` as `refused` with
its rule, and nothing runs after it), then the zero-row check and the statement through
`execute_sql.execute_guarded` with the built-in executor, the semantic-model verbs in process, and
the probes through `execute_sql`'s batch door, each still through the guard on its own. It writes
the same files the ledger read before, plus `probes.folded.plan.json` when a near-miss probe runs.
The skill and `shared/statement-check.md` now call it once per row; the question-fit step stays
a judgment the session makes by reading. Grading now needs the database's Python driver in `$PY`
on every tier: without it the script exits `3` with `driver_missing` and the install line, and the
run stops. The run also stops when the semantic model declares an engine its credentials do not
connect to. Checking a row again first clears the files the last check wrote, so an earlier
statement's files are never graded as the new one's. The zero-row check now writes its own outcome
to `zero-row.run.json` whatever it was, so no execution in this phase is unrecorded, and the wrap
it runs drops the statement's terminating semicolon along with any comment around it: a statement
ending `; -- done` used to be wrapped as two statements, which the guard refused, and that refusal
was written nowhere. (ACE-155)
### Added
- **Reconcile can ask agami through agami's own tools, so a row's answer comes from the surface a
person uses.** `run_golden_eval.py --via mcp` serves the cold client the local stdio MCP server and
lets it work: it calls `get_datasource_schema` and `get_prompt_examples` itself, scoping them as it
sees fit, and runs its own statement. Until now that client had every tool switched off and was
handed one pre-fetched schema blob, so a reconcile run never exercised the things that decide
whether a real question succeeds: how a question gets scoped, whether the prompt examples surface
the one that would have carried it, `execute_sql`'s safety pass, or the server's own
instructions. A run could therefore pass on a semantic model nobody could query,
and fail on one that works, with no way to tell which. Two rules make the result comparable
anyway, and both are enforced by the server rather than asked for in the prompt, because an
instruction can be partly obeyed and then nobody knows what the run measured. A query that did not
run ENDS the row: the failure is the finding, since a statement the database rejected or
the safety check blocked says the semantic model or the tool fetching is broken, and a client free
to retry would paper over exactly that. Successful queries are NOT capped, because a client reading a
column's values before it filters on one is a person doing the same; capping them would end the
row and report a defect against a model that works, which is the most expensive error this report
can make. The count of those probes is the measurement instead: it reads how much the semantic
model failed to say up front. The client names the query it answered from and the run checks that
against the server's own trace, so reporting a number with no query behind it, or a statement it
never ran, is an error row rather than an ordinary answer. The run also writes agami's result for
reconcile itself, in code: a statement the server did not run is never run again, and one that ran
is run once more, only for its result, through `execute_sql`'s guarded envelope, from the server's
own record of the statement rather than the client's copy. Leaving that run to
the skill let it pick a command-line tier with no guard, so a statement the server had blocked could
have reached the database another way. `--via context` stays the default, so every golden score
remains comparable with the runs before it.
### Fixed
- **A reconcile card showed nothing of what agami answered.** On a run read from a dashboard
screenshot, every card's Data section read "1 row" and the rows that could not be compared read
"failed", so a reader could not see the numbers that in fact matched. Three display faults, none of
them touching a verdict. The data sample was fetched only for a row whose own query was being
graded, which a screenshot row never is. "Failed" was written for any error row, including one where
agami's query ran and answered and only the comparison could not be made — that row now reads "not
compared", and "failed" is kept for a query that did not run, read from the row's attempts or from
the run record beside the result. And a row where only agami wrote a query rendered one SQL pane,
sitting where the person's query sits in the grid above it, so agami's statement read as theirs:
both panes now render, and the empty one says why it is empty.
- **A reconcile row whose query never ran said something else happened.** When agami's query was
refused or failed, the run still left an empty `actual.csv` behind, and `reconcile.py record` read
that empty file as a result with no columns and no rows. So the row skipped the sentence "agami's
statement did not run" and said the two results were tables that were not compared. On a row that
was only a question, it was worse: the row waited for a person to judge an answer agami never
gave. Now an empty result file is never a result, because a real one always has a header row that
names its columns. A file holding only a blank line counts as empty too. A run record that is
there and does not say `ok` means there is no result either, whatever file sits beside it. A run
record whose own file is empty or not JSON says nothing about the run, so the result file decides.
That is the file being unreadable, not the session saying the run went wrong: a run record holding
only an error the session wrote is a record of a run that failed, and the file beside it is not its
result. The same holds for your own query: when it did not run, the page no longer shows your
result as "0 rows". A query that ran and returned a header with no rows is still a result, and is
still compared.
- **An identical answer no longer scores as different when two of its columns hold the same values
on different rows.** When row order is not compared (reconcile always, and a golden run whose
answer key has no ORDER BY), each column's values are sorted before columns are paired. Two
flags that are each true on half the rows then look the same, and the comparator paired
whichever came first. Pairing each flag with the other's partner misaligned every row, so a
right answer scored below 1.0. A golden column now takes a generated column of its own name
first. A name can mislead too, when a statement swaps two labels or aliases one of two columns
that share a label, so that pairing is checked against the old one and the one that lines up
more rows is kept.
- **A column that mostly repeats one value no longer pairs with a different column.** A flag that
is N on every row agreed with any other mostly-N column on nine rows of ten. So a report said
the rows differed and blamed a column that was right, when the real finding was a missing
column. A column with no same-named partner now pairs on its values only when those values can
tell it apart. When row order is compared, no one value may fill more than half the rows. When
it is not, more than half the values must be distinct, because two flags with similar counts
overlap on most rows as multisets whatever they hold. Otherwise the column is reported missing.
Two costs are accepted. A renamed column of that kind that is wrong on one row is now reported
missing too, rather than as a column that differs on one row. And a score without row order can
change for such items. A different column that holds exactly the same values still pairs.
- **A reconcile table that matched only because a different column holds the same repeated value
is now marked unverified.** Reconcile compares two tables by pairing each of your columns with the
column of agami's that holds the same values. So when your `is_gift` is "N" on every row and
agami returned `is_express`, also "N" on every row, the two paired and the row scored as a match.
Values alone cannot tell a renamed column from a different one, so the comparator is unchanged.
Reconcile has the names and your result. The answers still match (`match` stays `true`), but the
row's status is now `match_unverified`, not `match`, and it reads "same answer, but part of your
query could not be checked". The card says why, naming both columns, and asks you to check that
agami returned the column you meant. The rule is narrow, so a legitimate rename stays a match: it
needs a full match, a pair whose names differ (ignoring case and a table prefix), and a column of
yours where one value fills more than half the rows. A column whose values vary, a one-row result
and a single number are left as they were.
## [0.9.3] — 2026-09-17
### Added
- **A hosted `execute_sql` now requires proof the client looked at the examples (#376).** Clients
skipped `get_prompt_examples`: the schema is indispensable for writing SQL, the examples feel
optional, and the pointer and instruction line from #301 were advice a client could weigh below its
own judgement. After a model change this also meant a change to the examples never reached the
conversation. When a datasource has stored examples, the call now carries `example` — an `id` the
lookup returned and `use`, either `followed` or `shown_only`. A missing example, an id the
datasource does not store (including one a model change removed), or any other `use` is refused on
the new rule `example_required`, and the remediation says exactly what to send. `shown_only` is
always accepted, so nothing pushes a statement toward a poor match, and `followed` is not verified.
Both values are recorded on the call (migration `026`), so the activity log shows how often the
examples fit. A datasource with no examples, and the local path, require nothing. Like
`model_version`, the field is optional in the input schema and enforced in the handler.
- **Every refused `execute_sql` writes one line to the server log.** Refusals were recorded only in
the app database, so an operator watching the server's own log (a container's stderr, or a cloud
log sink) saw a blocked query as an ordinary `200`. The line names the rule, the datasource, the
organization and the `audit_id` that joins it to its `query_executions` row. The datasource is the
caller's own text, so it is written escaped and cut to 200 characters: a newline in it cannot
start a forged line. The line never carries the statement, the refusal's own sentences or the
caller's identity. It is written at WARNING because the served entrypoint configures no logging
and Python's fallback handler drops anything lower. For `audit_unavailable`, which writes no row
by design, it is the only trace.
### Fixed
- **A served deployment no longer names a datasource the organization does not have.** With no
`datasource` named and `AGAMI_PROFILE` unset, the server resolved a profile from
`.config.active_profile` — a setting the local CLI writes — before looking at what the deployment
actually serves. A leftover `.config` on a dev box or a mounted artifacts directory therefore won:
on an organization with one datasource, an omitted call ran against a name from someone's
machine, and `list_datasources` reported that name as active. A served deployment now ignores
`.config`, and `list_datasources` reports `active_datasource` as `null` rather than `default`
when several datasources are served and none is named. The local CLI is unchanged. (#253)
- **`AGAMI_REQUIRE_THREAD_ID` no longer accepts a blank `thread_id`.** A required field only has to
be present, so `""` or whitespace satisfied it without naming a conversation. With the flag on, a
blank id is now rejected as an input validation error — a deployment turning the flag on should
confirm its clients send a real id. Deployments with the flag off are unchanged. (#257)
## [0.9.2] — 2026-09-16
### Added
- **A `sign_in_required` failure kind, for an executor that connects as the person asking.** An
injected executor that exchanges the signed-in person's own credential for a warehouse one had
only exit code 4 to report a missing or unrenewable credential. That reached the caller as `auth`
and "The database rejected the connection's credentials.", and a client relaying it told the
person their warehouse was broken when signing in again was the whole fix. Exit code 11 now
arrives as `sign_in_required` on every transport, with a sentence telling the person to sign in
again (reconnect the connector) and start a new conversation. The `execute_sql` description tells
the agent not to retry and not to describe the database as failing. The executor's own text is
never relayed and cannot reclassify the code. The built-in executor never raises it. An executor
should use `FAILURE_KIND_TO_EXIT.get("sign_in_required", 4)` rather than the literal, so it falls
back to `auth` on an older core.
## [0.9.1] — 2026-09-16
### Added
- **A resumed conversation can no longer query a model that has since changed (#364).** A client
keeps `get_datasource_schema` and `get_prompt_examples` output as text in its context, so a user
who returned to an old conversation after the model was edited, or after an older version was put
back, got SQL written against the model as it used to be. Both tools now return `model_version`,
and on a hosted deployment `execute_sql` takes it back: a missing or different version is refused
on the new rule `stale_model`, whose remediation names the live version and says to fetch the
schema and examples again. The comparison is equality, so putting an older version back is caught
the same way as deploying a new one. The property is optional in the tool's input schema on
purpose — a required one fails validation before the handler runs, with no fix named. A
deployment with no recorded version, and the local path, never refuse on it.
- **The instructions say which columns may be queried, and what a scope refusal means.** Neither
rule was stated anywhere a client reads before writing SQL. "Only columns declared on the model's
tables may be queried" existed solely inside the refusal a statement received for breaking it, so
an agent learned the rule by being refused: on one deployment, five refusals in a row were
plausible names that the model simply does not declare. And the two failure channels taught
opposite lessons for one mistake. A column the DATABASE lacks is documented as repairable
("correct the statement and retry SILENTLY"); a column the MODEL does not declare said only to
relay the refusal's `remediation`, which reads as a dead end and hands the user an instruction to
go and edit the semantic model. Both now say the same thing, and say which fix belongs to which
rule rather than stating one default and hedging it: on a scope refusal the schema the agent
already holds is usually enough, so rewrite with declared names and retry, and on every other
rule the remediation is the fix. Two cases a rewrite does not repair are named, because getting
either wrong is worse than the refusal: a table name matching more than one schema is declared
twice and is repaired by qualifying it, not renaming it; and a column left out of the model ON
PURPOSE is how an author makes one unreadable, and arrives as the same refusal as a typo, so
answering with the nearest declared column instead answers a different question than the one
asked. The columns rule is keyed to the table's own entry rather than to the response as a
whole, since a response sized down to `summary` or `index` carries no columns at all and a rule
keyed on what was read would tell that agent every column it needs is out of scope.
### Fixed
- **A self-hosted server could report an old model version as the live one (#364).** Each deploy
added a version row and never removed the previous ones, wrote them without a date, and the
lookup picked the "newest" by that date. With every date blank, the live version was whichever
hash sorted last; on Postgres, which sorts blanks first, a single undated row also outranked every
dated one. The trust receipt's `model_version` came from that lookup. The table now holds one
row per datasource, replaced and dated on every write (a restore included), and the lookup sorts
any undated row left from before last, on both engines. A deploy also records the hash of the
files it deploys rather than the newest snapshot's name, so an edit without a new snapshot is a
new version, and a model that was never snapshotted no longer deploys as the constant `deployed`.
- **A reconcile card showed nothing in its SQL section when agami's query failed.** The row record
dropped agami's statement whenever the row's status was `error`, so the one row where reading the
statement is the entire diagnosis was the one row that did not show it: a semantic model declaring
a column the warehouse does not have looks identical, from the card, to agami writing a name that
was never there. The rule it came from is kept and narrowed to what it was actually protecting.
An error row still carries no RESULT anyone could mistake for a verified answer (`actual`,
`recorded` and `delta_pct` stay null), and now carries the statement, which cannot be mistaken for
one that answered because the row's own error sentence says it did not. A multi-statement answer
keeps all of its statements for the same reason. `check-run`'s demand for a result file is now on
rows that answered rather than on rows that hold a statement, since a query that never ran has no
result file to point at. Two smaller consequences, both deliberate: a card showing several
statements for an error row no longer marks the last one "compared", because on that row nothing
was; and a row whose comparison failed while agami's query ran now reads "the run's files do not
say why" rather than "agami's query failed", which is what the files actually support.
### Changed
- **`get_prompt_examples` no longer tells a client to leave `area` out.** The steer landed in 0.9.0
so that an `area` call could not hide a better match sitting in another area, and it overshot: on
a served deployment no client passed an `area` at all, so a question that plainly belongs to one
subject area was ranked against every area's library. All three copies of the sentence are gone
(the server instructions, the tool description, and the reminder stamped onto a schema response).
What a call does is unchanged: an `area` still returns that area's examples, plus the cross-area
bucket on a served deployment, and omitting it still searches the whole library.
- **The `area` parameter now describes both serving paths.** It said "cross-area examples still
included", which is true of a served deployment and false of the local file path, where no
cross-area bucket exists on disk and an `area` returns that area alone. One schema described one
path, so a local caller was promised examples that path cannot return.
## [0.9.0] — 2026-09-16
### Added
- **`python -m execute_sql --batch` runs a plan of statements in one process.** Every statement paid
an interpreter start, a full semantic-model load (twice) and a connect, about sixteen seconds for
a query of a tenth of a second, and reconcile's statement check issues about ten per row. The
batch door takes a JSON list of `{id, sql | sql_file, out, area?}`, resolves the semantic model once (a
per-process memo keyed by the model files' count and newest mtime, never caching an absent
model; the hosted path is unchanged), keeps the connection open across the items on Postgres,
Redshift, Supabase and SQLite (dropped after a statement that broke it), and still runs every
item through the guard on its own; each CSV, each `<out>.run.json` and a manifest are written by
the door. Reconcile's statement check runs a row's probes this way on the `execute_sql` tier.
(ACE-137)
- **The reconcile report is built by code from the run directory, never typed.** `reconcile.py
record --run-dir --row` assembles the row record from the row's files and appends it to the
checkpoint (replacing an earlier record for the row; a question-only row nobody graded is
refused, since it waits for the grading page). `render_reconcile_report.py --run-dir` builds the
items itself, takes from the session a words file with `sentence` and `change` only (any other
field is refused), writes `report-items.json` beside the page, stamps the page with the digest
of the items it rendered, and prints the three lines the skill says. `reconcile.py check-run`
refuses to let Phase 3 speak while a row's files are missing or the page's stamp is not the
current items' digest. The skill's 2c writes `diff.json`, 2b writes `agami-run.json`, and 2d and
3a.5 name the verbs. (ACE-136)
- **Agami's answer may be several statements, and the reconcile page shows every one.** The cold
client's reply was read as one string under `sql`; a list, or several statements in one string,
lost everything but the first object or read as unreadable. The generator now keeps every
statement in order (`statements`), answers with the last, and the prompt says so; the ask door
writes `statements` beside `sql`; the row record carries `agami_statements`; the report card's
SQL block lists them numbered with the last marked "compared", and the rows check notes that
agami ran N queries. Only the last is run and graded. (ACE-135)
- **An eighth claim, `outputs`, says what a statement selects.** `sm claims` and the golden run
compared tables, filters, date window, group keys, join keys, ordering and limit, and never the
projection, so two statements selecting different expressions could still read as the same
query. The new claim reads each output expression with its alias peeled (`total` and
`o.amount_total` over one `SUM(orders.amount)` agree; `SUM` against `AVG` differs; `SELECT *` is the
one key `*`). It reports and never gates. Every reader that counted seven now counts eight.
(ACE-131)
- **`sm compare-results --unordered`** compares the rows as a set whatever ORDER BY either statement
wrote, for a caller whose ordering is a claim of its own (reconcile's 2e, in ACE-134). The
comparator's `compare_result_sets` takes the same as `ordered=False`; the golden run is unchanged.
(ACE-131)
- **Four `sm` verbs that grade a statement a person supplied, part by part.** `agami-reconcile` is
learning to take a trusted query as evidence rather than as the answer, and these are the
deterministic checks it will lean on. `sm claims` reports where two statements differ, in the seven
claims the golden runner already compares. `sm compare-results` says whether two result CSVs say the
same thing, through the golden comparator, so a table-shaped answer is judged the way an answer key
is. `sm join-probes` names, for every join a statement wrote, whether the semantic model declares a
relationship between those tables and whether the written key matches the declared one, and emits
the overlap and cardinality probes that would show whether the keys really resolve. `sm filter-values
plan` names, for every value typed into a filter, the column it binds to, whether the semantic
model's list of values holds it, and the probes that would settle it; `sm filter-values judge` reads
the probe results back and grades each value `confirmed`, `model_gap`, `query_defect` or
`unresolved`. None of the four runs SQL: the skill runs every probe through the same execution tier
a question takes. Joins are classified with the receipt's own flags, so a CTE that shadows a
declared table, a `USING`, a comma join, or a declared `on:` this layer cannot read all come back
as open states and never as a settled claim about a key nobody read. A probe that came back empty
is graded as a probe that failed, never as a column that holds nothing. A near miss is a case and
whitespace fold only, an empty list of values reads as not yet decoded rather than as no legal
values, a column marked sensitive is never probed for a value, and a value carrying a backslash or
a control character is never sent to the warehouse, because engines quote it differently. The
overlap probe is the one introspection already trusts, now shared as `introspect.overlap_sql` and
bounded to its 50-row sample on every engine through the new `Dialect.limited`, where it used to
be bounded only where the row-limit keyword was `LIMIT`. (ACE-114)
- **`reconcile.py` reads any input a person brings, and grades a supplied statement part by part.**
Three new verbs beside `parse`, `diff` and `band`, which are unchanged. `intake` reads the four
shapes the reconcile skill will accept, a list of questions, questions with the SQL the person
trusts, labels with numbers, and labels with numbers and the SQL behind each tile, into one row
shape with the question, the statement and the expected value, any of which may be missing. A CSV
whose third column is SQL now yields a statement instead of a label with SQL glued onto it; a
statement whose label matches a tile joins that tile's row. `ledger` grades every part of a
supplied statement from the files the skill wrote beside it: what happened when it ran, what
`sm prepare` and `sm receipt` said, what `sm join-probes` and `sm filter-values judge` reported, and
the probe CSVs the execution tier returned. Four grades, and only measurement earns `model_gap`; a
join that could not be graded leaves the fan-out check on its aggregate `unresolved`, said out
loud. `findings` writes a run's findings, the person's own defects listed apart, and every row's
ledger. Three shared references describe the row, the ledger and how a supplied statement is run
the way the AI's own SQL runs. Nothing here runs SQL or writes to the semantic model. (ACE-115)
- **`agami-reconcile` takes the evidence a person brings and grades their SQL part by part.** Four
input shapes, detected and never asked for: a dashboard screenshot, a table of numbers, SQL the
person trusts (alone, beside a question, or as the third column behind each tile), and a list of
questions with no answers. A supplied statement runs on the road agami's own SQL runs, with the
same guards, and every part of it is graded before anything is compared: a scope refusal is a
finding about the semantic model rather than a crash, a miscased value or a join on the wrong key
is the person's defect, and a part nobody could check says so. Two new row statuses keep a lucky
match out of the keep-offer (`match_unverified`) and keep a doubtful expected value out of the
mismatch count (`expected_doubtful`); `reconcile.py status` applies the rules. Tables are compared
through the golden comparator and the two statements' differences are named by claim. The run
writes its findings and the person's defects under `local/reconcile/<ts>/`. The skill still never
writes to the semantic model; a single fix goes through `agami-save-correction`, which now grades a
pasted statement with the same ledger. The plan-mode refusal no longer assumes a CSV path.
(ACE-116)
- **A grading page for a list of questions.** When a person brings questions and no answers, agami
answers each one and there is nothing to compare against, so the person grades. One page lists
every question with the AI's answer as one cell or a shape, the receipt's signals, and right,
wrong or unsure, with a box for the right SQL or for words; one block comes back. A right answer
becomes the expected value. A wrong answer with SQL is graded like any statement the person
supplies. A wrong answer with words becomes a finding carrying them. Unsure changes nothing.
`render_reconcile_grades.py` and `parse_reconcile_grades.py` follow the model explorer's
paste-back pattern; the page never renders a result row, and a grade decides nothing on its own.
An end-to-end test walks the flawed-inputs chain over the sample store: the wrong join key, the
miscased value, the omitted required filter, the clean count. (ACE-117)
- `sm mentions` quotes every description, caveat, glossary line, narrative paragraph and prompt
example that mentions a table or column a statement reads, and the reconcile ledger puts those
words beside every part that fell short, with a flag when two of them name different values for
one column. Quoted, never graded. (ACE-119)
- Reconcile Phase 1.5g reads a person's question beside their statement and writes
`question_fit.json`; the ledger's `question_fit` part withholds a doubtful row from the keep-offer,
and a fit that was never checked is an open part rather than a silent pass. (ACE-120)
- `shared/plain-language.md` says how reconcile talks to the person in Phase 3: four actors and no
fifth, the thing and never the mechanism, one idea per sentence, the semantic model quoted when it
decided something. Phase 3 points at it. (ACE-121)
- Reconcile's Phase 3 tells every row in four beats, in the reader's order: how we read and checked
the input, what agami did with the question and answered, how it got there, and what to change or
keep. The tables and the offer keep their text. The ledger runs once per row, after the
comparison. (ACE-122)
- One reconcile report page per run, on the plugin's theme, tells every row in the four beats and
takes the person's decisions back as one pasted block; a keep is offered only where the run said
match, and every decision goes through a door that already exists. (ACE-123)
- Reconcile takes input the way connect does: an options prompt for the four shapes, a template CSV
the person fills with a hand-off, and an intake page that shows what was read before anything
runs, with one block back applied by a parser. The grading, intake and report pages share one
stylesheet and the same four beats. (ACE-124)
- Reconcile works five rows at a time: `reconcile.py next-chunk` reads the run's checkpoint
(rows.jsonl) and hands back the next rows, so a long run resumes and no row runs twice; the
report, intake and grading pages filter by status, owner, shape or a word. (ACE-125)
- The reconcile report card is a diff grid built by code: `reconcile.py report-items` templates one row
per check from the run's files, the check named first, yours and agami as values with the differing
tokens highlighted, the sides aligned by construction; one sentence and one action per card. (ACE-126)
- The reconcile report card carries two facts read by code: the result (does the data match; are the two
queries the same) as the pill, and the fix (your query, the semantic model, the examples, the question,
agami again, nothing) as the action. A query written differently is a noted fact, no longer a blocker
on a matching answer. (ACE-127)
- The reconcile card after a second read: the question is the title, columns are compared by the data
they carry (the compare-results score names `column_pairs` and `unmatched_generated_columns`), a bare
column reads as its table's column in the claims reader, the change text and the decision boxes derive
from the one fix, an `example` decision joins the block, the checks panel folds. (ACE-128)
- Reconcile asks agami cold: `run_golden_eval.py --ask` answers one question and `--ask-file` a chunk of
them with the golden run's own generator and context, fetching the context once and spawning the
client per question several at a time; Phase 2b never writes agami's SQL in the reconcile session, and
a missing client is an error row, never the session's own statement. (ACE-129)
- From the first test of the grid: a plain column in a list query is no longer graded as a missing
metric (the receipt's output items say whether they aggregate); a date window written against the
clock (`date_trunc('year', current_date) + interval`) resolves and compares against another such
window; the claim keys read as words on the page; the owner order is your query, then the semantic
model, then the question, with the change text naming the gaps or the extra columns; the two SQL
statements sit collapsed under the grid. (ACE-126)
### Changed
- **The reconcile card shows a verdict and what to do, and nothing else until asked.** Five
statements run against a real warehouse found the card burying the answer: the verdict sat as plain
text between two panels, the same conclusion was written four times in four registers, and a check
saying "the column holds more than 25 distinct values" had the same weight as one that found a
mistake. A card is now the question, one verdict carrying the measurement behind it ("9 of 10 rows
match"), the one thing to do about it, and three closed sections: the answer, the two queries, and
the model checks. Each section carries a one-line summary that names the exception when there is
one and a count when there is not, so most cards need no opening and every card is the same height
at rest. Opening the answer shows up to five rows of the two results side by side, differing rows
first, read from the run's own CSVs at render time so the checkpoint still carries no result row
and the renderer enforces the five-row cap rather than trusting its producer. Three kinds of line
that were true and useless are gone: a wide column having no declared value list (the default state
of every free-text column, and reported even for a query pinned to one city, because the probe
reads the whole column), a join that dropped nothing, and twelve near-identical value checks that
now roll into one counted line. Anything that did not pass is kept whole and in place. The pages
also went from eleven font sizes, including 12.5 and 13.5, to five plus one for data, where
monospace now means "this came from the warehouse, or it is a statement".
- **A verdict a person reads is a sentence, not a code word.** A row could read `expected_doubtful`,
which is the row where the analyst's own query is the thing in doubt, and a check could read
`fan_out: SUM(total)`, the one word the skill's own plain-language rule forbids. The tokens stay
on the wire, where `rows.jsonl`, the keep gate and the `status` verb are pinned to them; what a
person sees now comes from one table in `reconcile.py` and nowhere else. A row reads "same
answer", "same answer, but part of your query could not be checked", "different answer",
"different answer, and your query has a problem", or "could not compare", so the status says both
of the facts it carries: whether the two answers agreed, and whether the analyst's own query
checked out. The report page is handed the colour family and the filter-chip words with its items
and keeps no glossary of its own, so the card, the chips and the chat cannot drift into calling
one row three things. `shared/plain-language.md` points at the table rather than restating it, and
a test refuses a status token in any line the skill puts in front of a person.
- **The result comparator pairs a column that mostly agrees instead of calling it missing.** One
differing cell used to unpair a column: the score fell to 0 with "no generated column carries the
values of: total" for a column agreeing on nine rows of ten, and every reader keyed on the pairs
saw an empty list. Columns still pair on whole-vector equality first; what is left pairs by name
when the two sides spell one (the qualifier and case dropped), or by the highest share of agreeing
rows when more than half agree. The score then counts rows ("9 of the answer key's 10 rows
matched") and carries `column_agreement` beside `column_pairs` and `paired_row_share` over the
paired columns. An item still passes at exactly 1.0; a golden column with no partner at all still
scores 0 with its name. Three pins moved with it: a same-named column of another type, a null
against an empty string, and one differing row now read as a pair that disagrees, not a column
that is absent. (ACE-131)
### Fixed
- **Text past plain ASCII survives `sm` on Windows.** Five `sm` commands — `set-terminology`,
`curate`, `add`, `add-example` and `seed-examples` — read their JSON file in the platform's
default encoding, which on Windows is not UTF-8. An em dash became `—`, and was saved as valid
UTF-8, so nothing flagged it. Worse, that garbled text holds a byte Windows' encoding cannot read
back, so the next read of the file failed — and because `get_prompt_examples` reads every subject
area into one response, one such file dropped the curated examples for every area. Both reads now
use UTF-8, and a file that still cannot be read is skipped on its own rather than failing the
rest. Text already garbled by an earlier `sm` run is not repaired by this; re-save it. (#236)
- **The reconcile card reads how many rows agree, names the side that failed, and still says what
the two queries are when the data could not be compared.** One differing cell used to print "0% of
the values match" for a column that was there; the values row now reads the comparator's share
("9 of 10 rows match, differs in total on 1 of 10 rows"), and "same rows, different columns" needs
the paired columns to agree on every row. Every error row said "agami's query failed" although the
cause was often the person's statement, two empty results or a bare question; the card now reads
the cause from the row's files ("This row could not be compared: your query did not run: …"),
offers agami again only when agami failed, and sends a question-only row to the grading page. The
result pill reads "could not compare", or "same query, answer not compared" and "different query,
answer not compared" when the claims could be read, the new `outputs` claim ("selects") included.
Phase 2e passes `--unordered`, so a different sort is a different query and never a different
answer. (ACE-134)
- **A join on the key the model declares one-to-one is no longer reported as a fan trap because a
second edge exists between the same two tables.** The fan and chasm pre-flight matched a declared
edge to a join by table pair and never read the columns the join wrote, so a subclass view joined
to its base table on its id fanned whenever the model also declared a many-to-one between the pair
on another key, and the same receipt's joins section, which does read the key, called the join
one-to-one. The pre-flight now keeps only the edges whose declared columns the written join
matches; a join on a key the model does not declare, or two tables in scope with no join between
them, keep every edge and today's verdict. (ACE-133)
- **A database failure nobody could read is no longer reported as a syntax error.** Every engine
raises its execution failure with the same exit code, and the classifier read that code back as
`syntax` whenever none of its rules matched the message, so a connection dropping mid-statement
("server closed the connection unexpectedly", "SSL SYSCALL error: EOF detected") told agami-query to
regenerate a correct statement twice and told reconcile that the person's statement was wrong. The
wire-drop messages now read `network` (stop, no retry), and an execution failure no rule reads is
`other`, never `syntax`. The connection reference's exit-code line now matches the code. (ACE-132)
- **A filter value the warehouse spells differently is no longer graded as a mistake in your query.**
The value grader compared the literal your statement wrote, as text, against the column's distinct
values, as the CSV rendered them, and returned a defect on a miss before it read the existence
count it had already run. `WHERE is_active = 1` over a column whose values render as `True` and
`False` graded "not a value the column holds" while its own probe had counted thousands of rows
with it, and the reconcile report told the person their working query was wrong. The count now
decides: a value missing from the list but matched by the statement's own predicate is confirmed,
with a note that the spelling check did not decide and, when the list spells it another way, what
that spelling is. The grade stays a defect when the count was zero, or when the probe did not run
(the note says which). (ACE-130)
## [0.8.8] — 2026-09-15
### Fixed
- **A same-named table in another schema no longer passes table or column scope** (#332). With
`sales_data.orders` declared, `SELECT … FROM staging.orders` used to pass because only the bare
name was compared. A schema-qualified reference must now match a table declared with that schema
(a table declared without one is reachable only unqualified), and an unqualified name the model
declares under two or more schemas is refused as ambiguous, asking for `schema.table`. Column
scope checks a column against the schema actually read, not every same-named table.
- **A CTE name no longer hides a physical table its `WITH` does not enclose.** CTE references are
resolved per reference: a body sees earlier siblings and enclosing `WITH`s, its own name only in
the last arm of a `WITH RECURSIVE … UNION` (its recursive term), a schema-qualified name is never a CTE, and a
quoted name binds only a quoted reference spelled exactly the same.
- The validator warns when one table name is declared under two or more schemas: lookups and schema
serving still treat such names by bare name, so queries must use the qualified form.
## [0.8.7] — 2026-09-15
### Added
- **An organisation can have its own row cap and statement time limit** (#329, engine half). Core
stores no such setting: an embedder registers a provider, `(org_id) -> {"max_rows", "timeout_s"}`,
through `Adapters.statement_limits` or `tools.set_statement_limits_provider`. A missing, `None` or
unusable value (not a positive whole number, or a provider that raises) falls back to
`AGAMI_SQL_MAX_ROWS` / `AGAMI_SQL_TIMEOUT_S`, which stay the deployment default, with a warning in
the log. There is no policy ceiling, only what the engines can represent: a time limit whose native
setting — the limit plus the executor's 5-second skew — would pass seven days (604,800 seconds,
Snowflake's own maximum and the smallest among the supported engines; so 604,795 is the largest
usable limit) and a
row cap of 2,147,483,647 or more (the drivers fetch one row past the cap, in a 32-bit count) are
treated as unusable, from the provider and from `AGAMI_SQL_TIMEOUT_S` / `AGAMI_SQL_MAX_ROWS` alike.
- An evaluation run scores both statements of each case under the named organisation's limits.
- `tools.statement_limit_is_usable(key, value)` is the rule a provider's values are held to (a
positive whole number within those two bounds), public so a settings screen can refuse at save
time what the executor would otherwise decline on every statement.
- The limits are resolved once per `execute_sql` call and held for the whole call, and the forked
child is handed the same two numbers in its environment, so the watchdog, the native bound, the
outer bound and the supervisor still derive from one budget on both sides of the fork.
- `tools.statement_limits(org_id=None)` reports the limits in force for the current (or a named)
organisation; `tools.statement_limit_defaults()` reports the deployment values and the
recommended ones (1000 rows, 30 seconds). `tools.pinned_statement_limits(org_id)` lets a direct
caller of `execute_guarded` apply an organisation's limits.
- The `execute_sql` description states the caller's organisation's numbers, built when tools are
listed rather than once at start-up. A client keeps the list for its session, so a changed limit
reaches new sessions; an existing session meets it in the refusal, which names the number per call.
- **`sm set-description`, and onboarding asks for a datasource description** (#327). The one line
`list_datasources` shows an agent to route a question by could only be hand-edited into
`datasource.yaml`. `sm set-description <root> --description "…"` writes it (validated, committed),
`agami-connect` asks for it on every onboard — with an option to generate it from the enriched
model, as it does for the database narrative — and `model_deploy` warns when a datasource is
deployed without one. A generated line is passed to the command through a quoted heredoc, never
pasted into a shell argument, since database metadata can contain quotes or `$(…)`; a re-onboard
can keep an existing description.
### Fixed
- **Follow-ups to per-organisation statement limits** (#334, #338):
- A row cap too large for the drivers to fetch (2,147,483,647 or more) and a time limit over seven
days now fall back to the deployment value, instead of failing every statement with an
`OverflowError` or a native timeout the engine rejects before the query runs.
- A provider mapping that raises when read falls back like a provider that raises, instead of
escaping the resolver.
- With a provider registered, the HTTP server's tool-visibility predicate runs in the request task
again, as `build_server` documents; only the descriptions are computed off the event loop.
- The `execute_sql` description no longer names "the deployment row ceiling" before stating the
caller's own row limit.
- **A table outside the connection's default schema resolves when the client names it without its
schema** (#258). A large model is served at the `summary` tier, whose area table lists carried a
bare name, so the client wrote `FROM orders` and a warehouse keeping it in `sales_data` answered
`relation "orders" does not exist` — a failed statement and a retry on every such question. Two
changes:
- `get_datasource_schema`'s summary table list now carries each table's `schema`, as the full
and table-scoped tiers already did.
- On Postgres, Redshift and Supabase, when every table in the model declares the same schema and it
is not `public`, a statement runs with `SET LOCAL search_path` set to that schema (plus `public`),
so a bare name still resolves. Any other model — two or more schemas, a table in `public`, or a
table with no schema — gets no path: the path would outrank the schema those tables resolve
through, and a bare name meant for one could silently read a same-named table the model does not
declare. It uses the model the semantic-model pass already loaded, so it applies only when that
pass is on — which is off by default on a server (see `SECURITY.md`); there, the summary tier's
`schema` is the fix.
- **A query that names no datasource no longer runs against a guessed one** (#327). On an
organization serving several datasources, an omitted `datasource` fell through to a fallback (an
env var, an active profile) and the statement ran there — so SQL written for one datasource reached
another and was refused as out of scope. `execute_sql`, `get_datasource_schema` and
`get_prompt_examples` now refuse an omitted `datasource` when more than one is served, naming the
choices (new `datasource_required` rule, decided before any model is consulted). One served
datasource still resolves as before.
- **A table-scope refusal names the datasource that declares the table** (#327). It said "add the
table to the model" when the table was already declared in another of the organization's
datasources; it now says which one to run the query against — only a datasource that declares every
table in the statement — or that the tables live in different datasources and one statement cannot
join them. Only the same organization's datasources are named. Table-scope refusals come from the
semantic-model pass, which is off by default on a server (see `SECURITY.md`), so there the hint has
nothing to rewrite.
## [0.8.6] — 2026-09-14
### Added
- **A Redshift schema response tells the client what Redshift rejects** (#325). The client was told
the engine and still wrote PostgreSQL — `FILTER` on aggregates, `SUBSTR`, date arithmetic in
PostgreSQL argument order, boolean casts — and each statement failed at the warehouse and cost a
retry. `get_datasource_schema` now carries `dialect_rules` for an engine with known gaps (Redshift
today): each construct it rejects and what to write instead, read before the SQL is written. Rules
are per engine and read-path only; an engine with no entry gets nothing.
- **`execute_sql`'s description states the row cap and statement deadline** (#326). It said a result
over "the deployment row ceiling" was refused and never said what the ceiling was, so a client
learned it by being refused. The numbers now come from the same settings the executor enforces
(`AGAMI_SQL_MAX_ROWS`, `AGAMI_SQL_TIMEOUT_S`), with what to do about each. `tools.statement_limits()`
returns both, so an administration screen can show the limits in force.
### Fixed
- **The activity log records the datasource a call ran against** (#328). `tool_calls.datasource`
was the argument the client sent, so every call that omitted it — and the server resolved one —
was logged with an empty datasource. It is now the resolved datasource, and the new
`datasource_source` column (migration 025) says whether the client named it (`explicit`) or the
server chose it (`resolved`).
## [0.8.5] — 2026-09-14
### Added
- **The import door reads an Excel workbook.** `golden_author.py parse --file bank.xlsx` reads one
sheet of an `.xlsx` into the same parse a CSV goes through, with the standard library alone — a
workbook is a zip of XML, so no dependency is added to a plugin that installs with none. A workbook
with several sheets stops and names them, because which tab holds the questions is the person's
call; `--sheet` names it. In a workbook the header no longer has to be the first row: the first row
within the top 20 that names a question column is taken, so a title block above the table is fine,
and every row above the header is reported rather than silently dropped. A CSV's header is still
its first row. Rows keep Excel's own numbering, so a skipped row is reported at the number the
person sees, and trailing rows Excel formats but never fills are dropped rather than reported as
empty questions. Both the Transitional and Strict flavours of `.xlsx` are read. Every coordinate
in the file is checked against Excel's own limits before it is used, and a part declaring a DTD is
refused in any encoding, so a crafted workbook costs a refusal rather than memory. An `.xls` file
— Excel's older binary format — is still refused, with how to get past it, and `--csv` keeps
working. A column the import doesn't read, but that looks like a field it does — a header with
`sql` or `question` in it, for a field the sheet hasn't supplied — is reported rather than
silently ignored, the skill asks the person about it, and `--column sql="<header>"` reads it
without renaming anything. (#261)
### Changed
- **A schema response says when a datasource has stored examples.** Clients often skipped
`get_prompt_examples` (#301): the only instruction to call it lived in the server instructions,
which a host may weight below its own, so curated corrections never reached the SQL.
`get_datasource_schema` now carries `prompt_examples` — how many examples the datasource has, and
a one-line reminder to fetch them — whenever it has any. It carries a count, never the examples:
ranking and returning them stays `get_prompt_examples`' job. The count is datasource-wide even on
an `area`-scoped call, and the instructions and both tool descriptions now say to pass the
question as `query` and leave `area` out unless sure, because an `area` drops every other
area's examples, however well they match.
- **A golden run pays for the model's description once, not once per question.** Every question
starts its own client, and every one re-sent the whole model — about 35k tokens on a 22-area
profile — in a prompt that could not be reused, because it began with the client's own system
prompt naming a fresh temporary directory. So nothing was ever read from cache, and a dozen
questions could spend a subscription's session limit. The part of the context that is the same
for every question — the tool's description of the model, plus any glossary paragraph it lacks —
now goes in the child's system prompt, which the client caches — fenced as reference data, so
text people wrote into a narrative or a note cannot act as an instruction. Only what the question
adds goes with it: metrics the cached description lacks, and its ranked examples. Measured: a second call sharing an 11k-token system
prompt read all of it from cache and wrote 134 tokens. Two smaller savings come with it: the
client's own ~6k-token default system prompt is replaced, and glossary paragraphs the tool already
sends are no longer sent twice. What the generator is told, and how the answer is scored, are
unchanged.
- **A golden run can set how hard the generator reasons.** `--effort low|medium|high|xhigh|max`
passes the client's own reasoning level to every generation; unset keeps the client's default,
as before. Once the model's description is cached, reasoning the answer never shows is most of
what a question costs — measured at 90-97% of its output tokens, against a statement of 50-80 —
so a lower level is the next large saving. The level is recorded in the run's JSON and artifact,
because a score measured at one level says nothing about another.
### Fixed
- **A chain of joins is no longer reported as a chasm trap.** The aggregates section flagged two
measures as inflating each other through a shared dimension whenever the model declared both of
their tables many-to-one to it — even when the statement joined one table to the other and only
then to the dimension, so no cross-product could occur. The receipt then named a join the SQL never
wrote. Two measures now count as a chasm only when the statement did not join their tables to each
other; a statement joining each of them to the dimension separately is still reported.
## [0.8.4] — 2026-09-12
### Fixed
- **A tool result now carries the verified caller's identity.** The hosted HTTP transport
(`mcp_http.py`) stamps a `caller_identity` field onto every tool result that is a JSON object, and the shared
instructions tell the model to resolve a self-referential term ("my", "me", "I", "mine") against
it rather than against a stored example's SQL — so a self-referential question has a real
identity to anchor on instead of inheriting whichever past example happens to match closely
(ACE-118). `caller_identity` is reserved and server-injected: it always overwrites a same-named
key a tool's own JSON body might already carry.
- **The eval's generator can start on Windows.** The client's Windows runtime (Bun) needs
`SystemRoot` set to initialize WinSock before its first network call; without it the child exited
immediately on a message `stderr=subprocess.DEVNULL` discarded, so every item in a Windows run
errored identically with "the generator exited without answering" — indistinguishable from a real
client or model problem. `SystemRoot` joins the child's environment allowlist, inert everywhere
the key doesn't exist. (#280)
- **The save door stops committing a query's result rows to git.** Confirming a golden-dataset item
wrote the confirmed query's actual result rows into a `recorded` block, into a path with no
gitignore exclusion — any real data a golden question's query returned ended up in version
control as a side effect of confirming an answer key. Nothing needed those rows: scoring
re-executes `expected.sql` live and never reads `recorded`, and the explorer already refused to
render row content for the same privacy reason. The save door now stamps only `recorded.at`. (#281)
- **The eval sources its context from the product's own `get_datasource_schema` tool, not a
hand-rendered copy of it.** The script's own rendering of the whole semantic model measured ~60k
tokens per question on a live 22-area profile — a 43-case run cost ~2.6M tokens, enough to
exhaust a subscription twice. Worse than the cost: it scored a generator against a context no
real session ever sees, since the shipped product always narrows through this same tool. Calling
it in-process, with no `mode` forced so `auto` picks verbosity the way a real session does, cuts
token cost by roughly half and — verified live against a real warehouse, twice — reproduces the
correct join every time. Known gap: called this way, with no `dataset_names` narrowing, the tool
does not return join cardinality or entity aliases; a session that narrows to specific tables
sees more than this single unnarrowed call does. (#277)
### Changed
- **Importing a question bank confirms the rows that already carry an answer.** A row with no `sql`
is still written unconfirmed — a question with nothing yet to check — but a row whose sheet
already carried a statement now lands `sql_confirmed: true` on import, with `confirmed_by.method`
naming the sheet as the source. Previously every imported row was written unconfirmed regardless,
which meant a pre-validated question bank could never gate a run without re-confirming every
answer a second time through the save door — reporting scores nobody's CI could act on. The
statement is still carried through verbatim and never generated by the import itself.
## [0.8.3] — 2026-09-08
Three additive fields on the activity record, each answering a different half of "what produced this
answer?" — the database identity a statement ran as, the warehouse's own id for it, and the AI model
a client says it is running. All three are nullable, written by the insert that already writes the
row, and read by nothing that makes a decision.
### Added
- **An execution record can name the warehouse's own id for the statement.** Some engines mint a
per-execution id, and it is the key to their own logs — the plan the statement got, how long it
queued, how much it scanned. Recording it lets somebody follow a row from here into a system this
database does not own and cannot replicate.
The column is `warehouse_query_id`, and this side treats it as **opaque**: nothing joins on it,
parses it or fails without it. That is a limit on purpose rather than an unfinished thought — the
moment the value means something here it becomes a thing that can disagree with the warehouse, and
a pointer that argues with what it points at is worse than one that says nothing.
**Which engines can answer is not decided in this library.** No database type or engine list
appears in the recorder or the seam: an executor that can name its statement reports one through
`ExecResult.warehouse_query_id` or `report_warehouse_query_id()`, and one that cannot never does.
Adding an engine is therefore a change where the connection lives, with no migration and no change
to the record — and there is a test asserting the field carries no engine name near it.
Named for the concept rather than for whichever warehouse was first, because a vendor-named column
becomes a schema change the day a second engine has one. Null where the engine has none, where the
executor did not ask, or where the ask failed — all four nulls, the pre-migration row included,
are claims rather than gaps.
Migration `023_query_executions_warehouse_query_id.sql`, one nullable TEXT column, portable across
SQLite and Postgres. Nothing reads it yet; nothing updates it.
- **An execution record can name the database identity that ran it.** `query_executions` said what
ran, when, for whom, against which model and with what verdict, and never said who the warehouse
authenticated. Where every statement runs as one shared account that is worth little; where a
deployment runs each statement as the person who asked, it is a different identity per row and
the record could not say so.
The column is `executing_identity`, and it holds **what the connection says it is** — not the
signed-in principal, not the subject of the token that opened it, not a namespace configured on
the profile. Those are statements of intent, and a row built from intent would report the account
we meant to run as even on the day a misconfiguration ran everything as somebody else, which is
the day anybody reads this column.
An executor reports through one of two seams: `ExecResult.executing_identity` for the ordinary
path, or `report_executing_identity()` at connect time, which is what lets a statement that
**fails after its connection was opened** still be attributed — the case the column is most worth
having for. Both are optional; an executor that reports nothing is not doing anything wrong.
**The built-in executor reports nothing, deliberately.** Obtaining an identity would cost a round
trip on every query in every self-hosted deployment, to record the same shared account each time —
a price paid by the people who get the least from it. Those rows stay null, and null here is a
claim (no executor reported one) rather than a gap, alongside the two other honest nulls: a row
written before the column existed, and a driver that could not answer. On the forked stdio surface
the column is null too, by the same construction that already leaves `error_detail` null there —
the child holds the connection and the parent writes the row.
Migration `022_query_executions_executing_identity.sql`, one nullable TEXT column, portable across
SQLite and Postgres. Nothing reads it yet; nothing updates it — the row stays append-only and the
value is written by the insert that already writes it.
- **A tool call can record the AI model the client says it is running.** The activity log recorded
who called, which tool, the statement and the outcome — everything the server observed — and never
what was driving. Over MCP the model belongs to the client and the server never sees it, so an
operator reading a run of unusually poor statements could not tell whether the client had moved to
a different model that week. That is the first question somebody asks, and the log had no answer.
`client_model` is an optional property on all four tools and a nullable column beside the other
self-reported ones. It is **evidence of a claim and never a fact**: models self-identify
unreliably, nothing checks the value, and nothing branches on it — it is never a basis for an
access decision, a bound, a bill or an audit conclusion. A client-controlled value that could
change what the server does would be far worse than an empty column, so a test asserts against the
source that no reader of it exists outside the record, the writer, the read list and the render.
There is deliberately **no list of known models** to check it against. A closed set would refuse a
model that shipped after the set was written, which is a certainty rather than a risk; storing what
was said, unvalidated and marked as unvalidated, is the only account that stays true. The admin
activity view shows it marked `· self-reported`, beside the agent's own framing.
Bounded by the writer at 200 characters — many times the longest honest model id — and with no
truncation flag, on the same reasoning the refusal detail already carries: nobody re-runs a model
id, so a flag would be a column false on every row ever written. Null is the ordinary case; a
client that reports nothing is behaving correctly, and so is one built before this existed.
Migration `024_tool_calls_client_model.sql`, one nullable TEXT column, portable across SQLite and
Postgres.
## [0.8.2] — 2026-09-07
Two reliability fixes to the eval, both found by running it for real: the client couldn't be found
on a non-interactive shell's `PATH`, and the test suite could be talked out of substituting a
scripted generator for the real one — silently, the way it already happened once.
### Fixed
- **The eval finds the client where it installs, and says so early when it cannot.** A run whose
`claude` client was not on the shell's `PATH` spawned it once per case, failed identically every
time, and presented a wall of "the generator command could not be started" — which reads as a
catastrophic model regression and is a `PATH`. It cost a 43-case run in the field.
Two changes. The client is now **resolved** rather than assumed: `AGAMI_CLIENT` if set, then
`PATH`, then the handful of directories it actually installs to. That is `scripts/sm`'s idea
applied to the other executable this package spawns, and for the same reason — a user-local
install directory is not on every non-interactive shell's `PATH`.
And when it still cannot be found, **one probe before the loop** turns N failed spawns and up to
2N warehouse queries into a two-second refusal that names the cause and what to do about it. Only
an unstartable client stops a run: a generator that starts and declines the probe is one answer
short, which the loop already reports per item. `--skip-preflight` refuses the probe.
- **The eval names its generator once, so the test suite cannot be talked out of substituting it.**
`run_golden_eval` constructed its generator in two places, and every test replaced the class by
the name those places used. Swapping the default meant editing the two construction sites — which
left the substitutions pointing at a class the command no longer named. Nothing failed: the suite
went on passing while spawning a real client, once per case, against the operator's own account.
The generator is now chosen at a single module-level name, `GENERATOR`, that both construction
sites and every test go through, so a changed default carries the substitutions with it. Two
tests hold that shut — one parses the command's own source and fails on any generator built
outside that name, the other pins what the name is currently set to, so moving it is a decision
someone makes rather than a diff that passes.
## [0.8.1] — 2026-09-07
Everything below came out of running the golden-dataset feature against a live warehouse for the
first time. One fix is the difference between the feature working and not working at all; the rest
is what a first real run and its review turned up.
### Added
- **Saving a golden item checks it against the profile's own examples first.** The curated example
library is where a team's conventions live — which of two equally correct date columns they answer
with, how they phrase a join — and it is what agami reads when it answers a question for real.
Nothing in the write path had ever looked at it, so an answer key could contradict the convention
and then fail every future run while blaming the model.
That is measured rather than imagined. On the first real dataset authored against a live
warehouse, nine of fifteen items failed and six were one mistake: the keys filtered on one
timestamp column where that profile's examples used another, 24 times out of 36.
The save door now ranks the nearest example for the question, reads both statements, and stops
with exit `1` and a `needs_confirmation_convention` payload when they disagree — before writing,
because a warning that arrives after the key is on disk is a warning about a file somebody now has
to decide whether to undo. `--confirm-convention` writes it anyway: a golden item may legitimately
depart from convention, and that is sometimes exactly why one is written.
Only three claims can stop a save — the tables read, the filters written, the resolved date window
— because those are the ones that change which rows are counted. A save only stops when the CLI
itself reports `high_confidence`, so a library that does not cover the question has no opinion to
hold anyone to. And the check is total: no model, no CLI, a profile mid-rebuild, all return no
opinion rather than blocking a correct answer from being written down, and the whole check is
bounded by one wall-clock budget rather than a timeout per subject area.
`agami-reconcile` is taught the second meaning of exit `1` in the same change, since a promotion
that met it and answered with `--confirm-replace` would loop on a payload with no before and
after to render.
- **`/agami-eval` says what a pass rate is not.** A run scores a generator with every tool switched
off and one attempt, because that isolation is what keeps it from reading the answer key. The
agent that answers the same question in production has four tools and can iterate. A dataset is
also deliberately not a random sample. So a run measures regression, and quoting its pass rate as
agami's accuracy is wrong in both directions.
### Changed
- **A reconcile run now teaches as well as tests.** Its only write was promoting agreeing rows to a
golden dataset. Nothing went to the curated example library — the one path there was the referral
to `agami-save-correction` when a number *disagreed*. So a run that proved twelve statements
correct against the customer's own dashboard taught agami nothing from any of them, while an
argument taught it something.
Agreeing rows are now split between the two. They cannot go to both: an item that is also an
example ranks first for its own question, the model reproduces it, and the test can only ever
pass. They came from one dashboard, so they are the same family of question, which is exactly what
makes holding some back meaningful.
**The product does the splitting.** Which rows should teach depends on what the example library
already covers, and asking a user that is asking them to guess at a library they have never
opened. `sm examples --query` already answers it: its `high_confidence` flag is the product's own
judgement of "we have a close example for this", so a row it does not cover teaches and a row it
does becomes a test. Below four agreeing rows nothing is split, and never more than half the
batch is sent to the examples so onboarding runs still write tests.
**The user still makes one decision, and it is not the split** — one yes for the batch, with the
breakdown shown. Saying what went where is mandatory rather than decorative: adding examples
changes how agami answers future questions, and that must not happen silently behind a button
that says "keep these numbers".
A promoted row passes `--confirm-convention` to the save door. The statement being saved is the
one agami generated *from* those examples minutes earlier, so the convention check would ask the
person to confirm a departure they never made.
- **The golden run report shows the difference it already worked out.** Reading a failure meant
diffing two SQL statements by eye, from a page that had computed the answer and kept it in a file
beside itself. Six things changed, all in the renderer and its template, none of them touching the
comparator, the claim reader or the contract:
- **A gated item is no longer described as failing to reproduce the answer key.** The verdict was
derived from `passed`, which a gate turns false — so an item whose rows matched the answer key
exactly was announced as not having reproduced it, next to a perfect score. Reproducing and
passing are different questions, which is why structure gates separately from rows, and the page
now says both: *"Same rows as the answer key, but not the same question."* The verdict is also
phrased as what happened to the data rather than in the dataset's own vocabulary — the pill
beside the question already says passed or failed, so the line's one job is whether the rows
matched.
- **The accuracy is gone from the page.** Over a real fifteen-case run every value was exactly
1.0 or exactly 0.0, which is structural: a row-count difference, an unpaired column and an extra
column each short-circuit before an overlap is computed, and columns pair only on equal value
vectors. On the narrow occasion a value in between is reachable, the comparator already writes
the same fact in words. So the number was redundant when it meant something and misleading
otherwise. The score itself is unchanged and still decides the verdict.
- **A gate names the column it fired on**, instead of one fixed sentence saying that *a* filter
was missing. It also stops describing a differing date window as a missing filter: structure
gates twice, and the page assumed one.
- **Every claim that differs is drawn, not just the table set.** The page kept `claims[0]` and
dropped the other six. In a structural failure the table set reliably *agrees* — both statements
read the same table, which is why the comparison got far enough to gate on something else — so
the page reliably showed the claim that said nothing while `filter_predicates`, which explains
the failure, was written to the JSON artifact and never rendered.
- **The run timestamp reads as a date** rather than `2026-09-06T11:47:38+00:00`.
- **The summary shows three tiles, not five**, surfacing "not compared" and "couldn't run" only
when they are non-zero and giving them a sentence when they appear. They are deliberately not
folded into "failed": a run of nothing but errors would then read as green, which is exactly
what the `1` / `2` exit codes exist to tell apart.
- **A claim reads the way a person writes it, in a table rather than a run of text.** A
difference was one separator-joined line per claim, holding the claim reader's own field names
(`filter_predicates`) and its functional rendering (`gte(created, '2024-01-01')`). Both are
deliberate where they are written — a claim rides on tool output a model reads as
server-authored, and something that looked like SQL would be read as SQL — but that argument is
about a model's context and does not reach a local page a person opens.
So the report now prints `created >= '2024-01-01'`, labels each claim in English ("Filters",
"Grouped by", "Date range"), and lays the two sides out as a table so they line up under one
another. Three shapes that rendered as nothing or as noise are handled: a join key
(`customers.id = orders.customer_id`), an ordering with its direction, and a subquery, which
arrives as its whole parse tree and now reads `customer_id IN (subquery)`. `GROUP BY 1` is
labelled "column 1 of the select list" rather than shown as a bare digit — under `group_keys`
only, since a `limit` of 100 is a row count.
- **A golden run report can be filtered by outcome.** A corpus-sized run is hundreds of cases and
was a flat scroll. Display only — nothing is recounted or re-sorted, so the tiles and the section
headings keep describing the whole run.
### Fixed
- **`/agami-eval` can generate again for an operator who signed in rather than exporting an API
key.** The generator child is given an allowlisted environment, and `USER` was not on the list. On
macOS the client resolves its stored credential through it, so the child started, reported that it
was not logged in, and exited — every case in the run coming back "the generator exited without
answering" and the run exiting `2`.
Anyone with `ANTHROPIC_API_KEY` set was unaffected, which is why it shipped: that variable *is* on
the list, and a machine with one set never reaches the failure. The tests could not catch it
either, because they inject their own generator and never spawn the real client.
`USER` names the account and not a path, so it opens nothing the no-tools invocation had already
closed. The allowlist is still an allowlist, and a new test holds it to that.
## [0.8.0] — 2026-09-05
One change: the server decides which tool calls belong to one conversation, instead of asking the
model for an id it cannot supply reliably. `tool_calls` gains a column and the Activity view groups
by it; nothing existing is removed and no argument, return shape or refusal rule moves.
### Added
- **The server decides which tool calls are one conversation, instead of asking the model.**
`tool_calls` gains `conversation_id`, worked out when the call is recorded from the authenticated
actor, their organization and the clock: calls by one person continue a conversation until a pause
longer than `CONVERSATION_IDLE_MINUTES` (30), and the next call after that pause starts a new one.
Nothing a caller sends can influence it.
`thread_id` was meant to be this and is minted by the model, which cannot be relied on. Measured on
one deployment: asked in prose it arrived on 2 of 10 calls and then 0 of 8; made a required
property (0.7.1) it arrived on 9 of 9 **and collided** — a model forced to invent an id it does not
otherwise track emits the shortest string the schema accepts, so two conversations two days apart
both arrived as `t1` and were shown as one, with their turns merged. Handing it a server-minted id
to echo back was ignored as well; nothing carried in prose is honoured, wherever the prose sits.
The transport cannot supply one either — MCP removed `MCP-Session-Id` in the 2026-07-28 revision,
and this server already runs stateless for the reason the spec dropped it.
The rule is a judgement, not a fact: two conversations a minute apart are one row. What it
guarantees is that the answer is the SAME every time, computed rather than remembered, and that the
mistake it can make is bounded to one person's own back-to-back conversations — it can never merge
two people and never join two days.
`list_sessions` groups by it, falling back to the old `(actor, thread_id)` pairing for rows written
before the column existed — so the Activity view stops blending two conversations that reported the
same self-reported id, without turning history into singletons.
`thread_id` is unchanged and still recorded; this is an addition. Deriving the value costs one
indexed lookup per recorded call, on the connection the recorder already holds.
## [0.7.1] — 2026-09-04
One opt-in change, off by default: a deployment can require the `thread_id` that links a
conversation's tool calls together. Nothing about the served tool surface moves unless the flag is
set.
### Added
- **A deployment can require `thread_id`, so conversations stop fragmenting in the activity log.**
The id that groups a conversation's calls is self-reported — the activity-log directive asks for it
on every call and ends "Best-effort; omit if unknown" — and models take the omission. Measured on
one deployment across two consecutive conversations: 2 of 10 calls carried one, then 0 of 8, not
one including the first. `list_sessions` degrades gracefully (each orphan becomes its own
singleton, so the Activity view stays audit-complete), but the conversation those calls belonged to
is gone. Setting `AGAMI_REQUIRE_THREAD_ID` promotes the property from optional to **required** on
every tool declaring it, which the MCP SDK enforces before dispatch; the same client then sent it
on 9 of 9 calls. Deriving it server-side is not available — the OAuth `sid` identifies one
authorization and survives token rotation, so it would file every conversation under one thread,
and the streamable-HTTP transport is stateless, so there is no `mcp-session-id`. **Default off**,
because the change cannot fail softly: a call omitting a required property returns an input
validation error rather than a degraded record, so it should be turned on somewhere observed before
being rolled forward. Existing deployments are unaffected until they opt in.
## [0.7.0] — 2026-09-02
The release is one feature: **golden datasets** — a way to write down questions whose answers you
have already agreed on, and then find out, on demand or in CI, whether the model still answers them.
Everything below is additive. No argument, return shape, refusal rule, receipt key or CLI flag from
0.6.x moves.
### Added
- **A golden dataset: the questions you have agreed on, and the answer key for each.** A golden dataset
is a file of questions where an author has written the question, the SQL they accept as the answer,
and how strictly a run has to match it. Datasets live under the profile and are read by
`semantic_model.golden`, whose whole job is to hand a runner records it can trust — and to say out
loud which files and cases it dropped getting there.
Two properties hold and are the reason it can be trusted. **One bad file does not cost you the
run:** a malformed file, or one malformed case inside a good file, becomes a finding and the read
continues. **Nothing returned carries a filesystem path** — not on a record, not in a finding — so
a runner downstream cannot forward a dataset location into a subprocess.
`plugins/agami/shared/golden-dataset-shape.md` is the authoring reference, and a test parses its
own example through the real reader, so the document cannot drift from the parser that has to
accept it. Its hard rule against reading a sibling profile to learn the shape binds harder here
than anywhere else: a golden dataset holds the business definitions *and* the answer key in one
file, so a glob across profiles reads another tenant's questions together with the SQL that
answers them.
- **Scoring is deterministic, not a model's opinion (`semantic_model.comparator`).** Given the answer
key's result set and the one the generated SQL produced, `compare_result_sets` returns a score. It
executes nothing, writes nothing, opens no connection, and cannot make a model call. That is what
makes a run reproducible rather than a second opinion.
Three decisions in it are worth knowing, because they decide what counts as the same answer.
**Columns match on their values, never on name or position** — a correct statement that renames and
reorders its projection is still correct, and that is precisely the case a name-matched comparator
fails. **Cells canonicalize to a `(type_tag, value)` key**, which is not stylistic: Python collapses
`True`, `1`, `1.0` and `Decimal(1)` into one bucket, so without the tag a boolean column would
compare equal to an integer column of zeroes and ones. **Ordering is read from the golden statement
alone**, parsed rather than pattern-matched, so a generated statement that drops the `ORDER BY`
still scores order-sensitively — and an `ORDER BY` inside a subquery, a CTE, an `OVER (…)` or an
aggregate does not make a result ordered.
- **`/agami-eval` — run a dataset and read the verdicts, failures first.** The developer surface for
all of the above: no admin, no server, no browser. `plugins/agami/scripts/run_golden_eval.py`
takes `--profile`, `--dataset`, `--tag`, `--rerun-failures`, `--top-k` and `--timeout-s`, and
`--list` answers "what is there" with no database, no generator and no credentials.
**The generating context never holds the answer key.** A model grading itself against a key it can
see reports a pass rate that means nothing, and no assertion over the scores would reveal it. The
generation runs in a sandboxed child process, and this is the one path in the product that does not
generate SQL in the caller's own context — the isolation is the point. The child's flags **are** the
sandbox and are asserted present by a test, because in that client the absence of a flag is the
permissive default.
**Both statements execute through the guarded chokepoint**, with an injected executor rather than a
raw connection — so a golden run is bound by the same read-only guard, row cap and statement
timeout as any other query, and works on every engine rather than one.
- **An exit code CI can gate on.** `0` every confirmed case passed, `1` a confirmed case failed, `2`
no verdict could be produced. `2` means *go look at the harness*, never *the model regressed*: it
covers a preflight refusal, a run that stopped partway, a run where every generation errored, and a
selector that matched nothing. A run that is both incomplete *and* carries failures reports `2`,
because sending someone to debug a model change against a run that never happened is the expensive
mistake.
The verdict rests on **confirmed** cases only. An unconfirmed item's answer key is one nobody has
reviewed, and gating CI on it would fail a build over a case no human has agreed to.
- **`/agami-save-golden` — two doors into a dataset, kept apart on purpose.** A dataset that has to be
hand-written stays empty, and an empty dataset gates nothing. The **import** door turns a CSV
question bank into items written unverified — volume arrives here and it gates nothing. The **save**
door writes one answer a person just accepted, with the statement that produced it and the result
as its receipt, marked verified.
They are separate because a spreadsheet holds questions, not the statement that answers them. An
import that marked its own rows verified would forge the gate: the suite would fail on its own
errors and teach people to ignore it.
Every write is **append-only** — a write that would change an existing item renders the before and
the after and stops for an explicit yes — and every write funnels through one function, so the rule
holds for both doors. A write proves itself by re-reading the file through the runner's own reader
and rolling back if that read refuses.
- **A report for a run, and a page for a dataset.** When a golden question stops matching, the useful
artifact is the confirmed statement and the generated one read against each other, which does not
fit in a chat message. A run now writes a self-contained HTML report with both statements side by
side — no rows, no network — beside its machine-readable artifact, sharing one filename stamp so the
pair is findable by name.
The **golden dataset explorer** answers the other question: not what broke, but what the dataset
*is* — how many items exist, how many can actually gate a run, which have rotted. Four tabs (Items ·
Coverage · Lint · Queued) with live search, and edits queued in the page and handed back as one
block a deterministic parser applies. **The Coverage tab is the reason the page exists:** forty
questions that never touch the revenue metric read as coverage they do not have, and the model
change that breaks revenue passes clean. Coverage is computed from what the statements actually
read, so that gap is a fact rather than a judgement.
### Changed
- **A reconcile run keeps the answers it already proved.** A run that matched nine of twelve numbers
has produced nine verified answers — a person read each one and accepted it — and then threw them
away: the statement that produced each number lived only inside the report HTML, and the row record
never held it. The row record now keeps the statement and the result (additive; every existing key
stays), and after the run's summary the rows that agreed are offered once for promotion into a
golden dataset. It writes through the save door above and nothing else. Declining writes nothing.
Only rows that actually matched at the run's own tolerance are eligible, and a promoted item is
written as a **band around the observed value** rather than an exact match, so it passes its own
next run by arithmetic rather than by hope. A promotion edits the question and never re-anchors the
statement to `CURRENT_DATE`: the band was drawn around today's value of a window that slides, so
anchoring it would drift the item out of its own band and fail later as a false alarm.
## [0.6.9] — 2026-08-27
### Added
- **An existing password account can move onto an identity provider, keeping everything attached to
it.** A deployment running on usernames and passwords could not switch to single sign-on:
`_resolve_oidc_user` refuses an account not already bound to the provider, so on the day it was
turned on every person already using the product would be locked out, with their conversations
orphaned behind an account nobody could reach. Verified against a real account rather than
reasoned about — it came back `None`.
`migrate_password_user_to_oidc` keeps the row, so the id, the address and everything keyed on it
survive. **The password is cleared**, because left in place the directory would not actually be in
charge: revoking somebody there would not stop them signing in with a password they chose months
ago, and nothing would show it. Guarded so it can never take an account already bound to a
different provider — the identity-confusion guard has to survive the migration that relaxes the
password case.
Deciding *whether* to adopt is deliberately not this function's job. It is safe only where the
caller has already verified the token against a pinned tenant and checked the address against that
company's own domains; core has neither of those facts, and its consumer does.
- **`set_user_names`** fills in a display name from whatever the provider now says, so a name
corrected in a directory reaches the product without anybody retyping it. A blank never erases
what is there: a provider that omits a name is silent rather than authoritative, and treating
silence as an instruction to delete would wipe a name an administrator typed in.
### Changed
- **`http://localhost` is accepted for `PUBLIC_BASE_URL`.** TLS is still required everywhere else,
for the reason it always was — a browser drops a `Secure` cookie over http. Browsers make a
specific exception for loopback, which is a secure context by their own definition, so nothing
that rule protects is lost. It is also the only thing that works: identity providers permit a
plain-http redirect only on loopback, so a developer signing in against a real provider from their
own machine had no https option to choose and could not run the server at all. The host is parsed
rather than prefix-matched, because `http://localhost.example.com` is somebody else's domain.
- The sign-in consent line now reads **"Allow `<client>` to connect to your data"** rather than "to
access your data". What is granted is a connection the person can see and remove, not a handover
of the data itself, and on a screen read in two seconds before deciding that is the whole
decision.
## [0.6.8] — 2026-08-27
### Fixed
- **A database connection the server closed is now reopened, instead of poisoning the process until
it is replaced.** `Store` opened one Postgres connection at startup and held it for the life of the
process. A deployment that sits idle for about an hour has that connection reaped from the server
side, and nothing told the process: `execute` called `conn.cursor()` with no liveness check and
there was no reconnect path anywhere in the package.
The instance was then poisoned for as long as it lived — **every** request, not only the ones that
touch data, answered in about three milliseconds having attempted no round trip at all. The three
milliseconds are the tell. A downstream deployment hit this four times in four days, one to two
hours each, across two revisions; thirty days of that service's error logs contained 123 tracebacks
and every one was this.
It fails closed on all traffic because the first thing a request does is resolve the caller's org,
and that read goes down the dead connection — before authentication, before routing. Somebody who
has just signed in successfully sees a connector error, which reads as a login failure and gets
reported as one.
`Store` now keeps the URL it connected with and retries once when the connection is gone. Three
things bound that retry, and each of them was a real defect in an earlier draft rather than
caution for its own sake:
- **It refuses while an uncommitted write is outstanding.** Callers write two rows that must land
together — a role change and the audit line naming who granted it. If the connection dies between
them the first is already lost, and reconnecting silently would let the second commit **alone**:
an audit line for a grant that never happened, which is exactly what that line exists to make
impossible.
- **A read does not count as an outstanding write**, and that distinction is the difference between
fixing this and appearing to. The path this exists for reads five tables in a row and never
commits, because there is nothing to commit. Counting reads would have reconnected once and then
refused for the life of the process.
- **`OperationalError` alone is not enough to act on.** It is a broad base class — a cancelled
query, a statement timeout, a full disk. Retrying on any of them re-runs the statement, and for
the first write of a transaction that lands the row twice. Only the driver marking the connection
closed counts. `InterfaceError` needs no such test, since the driver raises it only on a
connection it has already closed.
It also refuses while a session-scoped advisory lock is held, so a reconnect part-way through a
migration cannot continue without the lock and let a second instance apply the same files.
`Store.rollback()` is new, and callers that reached past it into `store.conn.rollback()` now go
through it — a rollback that bypassed it would leave the connection marked mid-transaction forever
and unable to reconnect again.
## [0.6.7] — 2026-08-18
### Added
- **A curated example keeps the same id across deploys, and the caller can see it.** `prompt_example`
has had an `id` column in its primary key since the table existed, and it was unusable three times
over: an example with no `id` in its YAML got a minted uuid4, nothing wrote that back, so the next
deploy minted a different one for the same example — and the read selected `question, doc`, never
the `id`, so no identity reached a caller even where one existed. Across a real deployed model of
16 subject areas and 46 curated examples, not one carried an id.
So nothing could name an example, count one, or say whether the one returned yesterday is the one
returned today. `agami-save-correction` cannot tell a correction that created an example from one
that should have replaced it; ranking returns a subset and which subset is unrecordable; the library
only grows, because no two observations of an example can be tied together.
The id is now **derived from the example's own content** — `question` and `sql`, each NUL-terminated,
SHA-256, twelve hex characters, the construction `compute_model_hash` already uses. Derived rather
than minted, because a minted id changes every deploy; derived rather than authored, because
authoring gives the id two homes that can disagree and needs a pass over every deployment. An
authored `id` in the YAML still wins, which is the escape hatch for an example deliberately filed
under two subject areas — those otherwise resolve to one id, and the first now wins rather than the
second aborting the deploy.
`select_examples` returns it, and `example_by_id` reads one back, scoped by org and datasource like
every other read in the module. The derivation is one-way, so a consumer holding an id needs the
lookup for the id to be worth having.
- **An agent can say what it based a query on, and why.** The activity log records what ran and two
self-reported columns carrying the caller's framing of the question. What it never recorded is the
reasoning in between: why that table, when the model declares three that look similar; why that
join, when it declares no relationship between those two tables; which curated example this
followed, or whether there was none. An operator could see that a query was wrong and not where the
reasoning went wrong — and a query against the wrong table is indistinguishable, in the log, from
one against the right table.
`execute_sql` takes an optional `basis`: a list of `{kind, ref, why}`, recorded on `tool_calls` and
rendered in the activity log beneath the statement it explains. Absent is the ordinary case, the
field is not `required`, and a client that has never heard of it is unaffected.
**The property is declared as a bare array, and that is a correctness constraint rather than a
simplification.** The MCP SDK validates arguments against `inputSchema` before the handler runs, so
a `maxLength`, `maxItems` or `enum` there does not trim a bad entry — it refuses the whole call and
takes the caller's answer with it. The kinds are advertised in the description and enforced at the
boundary, where an over-long `ref` or an unknown `kind` costs its own entry and nothing else. The
stored value says when it was cut, so a reader never takes a truncated list for the whole claim.
Self-reported, like its neighbours: core records the claim and adjudicates nothing. Whoever holds
the receipt can check `ref` against the SQL themselves.
## [0.6.6] — 2026-08-14
### Added
- **The activity log can name the execution it ran.** `tool_calls` records that a tool ran and
`query_executions` records what the warehouse was asked and what the guard decided; both are
written for every `execute_sql`, they are 1:1, and nothing joined them. `018`'s own comment
recorded the gap — *"there is no key between them (`Envelope.audit_id` is `query_executions.id`,
which `tool_calls` does not carry)"* — and chose to duplicate a column rather than add one.
That holds for a single sentence-shaped field. It stops holding once anything has to be attached to
a **statement**: `correlation_id` names the turn, one turn runs several statements, and no column
told them apart. A reader wanting per-statement facts had to guess, and the only material to guess
with was the agent's own framing text — which repeats whenever a turn retries a sub-question, so it
mis-attributes in silence.
`tool_calls` now carries `audit_id`, and the value is one the caller already received: `_emit`
stamps it onto every envelope, and `query_executions.id` is that same id. The column reconciles
nothing and mints nothing.
It is read off the body `record_tool_call` already parses, on `ok`, `refused` and `failed` alike —
a blocked or broken call is the one an auditor most wants to trace. A caller that hands over no
body can state it instead, like the outcome fields beside it. NULL is the ordinary case, not a gap:
a schema read or a datasource list runs no statement and has none.
### Fixed
- **A client asking what this server looks like gets the icon, not a 401.** `/favicon.ico` was not
merely absent, it was gated: the bearer middleware challenges anything off its public list before
routing can 404 it, so the one unauthenticated request a client makes to learn what a server looks
like came back as an auth prompt. An MCP client showing a server in a connector list has no
credentials to offer for an icon fetch, and a deployment that redirects `/` elsewhere leaves this
the only path left to ask on — so such a server drew a letter avatar beside servers that answered.
## [0.6.5] — 2026-08-13
### Fixed
- **A model whose subject area joins one pair of tables twice now loads.** It did not before: each
relationship row was keyed by its two table names, and that key is the table's primary key — so the
second join collided with the first and the INSERT raised part-way through the write. The failure
was total rather than partial. Nothing commits, so the deployment ends up serving **no semantic
model at all**, and the error names a column tuple rather than the join that collided.
Self-referencing tables make this ordinary rather than exotic: an employee row pointing at another
as its manager, and again as its mentor, is one pair of tables and two genuinely different joins.
Two `on:`-expression joins between one pair, and same-named tables living in two schemas, failed
the same way.
The key now carries the relationship's position within its subject area, so a collision is
impossible for any model that validates rather than impossible for the shapes somebody enumerated
— which is what the two previous versions of this key each attempted, and each missed. Nothing
reads the value (a relationship is rebuilt from its stored document), and a redeploy replaces a
datasource's rows wholesale, so models written under the old key need no migration and no backfill.
## [0.6.4] — 2026-08-12
### Added
- **The claim page says whose account it is.** Somebody following a setup or reset link was asked to
choose a password with no indication of which account it belonged to. A link is shared out of band,
so the person holding it may have been sent the wrong one, or two of them — and now that a link can
REPLACE a working password rather than only set a first one, following the wrong one silently locks
somebody out of their own account with nothing on the page to warn them. It discloses nothing new:
whoever holds the link already holds a token naming that account.
- **The MCP surface states what the server already does.** Four capabilities shipped working and
undocumented, and a silent feature is worse than a broken one — a broken one gets reported.
`execute_sql` returns three statuses and described two; the third carries the whole database-error
channel with a `kind` from a declared enum, which tells an agent whether a failure is repairable
from the schema it already holds or whether retrying is pointless. Documented, so it can be used.
### Fixed
- A password refused for being **too long** said "use at least 8 characters" — the same message as
one refused for being too short, and advice that makes the problem worse the more carefully it is
followed. Each bound now names itself.
- Seven store-step tests resolved the profile from the developer's own `~/agami-artifacts` rather
than from the temporary directory they had set, so they compared against a value from a file the
test never mentions. CI has no such file, so they passed there and failed only on a machine that
had run the CLI.
## [0.6.3] — 2026-08-11
### Added
- **An administrator can give a colleague a new password.** The setup link works exactly once —
claiming an account flips it out of `pending`, and every later use is refused — so an administrator
could create somebody's account and then never help them back into it, and with no mail path there
is no self-service reset either. `/claim` now carries a second purpose: `setup` still means a
pending account choosing its first password, and `reset` means a claimed account being given a new
one because an administrator asked. The two are matched against the state of the account rather
than trusted from the link, so neither can do the other's job.
- A reset link is **single-use**, by a mechanism the setup link does not need: a reset leaves an
account exactly as claimed as it was, so nothing about it would change and the link would work
forever. The link is bound to the credential it was minted against, and that credential is a
condition of the write — so the first successful reset retires it, a password change by any other
route retires it, and two simultaneous uses cannot both succeed.
- A reset **ends the sessions running on the old password** by revoking that person's refresh
tokens. Stated precisely because a security control believed to cover more than it does is worse
than a narrow one: a session already holding an access token lasts until that token expires, and
the `/admin` console cookie and any outstanding authorization code are untouched.
- A reset can never give a password to an identity that signs in through a provider, and can never
revive a switched-off account.
### Changed
- The claim page is worded for what the link is for — somebody with an account is not told to
"finish setting up" — and `setup_page_html` / `setup_done_html` are now `claim_page_html` /
`claim_done_html`, which take the purpose.
- Password hashing on `/claim` runs off the event loop. It is deliberately slow and memory-hard, and
this is a public endpoint, so hashing inline stalled every other request in flight.
- `get_datasource_schema` answers **within the scope the caller declares**, and the never-hide
guarantee is now stated relative to that scope: within it, nothing is hidden — the scope's own
metrics plus the cross-area bucket. A new `area` parameter narrows to one subject area
(completing the set `execute_sql` and `get_prompt_examples` already had); `dataset_names`
narrows to those tables. `query` and `metric_names` rank and select detail and do **not** scope,
because narrowing on something the caller never declared is silent deprivation. The response
echoes the `scope` it resolved.
- The appended domain context no longer repeats the subject-area listing that the same response
already carries as structured `subject_areas`.
### Fixed
- A table-scoped `get_datasource_schema` call returned columns and **no way to join them**. It had
resolved the relationships and metrics for those tables and discarded them, while the tool's own
description promised both. It now returns them, with each metric carrying the binding for the
deployment's engine — so one call carries the columns, the joins and the declared expression.
- **The instructions named a metric field the payload does not send.** Both instruction surfaces
told the agent to reuse a metric's `calculation`/`bindings` VERBATIM; the payload key is
`binding`, singular, already resolved to this deployment's engine. An agent told in capitals to
reuse a field it never receives hand-rolls the SQL instead — and hand-rolled SQL does not reduce
to the declared binding, so the receipt reads `unmatched` on a column that does compute the
metric. The text and the payload are now pinned to each other by a test derived from the
projection itself.
## [0.6.2] — 2026-08-09
### Fixed
- **An omitted `datasource` names the org's real datasources instead of one nobody has.** #216 (also in
this release) removed the invented `'default'` for a deployment serving exactly ONE datasource;
serving several, resolution still fell through to that literal and the caller heard
`no such datasource: default` — so the one-datasource case and the several-datasource case are fixed
together in this release, and nothing in 0.6.1 or earlier ever consulted the store to resolve a
profile at all.
Two audiences read that sentence and both were misled: an administrator sees a name no customer has
and concludes their data has gone missing, and a model sees a failed lookup and invents another name,
because the tool's own description told it a default existed. So the description is fixed as well as
the refusal — all three tools now say what is true on each install and point at `list_datasources`.
An omitted argument gets the catalogue back (`datasource_required`); a NAMED one that does not exist
is still told so; an empty string counts as omitted, matching `resolve_profile`'s own
`if explicit:`. (#218)
- **A CASE predicate is not what a SUM aggregates, and a shared expression names no metric.** (#215)
### Changed
- **The tool surface reports what it checked, and accepts what it advertises** — nine findings from a
connector audit, all one shape: the server stated something it had not established, or named a field
the agent could not reach. `model_present` dropped from the served listing (it was the literal `True`);
`resolve_profile` consults the store; a hosted server no longer claims "All execution is local";
`bindings` is actually sent; `area` reaches `get_prompt_examples`; `list_datasources` returns
`description`; `for_questions_about` dropped from the payload. (#216)
**Released as a patch deliberately, and it is the one judgement call in this version.** #216 REMOVES
fields from emitted tool payloads, which a consumer reading them would notice — normally a minor bump
under 0.x. It ships as 0.6.2 because every field removed was provably dead: `model_present` could only
ever be `True` on that path, and `for_questions_about` had no writer anywhere in the product. A
consumer reading either was reading a constant or an empty list. The field stays DECLARED on the model
format, so no model on disk fails to load.
## [0.6.1] — 2026-08-08
### Fixed
- **A refusal's own sentence reaches the audit row again, so a console reader can act on it.**
`tool_calls.refusal_detail` and `tool_calls.refusal_remediation` — the two columns added in 0.6.0 so
that a row says what the decision was made against and what you were told — were **NULL on every
refusal a real deployment recorded**. The rule was recorded; neither sentence was.
The cause was the coherence rule that shipped beside them. It cleared both whenever a caller
*stated* the outcome rather than leaving it to be parsed out of the response body, reasoning that
such a caller is not offering these sentences. That is true of a caller who **contradicts** the
body, and false of every real one: the served MCP transport states the outcome for every tool that
speaks the Envelope, which `execute_sql` does, and a consumer's own sink states what it observed
with no body to hand over. So the sentences survived on exactly one path — a caller that hands over
a body and states nothing — and nothing in production takes it.
They are now kept when the stated outcome **agrees** with the body, and dropped when it does not.
What the rule guarded is unchanged: a stated success has no refusal to explain, and a stated kind
naming a different failure is describing something these sentences are not about. No consumer
change is needed — both surfaces already state the rule itself.
The tests are worth naming, because they were the real defect. Every test that asserted these
sentences were **present** drove the recorder the way no production caller does — handing over a
body and stating nothing — so they went on passing while every path the world actually takes
recorded nothing. A test can be demonstrably wired to the line it covers and still say nothing
about whether that line is reached the way the world reaches it. The new test that states an
outcome agreeing with the body is the one that fails against the shipped behaviour.
## [0.6.0] — 2026-08-08
### Added
- **A server can now run with the semantic-model pass off, and always says so
(`AGAMI_GOVERNANCE_ENFORCED`, ACE-101).** The model-scoping checks refuse on facts about *our*
parser and *our* model resolution rather than about your SQL, so a dialect drift or a model that
will not resolve could refuse every query on a server until an operator intervened, and there was
no way to bring that server up without them. There is now one switch, read per request so flipping
it takes effect on the next query with no redeploy.
**It is off by default,** which means a fresh server does not enforce table scope, column scope,
the `SELECT *` ban, or the engine-mismatch check until you turn it on. With it off, a query may
read anything the connecting role is granted, including columns you excluded from the model, and
may enumerate your schema through catalog relations. Turn it on once the checks have been
validated against your data; see the [self-hosting reference](docs/self-hosting.md) and
[SECURITY.md](SECURITY.md).
**What the switch cannot reach:** the read-only guard, the dangerous-function guard, the statement
timeout and the row cap. Those are composed outside the pass and enforce in both postures, as does
the read-only database role. And nothing ever claims the checks ran: every answer and every audit
row carries "the semantic-model checks are turned off in this deployment", on all five receipt
sections, with no findings attached.
The server logs one warning at startup naming the variable and the exposure whenever it boots with
the pass off.
- **The receipt now tells you which of a table's declared filters your statement actually applied
(ACE-099).** Declaring `default_filters` on a table has never applied them to your SQL, and since
the filter injector was removed nothing reported on them either — so `SELECT COUNT(*) FROM orders`
returned every row where it once returned the undeleted ones, and nothing on the answer said the
number meant something different from what the model says the table means.
Each entry in the receipt's `tables` section now carries `filters`, one `{expr, status}` per
declared filter, with `status` one of `applied`, `omitted` or `undetermined`, plus a `scope`
naming where in the statement that reference sits.
The determination is **per reference, not per table**, which is the part that makes it worth
trusting. A filter satisfied inside a CTE and absent from the outer query is two different answers
about the same table, and reporting one verdict for both is what made the old injection unsafe:
```
WITH recent AS (SELECT id FROM orders o WHERE o.status != 'cancelled')
SELECT o.id FROM orders o JOIN recent r ON o.id = r.id
```
```
orders scope=cte:recent filters=[{expr: "o.status != 'cancelled'", status: applied}]
orders scope=main filters=[{expr: "o.status != 'cancelled'", status: omitted}]
```
`applied` means the declared predicate is one of the top-level `AND` conjuncts of that reference's
own scope; extra conditions beside it do not weaken that. Anything the check cannot stand behind
is `undetermined` rather than a verdict — a predicate on the same column that is not identical, one
reachable only through an `OR`, one that only appears in an outer join's `ON`. Only an outright
absence is `omitted`, because a confident "you left this out" that turns out to be wrong is worse
than saying nothing. **An omitted filter is never a refusal**: whether it matters depends on the
question, which only you have.
The shipped sample declares one (`orders`: `status != 'cancelled'`), so this is visible the first
time you run it.
### Changed
- **The receipt now reports on every number your query computes, not just the ones with a problem
(ACE-060).** The `aggregates` section listed findings, so a total no join had multiplied produced
no entry at all — and an entry that is absent looks exactly like a check that never ran. It also
named the measure *table*, so a query computing two numbers over one table told you a join
multiplied "orders" and left you to work out which of your two numbers it meant.
There is now one entry per aggregate, saying the aggregate as parsed, whether a join multiplies
the rows behind it, and which join does. A number nothing multiplied **says so**, which is the
point: reading "not multiplied" beside a total is the difference between a clean answer and an
unchecked one.
An aggregate whose reads could not be resolved reports `undetermined` rather than clean.
`COUNT(*)` is the case that matters: it names no column, so nothing tells us which table's rows it
counts, and a fan-out around it is invisible to the check. Reporting that as clean would put a
clean bill of health on the one number a join had multiplied.
The section's `undetermined` line is now composed per query from what *that* query left open, so
it can finally be empty. It used to carry a sentence on every answer, including "whether this is a
problem depends on the question" — true of every answer forever, and the reason the section could
never say "checked, and complete". A trap is still reported and still never refused.
- **The audit row now says what the decision was made against, and what you were told (ACE-098).**
A row recorded the verdict — `ok` / `refused` / `failed`, and for a refusal which rule under which
reason — but nothing about the basis for it, so nobody could take a row and check the decision
again. Three columns close that: `detail` (the refusal's own sentence, which is where "which bound
fired and what it was set to" lives, since the statement timeout and the row bound share one rule),
`receipt` (everything the trust receipt reported, including its `undetermined` markers), and
`model_version` as a column you can filter on.
A test now takes those rows and **re-derives each refusal with no database connection at all**,
matching what was recorded. That is the check that tells whether the fields are sufficient rather
than merely present. The two runtime bounds are exempt and stay exempt: whether a statement
outruns its budget is a property of the run, not of the SQL, so it is not reproducible offline by
anyone.
Also, the **tool-call log now reads the verdict rather than re-reading the answer**. It used to
parse the response body to work out whether a call failed and why, which made the audit trail
depend on the wire format; it now takes the classified outcome directly.
- **A self-hosted server that cannot record a query no longer runs it (ACE-097).** Recording was
best-effort in three places, and two of them were silent, so a deployment could execute SQL
against your database and keep no record of having done so with nothing anywhere saying the
record was lost.
On a **server** (one with `AGAMI_DB_URL` or `APP_DATABASE_URL` configured), the audit store is now
checked before the statement runs. If it cannot be opened the call is refused, with
`rule: audit_unavailable` and a remediation naming the operator action, and the statement never
reaches your database. If the store was reachable at that check and the write fails afterwards,
the call fails rather than returning an answer whose statement left no trace. The connection is
read-only, so nothing was changed and re-running costs only the round trip.
**Local single-player use is unchanged.** With no database configured there is no audit store to
reach: the log is a local jsonl file, a write failure is still logged and never breaks your query,
and a read-only artifacts directory cannot stop you asking questions.
For operators this is an availability change, and a deliberate one: a briefly unreachable audit
database now produces refusals rather than unrecorded answers.
- **A result too large to return is refused, not trimmed.** A query whose result exceeded the
deployment ceiling (`AGAMI_SQL_MAX_ROWS`, default 1000, unchanged) used to come back cut down to
that many rows with a flag saying so. It now comes back as a structured refusal carrying no rows.
The trim was unsound before it was anything else. Without an `ORDER BY` a SQL result has no
defined prefix, so what you got was whichever rows the engine happened to emit first — different
between runs, different between engines, and presented as the answer. It was not a smaller version
of your result; it was an arbitrary sample of it.
The refusal tells you which fix applies to the statement you sent, because the wrong one is worse
than none: a row listing should be bounded with a `LIMIT` and an `ORDER BY`, while an aggregate
should have its grouping narrowed or a filter added — putting a `LIMIT` on a grouped result drops
groups, and the breakdown you get back reads exactly like a complete one.
- **A `#` is no longer mistaken for the SQL it hides, and is now refused wherever it appears
outside a string (ACE-096).** `SELECT a FROM t # DROP TABLE t` came back as *"keyword 'DROP' is
not allowed"*, and `... # note; more` as *"multiple statements are not allowed"* — both refusals
of valid MySQL, and both naming a fix that would not have helped, because neither the `DROP` nor
the second statement was ever going to run.
The guard reads every statement with one grammar and no engine, and `#` means four different
things across the engines it speaks: a line comment in MySQL and MariaDB, the `#` / `#>` / `#>>`
operators in PostgreSQL, a temp-table prefix in SQL Server (`#tmp`, `##global`), and an ordinary
character inside a backtick- or bracket-quoted identifier. It cannot tell them apart, so it now
declines to pick a reading and says which ambiguity it hit — the same call it already makes for
a bare `--x`.
**This is a widening: all four shapes ran before and are refused now**, not just the comment.
If you use jsonb path operators, SQL Server temp tables, or a `#` inside a quoted identifier,
those statements will start coming back refused. It is the accepted cost of one grammar; the
other direction lets a trailing `;DROP` ride through inside text the guard decided to ignore.
Rewrite a `#` comment as `-- ` or `/* … */`; a `#` inside a name has to be spelled another way.
- **The trust receipt is five sections, and each one says what it did NOT establish (ACE-088).**
Every answer's receipt now carries `columns`, `tables`, `joins`, `aggregates` and `assumptions`,
always all five, each an object `{items, undetermined}` beside the `model_version` pin.
`undetermined` is a plain sentence naming what that section did not check; it is `null` only when
the section is complete. **This is the point of the change**: before it, a section nobody had
checked and a section that found nothing were both the empty list, so silence read as clean.
Aggregate fan-out is the live example — the receipt now states, where you read the answer, that
whether a join multiplies the rows an aggregate is computed from was not checked, instead of
shipping an empty section you would read as "no problem". `assumptions` is the section that most
often has nothing to admit, and it earns its `null` rather than assuming it: it lists at most
three AI-written column meanings and counts any beyond that onto its own marker, because a
truncated list under a null marker is a positive claim of completeness.
The receipt also rides on **every** status, not just `ok`. Every non-ok body carries the bounded
form — the caller's own identifiers and, per reference, whether the model declares that name;
nothing else about the model. `tables` is now **one entry per reference** rather than per table,
so a table read twice is listed twice and a reference the model does not declare says so. A name
the statement defined for itself — a CTE, including one that shadows a real table — is not a
declared table in ANY section: it borrows no row estimate, no schema-qualified column label and
no model-written column meaning, in every spelling of a column reference.
**Breaking for anything reading the old flat keys.** `tables_used` → `tables.items[]`;
`relationships` → `joins.items[]`; `metrics` → the `columns.items[]` entries whose `metric` is
non-null; `assumptions` → `assumptions.items[]`; `warnings` → derive it from `joins.items[]`
filtered to `review_state != "approved"` (a review state can be counted and linked back to its
join; a pre-rendered sentence could only be printed); `named_filters` and `sql` are gone (nothing
ever produced the former, and the statement is on the response body). The HTML report, the MCP
server instructions, `render_chart.py` and the `agami-query` skill all move with it, and
`render_chart.py` now **rejects** a receipt that is missing a section rather than silently
rendering nothing for it.
- **The answer report renders the whole receipt.** The provenance panel draws every section with its
marker, lists tables per reference (an undeclared one explains itself instead of showing blanks),
and adds the columns the statement referenced. The unreviewed-join banner is derived from the join
review states, so it can count them and name each one. The unapproved-metric banner and its
Approve / Change write-back are unchanged in behaviour. Two long-standing display bugs fixed while
repointing: an unreviewed entry read "confidence ?" to every user (the phrase tested `confidence`
as a number; it is a label), and a named-filters block that no producer ever filled is deleted.
### Removed
- **`max_rows` is no longer an argument to `execute_sql`,** and `--max-rows` is gone from the
command line. It could only ever *lower* the deployment ceiling, so the one case where a caller
knows better than the operator — wanting more data — was the case it could not serve. Ask for the
rows you want in the statement: `LIMIT 200` says what it means to everything that reads it.
- **`truncated` is no longer a field on a successful result.** With an oversized result refused, it
could only ever be `false`, and a field that is always `false` is one a client can only branch on
wrongly.
- **Agami no longer rewrites your SQL to fix a fan-out join.** A query that aggregated a measure
across a one-to-many join, touching the many side nowhere but the `ON` clause, used to have that
join silently dropped and the rewritten statement executed in place of yours. Your statement is
what runs now, byte for byte — comments, whitespace and quoting included — and that is asserted
rather than assumed.
- **Four correctness checks stopped refusing.** A fan trap, a chasm trap, a `SUM` of a rate or an
identifier, and a `SUM` of a balance across time were all refused. They **return a result** now,
and what the check found rides on the answer's receipt, under `aggregates`.
This is the point of the change rather than a relaxation of it. Whether a multiplied total is
*wrong* depends on what you asked: the same statement is wrong for order revenue and right for
line-item exposure. The check has your SQL and your model and never your question, so it describes
what it found and leaves the judgement to you — or to the assistant, which does have the question
and is asked to say out loud when it restructures a query because of a finding.
A statement that trips two conditions now reports both. The old code stopped at the first, so a
query that both fanned out *and* summed a rate was reported as having one problem.
- **`sensitive` is a description, not a gate.** Marking a column `sensitive` no longer blocks
projecting it. The answer's receipt reports which sensitive columns it projected, under
`columns`, and the authoring guidance asks the assistant to prefer aggregates and to say when it
did project them.
**If you relied on this to keep values from coming back, read this.** The gate was never a
boundary: it inspected the projection list and nothing else, so `WHERE email LIKE …` always
answered the same question one bit at a time. What it bounded was the *rate* of that, which is an
access policy, and Agami holds none of its own — it reads exactly as the connecting database role
reads. Two things do enforce, and neither changed: a column left out of the model is out of scope
and any statement naming it is refused, and the connecting role's grants and your warehouse's
masking policies apply as they always did. If a value must not come back, exclude the column from
the model or make sure the role cannot read it.
- **The `model_safety` refusal rule is gone.** It stood in for two branches that refused without
naming a rule. Both branches went, so every refusal now names the gate that chose it. A consumer
keying on `refusal.rule == "model_safety"` will stop matching, which is the point.
- **The `GovernancePolicy` port and the `Adapters.governance` field (ACE-095).** `GovernancePolicy`,
its `GovernanceVerdict` value type, and the `WarnOnlyGovernancePolicy` default adapter are gone,
along with the `governance` field on `ports.Adapters`. The port was declared but never called: no
core call site ever evaluated it, so removing it changes no behaviour and touches nothing on the
`execute_sql` enforcement path. `Adapters` is public API, so a consumer that constructed it with
`governance=...` drops that keyword; the remaining ports (`ActivitySink`, `OrgResolver`,
`AuthProvider`, `Executor`) are unchanged. **`Adapters` is not keyword-only, so a consumer that
built it POSITIONALLY must also re-check its call**: the 4th positional slot was `governance` and
is now `executor`, so `Adapters(sink, resolver, auth, my_policy)` still constructs but binds the
policy as the executor and fails later at query time with `AttributeError: 'MyPolicy' object has
no attribute 'execute'`. Construct `Adapters` by keyword.
### Fixed
- **Updating the plugin now updates the library it runs on.** The plugin's own files are refetched on
every version bump — the cache directory is the version — but the pip-installed `agami-core` was
not, and the loader prefers that installed package over the fresh copy bundled beside the scripts.
The readiness check only asked whether the library *imported*, which a stale one does perfectly
well, so updating the plugin left new skills running against an old library with nothing anywhere
saying so.
This release is the first where that combination breaks outright rather than drifting quietly: the
receipt below is five sections, and the chart renderer rejects a receipt shaped the old way, so
every charted query would have failed on `receipt.columns is missing` until the library was
upgraded by hand. The launcher now compares the installed distribution against the plugin's own
version and reinstalls with `--upgrade` when it is behind. A library at or above that floor is left
alone, so this costs nothing on an ordinary run, and a source checkout with no distribution
metadata is not disturbed.
- **Catalog and dictionary reads no longer hit the row cap that exists to bound your queries.** The
executor refuses (never truncates) any result over `AGAMI_SQL_MAX_ROWS`, default 1000. That bound
is sized for a question someone asks; a *catalog* read exceeds it on schema size alone. The visible
symptom was `sm enrich-metadata` dying on any platform whose data dictionary is a real table
(`RuntimeError: … "rule": "resource_limit"`), but the quieter cases were worse: the bulk
`information_schema.columns` read behind `sm discover`, and the table/foreign-key reads behind
`sm introspect`, discard a refused read instead of reporting it — so on a wide catalog they
degraded to one round-trip per table, or produced a model with no join graph, and said nothing
about either.
The connect skill now runs those three commands with a raised cap and **tells you it did, along
with your unchanged query-time cap**. Nothing about the bound on an ordinary question changes: a
query returning more than the cap is still refused rather than quietly truncated, and no new lever
is reachable from a generated query.
- **The guard now reads your SQL in your database's own grammar, and refuses what it cannot read
(ACE-079).** Every model-scoping check decided by parsing the statement, and every one of them
parsed in a generic SQL dialect rather than your engine's. On MySQL, BigQuery, Databricks and SQL
Server that is not a subtlety: a backtick is not an identifier quote in the generic grammar, so
`` SELECT `ssn` FROM `customers` `` parsed to **no tables and no columns**. The scope checks were
not bypassed by a trick — they inspected the tree, found nothing to object to, and passed. So on
those engines a query could read any table in the database regardless of what your model declared,
and the trust receipt reported no tables read, which made the answer look clean.
The error posture was the other half. `error_level="ignore"` was not a lenient setting, it was no
setting at all: sqlglot compares that argument against enum members, so a string matched no branch
and every parse error was silently discarded, leaving a truncated tree that read as valid. Both
halves are fixed together, because either alone leaves a hole.
Four situations are now refused rather than run blind, each with the next step it actually needs:
- the datasource does not say which engine it runs on (undeclared, unmapped, or two connections
disagreeing) — `model_unavailable`, and the fix is the operator's: declare
`storage_connections[].storage_type`. It does not invite you to retry the query, because no
rewrite of the query helps.
- the statement does not parse in that engine's grammar — `unparseable`, and you can re-emit it.
- a double-quoted token on a backtick-quoting engine, which means a column under `ANSI_QUOTES` and
a string literal otherwise. The server setting is not visible to the guard, so rather than guess
it asks for the statement in the engine's own quoting.
- the statement parses, reads from something, and resolves to no named table at all — `unscopable`.
A backstop that does not depend on the engine map being complete.
A model that declares one engine while its credentials connect to another is also refused
(`engine_mismatch`): those are two independent pieces of configuration, and a mismatch means the
statement was checked against the wrong grammar.
**If your model does not declare a `storage_type`, queries against it now refuse** with the
message above. This is deliberate: a datasource whose engine is unknown cannot be governed, and
the alternative was to keep parsing it in a grammar no engine uses.
### Contract changes
- Receipt `tables` items carry an **arm ordinal on `scope`** (ACE-043). When a reference's scope is
one of two or more arms of a `UNION` / `INTERSECT` / `EXCEPT`, its label gains a trailing 1-based
`#<n>`: `main#1`, `main#2`, `cte:recent#2`. A plain `SELECT` and a single-arm CTE body are
unchanged and carry no suffix, and `subquery` never takes one. **If you branch on
`scope === 'main'` or `scope === 'cte:x'`, that branch stops matching inside a set operation —
strip the ordinal with `scope.replace(/#\d+$/, '')` (or `scope.rsplit('#', 1)[0]`) and branch on
that.** Split from the RIGHT, not the left: the CTE-name half is caller-written text, and it is
only sanitization to an identifier alphabet excluding `#` that keeps a left split working today. The ordinal is the arm's position in the SQL, which
is not the order of this list: items are in parse-walk order, so a capped receipt can list
ordinals that are neither contiguous nor monotonic, and the largest one is not the arm count.
- Receipt `tables` items gain **`scope`** and **`filters`** (ACE-099). `ref` is unchanged and is
still a string. A `refused` or `failed` receipt is unchanged too — it carries `{ref, declared}`
and neither new field, because a declared filter names the columns and literals the model author
wrote and a refusal is the one outcome a caller can provoke on purpose.
- `tables.undetermined` is now **`null` when the section is complete** (ACE-099). It used to be a
fixed sentence on every receipt saying the filter accounting was not done. It now names only what
was genuinely not established — references whose filters could not be accounted for, references a
shadowing CTE name stopped resolving, and the count the reference cap dropped. If you branch on
this field being present, that branch changes meaning: present now means something really is
missing.
- `sm receipt --applied-filters` is **gone**, and the receipt no longer emits a top-level
`default_filters_applied` key (ACE-099). Nothing had produced either since the filter injector was
removed; the fact lives in `tables.items[].filters` now, in one shape rather than two.
- `sm prepare` returns `{sql, findings, units}` and **always exits 0**. It previously returned
`{action, risk, sql, units, reason}`, or exited 1 with a refusal. It runs the reporting checks,
not the refusing gates — do not pair it with `--no-safety`.
- `sm preflight` returns `{findings: [...]}`, replacing the single
`{risk, action, reason, suggestion, triggering_joins}` verdict.
- The receipt's `aggregates` section can now be non-empty, and its `undetermined` sentence changed:
it says the checks ran and names what they still do not reach, rather than saying the check does
not happen. `columns` items may carry `sensitive: true`.
- The `{"error": {"kind": "preflight_refused"}}` and `{"kind": "sensitive_columns"}` diagnostics no
longer appear on stderr. Every refusal is a single JSON object on every path.
- A refused `SELECT *` reports `reason: "undetermined"`, not `reason: "out_of_scope"`. The refusal
and its message are unchanged; only the reason moves. If you route, count or alert on `reason`,
this row changes bucket.
### Docs
- **Read-only datasource role is now a stated deploy obligation, not a suggestion (ACE-036).** The
self-host guide and `readonly-grants.md` now frame the SELECT-only role as the **required**
`DATASOURCE_URL` posture for a deploy (single-player stays *recommended*), spell out the app-role vs
operator/owner-role split, and clarify that the role guarantees integrity/confinement but **not**
availability/recon — bounding runaway work and recon is app-side, and the docs now state which of
those bounds exist today (the result-row cap) and which do not yet (a per-statement query timeout,
error-text/recon hardening). Docs only — no behaviour or config change.
## [0.5.3] — 2026-07-31
### Added
- **Override seam on the tool-call audit log (`record_tool_call`).** An embedder that dispatches the
tool handlers itself can now supply the values the log would otherwise infer from the model's own
tool arguments and result body — `source`, `thread_id`, `correlation_id`, `user_question`,
`org_id`, and the outcome (`success`, `row_count`, `error_kind`). Each of these override
parameters defaults to `None` = "derive it the way you always have" (the recorded fact `raised` is
a separate boolean, default `False`), so a caller that passes none of them gets byte-identical
rows — the in-repo transport path is regression-pinned identical to before the seam existed. `_record_query`'s source is likewise
readable from a context var, so the query log and the tool-call log can't disagree about what drove
one execution.
### Changed
- **The audit row now fails toward honesty.** A caller with no result body to parse can now record a
failure (previously the row defaulted to success — the one direction an audit log must never fail
in); a raise outranks every override; naming an `error_kind` is itself a statement of failure; and
a success never carries an error kind, so the row can't contradict itself.
- **Honest question provenance in the Activity drawer.** The `· self-reported` trust marker was
stamped on every question unconditionally (the `source` column was never even selected). The column
is now projected and a per-turn flag derived, so the marker is dropped only where the caller
observed the question directly — and kept under any disagreement or uncertainty, because
overstating trust is the one direction it must not fail.
## [0.5.2] — 2026-07-30
### Added
- **Per-request tool-visibility seam (`Adapters.tool_visibility`).** A consumer can now narrow the
advertised MCP tool surface per caller via an optional `tool_visibility(tool_name) -> bool` on the
adapters, applied in `build_server`. It is applied at *both* the list and the call seam — filtering
only the list would leave an unlisted tool callable by name, a surface that looks narrowed while
remaining open. A hidden tool answers as *absent* (the same `Unknown tool` a typo gets), not as
refused, so the list can't be used as an oracle for what exists but is withheld. A predicate that
raises hides the tool rather than granting it, and is logged rather than propagated, so one broken
classification can't become an accidental grant or a dead transport for every other caller.
Subtractive only — a surviving tool's description and input schema pass through untouched. `None`
(the OSS default) is byte-identical to prior behaviour.
## [0.5.1] — 2026-07-30
### Security
- **Closed a read-only-guard bypass via a welded quoted identifier.** A double-quoted
identifier is self-delimiting in SQL on **both** ends, so `SELECT*FROM"pg_read_file"(…)`
and `SELECT "x"INTO evil FROM t` are valid statements with no whitespace either side of
the quote. The guard's lexer dropped the quote characters without re-supplying those
boundaries, fusing neighbouring tokens into one (`FROM"pg_class"` → `FROMpg_class`,
`"x"INTO` → `xINTO`) and destroying the word-boundary anchor every deny-list pattern
matches on — so the gate stopped *seeing* the token rather than allowing it, and returned
no rejection. Verified against PostgreSQL 16: the leading form reads a server-side file
and the trailing form creates a table from `SELECT … INTO`, both while the guard passed
them. Row locks (`FOR SHARE`) were reachable the same way. The lexer now re-supplies a
separator on either side, when and only when the quote was actually separating two word
characters — so a qualified name (`t."current_user"`) still neutralizes to one token,
`t.current_user`, rather than being split. This restores an invariant the lexer already
documented for comments and literals ("never empty"); the identifier branch was the one
place not honouring it. Prior corpus cases all happened to carry a space before the
quote, which is why this stayed invisible; the corpus now pins both weld directions and
asserts the neutralized token structure directly, so neither a one-sided fix nor a
blanket separator can pass.
## [0.5.0] — 2026-07-25
### Changed
- **Renamed the per-datasource model files** so their names say what they are, now that a
company-wide `organization.yaml` exists at the artifacts root. In each profile,
`org.yaml` → `datasource.yaml` and `ORGANIZATION.md` → `datasource.md`. The model root's
display-name field `organization:` is likewise `datasource:`, and the served per-datasource
memory row uses `kind='datasource'`. The company record (`organization.yaml`, `org_id`,
tenancy) is unchanged. No on-disk migration is provided — re-run `agami-connect` (or rename
the two files by hand) for any pre-existing profile.
## [0.4.5] — 2026-07-15
Hosted/self-hosted server hardening: a real-wheel packaging fix, a multi-tenancy seam, and an
append-only instructions hook. **The local plugin path is behaviour-preserving** — everything
resolves to a single `local` org and existing deploys are byte-identical by default.
### Fixed
- **Migrations and static assets now ship inside the wheel.** In a real (non-editable) `pip install`,
`store.MIGRATIONS_DIR` resolved outside the package, so the server could boot on an **empty schema**
with no error, and the missing `static/` dir made app construction fail. Migrations moved into the
package (`packages/agami-core/src/migrations/core`), both are packaged as package-data, and `run_migrations`
now **raises** on a missing core-migration root instead of silently applying nothing. (Editable
checkouts and the `pip install -e` Docker deploy were unaffected — which is why CI stayed green.)
### Added
- **Multi-tenancy: `org_id` scoping across serving, runtime logs, and credentials.** One deployment
can host many tenants whose datasources collide on name (e.g. `prod`). `org_id` (default `local`) is
threaded through every serving/runtime read+write; a redeploy DELETE is org-scoped (one tenant's
reseed can't wipe another's same-named datasource); per-tenant credential env vars
(`<ORG>_DATASOURCE_URL[__<PROFILE>]`) with a **fail-closed** rule (a named tenant never falls back to
the org-less DSN); and a resolver may raise `PermissionError` to refuse a caller (clean 403, not 500).
A plain OSS/self-host deploy is unchanged — everything resolves to `local`.
- **Append-only `extra_instructions` seam on the HTTP composition root.** `build_server(...)` /
`create_app(...)` accept `extra_instructions` so a consumer can add to the model-facing MCP
instructions without forking core. **Append-only, never replace** — so a consumer can't silently drop
a safety directive (e.g. the sensitive-column output rule); no-op and byte-identical by default.
## [0.4.4] — 2026-07-14
Onboarding fix for the examples-validation (NL→SQL) dashboard.
### Fixed
- **Examples-validation dashboard: Edit/Add-note no longer fires on every card.** Each example's
interaction state was keyed on its display number `n`, which `sm seed-validate` assigns **per
subject area** (1..k) — so a dashboard combining multiple areas carried duplicate `n`, and clicking
**Edit** (or **Add note**) on one card opened every card that shared that number. It also made the
"Generate feedback" block ambiguous (`edit N` could match more than one example). The renderer now
assigns a **stable global `1..N`** in render order — the single numbering shared by the interaction
key, the `#N` label, the feedback block, and the apply lookup — and normalizes the items file to
match, so `edit N` resolves unambiguously.
## [0.4.3] — 2026-07-14
Documentation-only release. No behavior changes; the executor and skills from 0.4.2 are unchanged.
### Changed
- **`agami-core` PyPI page is now a readable landing page.** Reframed
`packages/agami-core/README.md` (the PyPI `long_description`) to lead with the value proposition
and clarify the plugin-vs-`pip` audiences, and trimmed the deep HTTP-server internals down to a
summary plus links to `deploy/README.md` and `docs/`. Added `[project.urls]`
(Homepage/Repository/Documentation/Issues) so PyPI shows sidebar navigation. Publishing this
version is what refreshes the live PyPI page.
## [0.4.2] — 2026-07-14
Onboarding hardening for the public launch — fixes to the first-run `/agami-connect` path — plus a
documentation pass. No breaking changes; the executor internals from 0.4.1 are unchanged.
### Fixed
- **Seed validation no longer rejects every seed example.** The zero-row validation probe wrapped each
seed as `SELECT * FROM (<sql>) WHERE 1=0`; its own `SELECT *` tripped the `SELECT *` ban and every seed
was rejected regardless of its SQL. The probe now projects `SELECT 1` — it still parses and plans the
inner query, but no longer trips the ban.
- **DuckDB readiness now requires `pytz`.** DuckDB needs `pytz` to materialize `TIMESTAMP WITH TIME ZONE`
values; the driver probe only checked `import duckdb`, so an interpreter missing `pytz` scored as ready
and then failed at query time on any `timestamptz` column. `pytz` is now part of the DuckDB probe.
- **Approve operations auto-stamp their timestamp.** An approve op without a `signed_off_at` recorded
`null` and the validator rejected the whole batch. The timestamp is now stamped at the CLI boundary
(where the clock is available), so sign-off batches apply cleanly.
### Added
- **Headless sign-off (`sm approve-queue`).** A no-browser path that reads the pending review queue
(Rule 1 + Rule 2), builds a self-stamped approve op per item, and applies it (`--kind` to narrow,
`--dry-run` to preview) — so onboarding can complete without opening the review dashboard.
- **The no-DB sample clears its own pre-seed gate.** The sample's silent build auto-approves its pre-seed
queue as `signer=system` before seeding; real databases keep the human sign-off gate.
### Docs
- **Launch positioning.** The self-hosted team server (`/agami-deploy`) is labeled **Early access (in
testing)** throughout; the free-vs-paid copy leads with the value the hosted cloud adds.
- **README.** A **Databases supported** section (all engines + how each executes), VS Code/Cursor install
clarified as Manage-Plugins-UI (not the CLI slash-command form), and the sample-query copy made generic.
- **Guides.** Onboarding docs (`duckdb pytz` install, explicit render flags, the headless sign-off path),
a plain-English trust-layer intro, an `/agami-serve` (Claude Desktop) usage section, and an accurate
`migrations/core` README (the self-hosted server schema).
## [0.4.1] — 2026-07-12
The self-hosted HTTP server now runs SQL **in-process** by default — no per-query subprocess fork,
no CSV round-trip — behind a swappable execution seam. Plus a correctness fix for Postgres/Redshift.
### Added
- **Executor seam (`ports.Executor`).** `execute_sql` is split into a shared, un-bypassable *guarded
envelope* (read-only guard → semantic-model safety → resolve datasource → execute) and a swappable
**`Executor` port**. The built-in executor is the default and behaviour is unchanged; a consumer can
inject a custom executor (e.g. pooled / per-user-RBAC) via `create_app(adapters=…)` **behind the same
guard** — one execution implementation, never forked.
### Changed
- **The HTTP server executes queries in-process by default.** Previously every query forked
`python -m execute_sql`; now the served path runs through the executor seam in-process — no fork, no
CSV serialize/re-parse round-trip, native rows. The local stdio path and the `python -m execute_sql`
CLI still fork (the throwaway-process isolation is kept for the single-user tool). Successful query
results are identical to before.
- **The per-call row cap is request-scoped.** `--max-rows` now rides a `ContextVar` (was a module
global), so concurrent in-process queries with different caps can't affect each other.
### Fixed
- **Postgres/Redshift queries returned 0 rows through the Python executor.** A psycopg2 server-side
(named) cursor reports `description = None` until the first fetch, and the result collector read it
*before* fetching — so it concluded "no result set" and returned empty. It now fetches first, then
reads the description. SQLite/MySQL/etc. (client-side cursors) were unaffected. (Present since the
server-side cursor was introduced for bounded transfer.)
### Performance
- **Tool handler runs off the event loop.** The heavy query handler is offloaded via `run_blocking`
(completing the async-offload work), so one slow query no longer stalls the server's event loop.
## [0.4.0] — 2026-07-12
Runtime scalability & safety hardening: the server stays responsive and bounded as model size,
result size, and concurrent load grow. Behaviour-preserving unless a note says otherwise.
### Added
- **Bounded result sets.** A query now materialises at most a row cap instead of the whole result:
`AGAMI_SQL_MAX_ROWS` (default 1000) is the deployment cap; a per-call cap is available via the
executor's `--max-rows` (which can only lower it). Truncation is flagged (`result.truncated`) so a
cut-off result is never presented as complete. The SQL is never rewritten (no injected `LIMIT`);
Postgres uses a server-side cursor so the cap bounds transfer, not just what's written.
- **Multi-worker HTTP server.** The server can run with `--workers=N` (uvicorn import-string
factory). MCP session state is already stateless (JWT + Postgres), so it scales horizontally.
### Changed
- **OAuth refresh-token storage is now configurable — default `overwrite`.**
`AGAMI_REFRESH_TOKEN_MODE` selects `overwrite` (default: each refresh UPDATEs the session's single
token row in place — one row per session, no growing heap of dead tokens) or `rotate` (the prior
behaviour: insert-new + revoke-old, keeping OAuth 2.1 **stolen-token reuse detection**, plus a
cleanup that prunes only already-expired revoked rows). **Upgrade note:** the new default
`overwrite` trades away reuse detection — a replayed stolen refresh token simply fails to
authenticate instead of revoking the whole family. A deployment that wants family-revocation must
set `AGAMI_REFRESH_TOKEN_MODE=rotate`. Also: used/expired one-time authorization codes are now
cleared at authorize, and the query/activity logs (`query_executions` / `tool_calls`) are
explicitly **retained** — never deleted by any default path — with a new `idx_query_executions_ts`
index to keep newest-first reads fast as history grows.
- **Hosted safety guard is now fail-closed and DB-backed.** On the hosted server the fan/chasm-trap,
table/column-scope, SELECT-\* and PII guards resolve the semantic model from the database (not only
the `/artifacts` disk mount) and **refuse** a query when no model can be resolved — instead of
silently running it unguarded. The local single-player path is unchanged (a not-yet-built model is
still fine, not an error).
### Performance (behaviour-preserving)
- **Per-process semantic-model cache + single SQL parse.** The model loads once per process and the
SQL is parsed once per query, with the safety-guard indices built once and shared — down from a
reload/re-parse per query on a long-lived server. Biggest latency win.
- **Blocking work runs off the event loop.** Password hashing (argon2), OIDC HTTP calls, and the
per-call audit write no longer stall the async server — one slow login can't freeze all traffic.
- **Incremental model-authoring validation.** Curation/enrichment re-validates only the edited area,
and snapshots read each file once, so authoring a large (many-area) model no longer grows
super-linearly. Same verdicts and snapshots.
- **Faster schema discovery.** `get_datasource_schema` resolves tables via an O(1) index instead of
re-scanning the model per table — byte-identical output, faster on wide models.
## [0.3.9] — 2026-07-10
### Added
- **Composition seams for downstream extension (no-op by default).**
`mcp_http.create_app(extra_tools={}, adapters=None)` lets a downstream consumer add MCP tools and
inject the `ports.py` adapters without forking or monkeypatching core, and `tools.register(...)`
adds a tool to the shared registry with a duplicate-name guard. An existing deploy is unaffected —
`create_app()` with no arguments behaves exactly as before, and `execute_sql`'s schema is unchanged.
(#100)
- **Migration-overlay seam.** The store can layer additional migration roots on top of core's, so a
downstream package can ship its own migrations alongside agami-core's (empty/duplicate namespaces
and non-directory roots are rejected). No change for a default install. (#101)
## [0.3.8] — 2026-07-06
### Added
- **OAuth refresh tokens — no more hourly re-login on the self-hosted server.** The token
endpoint now issues a `refresh_token` and supports the `refresh_token` grant (RFC 6749 §6),
so a connected client (claude.ai) silently renews the short-lived access token instead of
redoing the full login every hour. Refresh tokens **rotate** on each use with **reuse
detection** (replaying a rotated/stolen token revokes the whole family), are stored **hashed,
never in plaintext**, and are revocable. Access tokens stay short-lived (1h). Both lifetimes are
now env-configurable (`AGAMI_ACCESS_TOKEN_TTL` / `AGAMI_REFRESH_TOKEN_TTL`, seconds) with the
same defaults when unset (access 1h, refresh 30-day idle). No action needed on upgrade — the
new `oauth_refresh_token` table migrates in automatically on boot.
## [0.3.7] — 2026-07-06
### Added
- **Read-only database user guidance.** `/agami-connect` and `/agami-deploy` now
recommend connecting agami with a **read-only** database user — agami only ever
runs read-only SELECT queries, so read access is all it needs. A new
[readonly-grants.md](plugins/agami/shared/readonly-grants.md) ships copy-paste
`CREATE USER` / `GRANT SELECT` SQL for every supported dialect (Postgres/Redshift,
MySQL, Snowflake, SQL Server, Oracle, Databricks, Trino, BigQuery). Ask agami for
"the read-only grant" to get the exact SQL for your database.
### Changed
- **Self-host compose caps container log growth.** Every service now uses the
`json-file` driver with `max-size: 10m` / `max-file: 3` (≤30 MB per container), so
a long-running deploy on a small VM can't silently fill the disk — no VM-side
`daemon.json` step needed. Also silenced a harmless `CLOUDFLARE_TUNNEL_TOKEN … not
set` warning on non-tunnel deploys.
### Fixed
- **`list_datasources` no longer reports empty on a self-hosted server.** On a
served deployment the warehouse/model is reached through the store, and the local
`credentials` file never ships to the container — but `list_datasources` was the
one tool still reading only that file, so it always returned "No profiles found …
run agami-connect", even while `get_datasource_schema` and `execute_sql` worked
against the deployed model. Because clients are told to call it first, they'd
conclude nothing was connected. It now enumerates the served models from the store
(the same seam every other tool already uses), and only falls back to the
credentials file for the local plugin.
### Security
- **Hardened the read-only `execute_sql` gate.** SQL execution now runs through a
single guard (`sql_guard`) at the shared executor, so the stdio server, the hosted
HTTP server, the skills, and cron are all protected identically (previously the
check lived only on the MCP tool path; a direct `python -m execute_sql` call — used
by the skills and cron — was unguarded). Beyond "must start with `SELECT`/`WITH`",
it now rejects multi-statement SQL (including bypasses hidden in string literals,
comments, or double-quoted identifiers), data-modifying CTEs, transaction-control /
session-state / prepared statements, `SELECT ... INTO`, row-level locks, and
dangerous server-side functions (`pg_read_file`, `lo_export`, `dblink`,
`copy_program`, `pg_sleep`, advisory locks, `query_to_xml`, …). Legitimate analytics
SQL is unaffected — a large false-positive corpus pins that. Enforcement is not
bypassable via `--no-safety` (that flag only skips the semantic-model pass).
- **Closed a dollar-quote statement-stacking bypass in that gate.** A `'` inside a
Postgres/Snowflake/DuckDB `$$…$$` (or `$tag$…$tag$`) string desynced the literal
stripper and could smuggle a second statement (`SELECT $$'$$ ; DROP TABLE x -- '`)
past the multi-statement check. The gate now neutralizes comments and string /
dollar literals in a single lexer-faithful pass (first-opened construct wins),
refuses dialect-ambiguous MySQL comment forms (a bare `--x` and executable
`/*! … */` comments), and also blocks sequence writes (`setval`/`nextval`) and
server/replication control
(`pg_stat_reset*`, `pg_switch_wal`, `pg_drop_replication_slot`, …). The guard module
is also now packaged in the built wheel (it was missing from `py-modules`, which
would have broken `import sql_guard` in an installed/containerized deploy).
## [0.3.6] — 2026-07-04
### Changed
- **`/agami-deploy` is easier to find and safe to re-run.** The config file is now
a **visible `agami.env`** (not a hidden `.env`), and the skill opens it for you.
A **re-run upgrades in place, non-destructively**: your typed password/secret and
DSN are kept, any setting new in a version is surfaced (e.g. `DATASOURCE_URL`),
and the image tag bumps only when you pass one — so a model update is just
re-run + restart, and a version upgrade tells you exactly what's new.
- **Multi-datasource deploys are an explicit choice.** With more than one model,
the skill asks which to deploy (all or a subset) and names the per-datasource
`DATASOURCE_URL__<NAME>` to set; dropping one on a re-run removes it cleanly.
## [0.3.5] — 2026-07-04
### Fixed
- **Self-host deploy no longer crash-loops on artifact permissions.** The team
server runs as a non-root container user; the deploy now stages the model
**world-readable**, so the boot-time model load can't fail `Permission denied`
on `datasource.md` under a mismatched host owner.
- **claude.ai connects to a self-hosted server.** The `/mcp` endpoint no longer
answers the bare (no-trailing-slash) URL with a `307` redirect that the MCP
client won't follow — the server normalizes it internally, so `{base}/mcp`
works on every deploy profile (including the Caddy-less Cloud Run one).
### Changed
- **Warehouse credentials come from the environment (`DATASOURCE_URL`), not a
mounted file.** The executor resolves a connection DSN from
`DATASOURCE_URL[__<datasource>]` env-first, falling back to the local
`credentials` file — one code path, no fork. The self-host bundle now carries
the DSN in `.env` and **ships no secret**: `local/` (credentials, `.pgpass`)
is never staged, and a re-run purges any stale copy from an older bundle.
## [0.3.4] — 2026-07-03
### Fixed
- **Table-prune step of a real-DB onboarding no longer crashes on an installed
build.** The `discover` pass (which renders the prune page where you pick which
tables to model) failed with `ModuleNotFoundError` on a pip/marketplace install;
it now resolves its renderer via the plugin root and works everywhere.
### Docs
- Refreshed for the current release: README slimmed (self-hosting moved to
`docs/self-hosting.md`), the published PyPI install surfaced
(`pip install "agami-core[model]"`), and the changelog backfilled.
## [0.3.3] — 2026-07-02
### Fixed
- **Marketplace-install reliability.** Credential promotion and the Claude Desktop
setup (`/agami-serve`) no longer fail on a fresh marketplace install — they resolve
the bundled library the same way every other script does, and install the model
engine through the single `sm install` path.
- **Externally-managed Python + package shadowing.** The installer now works on an
externally-managed interpreter (Homebrew / PEP 668) and can no longer be shadowed by
a partially-installed package (the model CLI is verified from a neutral path).
### Changed
- **Sample "watch it build" opens the model explorer** when the build completes, and
skips the prompts that don't apply to the curated sample (no table-prune / org /
data-dictionary questions).
- **First-time setup no longer shows a placeholder profile name** — it reads as
"first-time setup" until you name your profile.
## [0.3.2] — 2026-07-01
### Added
- **Published to PyPI.** `pip install "agami-core[model]"` (and `[server]`) installs
the library from the index, and the plugin's model-build step uses it automatically.
Published via GitHub trusted publishing (no stored token). The self-host server image
is published to GHCR (`ghcr.io/agamiai/agami-core`) so a deploy pulls it — no clone,
no build.
## [0.3.1] — 2026-07-01
### Fixed
- **Marketplace installs can query and build models with no dev checkout.** Bundled the
stdlib query library into the plugin, so a marketplace install answers questions with
no `pip install`; and the model-build step installs the engine from a source that
exists in a marketplace layout (the published package, else git) instead of a
dev-only path.
## [0.3.0] — 2026-06-24
### Added
- **No-database sample (`/agami-connect sample`).** Ships *Acme Store*, a small
local SQLite dataset (commerce + subscriptions) with a ready-made, signed-off
semantic model. Goes from install to a governed, receipted answer in under a
minute — no connection, no credentials, nothing leaving the machine. The
bootstrap (Phase 0s) offers a fast copy-the-model path and a "watch it build
live" rebuild path. Builds deterministically via the `sqlite3` CLI or a pure
Python-stdlib fallback (no install required).
- **Model snapshots / `model_version`.** A model write now stamps a content-hashed
snapshot under `<profile>/.snapshots/<hash>/`, so every answer's receipt pins a
real `model_version` (previously `null` for all profiles) and old answers stay
reproducible. New `sm snapshot <root>` CLI.
- **Deterministic interaction spine.** The mechanical parts of the skills are now
scripts that emit a uniform `{ok, data, anomalies, needs_judgment}` contract, so
the agent only makes judgment calls on genuine ambiguity:
`connect_resolve.py` (one call resolves profile / credentials / interpreter +
next-phase decision — fixes choosing a Python that can't connect),
`parse_prune_block.py` (fixes a shell word-split that mangled table lists),
`parse_model_feedback.py` (the dashboard back-channel), `csv_to_sections.py`
(charts/tables get their numbers from the result CSV, not the model), and the
`sm receipt` / `sm cura