AGENTS.md
Rama main · réplica de lectura del repositorio original.
# AGENTS.md
Guidance for Codex when working in this repository.
## What Morgan is
A **project-scoped memory for the owner's AI tools**, consolidated into facts by a local
model. One SQLite database under `MORGAN_DATA_DIR` holds memories, facts, vectors
(sqlite-vec), the FTS5 index, the entity index and session history. Two surfaces, the `morgan` CLI and the `morgan-mcp` server, are thin adapters over
one `MemoryGate`. The model server is any OpenAI-compatible endpoint (llama-server by
default). One process, no queue, no worker, no scheduler.
The optional `morgan_brain.tasks` SDK is a separate, explicitly invoked task-execution stage.
It uses a caller-selected execution journal, never the memory database, and runs only registered
trusted handlers. It has no service, scheduler, retry or provider client. See
`docs/decisions/explicit-task-execution.md` for the authority, cancellation and recovery contract.
The one-database invariant below applies to memory; task receipts have a separate lifecycle and
are not erased by `morgan forget`. Never derive execution authority from remembered text.
The opt-in `python -m morgan_brain.local_notes_cli` workflow delivers account-approved Markdown
notes into a private workspace with finite limits, STOP control and receipt-based recovery.
It does not start a service or call a model. `enqueue` appends exact account-approved notes;
`status`, `revoke`, STOP/resume and SIGTERM manage its finite lifecycle. See
`docs/decisions/local-approved-notes.md` and `docs/decisions/local-note-intake.md`.
`approve --not-before TIMESTAMP` adds one explicit delivery window; STOP, expiry and receipts
still govern resumed runs. See `docs/decisions/scheduled-local-notes.md`; no service is installed.
`repeat` compiles an explicitly approved finite UTC interval plan into exact occurrences.
Missed windows are skipped; unresolved attempts block later occurrences in the same plan.
See `docs/decisions/finite-recurring-notes.md`; no automatic renewal or service is added.
`diagnose --user OWNER --project PROJECT` reads payload-free run observations and scoped journal
counts; it never retries or changes tasks. See `docs/decisions/local-worker-diagnostics.md`.
`supervise` explicitly owns a finite fixed worker subprocess with bounded crash restarts and
sequential shutdown; it installs no service. See `docs/decisions/bounded-local-supervisor.md`.
`daily_brief_cli` prepares an unapproved focus plan from explicitly selected current commitment
evidence through the gate, without model calls. See `docs/decisions/memory-backed-daily-brief.md`.
`python -m morgan_brain.eval.note_selection` is an explicit synthetic-only Ollama experiment.
It neither reads memory nor authorizes tasks; see `docs/decisions/synthetic-note-selection.md`.
`daily_brief_cli --follow-revisions` resolves selected commitment families to one current version,
so explicit completion/correction updates the next plan without changing the selected IDs.
See `docs/decisions/current-personal-plan-pilot.md` for the full synthetic pilot and remaining gaps.
`daily_brief_cli --unavailability ID --timezone IANA_ZONE` conservatively defers a day when
reviewed, sourced unavailability overlaps it. Constraints follow explicit immutable revisions;
see `docs/decisions/confirmed-unavailability.md`. It does not infer location or schedule hours.
`guided_plan` provides one foreground review/plan/completion conversation over the existing
model cycle, capture path and current-revision planner. It uses a separate private session
workspace and explicit confirmation; see `docs/decisions/guided-personal-plan.md`.
Its `busy` / `change N` / `cancel N` dialogue journals reviewed source statements and intervals
and carries them into each session plan; see `docs/decisions/guided-unavailability.md`.
`renew N` / `повторить N` renews a selected open task after source/date review, with exact
revision replay and stale-parent checks; see `docs/decisions/reviewed-task-renewal.md`.
It changes a plan date only, never sends a reminder or infers a target from message text.
`guided_plan WORKSPACE --status` reads current scoped state as JSON without prompts, memory
updates or model calls; pending review and fixed-day freshness remain explicit.
Its `--preview-day DATE` option evaluates current evidence for an explicit day without changing
the saved session date. See `docs/decisions/guided-status.md` and the bounded personal pilot plan;
it installs no user messaging channel or service.
`owner_input` provides one-shot local message submission, exact review/confirmation and STOP
without a model, listener or service. It reuses the private guided workspace and MemoryGate;
messages alone grant no task changes. The finite pilot is capped at 14 messages, eight tasks
and seven days. See `docs/decisions/local-owner-input.md`; no automatic language matching is added.
`local_alert_cli` explicitly displays one bounded macOS dialog with account-approved text,
STOP and journal fencing; see `docs/decisions/local-alert.md`. A timeout is not a read receipt.
It installs no service or standing permission and never retries a started attempt.
`owner_push_cli` explicitly publishes one reviewed ntfy message with a transient prompted token;
server acceptance is not phone delivery or reading. `launch_agent_plan` only renders a disabled
plan for the bounded local-note worker. See `docs/decisions/owner-channel-hosting.md`.
Neither installs credentials, services or recurring external-send authority.
`note_proposal_cli` makes an explicit local-model draft from piped text, without memory writes.
`morgan commitment` validates reviewed planning fields and reuses `remember` capture. See
`docs/decisions/personal-note-intake.md`; neither path grants execution authority.
`model_cycle` owns one finite local model operation with an independent expiry guardian,
private Unix socket and the existing workspace locks/STOP. No service is installed; see
`docs/decisions/bounded-model-cycle.md` for limits and measured extraction failures.
`morgan review-proposal` previews full source context and binds explicit acceptance to the
exact claim and destination before reusing memory capture. The digest is not authentication;
see `docs/decisions/reviewed-proposal-intake.md`. No execution authority is granted.
Read first: `docs/ARCHITECTURE.md` (the package), `docs/WIRING.md` (running it),
`docs/ROADMAP.md` (what is not in the code and why, what retrieval measures, what is next),
`docs/decisions/` (decisions, and what the code does about each). The archived agent kernel is
at the tag `legacy-v0.1.0-kernel`, its designs under `docs/archive/`.
## Package map (`morgan_brain/`, ~11,200 lines)
The tree is grouped by what a file does, so "where does a write go" and "where does a request
come in" are answered by the directory names.
- `config.py` — the single `MORGAN_`-prefixed settings source (`settings_for(surface)`). The
CLI reads `~/.config/morgan/.env`, then `./.env`, then the environment; `morgan-mcp` reads
the user file and the environment only, because its working directory is the client's.
`doctor` lists the files read. The database defaults to `~/.local/share/morgan/`.
`MORGAN_EMBEDDING_ENDPOINT` addresses embeddings separately when the chat server does not
serve them.
- `models.py` — the domain models. Everything that persists is `user_id`- and
`project`-keyed. `Memory` carries a `MemorySource`; `TemporalFact` carries
`valid_from`/`valid_to`/`superseded_by`.
- `logging_setup.py` — process output. stdout is UTF-8 because the protocols on it are; logs
go to stderr. One call per entrypoint.
- `composition.py` — opens the database and wires the above. `build_memory_context` needs no
chat model and sends no request: it registers the settings' embedding model and width on a
writable database that has no embedding space, refusing instead when the vector table was
created at another width (the hash backend registers none), and it refuses a space of
another width than `MORGAN_EMBEDDING_DIM`. While a heavy migration step waits, it opens the
gate read-only and registers nothing. `build_app_context` adds the chat model.
- `memory/` — the core. `gate.py` is the only door and `module.py` is the one write path and
the fused recall; `embedder.py` is the embedding seam; `fingerprint.py` is the five frozen
strings that identify an embedding space and the pure arithmetic (`cosine`, `compare`,
`pack`/`unpack`) over their vectors, with no I/O of its own; `checked_embedder.py` wraps an
embedder so that a process's first request also carries those strings, and refuses with
`EmbeddingSpaceMismatch` when the model answering is not the one that wrote the active
space; its `check()` re-sends them on demand, never short-circuited by that first-call
cache, for `morgan import`'s canary; `migrations.py` upgrades a database written by an
older version, its light steps when it is opened and its heavy ones under `morgan migrate`;
`snapshot.py` writes and lists verified `VACUUM INTO` copies of the whole database, and
restores one behind a safety
snapshot of its own. Below them:
- `store/` — persistence only: `db`, `episodic`, `temporal`, `vectors`, `fts`, `entities`,
`history`, `spaces` (the `embedding_spaces` table and its one-active partial index),
`projects` (the `projects` table, keyed by name: classification, remote, root and the
per-project capture/consolidate switches; `get` and `list_all`, `seed` for migration step 7's
one row per project already named in `memories`, `facts` or `session_history`, `register`
for every project-keyed write after it, and `record` for the classification, remote and
root a CLI write from inside a repository fills in),
`tables` (`PROJECT_TABLES`, the one list of project-keyed tables `forget` reaches). Each
owns its schema and its queries; none of them ranks anything. Every write goes through
`db.write_transaction`.
- `recall/` — `fusion` (reciprocal rank over vector and keyword search, rank-only),
`floor` (the relevance floor, judged on vector scores) and `language` (a query's language
by script alone, no model call, logged on every recall).
- `knowledge/` — `extract`, `surprise`, `fact_ops`, `consolidation`.
The work that costs a model call or a full pass, and never runs inside a recall.
- `eval/` — measuring what recall returns: labelled probes, recall@k, MRR and leak rate,
scored per probe kind, printed beside the run's configuration. Run with `pytest --live`
against a real embedding endpoint.
- `providers/` — the only place a model SDK is imported: `openai_compat.py` (chat),
`embeddings.py`, `structured.py` (JSON-validated output), `factory.py` (settings → adapters,
where embeddings are sent, the `CheckedEmbedder` around the embedding model, and which
retry budget an embedding call gets), `wire.py` (message types, `ChatClient`,
`ProviderUnreachable`, `ProviderRefused`, `EmbeddingSpaceMismatch`, and `is_refusal`,
which statuses refuse a request, for the embedder and `doctor` alike).
- `app/chat.py` — one turn: recall, answer, remember. Not a surface: the one use-case both
surfaces call. `app/chatgpt_import.py` seeds memory from a ChatGPT export, routes a fifth of
the conversations to a holdout project, and runs the import canary.
- `surfaces/` — where requests come in. `cli/` (`__main__` parses and dispatches, `commands`
answers, `maintenance` answers `morgan snapshot`, `morgan restore` and `morgan migrate`,
`payloads` shapes the result, `render` prints it, `doctor` diagnoses by reading only -- it
opens the file read-only, builds no store, creates no table and runs no migration step --
and probes the chat and embedding servers separately, telling reachable, slow, refused and
unreachable apart, `project` resolves the enclosing git repository once -- its name, its
root and its remote, read by parsing the repository's own git config rather than spawning
`git` -- and holds `classify`, `install_skill` writes the packaged `skill/SKILL.md`
into the coding agents installed here), `mcp_server.py` (five MCP tools over stdio or
streamable-HTTP, calling those same command handlers), and `network.py`, the bind guard
that protects the HTTP one.
## Invariants
- **All memory access goes through `MemoryGate`.** No caller holds the `MemoryModule`.
- **Every read and write is project-scoped.** `Memory` and `TemporalFact` carry a required
`project`; the gate rejects an empty one. A write that names no project lands in `personal`,
and the result says so; there is no silent default. `all_projects=True` is the explicit
cross-project escape hatch, never the default.
- **One write path.** `MemoryModule.store` writes every index in one transaction: episodic
row, vector, FTS5, entity index, and the project's `projects` row. Entities are extracted
there when the caller gave none. Every row carries its provenance: origin, client, session,
working directory, author and scope, with defaults; every writer names its origin. A memory
visible to one index and not another is found by one search and missed by the next.
- **`forget` reaches every project-keyed table.** `store/tables.py::PROJECT_TABLES` is the one
list; a store that adds a table registers it there, and a test fails on any table with a
`project` column missing from it. `forget` walks the registry and erases each table through
the deleter its store owns, by the memory ids and by the owner's `user_id` and `project`
columns wherever a table has both; a space's vec0 table other than `vec_items` is erased by
those columns alone. A registered table it cannot erase that way stops it by name before
anything is erased. The forgotten words leave the database files: FTS5 is optimized, the
database vacuumed and the write-ahead log truncated; while another connection's read blocks
that checkpoint they stay until a later one, and a warning says so. The snapshot `morgan
forget` takes first is the undo and keeps them until the owner deletes it.
- **Every write holds the lock from its first statement.** Other processes share the database
file, so a write that reads before acting takes the lock before the read:
`store/db.py::write_transaction` opens `BEGIN IMMEDIATE`, and a write inside another one
joins it as a savepoint. Nothing awaits while the lock is held -- the embedding happens
before it is taken -- and no store method commits on its own.
- **Recall ranks the whole scope.** Vector and keyword search each rank every memory in the
project (or every project, with `all_projects`), fused by rank. Nothing narrows the
candidates first: on a real archive a narrowing pool cut answers out and found none. The
entity index is the relevance floor's evidence, not a third ranking.
- **Light steps run on open; heavy steps run only under `morgan migrate`, behind a snapshot.**
Derived data is re-derived by numbered steps in `memory/migrations.py`, counted in SQLite's
`user_version`. A step that only adds a table or a defaulted column runs when the database
is opened; a step that rewrites, moves or deletes rows runs only under `morgan migrate`,
which takes a `VACUUM INTO` snapshot first and runs every pending step in one write
transaction. Until then the database opens read-only and every write says so by name.
`morgan restore` puts a snapshot back.
- **Nothing destructive runs without a snapshot.** A migration wave, a project-grain `forget`
and any re-embed take a `VACUUM INTO` snapshot first, into `MORGAN_SNAPSHOT_DIR`, which
Morgan never prunes.
- **Facts evolve, they don't overwrite.** Update = close the old interval, open a new one. A
key has at most one current fact, and a unique index on `facts` enforces it. The key is
(`user_id`, `project`, `subject`, `predicate`); it gains `author_id` and `scope` once
supersession is assembled deterministically
(`docs/decisions/0001-fact-key-and-forget-reach.md`).
- **Facts are surfaced alongside episodics, never instead of them.** Recall budgets the
fact block so a matching memory cannot be pushed out of the window by fact volume.
- **Actor attribution.** Every memory records its `MemorySource`. The reply to `ask` is
stored as `agent_inferred`; never treat an inference as a user statement.
- **One embedding space is active, and it is fingerprinted.** `embedding_spaces` records
model, width, prefixes and the vectors of five fixed strings; a model that answers outside
the measured tolerance of that fingerprint is a named error, never a silently wrong search. A
same-width swap is caught like a width change.
- **A model server that is down is reported by name.** Adapters raise
`providers.wire.ProviderUnreachable` carrying the endpoint and the setting that addresses
it; the CLI and MCP tools print its message. A transient failure is retried first, within a
budget: short when the host refuses connections; longer when it answers slowly or drops one,
as a cold host loading its model does; longer still for an import. The message says which. A
4xx is `ProviderRefused`, naming the key setting. A bare `HTTPStatusError` is a regression.
The factory hands each adapter the setting that addresses its endpoint, because embeddings
go to the chat endpoint unless `MORGAN_EMBEDDING_ENDPOINT` is set. A bare traceback is a
regression, and so is sending the owner to check a server that works.
- **stdout is a protocol on both surfaces.** `--json` output and the MCP stdio transport are
parsed by machines; all logs go to stderr. Never `print()` diagnostics from library code.
- **Nothing runs a model unasked.** `ask` and `consolidate` call the chat model; nothing else
does, and nothing runs on a schedule. Consolidation is on demand (or the owner's own cron).
Nothing embeds at open, in a hook or in a sweep. The active embedding space is fingerprinted
at a process's first embedding call, riding on that call, or by `morgan migrate` once its
wave has committed; `doctor` embeds when it is run, and `doctor --vectors` re-embeds a
sample when asked.
- **Every MCP tool declares what it does.** `TOOL_ANNOTATIONS` states all four hints for every
tool, and only a tool that changes nothing claims read-only: a client may run those
unprompted, and `install-skill` allows exactly those in Claude Code. `ask_morgan` stores the
exchange, so it is a write.
- **A project is a repository.** The CLI names it after the enclosing git repository; a linked
worktree belongs to the repository it was created from, a submodule is its own. Outside one,
the project is `personal`. One resolution answers the name and the repository together, so
the two can never come from different checkouts.
- **Every project written to has a row, and a CLI write from inside a repository says what
that repository is.** A memory, a fact and a turn of session history each register their
project in `projects` in their own write transaction (`INSERT OR IGNORE`, `unclassified`),
below the gate, so both surfaces do it and a failed write leaves no row. `remember`, `ask`
and `consolidate` then record `classify(remote, MORGAN_WORK_REMOTE_GLOBS)`, the remote and
the root through the gate -- only when the project was named by the repository they ran in
(not `--project`, not the `personal` default, not `--all-projects`), recomputed every time,
and refused like any other write while a migration step waits. A read records nothing. Only
those three columns are written: the owner's capture, retention and consolidate switches are
theirs. A config Morgan cannot read is not a repository without a remote: the reader says
which of the two it found, and only the second is recorded -- the first leaves the row as it
is. A recording that fails never fails the command whose write already committed; it warns
on stderr, naming the project. The remote and the root are the owner's data -- never logged,
and `doctor` prints neither.
- **No listener beyond loopback without a key.** `network.assert_safe_bind` refuses to start
`morgan-mcp --transport http` on a non-loopback host while `MORGAN_API_KEY` is unset or the
placeholder.
- **One of each.** One settings object, one database, one way to write to it, one gate, one
entity extractor, one logging configuration.
- **Never hardcode the owner.** Everything is keyed by `user_id`; single-owner is a config fact.
## Credential admission
`memory/credentials.py` refuses a narrow set of explicit provider-token/private-key shapes in
new memory content and fact text, before embedding or writes. `ask` checks its question before
counting/recall; import checks the whole visible turn before embedding its first new piece.
Atomic turns preflight both
events. Accepted evidence is never redacted or reattributed; exact store replay is unchanged.
This is not a complete secret detector or egress filter: historical data, metadata and separate
task journals remain outside it. See `docs/decisions/credential-admission.md`.
## Known limitations
- `recall` declines to answer only when `MORGAN_RECALL_FLOOR_MARGIN` is set. The value belongs
to the embedding model: 0.08 for `qwen3-embedding:8b`, measured on a 3,010-memory archive
with 128 labelled questions (`docs/measurements/2026-09-phase0-baseline.md`); 0.11 for
Qwen3-Embedding-0.6B, measured on the bundled probes and on a real archive. Other models are
unmeasured.
- `consolidate` reads a project's episodics with `recall(text="", top_k=50)`: the memories
nearest an empty query's embedding, not the most recent, and fewer than 50 when the project
holds facts, which take up to half of those places. With `MORGAN_RECALL_FLOOR_MARGIN` set,
the floor can decline that query, and consolidate then has nothing to read.
- The import canary catches a model that is still answering wrong when a check runs; a single
transient wrong vector between two checks passes it. Suspect memories stay stored, a re-run
of the import skips them, and `doctor --vectors` samples rather than checks every row.
Nothing re-embeds named memory ids.
- A database keeps the embedding model its vectors were written with. Once its space is
fingerprinted, a different model -- same width or not -- is refused on every path that
stores or searches; nothing retires a space or re-embeds, and a snapshot holds the same
model's vectors. To change the model on purpose, point `MORGAN_DATA_DIR` at a new database, with
`MORGAN_EMBEDDING_DIM` at the new model's width; the old one keeps working with its own.
- The remote is read from the repository's own config file: `url.<base>.insteadOf` rewriting
is not applied, and `include`/`includeIf` files are not followed. A work host reached only
through a rewrite is classified by the URL as written, and a remote declared in an included
file is not seen at all, leaving the project `unclassified`.
- `projects` is keyed by the project's name, and a project is named after its folder, so two
checkouts called the same thing share one row: each CLI write from either rewrites the
other's classification, remote and root. Nothing acts on the label yet, and recall
and consolidation are unaffected -- they are keyed by that same name, so the two checkouts
share their memories as well.
- A project written to only through `morgan-mcp` has a row and stays `unclassified`. The
server may run on another machine than the client, so it never sees the repository a call
came from, and its `project` argument is a name, not a checkout. The classification, remote
and root are recorded by a CLI write from inside the repository; the walk over
`MORGAN_CODE_ROOTS` that would classify the rest is not built.
- `morgan-mcp` builds a FastMCP server, and FastMCP's own settings read a `./.env` in the
folder the client starts the server in. FastMCP passes every one of its settings
explicitly, so the `FASTMCP_*` values in that file change nothing, and Morgan's settings
never read it; but a `./.env` there that is not UTF-8 stops `morgan-mcp` at start with a
`UnicodeDecodeError`.
- Entity extraction is deterministic and cased-script only; scripts without letter case
(Chinese, Japanese, Arabic, Hebrew) yield nothing rather than a guess. A capitalised word is
a name only where the text capitalises it away from a sentence, clause or line start, so a
name that only ever opens sentences in a memory is not indexed. Code in a memory still
yields words like `true` and `error`.
- A superseded memory outranks the current one in 7 of the 10 knowledge-update probes
(`qwen3-embedding:8b`). Fusion is rank-only and carries no recency term. Supersession lives
on facts; the probes store episodics, which carry none.
- Multi-hop questions are not answered. Recall ranks memories and has no mechanism to
compose two of them; measured recall@8 is 0.62 against 0.90 for single-hop
(`qwen3-embedding:8b`).
## Pinned local deployment
`scripts/runtime_bundle.py` exports one exact trusted Git commit into a normal wheel and
hash-pinned offline dependency bundle, then installs only into a new private runtime directory.
`python -I -m morgan_brain.runtime_check --expected-revision SHA` verifies installed origin,
revision and package hashes. Never deploy the editable development environment or infer a
revision from cwd. No runtime is activated and no service is installed automatically. See
`docs/decisions/pinned-runtime.md` for recovery and trust boundaries.
## Build, test, run
```bash
pip install -e ".[dev]"
mkdir -p ~/.config/morgan && cp .env.example ~/.config/morgan/.env # MORGAN_LLM_ENDPOINT
morgan doctor
pytest -q # See the latest CI run for verified totals
ruff check . && ruff format --check . && mypy morgan_brain && bandit -c pyproject.toml -r morgan_brain
```
Python 3.12+, line length 100, `ruff` is the linter and formatter, `mypy --strict`. Keep
`main` green. Root-cause fixes only; a workaround gets flagged, not silently applied.
`owner_ui` is an opt-in bounded macOS dialog adapter over `owner_input`, with explicit
review, STOP and saved-review recovery. No listener or service is added; see
`docs/decisions/native-owner-review.md`.
`owner_ui --server PATH --model PATH` explicitly offers a bounded local draft within the
same review. `owner_model` reuses model-cycle guardian locks and the owner STOP marker;
source-keyed draft recovery never grants task authority. See
`docs/decisions/integrated-owner-model.md`. No service or automatic retry is added.
`owner_loop` is an explicitly invoked finite background drafting stage over owner_input.
It prepares unapproved local proposals, never confirms tasks, and preserves the pilot's
seven-day expiry, STOP and attempt markers across restart. `owner_loop_plan` only renders
an uninstalled disabled login-agent plist. See `docs/decisions/finite-owner-loop.md`.
`owner_input review --cancel CURRENT_ID` and native `Отменить задачу` record an
explicit reviewed open-to-cancelled transition. It is distinct from completion;
never infer cancellation authority from message text. Older runtimes cannot read
cancelled commitments or pending cancellations: update every pilot reader before
exposing the action. See `docs/decisions/reviewed-task-cancellation.md`.
`owner_input review --reopen CURRENT_COMPLETED_ID --due DATE` and the native reopen action
explicitly revise a completed task back to open while preserving its family and sources.
Older readers reject persisted reopen reviews even after application; update all readers before
enabling it. See `docs/decisions/reopen-completed-task.md`. No installed pilot is updated.