CHANGELOG.md

Rama main · réplica de lectura del repositorio original.

# Changelog

All notable changes to **OpenNVR AI Adapter** (the reference adapter server in
this repo) are documented here. The standalone Python SDK has its own
changelog under [`opennvr_adapter_sdk/CHANGELOG.md`](opennvr_adapter_sdk/CHANGELOG.md).

Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
the project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

- **fast-plate-ocr: the whole-graph HTP pin is off by default.** The pin
  shipped in #89 compiles the OCR export whole for the Hexagon and, measured
  on the Ventuno Q, answers one constant string for every crop while the
  CPU path reads them all. Default back to the partially offloaded graph
  that reads correctly; `OPENNVR_LPR_PIN_BATCH=1` turns the pin on for
  anyone chasing the fp16 numerics.
- **fast-plate-ocr reads on the Hexagon NPU, not beside it.** The OCR
  export has a symbolic batch, and the QNN HTP compiles fixed shapes only,
  so on the qualcomm flavour it took 2 of the model's 377 nodes and left
  the rest to the CPU: 62 ms and 501 ms of CPU per read across 6.5 cores,
  measured on a Ventuno Q — enough for two reads in flight to slow the
  detector. The service now pins the batch to 1 before the session is
  built (one crop per call) and the SDK shim compiles a fixed-shape copy:
  every node on the HTP, 10 ms and 1 ms of CPU per read. Through core and
  KAI-C the per-call latency is unchanged (the frame decode and the proxy
  hop, and on a shared HTP the wait for the device, set it); the saving is
  the CPU. The SDK's shape pin now takes a `pathlib.Path` as well as a
  `str`: the library hands its model that way, and the pin had been
  silently skipped as "in-memory". The first start compiles once; a model
  with no on-disk path is left as exported and the log says so.
### Fixed

- **fast-plate-ocr's docs say what it does.** The README, the Dockerfile
  and the service docstring described a "YOLOv8 → crop → OCR" chain
  and a "pre-cropped plate image"; the adapter has localised the plate
  in a vehicle crop itself (open-image-models yolo-v9-t-384) since 0.1.5,
  its default floor is 0.45 (not 0.30), and it ships in CPU, -intel and
  -qualcomm flavours (the Dockerfile said "CPU only"). The qualcomm
  publish job's comment no longer says "-intel suffix, amd64 only".

### Added

- **`text_embed`: sentence embeddings for search's caption arm.** A new
  adapter (`adapters/text_embed`, image `text-embed-adapter`, port 9013,
  amd64 + arm64) advertising a new task, `text_embed`: a sentence in, a
  384-dimensional unit vector out, from `BAAI/bge-small-en-v1.5` (MIT) as
  the repo's own ONNX export, run by ONNX Runtime on the CPU with two
  threads — a caption costs a few milliseconds. The `embed` adapters keep
  the frame arm (a frame and a query in one space); this one exists
  because an image-text model's text tower is a poor judge of text
  against text: on the Ventuno Q SigLIP2's text-to-text cosines for
  relevant and irrelevant caption/query pairs overlap (0.76–0.87 against
  0.59–0.75, 2026-11-10), where bge-small puts the matching caption at
  0.76–0.82 and the rest at or under 0.63. `text` or `texts` (a batch of
  up to 64), `kind` query/document (a model's query instruction, if its
  card asks for one, goes on queries only), `dim` + `model` in the answer
  so a store never compares vectors from two models; `TEXT_EMBED_MODEL_DIR`
  takes any BERT-style export with `cls` or `mean` pooling. Weights baked
  in, no egress. OpenNVR's core picks it for caption vectors and the
  caption arm's query where registered, and falls back to the embed
  adapter's text tower where not.
- **`whisper_qnn`: Whisper-Small on the Qualcomm Hexagon NPU.** A third ASR
  adapter, for the camera agent on the Ventuno Q, where speech-to-text on
  the CPU (parakeet) took 0.7–2.3 s an utterance and had no confidence
  signal to gate noise. It drives Qualcomm AI Hub's precompiled QNN export
  of openai/whisper-small (w8a16, QCS8275 / HTP v75; the asset names the
  Arduino VENTUNO Q as its reference device) through ONNX Runtime's QNN
  plugin provider: numpy log-mel front end checked against HuggingFace's
  extractor to 1e-6, encoder → cached greedy decoder with the English
  prefix forced, `no_speech_prob` from the first step, the §5.3 result plus
  `encoder_ms`/`decoder_ms`. Hexagon only: anywhere else `/health` says
  `error` and `/hardware/evaluation` `blocked` with the reason, never a
  silent CPU fallback. Assets, tokenizer and mel filters are baked into the
  image (`--build-arg WHISPER_QNN_CHIPSET=` for another chip in the release).
  Published in the qualcomm flavour as `whisper-qnn-adapter`, port 9020.
  Not yet measured on the hardware.

### Added

- **litertlm: an OpenAI-compatible chat endpoint on the NPU engine.**
  `POST /v1/chat/completions` on the caption adapter, same engine, same
  bearer token: `messages` with system / user / assistant / tool roles,
  `tools` as OpenAI function envelopes, `tool_choice`, `temperature`,
  `max_tokens`; answers `choices[0].message` with `content` and
  structured `tool_calls`, so the camera agent switches to the NPU with
  `llm_provider: openai` and the adapter's URL. Measured on a Ventuno Q
  with the agent's real prompt and seven tool schemas: 10 of 10 right
  tool choices at 0.8–2.0 s a turn (the CPU model: 20–25 s). A prompt
  past the bundle's 4,096-token window is `400 context_length_exceeded`,
  never truncated; captions and chat share one lock; streaming is not
  supported.

- **litertlm: Gemma captions on the Qualcomm Hexagon NPU.** A new
  caption/VQA adapter (`adapters/litertlm`, image `litertlm-adapter`, port
  9020, arm64, published as `…-qualcomm`) runs Gemma 4 E2B compiled for the
  QCS8275 through Google's LiteRT-LM, with the image encoder on the NPU
  too. Same tasks and result keys as moondream/ollamavlm (`scene_caption`
  → `caption`, `visual_qa` → `answer`), so it drops into the
  `caption-adapter` slot. Measured on a Ventuno Q (2026-11-05): a 124 KB
  evidence photo in 2.5 s warm / 4.3 s cold, ~1.2 GiB resident, the
  detector sharing the NPU slowed from 38 to 44 ms per inference; the CPU
  captioner the box ran before (qwen3-vl 2B via Ollama) took 60–90 s per
  photo and outran every deadline. The 2.7 GB model is downloaded once into
  `/models` when `OPENNVR_LITERTLM_MODEL_URL` is set; Qualcomm's six AI
  Runtime libraries are fetched once at first start by range requests
  (pinned SHA-256; not redistributed in the image). Photos are downscaled
  to 768 px before the model. Without `/dev/fastrpc-cdsp` the adapter
  stays up and `/hardware/evaluation` is `blocked` with the reason.

### Fixed

- **litertlm chat: the agent's second iteration works.** A tool result
  goes back to the engine as one `tool` turn of `tool_response` parts
  (a plain-text tool turn fails the native send; a Python list fails the
  entry point), a conversation that ends with tool results sends those
  with the question and the call as history, and past tool calls in
  history keep the OpenAI `function` envelope (a flat shape fails the
  chat template). Measured on a Ventuno Q: result back in, answer out in
  1.07 s. These three fixes were pushed after #83 merged and missed it.

- **QNN on the HTP is refused, out loud, when the DSP node is not in the
  container.** The plugin lists an HTP device on any Qualcomm host and
  compiles the graph without opening it, so an adapter recreated without the
  Qualcomm overlay (no `/dev/fastrpc-cdsp`) built a session that named QNN,
  reported `accelerator_in_use: true`, and ran every node on CPU — 468 ms an
  inference against 15 on the HTP, health green (Ventuno Q, 2026-11-01).
  `plugin_device_missing` now checks the node before the plugin is attached;
  without it the adapter runs on the classic CPU list, `/hardware/evaluation`
  says `accelerator_in_use: false`, and the log names the missing file and
  the overlay that lends it. SDK 1.4.x.
- **ollamavlm: a failed auto-pull is a hard error with its cause.** A pull
  that died ("no space left on device") left `/infer` answering
  `model_not_pulled, transient, auto-pull is running — retry shortly` for an
  hour. It now answers `model_pull_failed`, not transient, with Ollama's own
  reason and the fix; `/hardware/evaluation` is `blocked` with the same text
  until the model appears (pulled by hand, or the adapter restarted).

### Added

- **InsightFace: a recognition miss carries its evidence.** The
  `face_recognition` reply for a face below the threshold now includes
  `similarity` (the nearest enrolled face, below the threshold by
  definition and deliberately unnamed; `null` when nobody is enrolled)
  and `confidence` (the detector's score for the face). A match reply
  gains `confidence` too. A caller can now hold its verdict on a
  side-on frame at 0.45 instead of calling a known person a stranger;
  the Smart Doorbell's per-visit decision reads both. Additive: the
  wire shape for a match is unchanged, and `recognized: false` means
  what it did.

## [0.1.10] — 2026-10-31

The 0.1.9 images under the number of the OpenNVR release they ship with
(open-nvr 0.1.10 carries one more detect-pipeline change; nothing changed
here). Every adapter at `0.1.10`, the five ONNX adapters also at
`0.1.10-intel` and `0.1.10-qualcomm`. SDK stays 1.4.0.

## [0.1.9] — 2026-10-31

### Added

- **OpenNVR Accelerate: Qualcomm flavour (`-qualcomm`) of the ONNX
  adapters.** yolov8, yolo-pose, insightface, fast-plate-ocr,
  package-detection and parakeet build with `ACCEL=qualcomm`: plain
  onnxruntime plus `onnxruntime-qnn`, the Qualcomm AI Runtime *plugin*
  provider (bundles the HTP user-space), arm64 only. The SDK registers it
  and routes every session — the adapter's own and its libraries' — to the
  Hexagon NPU for `ADAPTER_ONNX_PROVIDERS=QNNExecutionProvider:htp`;
  open-nvr's `docker-compose.accel-qualcomm.yml` sets that per adapter and
  lends the container the DSP. Parakeet now honours
  `ADAPTER_ONNX_PROVIDERS` like the other ONNX adapters (CUDA-when-present
  stays the default). Measured on an Arduino Ventuno Q (QCS8275), `/infer`
  end-to-end, NPU vs the eight ARM cores: yolov8 59–67 vs 152 ms, pose
  48–69 vs 113 ms, package detection 55 vs 103 ms, plate OCR 66–84 vs
  272 ms. Parakeet loads on the HTP but its dynamic-length graphs fail at
  inference (Expand node), so it is not published in this flavour and
  stays on CPU (0.6 s per 3 s of audio).

- **Adapter SDK 1.4.0 on PyPI.** The plugin execution-provider support
  above (`register_plugin_providers`, `plugin_session`, the constructor
  shim, `.opennvr-accel/` caches, `static_shapes` with `*`) for adapters
  built outside this repository; `opennvr_adapter_sdk/CHANGELOG.md`.

- **An accelerator that cannot start never takes an adapter down.**
  `build_with_fallback()` in `opennvr_adapter_sdk.providers` builds the
  session (or InsightFace / fast-plate-ocr model object) with the requested
  providers and, if that raises — device not passed through, driver absent,
  OpenVINO plugin failure — logs the error and builds again on CPU. All five
  ONNX adapters use it; `/hardware/evaluation` then reports
  `accelerator_requested: true`, `accelerator_in_use: false` and
  `accelerator_fallback_error` with the reason. A failure with no
  accelerator requested still propagates.
- **Static shapes on OpenVINO accelerators.** `ADAPTER_ONNX_PROVIDERS` with an
  OpenVINO device other than `CPU` now compiles the graph for the shapes of
  the first inference (`disable_dynamic_shapes`) instead of as a dynamic
  graph; `ADAPTER_ONNX_DYNAMIC_SHAPES=1` restores the old behaviour. The
  NPU plugin cannot run dynamic shapes at all, and an Intel iGPU driver was
  seen to hard-reset the host while compiling one. `yolo_pose`, whose
  export has dynamic axes, pins its input to `DEFAULT_IMGSZ` on such a device
  (`static_shapes` / `wants_static_shapes` in `opennvr_adapter_sdk.providers`)
  and reports `input_shape_pinned` in `/hardware/evaluation`; `imgsz` is then
  fixed, as it already was for fixed-size exports.
- **OpenNVR Accelerate: hardware flavours for the ONNX adapters.** `yolov8`,
  `yolo_pose`, `insightface`, `fast_plate_ocr` and `package_detection` build
  from one Dockerfile each with `ACCEL=cpu|intel`; the `-intel` tags carry
  `onnxruntime-openvino` plus Intel's OpenCL, graphics-compiler, Level Zero
  and NPU user-space, so inference runs on an Intel iGPU or NPU. One SDK
  resolver (`opennvr_adapter_sdk.providers`) replaces every hardcoded
  `["CUDAExecutionProvider", "CPUExecutionProvider"]`: `ADAPTER_ONNX_PROVIDERS`
  takes a `:device` per provider, CPU is always the fallback, ORT's silent
  fallback is logged once, and `/hardware/evaluation` reports requested vs
  effective providers. InsightFace no longer forces CPU on an accelerated
  build (its `ctx_id` now follows the build). The §8 `gpu` permission is
  declared truthfully for any accelerator provider, not only CUDA.

## [0.1.8] — 2026-10-16

Better ears and better memory. Speech to text moves to Parakeet (a third
of whisper's errors at a fifth of the time on four cores) and the
embedding adapter moves to SigLIP 2 (coco128 retrieval mAP 0.672
against 0.571); both measured with harnesses that now live beside the
camera agent in open-nvr. No SDK release: `opennvr-adapter-sdk` stays at
1.3.0.

Upgrading: the platform's `.env.example` pins `ADAPTER_TAG=0.1.8` and
`STT_ADAPTER_TAG=0.1.8` from OpenNVR's next release. The `clip` image's
vectors are 768 wide; the platform re-embeds history on its own
(open-nvr#659). Nothing else migrates.

### Changed

- **The embedding adapter is SigLIP 2** (`ViT-B-16-SigLIP2-256`, Google's
  `webli` weights, Apache-2.0, 768 dimensions) in place of CLIP
  `ViT-B-32`. Measured on coco128 text-to-image retrieval: mAP 0.672
  against 0.571, recall@5 1.00 against 0.94, 57 ms an image against 26
  on a 4-thread CPU. Vectors are 768 wide and not comparable with the
  old 512; the platform re-embeds when the model changes. `CLIP_MODEL`
  / `CLIP_PRETRAINED` still choose any OpenCLIP checkpoint.

### Added

- **`parakeet-adapter`**: speech to text with NVIDIA Parakeet TDT 0.6B
  v3 through onnx-asr (ONNX Runtime, int8, CPU first), port 9019. The
  whisper adapter's §5.3 result shape and the same `audio_transcription`
  task, so the camera agent's `whisper_url` points at either. Measured
  on the agent's own phrasings: WER 1.4 % against faster-whisper
  base.en's 3.3 %, 0.31 s against 1.42 s an utterance on four CPU
  cores. Transcription only (no translation; 25 European languages, no
  Hindi — whisper stays for those). Weights baked in, no egress.

## [0.1.7] — 2026-10-16

Detect what you can describe, and a faster detector. The open-vocabulary
adapter is published at last, with its weights baked in so it boots on a
sealed box; the yolov8 adapter learns the name of its weights file so the
platform can move to YOLO26n; and the published open-vocab image answers
`/infer` on transformers 5. No SDK release: `opennvr-adapter-sdk` stays
at 1.3.0.

Upgrading: the platform's `.env.example` pins `ADAPTER_TAG=0.1.7`,
`YOLO_POSE_ADAPTER_TAG=0.1.7` and `OPEN_VOCAB_ADAPTER_TAG=0.1.7` from
OpenNVR's YOLO26n change (open-nvr#654). Nothing migrates.

### Added

- **`YOLOV8_WEIGHTS_FILE`** names the ONNX file the yolov8 adapter loads
  from its weights directory (default `yolov8n.onnx`). open-nvr's weights
  image moves to YOLO26n — same `(1, 84, 8400)` tensor and decode, 17 %
  faster on a 4-thread CPU and +3.6 COCO mAP — and names the file
  through this variable, so an older install's `yolov8n.onnx` still
  loads. The README recipe exports yolo26n.
- **`open-vocab-adapter` is published** (`adapters/vlm`, port 9012). The
  OWL-ViT v2 open-vocabulary detector has been in the tree since June
  with its tests, its Dockerfile and its contract conformance, and no
  row in the publish matrix, so nobody could pull it. It now builds
  for amd64 and arm64 with the rest: `ghcr.io/open-nvr/open-vocab-adapter`.
  One task, `open_vocab_detection`: a box and a score for any phrase
  the caller writes — "red truck", "forklift", "person wearing a hard
  hat" — where YOLOv8 knows only its eighty COCO classes. The ~600 MB
  weights are baked into the image and the runtime is pinned offline,
  as blip and clip are (#79): no volume, no network, `network_egress:
  []`, boots on a sealed box. CPU inference takes a few seconds a
  frame, so this is an on-demand adapter, not a per-frame one.

### Fixed

- **`open-vocab-adapter` answered every `/infer` with a 500** on the first
  published image: transformers 5.0 dropped
  `Owlv2Processor.post_process_object_detection`
  (`post_process_grounded_object_detection` since 4.5x). The stubbed
  tests never reached it; found by running the image on a photo. The
  README's multipart example also used loose form fields where the SDK
  reads a JSON `params` field (contract §3.5). (#61)

## [0.1.6] — 2026-10-11

Better eyes and new senses. The plate reader moves to the v2 model and
reports the plate's region; YOLOv8 draws one box per object; the voice
adapters stop taking the whole box; and three capabilities arrive: parcel
detection, human pose, and CLIP embeddings that let search match words to
frames. `opennvr-adapter-sdk` is published as 1.3.0 alongside this release
(its own changelog), with the `Adapter` front door and the `opennvr-adapter`
CLI.

Upgrading: the platform's `.env.example` pins `ADAPTER_TAG=0.1.6` from
OpenNVR 0.1.6. InsightFace face files migrate on load.

### Changed

- **Plate OCR defaults to `cct-s-v2-global-model`** (was the v1 xs
  build). The v2 generation is trained on ~3x the data, recognises the
  plate's region for 65+ countries, and both v2 sizes exceed 0.99
  region macro-F1 on a 114k-sample held-out split, for ~0.35 ms more
  per plate and 3 MB more on disk. A more accurate single look is also
  what shortens core's consensus sweep. The result gains an additive
  `region: {code, confidence}` (None on v1 models). Override with
  `OPENNVR_LPR_MODEL`; the v1 ids still work.
- **Whisper decodes greedily by default** (`beam_size` 1, was 5). Beam 5
  costs 30-50% more CPU and buys nothing on a short, VAD-trimmed
  utterance — what a voice assistant sends. A request may still ask for
  any beam up to 32. (open-nvr #583: 14 s of a 71 s voice turn.)
- **Piper builds a thread-capped ONNX session** — `OPENNVR_PIPER_THREADS`
  (default 2) sets intra-op threads; unset by piper, one utterance took
  every core (~390% of a 4-core box, 25 s for two sentences on the field
  box) and fought the detector and recorder. Uses the
  `PiperVoice(session, config)` constructor of piper-tts 1.2 — the version
  the image pins (1.3+ also changed `synthesize`; the pin is deliberate);
  a piper without it falls back to `PiperVoice.load` and logs once that
  the cap is not in effect.
- **A requested Piper voice that is not on disk falls back** to the default
  voice, else a complete voice in the same language, else any — said once
  per missing name — instead of muting the box; the result's `voice` names
  the voice that actually spoke, and the requested voice is picked up the
  moment its files land. `OPENNVR_PIPER_VOICE` is validated at start-up
  (a path is refused with the variable's name, not as "weights missing").
- **`OPENNVR_PIPER_VOICE`** names the voice used when a request names
  none; **`OPENNVR_WHISPER_CPU_THREADS`** caps CTranslate2's threads on
  CPU (ignored on CUDA).

### Fixed

- **YOLOv8 boxes are de-duplicated, letterboxed and mapped back
  correctly.** The adapter returned every raw anchor over threshold — the
  Ultralytics ONNX export emits 8400 of them — so one person came back as
  a cluster of overlapping boxes (reported by QA as boxes "not accurate").
  Now: per-class non-maximum suppression (`iou_threshold`, default 0.45,
  overridable per request like `confidence_threshold`; the sibling
  adapters' spellings `iou`, `nms_threshold` and `conf` are accepted
  too); the frame is
  letterboxed into the model's square instead of squashed 1.78x on a 16:9
  camera, which the model was never trained on; a box crossing the frame
  edge is cropped rather than slid inward; and the decode is vectorised
  (the per-anchor Python loop cost tens of ms per frame on CPU). The
  legacy `YOLOv8Adapter.infer_local` path shares the same decode.
- **YOLOv8 and YOLO-pose streaming no longer freeze the process.** Both
  §6 WebSocket loops ran decode + inference inline on the event loop, so
  every other stream and every HTTP request on the adapter stalled for the
  duration of each frame. Inference now runs on a worker thread, as the
  SDK's `/infer` route already did.
- **`WHISPER_MODEL_SIZE` is honoured.** It has been in the README, the
  Dockerfile and open-nvr's compose stack since the adapter shipped, and
  nothing read it: every deployment loaded `base` whatever it set.

### Added

- **`package-detection` adapter** — contract-v1 `package_detection` on
  port 9010: parcels on a doorstep, the one class COCO does not have.
  A YOLOv8n fine-tuned on two openly licensed doorstep sets (*package at
  front door*, 1,293 images, MIT; Roboflow's *Packages*, CC0) on
  `onnxruntime`, returning §5.1 `detections` with the label `package`,
  normalized boxes un-letterboxed into the source frame, plus `count`
  and a `labels` echo. Advertises ONLY `package_detection` — a
  one-class model must not satisfy an app whose `object_detection`
  means people and vehicles. On demand over HTTP `/infer`, no stream:
  its consumer (open-nvr's `package-delivery` app) asks once when
  Tier-0 says somebody left the doorstep and on a slow recheck cadence,
  and picks this adapter over VQA and over the COCO bag-class stand-in
  the moment it registers. Trained 40 epochs at 416 px on 1,242
  images; on the held-out test split precision 0.974, recall 0.804,
  mAP50 0.914, mAP50-95 0.631 — precise before it is complete, which
  is the right way round for a consumer that re-counts on a cadence
  and must not invent a delivery. CPU is the design target: 11.7 MB
  ONNX, 23–34 ms per frame on two x86 cores. Per-call `conf`, `iou`, `imgsz`,
  `max_detections`; multi-class fine-tunes work unchanged through
  `PACKAGE_DETECTION_LABELS`. Domain metrics
  `adapter_package_frames_total{result=packages|empty}` and
  `adapter_packages_total{label}`. Weights are not baked: mounted, or
  fetched once from `PACKAGE_DETECTION_MODEL_URL` (the release asset),
  with `network_egress` derived from that setting. The training and
  export recipe ships as `adapters/package_detection/train_package_model.py`,
  including the hook for fine-tuning on a site's own frames.

- **`yolo-pose` adapter** — contract-v1 `pose_estimation` on port 9009,
  wrapping a YOLO11n-pose ONNX model on `onnxruntime`. Returns a
  COCO-17 skeleton per person (`persons[].keypoints` — 17
  `[x, y, confidence]` triples in the fixed COCO order, plus
  `keypoint_names` and `frame_dimensions`), over HTTP `/infer` or the
  full §6 WebSocket streaming protocol, which is the path a 10 fps
  camera uses. Built for apps that reason about body geometry rather
  than object location: wand-compliance verification (is the guard's
  detector tracking the visitor's arms?), fall detection, queue
  orientation.
  CPU-first by design — the nano model at a 448 px default input, one
  serial session, ~47 ms/frame and ~160 MB RSS measured on an 8-core
  x86 CPU. Coordinates are **pixels in the source frame**, not §5.1's
  normalized boxes: normalizing a 16:9 frame to a unit square distorts
  exactly the limb angles these apps measure. Per-call `conf`, `iou`,
  `imgsz` and `max_persons`; per-joint visibility exported as a domain
  metric (`adapter_pose_keypoints_visible_total{keypoint=…}`), which
  is how a camera that has stopped seeing wrists shows up before the
  app goes quiet. Weights are not baked into the image: the ONNX is
  exported locally (`python download_models.py --all`) and mounted,
  or fetched once from an operator-configured `YOLO_POSE_MODEL_URL` —
  and `permissions.network_egress` is derived from that setting, so the
  default deployment declares no egress at all and a configured fetch
  declares exactly its one host.

- **CLIP embeds words as well as frames** — the `embed` task puts a
  camera frame and an operator's query in the same space, which is what
  lets OpenNVR's search fuse a similarity ranking with its keyword one: a
  search for "truck" finds the visit a captioner described as "a lorry at
  the loading bay". A request without a frame is the query half, so the
  body shape is TEXT and the service decodes `frame_b64` itself, enforcing
  its own size limit on the encoded length.

- **InsightFace keeps several samples per person.** A record holds a list
  of embeddings and a match is the best of them, so a porch-light,
  30-degree view can still match a straight-on enrolment. `POST
  /faces/register` gains `append` (add a sample; name, category and
  metadata change only when given); `DELETE /faces/{id}/samples/{i}`
  removes one bad capture. Up to 32 samples, oldest dropped first. The file
  schema moves to 2 and a schema-1 file loads as one sample, so nobody
  re-enrols.

- **`PATCH /faces/{id}`** edits a person's name, category or metadata
  without a new photo; absent fields are untouched, metadata merges, and a
  key sent as null is removed. The embedding never changes here.

## [0.1.5] — 2026-09-10

The LPR adapter goes from unusable to correct. Everything below is the
same plate-reading path, fixed in four steps — packaging, localization,
confidence and the box's frame of reference. `opennvr-adapter-sdk` is
unchanged at 1.2.0 (already on PyPI); it is versioned by its own API
contract, not by this repo's release number.

### Fixed

- **The shipped LPR image could never read a plate.** `fast-plate-ocr`
  was installed without its `[onnx]` extra, so the runtime the model
  needs was absent from the published image — every read failed on a
  machine that was otherwise correctly configured.
- **LPR confidence was ALWAYS 1.0**, which made every precision floor in
  the adapter inert. The output parser expected a `.confidence`
  attribute, but `fast-plate-ocr` 1.1.0 (the shipped version) reports
  `char_probs` and has no `.confidence` — so `getattr` defaulted every
  production read to certainty: garbage like `'1023'` shipped as
  accepted, and raising the floor changed nothing. Confidence is now the
  **minimum** per-character probability (a read is only as trustworthy as
  its shakiest character); measured on the real models, a clean crop
  scores 0.997, noise-scene garbage 0.071 and a tiny-crop hallucination
  0.384 — min separates them where mean would blur them together. An
  unrecognised confidence shape now parses as UNTRUSTED (0.0) rather than
  1.0, so a read whose confidence cannot be seen can never outrank the
  floor. The unit tests missed this for a release because the stub
  returned `(text, conf)` tuples that parse fine — classic
  stub-blindness; the regression test now uses the real 1.1.0 shape and
  is sabotage-verified against the old behaviour.

### Added

- **Plate LOCALIZATION before OCR** — the missing middle stage.
  `fast-plate-ocr` is a pure OCR model expecting a single plate crop, but
  the platform's enrichment path sends the whole-VEHICLE best-frame crop;
  pointed at an entire vehicle it reads whatever character-like texture it
  finds, so nearly everything landed under the floor (silent misses) and
  the occasional hallucination landed over it (false reads). The adapter
  now localizes the plate with `open-image-models` (same author as the OCR
  model, same ONNX/CPU footprint, ~7 MB weights cached the same way) and
  OCRs the crop plus a small context margin. Measured end to end on a
  synthetic scene: whole-image OCR read `'L330'` at ~15% per-character
  confidence; detect → crop → OCR read the exact plate at 99.9%.
  Additive and fail-soft — a plate that isn't found falls back to
  whole-image OCR at a raised floor (default 0.75), a detector that fails
  to load never takes the adapter down (OCR is the core, detection the
  enhancer), and `OPENNVR_LPR_DETECTOR=""` selects explicit pure-OCR mode
  for callers who assert they send crops. Results gain
  `plate_detection {attempted, found, confidence, box, model_id}`.
- **Tiny-plate guard.** A localized plate narrower than
  `OPENNVR_LPR_MIN_PLATE_PX` (default 40) returns a clean non-read
  (`accepted=false`, `plate_detection.too_small`) instead of OCRing pixel
  soup — a distant car should produce nothing, not a wrong plate that
  fires a false unknown-vehicle alarm downstream.
- **`plate_detection.image_size`** — the box's frame of reference travels
  with the box. Consumers rejecting clipped (partial) reads judge the
  plate box against the image it was measured in (open-nvr#378), and
  multi-frame OCR sends candidate crops whose size differs from the
  visit's evidence frame, so a consumer guessing the denominator measures
  in the wrong image. The adapter knows exactly what it OCR'd — the
  `[width, height]` of the caller's bytes — so it now says so, and no
  consumer has to parse a JPEG header or guess.

### Changed

- **LPR precision floors are env-tunable**, so operators dial precision
  against recall without a rebuild: `OPENNVR_LPR_MIN_CONFIDENCE`
  (default raised 0.30 → 0.45 under min-character scoring) and
  `OPENNVR_LPR_MIN_PLATE_PX` (default 40).

## [0.1.4] — 2026-08-24

### Added

- **`ollamavlm` adapter** — contract-v1 visual_qa / scene_caption served
  by proxying to an Ollama endpoint (``OPENNVR_OLLAMA_VLM_URL``, default
  ``http://host.docker.internal:11434``; model
  ``OPENNVR_OLLAMA_VLM_MODEL``, default ``moondream``). Drop-in for the
  camera-agent's ``CAPTION_ADAPTER`` slot. Motivation: on macOS/Windows
  the Docker VM has no GPU access, so in-container VQA is CPU-only —
  pointing this adapter at a host-side Ollama gets Metal/GPU inference
  while every call stays inside the audited Adapter Contract, and model
  management collapses to ``ollama pull``. Lazy-ready by design (an
  unreachable endpoint is a transient per-infer error, not a boot
  failure), auto-pulls a missing model via Ollama's own API, ~80 MB
  image with no weights and no ML deps.
- **Model identity + per-task + domain metrics on `/metrics`** (SDK
  1.2.0, wired into every published adapter).
  `adapter_model_info{adapter, adapter_version, model, model_version,
  framework, fingerprint} 1` exports the served model's identity, so
  §11.3 fingerprint drift is visible from one scrape and a latency
  regression can be correlated with a weights change.
  `adapter_infer_total` / `adapter_infer_latency_seconds` gain a
  closed-set `task` label, and adapters register model-specific domain
  metrics (detection counts, plate-read confidence, …) through
  `load()`-time registration.

### Removed

- **The camera-agent-lite adapters** (`llamacpp`, `smolvlm`,
  `whispercpp`, `pipertts`) — companion to open-nvr's camera-agent-lite
  removal. llamacpp/smolvlm were amd64-only (blocked upstream on
  ggml-org/llama.cpp#19177 — the ARM audience lite targeted could never
  run them natively), and whispercpp/pipertts duplicated the `whisper`
  and `piper` adapters. STT/TTS stay covered by whisper/piper; the
  llama.cpp runtime stays covered through Ollama (`ollamavlm` + the
  agent's host-Ollama path). Every remaining published adapter image is
  multi-arch.

## [0.1.3] — 2026-08-17

ARM64 release. Adapter images are now `linux/amd64` + `linux/arm64`
manifest lists, fixing the Apple Silicon / Raspberry Pi 5 install
failure where a fresh OpenNVR `docker compose pull` aborted with
``no matching manifest for linux/arm64/v8`` on the first adapter image
it reached. (No 0.1.2 was ever tagged; this release follows 0.1.1.)

### Added

- **linux/arm64 images for 11 of 13 adapters** — `yolov8`, `piper`,
  `pipertts`, `whisper`, `whispercpp`, `fast-plate-ocr`, `insightface`,
  `blip`, `bytetrack`, `moondream`, and `voice` now publish multi-arch
  manifest lists. Every pinned dependency was verified to resolve to a
  pre-built ``manylinux*_aarch64`` wheel (no source builds under QEMU).
  `llamacpp` and `smolvlm` remain amd64-only: they derive from
  ``ghcr.io/ggml-org/llama.cpp:server``, which publishes no arm64
  manifest upstream (ggml-org/llama.cpp#19177).
- ``platforms`` is now a per-adapter field on the publish matrix, so
  arch support is declared next to the adapter it belongs to.
- **SDK release-alignment guard** — a ``v*`` release run now fails if
  the SDK version pinned in the tree is not on PyPI (the gap v0.1.1
  shipped with: tree said 1.1.0, PyPI had only 1.0.0). Version numbers
  stay decoupled (SDK tracks the contract major); the guard aligns the
  *release*, not the number. See ``opennvr_adapter_sdk/RELEASING.md``.

### Fixed

- **blip / voice-bundle arm64 builds** — both pinned
  ``torch==2.9.1+cpu`` from ``download.pytorch.org/whl/cpu``, an index
  with no aarch64 ``+cpu`` wheel (pytorch/pytorch#136275). The
  Dockerfiles now branch on ``TARGETARCH``: arm64 installs PyPI's plain
  ``torch==2.9.1`` aarch64 wheel, which is already CPU-only (all CUDA
  deps carry ``platform_machine == "x86_64"`` markers). The amd64 path
  is unchanged.

### Documentation

- README: supported-architecture matrix and the maintainer rule that a
  dependency bump must keep aarch64 wheel coverage, since a regression
  only surfaces as an arm64 build timing out under QEMU.

## [0.1.1] — 2026-07-31

Aligned with the v0.1.1 cut of the OpenNVR NVR — this release supplies
the adapter set behind the new `camera-agent-lite` example.

### Added

- **Four new adapters for camera-agent-lite** — `llamacpp` (LLM via
  llama.cpp), `whispercpp` (speech-to-text), `pipertts` (text-to-speech),
  and `smolvlm` (vision). All four download their models on first boot
  via the SDK 1.1.0 `ensure_model_file` helper (a pre-populated weights
  volume always wins, so offline installs never trigger egress) and are
  in the GHCR publish matrix.
- **AI Adapter Contract v1.1 (optional detector spec)** — adapters can
  now declare their accelerator and expected input.
- SDK 1.1.0 — see
  [`opennvr_adapter_sdk/CHANGELOG.md`](opennvr_adapter_sdk/CHANGELOG.md).

### Changed

- CI publishes adapter images from every branch push
  (`:<branch>` + `:sha-<short>` tags), not just release tags.

### Fixed

- SDK `__version__` now matches `pyproject.toml` (1.1.0), unblocking the
  release version check.

### Documentation

- README opening section rewritten for search discoverability.

## [0.1.0] — 2026-07-14

First public release of the reference adapter server. Aligned with the
v0.1.0 cut of the [OpenNVR](https://github.com/open-nvr/open-nvr) NVR. The
`opennvr-adapter-sdk` is structured for PyPI distribution; the upload wires
off the first `sdk-v*` tag, so until then install from source.

### Added

#### SDK & contract

- **`opennvr-adapter-sdk`.** Apache-2.0 SDK that adapter authors install to
  write a new contract-compliant detector in ~30 lines. Public API:
  `AdapterService` ABC, `AdapterApp` FastAPI builder, `ServiceError`
  envelope, `BodyShape` enum, and re-exports of every Pydantic wire type
  from the contract. PyPI publish wires off the first `sdk-v*` tag; until
  then, install from source via `pip install -e ./opennvr_adapter_sdk`.
- **AI Adapter Contract v1 compliance.** All six mandatory endpoints
  (`/health`, `/capabilities`, `/hardware/evaluation`, `/metrics`, `/infer`,
  `/infer/stream`), the five result conventions, the WebSocket streaming
  protocol, the typed failure envelope, and live model fingerprint drift
  detection are exercised by the reference adapters.
- **Conformance test suite** in `conformance/` for verifying any
  contract-compliant adapter, in-tree or third-party.

#### Reference adapters

The reference server ships with the following adapters auto-discovered at
startup:

- **YOLOv8** — ONNX object detection, CPU-friendly, no torch dependency.
- **YOLOv11** — PyTorch person counting with ByteTrack.
- **InsightFace** — face detection, recognition, and embedding extraction
  (Buffalo-L ArcFace). SDK-based; accepts multipart / base64 frames.
  Exposes face-DB CRUD (`/faces/register`, `/faces`, `/faces/{id}`) so
  external apps like the Smart Doorbell example can enroll known faces
  via REST without a shared volume. Embeddings live in a JSON-file
  face DB at `OPENNVR_INSIGHTFACE_FACE_DB`; raw face images are never
  persisted.
- **BLIP** — scene captioning via HuggingFace
  ``Salesforce/blip-image-captioning-base`` (default; ``-large`` available
  by env override). SDK-based contract-compliant adapter — accepts JPEG /
  PNG / WebP / BMP request bodies and returns ``result.caption``. The
  camera-agent example's ``describe_camera`` tool calls this adapter via
  KAI-C. CPU-friendly (~2-4s/image on a modern 4-core); GPU-accelerated
  when ``torch.cuda.is_available()``. Operators with strict sovereignty
  pre-bake the model into the image or mount a populated HF cache.
- **HuggingFace** — cloud inference proxy.
- **Whisper** — speech-to-text via faster-whisper (CPU or GPU).
- **Piper** — text-to-speech via ONNX voices. Default response shape
  returns ``audio_uri`` (the internal ``opennvr://audio/...`` scheme,
  suitable for shared-volume deployments). Callers without a shared
  filesystem — e.g. the camera-agent example's Pipecat TTS service —
  send ``inline: true`` in the request body to also receive
  ``audio_b64`` inline alongside the URI.
- **fast-plate-ocr** — license-plate recognition (Apache-2.0 upstream,
  ONNX, CPU-only, plate-specific). New `lpr` install extra; single-
  purpose OCR adapter designed to be chained downstream of YOLOv8 by
  the `license-plate-recognition` example app on OpenNVR. The adapter
  decodes request body bytes (JPEG / PNG / WebP / BMP via OpenCV)
  into a numpy array before invoking
  `LicensePlateRecognizer.run()`, since the upstream library's 1.x
  API accepts paths or ndarrays but not raw bytes. Garbage bytes
  surface as a typed `TRANSPORT_ERROR(invalid_image)` envelope with
  HTTP 400.
- **Ollama** — local LLM access over HTTP. Supports the OpenAI-style
  `tools` / `tool_choice` request fields and normalises Ollama's
  native tool-call response into the OpenAI-shaped `message.tool_calls`
  array (with synthesised call ids and JSON-stringified arguments)
  so Pipecat / OpenAI clients work without translation. Tool support
  requires a tool-capable model (Llama 3.1+, Qwen 2.5+, Mistral Nemo,
  others — see https://ollama.com/blog/tool-support).
- **ByteTrack** — multi-object tracking post-processor. Takes a frame's
  detections in (JSON, `BodyShape.TEXT`), returns the same detections
  with persistent `track_id` populated. Stateful per camera (TTL-evicted
  to keep memory bounded), tunable per-call via the `tracker_config`
  block. Composes with any detection-shaped upstream adapter — chain
  through KAI-C the same way the license-plate-recognition example
  chains YOLOv8 → fast-plate-ocr. Wraps
  [`supervision`](https://github.com/roboflow/supervision)'s
  ByteTrack (pinned `>=0.21,<0.30`; 0.30 removes ByteTrack from the
  package). The contract's `DetectionItem.track_id` field gains a
  populated home with this adapter — alert deduplication, dwell-time
  analytics, and track-stable state machines all become available
  to downstream examples.

#### Architecture

- **Auto-discovery + lazy loading.** Place an adapter file under
  `app/adapters/`, restart, the system finds it automatically. Models load
  into memory only when their first request arrives. Idle adapters consume
  zero memory.
- **Anti-bloat dependencies.** A minimal core install is roughly nine packages
  (~50 MB). Adapter ML libraries live in `[project.optional-dependencies]`
  groups so users only install what their deployment uses. The full install
  (`uv sync --extra all --extra cpu`) is ~4 GB; most deployments need
  `--extra yolo --extra face` (~750 MB).
- **Graceful adapter skipping.** If an adapter's optional dependencies are
  missing, `PluginManager` skips it cleanly with a helpful install hint
  rather than crashing the server.
- **Pydantic-validated response envelopes** catch invalid data at the
  boundary — confidence out of range, missing fields, count mismatches.
- **Async pipeline engine.** Chain multiple tasks sequentially via
  `POST /pipeline/run`. Output of step N feeds into step N+1.
- **Process isolation by default.** Each adapter can run as its own container
  with its own resource limits. A crash in one adapter doesn't take down the
  alerts pipeline.

#### Operations

- **Multi-stage Docker build** with CPU and GPU variants
  (`docker build --build-arg USE_GPU=true`).
- Built-in `HEALTHCHECK` against `/health` every 30 seconds.
- **Opt-in `X-API-Key` authentication** via environment variables.

### Security

- **Sovereignty enforcement at the middleware boundary.** KAI-C (in the
  `open-nvr` repo) refuses to register adapters that declare
  `network_egress` permissions under the default `local_only` policy.
- **Operator-gated permissions.** GPU, host filesystem, network egress, and
  shared-memory access are all declared in `/capabilities` and surfaced for
  approval rather than implicit.
- **Live fingerprint drift detection** built into the SDK. The middleware
  polls `/capabilities` every 60 seconds and emits an audit event when the
  declared fingerprint changes.
- **Typed error envelope.** Every failure path raises `ServiceError`, which
  the SDK translates to a wire-compatible response with category, code,
  transient flag, and `retry_after_ms`.

### Changed

- Migrated FastAPI lifecycle from the deprecated `@app.on_event("startup")`
  decorator to the lifespan async context manager pattern.

### License

The reference server is licensed under [GNU AGPL v3.0](LICENSE). The SDK
(`opennvr_adapter_sdk/`) is licensed under **Apache-2.0** so adapter authors
can publish under any compatible license, including proprietary. Model
weights are not AGPL-bound — ship them under whatever licence the model
permits.

---

[Unreleased]: https://github.com/open-nvr/ai-adapter/compare/v0.1.4...HEAD
[0.1.4]: https://github.com/open-nvr/ai-adapter/compare/v0.1.3...v0.1.4
[0.1.3]: https://github.com/open-nvr/ai-adapter/compare/v0.1.1...v0.1.3
[0.1.1]: https://github.com/open-nvr/ai-adapter/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/open-nvr/ai-adapter/releases/tag/v0.1.0