capcodec/DECISIONS.md

Rama main · réplica de lectura del repositorio original.

# Decisions

Choices made where the brief was ambiguous, newest last. Each entry: the decision, then why.

## Project

- **Repository name `capcodec`.** Nothing more specific was obvious. The encoder library, CLI and muxers are Tov; the benchmark harness and corpus tooling are Python 3 (they are tooling, not the product, and the brief's "no JavaScript/npm workarounds" rule is about gaps in Tov).
- **The Tov feature branch is delivered as a git bundle and patch series.** The token on this machine has no push access to `richiemcilroy/tov`, so `encoder-features` lives in the local clone and is exported to the artifacts directory.
- **Speed is the primary goal** (follow-up brief): the pitch is the fastest H.264 encoder for screen recordings at quality equal to x264 veryfast. Every design choice below that trades a little compression for a lot of speed is taken in that direction, and the headline metric is encode fps vs x264 veryfast at matched VMAF.
- **The OpenAI `math` repository was reviewed and is not used.** Its relevant results (an O(n (log n)^(1-1e-13)) exact DFT, `o(n log n)` Fourier circuits for unbounded lengths, matrix-multiplication exponent bounds) are asymptotic. H.264 fixes the transforms at 4x4 and 8x8 integer butterflies and SATD uses a 4x4/8x8 Hadamard, which already cost only additions and shifts; none of the results lowers the operation count at those sizes. Re-reviewed on 2026-10-07 (all 722 preprint titles, abstracts of the Fourier, tensor and matrix-multiplication papers): same conclusion. Speed comes from vector throughput, memory traffic, fewer evaluated candidates and threads, and from video-coding practice (global motion for screen content, weighted prediction, byte-oriented CABAC, decimation and fast skip), which is what the encoder applies.

## Benchmark harness

- **Colour conversion.** Lossless RGB captures are converted once to 8-bit 4:2:0 with BT.709 matrix, limited range, Lanczos scaling, and the VUI of every encoder is tagged BT.709/limited. This is what HD screen recordings and Cap's pipelines use.
- **GOP for reference encoders is 2 seconds** (`-g 2*fps`, `keyint_min 1 s`) so that keyframe cost matches Cap's 2 second chunking and HLS segments. Production Cap baselines keep their own production GOP.
- **Rate ladders.** x264: CRF 18/23/28/33. capcodec: QP 20/25/30/35 until rate control exists. Bitrate-driven encoders (OpenH264, browser encoders, hardware) use a ladder relative to an anchor, x264 veryfast's bitrate at CRF 23 on the same source, times 0.35/0.6/1/1.7/3. This keeps every curve overlapping the reference on clips whose bitrates differ by 100x (idle vs full motion).
- **BD-rate** uses piecewise cubic (PCHIP) interpolation of log-rate against quality over the overlapping quality range (the method of current codec evaluations), averaged over clips. PSNR is weighted YUV (6Y+U+V)/8, each component computed from the mean MSE over all frames, capped at 100 dB.
- **Matched-quality speed.** The target quality per clip is x264 veryfast at CRF 23. Each capcodec preset is re-encoded at the QP interpolated to the same VMAF, rounded towards higher quality, and both encoders are timed in serial runs using all cores. Single-thread CPU speedups are reported beside it.
- **Quality runs are single-threaded and parallel across cores; speed runs are serial and use each encoder's own threading.** CPU time and peak RSS come from `wait4` rusage. Production Cap reproductions keep their production threading.
- **VFR outputs are compared at the source frame rate**: decoded frames are mapped to source frames by presentation time (or by the encoder's frame map for elementary streams), repeating the last decoded frame, which is how a player shows a variable-frame-rate stream.
- **Regression gate**: smoke fails if capcodec-medium's BD-rate (PSNR-YUV vs x264 veryfast) is more than 1.0 percentage point worse than the committed `bench/best.json`, if its median single-thread fps drops more than 10%, or if any decode check fails.
- **Stage timing is batched.** Stages run as separate passes over each macroblock row (analysis, motion search, mode decision, transform/quant/reconstruction, entropy coding, deblocking), so the encoder reads the clock a few times per row instead of per macroblock. This costs well under 0.1% and makes the per-stage breakdown exact rather than sampled.
- **Change-proportional cost** is measured in single-threaded speed runs: static frame time (frames where nothing changed) against an explicit full-analysis run of the same clip (every macroblock forced through analysis, no frame dropping), plus the pixel-difference fallback without side info.
- **Colour tags go on the raw input, not the output.** Tagging only the output (`-colorspace bt709` after `-i`) makes ffmpeg insert a BT.601-to-BT.709 conversion in front of the encoder, so every ffmpeg-based encoder compressed a colour-shifted picture and was scored against the original (x264 lost about 5 dB PSNR-Y). Found by checking harness PSNR against a direct numpy computation; all ffmpeg adapters now tag the rawvideo input, and their cache identity carries a version marker so old encodes are discarded.
- **Production capcodec presets are the `--unchecked` build**, as Rust release builds are; `capcodec-medium-checked` (bounds and overflow checks on) runs beside them in every tier so that any memory-safety bug traps in the harness instead of corrupting output.
- **The OpenAI `math` repository** — see above; nothing applicable.

- **Texture AQ off by default (2026-10-07).** The text-aware AQ lowered QP by 3 on text macroblocks and raised it by 2 on textured ones. A harness sweep (standard tier, five clips, `/tmp/std8`) showed the texture raise costs VMAF on video (full_motion BD +39.0% with it, +35.3% without) and slides (−6.9% vs −8.3%) and changes nothing on the screen clips, while the OCR tier showed the text reduction lowers character error on scrolling text by about a third at equal bitrate. Default: text −3 kept, texture raise 0 (`--aq-texture N` restores it). Refresh and lifetime AQ stay on: dropping refresh costs slides +9 points, dropping lifetime AQ costs busy_ui/code_editor +2.5 points.

- **Benchmark builds target x86-64-v3 (AVX2).** `bench/build_encoder.sh` builds `build/capcodec` and `build/capcodec-unchecked` with `--target-cpu x86-64-v3` and keeps `build/capcodec-unchecked-baseline` for SSE2-only CPUs. x264 dispatches to AVX2/AVX-512 at run time on this machine, so comparing against a baseline build would understate capcodec; the AVX2 build is 10-18% faster (full_motion 23.6 -> 27.8 fps, text_scroll 154.7 -> 170.5 fps, identical output). On Arm (Apple silicon) the vectors compile to NEON either way. A shipping CLI would pick between the two builds at start-up.
- **Benchmark builds carry no unwind tables or symbols (Linux, without `-g`).** `bench/build_encoder.sh` adds `-fno-asynchronous-unwind-tables -fno-unwind-tables -Wl,-s` through `TOV_CFLAGS`, and `-fuse-ld=lld` when lld is installed: nothing unwinds a Tov program's stack and the instructions are unchanged, while the files shrink by 37.8 KB unchecked and 45.4 KB checked (460,240 and 608,256 bytes). A `-g` build keeps both for debuggers and profilers.

- **Lookahead QP from macroblock lifetime, buffered in the CLI (2026-10-07).** `--lookahead N` (default 10 on fast, medium and slow; 0 on live, which must not add delay) reads N frames ahead and sets each macroblock's QP from how long its content stays, x264's macroblock tree adapted to screen content. The ring buffer and analysis live in the CLI (`src/lookahead.tov`, `enc.setLookahead()` before each `encodeFrame`) so the encoder's frame flow stays as it is for B-frame scheduling; with more than one thread the analysis runs on the read-ahead thread. Only changed macroblocks are analysed (side-info dirty rects and scroll rects, otherwise the pixel-difference test). Intra cost is the SAD against the block mean; inter cost is the best full-pel SAD over zero, the scroll hint, neighbour and previous-frame motion and a two-step diamond, on even rows. The kept fraction 1 - inter/intra carries each macroblock's cost back along its motion over the window (x264's propagate step), and content still unchanged at the window end is assumed to last to the horizon, weighted by kept^12 where it changed. QP offset -3 + 1.75 log2(span / lifetime), at most +8, span = min(keyint, 2 s). IDR frames skip lifetime AQ (it counts the same persistence twice); P frames keep it (full_motion 1.9 points better) and text AQ stays. Measured alternatives (standard tier, VMAF BD-rate, five clips): N=30 gains 0.1-0.25 points over N=10 and N=60 no more, for 3-6x the delay and memory; strength 1.5 or 2, base -2.5 and a +10 cap are worse; applying it to IDR frames only loses 2-8% on full_motion; ending unchanged content at the window edge halves the gain; text AQ off under the lookahead leaves -1.9% instead of about -4%, and a floor on text macroblocks' offsets is also worse; a wider search (steps 8 to 1) and a half-resolution model with DC/H/V intra costs leave full_motion at -0.7% and +0.6%. Without the 2 s span cap a 250-frame GOP lost 8% on full_motion; with it the loss is about 2%.
- **Benchmark builds target x86-64-v3 (AVX2).** `bench/build_encoder.sh` builds `build/capcodec` and `build/capcodec-unchecked` with `--target-cpu x86-64-v3` and keeps `build/capcodec-unchecked-baseline` for SSE2-only CPUs. x264 dispatches to AVX2/AVX-512 at run time on this machine, so comparing against a baseline build would understate capcodec; the AVX2 build is 10-18% faster (full_motion 23.6 -> 27.8 fps, text_scroll 154.7 -> 170.5 fps, identical output). On Arm (Apple silicon) the vectors compile to NEON either way. A shipping CLI would pick between the two builds at start-up. On x86-64 it also builds `build/capcodec-unchecked-v4` (`--target-cpu x86-64-v4`, AVX-512, identical output). The kernels stay 256-bit in both: Tov has no compile-time query of the target's vector width, and a 32-lane 16x16 SATD that is 10-20% faster under AVX-512 costs 3.4% more instructions and runs slower under AVX2 (register spills), while 64-byte SAD is no faster than the two-row 32-byte form on AVX-512.

## B-frames

- **One reference per list; adaptive `--bframes 3` by default (fast, medium and slow since `9bce84f`, live since `c6225e5`; 0 with `--zerolatency`).** Each B-frame is coded after its future anchor with L0 = previous anchor and L1 = next anchor (with the pyramid, the middle B-frame replaces one of them for the outer B-frames, see below). Macroblock types are B_Skip, B_Direct_16x16 (spatial direct with `direct_8x8_inference`, colZero from the anchor's vectors), and 16x16 L0, L1 and Bi; no B sub-partitions. POC type 0 is only used when B-frames are on, so `--bframes 0` streams stay byte-identical to the P-only encoder. Live first stayed at 0 because B-frames add up to `bframes` frames of latency and reordering; it uses them since `c6225e5` (see "Adaptive B-frames on every preset" below). The screen clips gain nothing from them (adaptive B usage keeps them within ±0.5% of P-only). See PROGRESS.md for the recommended settings.
- **B-pyramid (`--b-pyramid`, default on fast, medium and slow, off on live).** In runs of 2 or 3 the middle B-frame (index (n-1)/2) is coded first as a reference (`nal_ref_idc` 2) at the anchor's macroblock QP + 5 (`--b-ref-qp-offset`; 4 and 6 were 0.1-0.3 points worse), and the outer B-frames predict from it: the earlier ones with L1 = middle, the later ones with L0 = middle. The next anchor reorders L0 back to the previous anchor and frees the middle B-frame with MMCO 1, so the P chain and its references are unchanged; the decoded picture buffer grows by one frame, `max_num_reorder_frames` is 2 and fragmented MP4/HLS delay composition by two frame durations. full_motion VMAF/PSNR BD-rate against no pyramid (120 frames, CRF 22-34, with the B search gates below): medium −2.26/−0.66%, fast −2.31/−0.59%, slow −1.15/−0.18%, live b3 −2.69/−1.02%, for about 1.5% more instructions. The outer B-frames wait for the middle one's padded picture and half-pel tiles, so on 4 threads part of the run is serial (CPU +11%, wall −5% at load ~20 before the gates). Live keeps it off because it doubles the decoder's reorder depth. The lookahead does not credit the middle B-frame; it inherits the anchor's lookahead offset through the anchor's macroblock QP.
- **B-frames are not deblocked.** B slices signal `disable_deblocking_filter_idc` 1, including the pyramid's reference B-frame and with `--recon`. At the B QP the filter cost more bits than it gave back in VMAF: full_motion BD-rate −0.26/−0.16% (medium), −0.27/−0.13% (fast), −0.37/−0.26% (slow), −0.25/−0.07% (live b3) without it, and ffmpeg decodes full_motion 12% faster.
- **The B-frames of a run are coded concurrently and alongside the next anchor.** They read only the anchors (and the middle B-frame), so each gets its own reconstruction and per-macroblock state (the source is the padded pending buffer); workers claim rows from any frame of the run (a wavefront per frame) and whoever finishes a frame entropy-codes it, so the output does not depend on scheduling. With more than one thread and outside VBR the run is coded asynchronously while the next anchor is coded, and joined after that anchor's frame or at flush, which is how x264 scales B-frames: full_motion 4 threads wall medium b3 7.34 → 5.59 s at load ~12 (CPU unchanged). VBR joins the run before the next anchor because its rate control spends each B-frame's bits before choosing the anchor's QP. The price is memory: at 1080p each concurrent frame holds about 8.7 MB of macroblock state (6.8 MB of it coefficients) and a 3.5 MB reconstruction, and an asynchronous run keeps its three sources while the next three arrive, so 4 threads peak at 151.8 MiB on medium and 114.5 MiB on live b3 where main's sequential B-frames peaked at 95.5 and 65.3 MiB. With one worker the frames are coded one after another, share one slot and reconstruct into the idle frame (112.6 and 72.0 MiB against 91.1 and 60.7; the rest is the pyramid's reference frame). Entropy-coding B rows as they finish would let the coefficients live in a few rows instead of a frame.
- **Concurrent B runs on thread-efficiency's wavefront (merge of main `7189fa4`, 2026-10-08).** B rows run through the same wavefront row function as P rows but skip its row prep and per-row QP assignment: a run's macroblock QPs are set from the anchor's when it starts, and its sources were padded when they were queued. Each B-frame is entropy-coded whole by the worker that finishes its last row, so a run never takes the main thread, and the anchor coded while a run is in flight keeps the pipelined entropy coding. B workers do not claim half-pel tiles: the run computes both anchors' tiles in one parallel pass before its workers start, so the next anchor's on-demand claims on the same reference find every tile done; the pyramid's outer B-frames wait for the middle one with the adaptive spin-wait. The anchor scan peek runs when the B queue fills, which with asynchronous runs is while the previous run is in flight (13 of 14 peeks on full_motion without side info, 2 and 4 threads); it reads only the previous anchor's source and reconstruction, which the run also only reads, and writes its own buffers. Output is byte-identical to the branch before the merge with 1-4 threads, and ThreadSanitizer finds nothing in 43 scenarios. A run with one worker (one thread, or frames under 8 macroblock rows) is coded on the calling thread: spawning its only worker and joining it at once cost two thread handoffs per run, which at load 20-30 left the 1-thread full_motion encodes waiting on the scheduler (medium wall 29.7 → 11.1 s best of 5, 488 → 53 voluntary context switches; main 33.0 s and 18). Memory is unchanged by the merge; at 1080p (90 frames, CRF 24) live peaks at 80, 117 and 126 MiB on 1, 2 and 4 threads (main 67, 72, 73) and medium at 115, 150 and 159 MiB (main 92, 96, 97), `--zerolatency` at 42 and 47 MiB either way. Of live's +53 MiB on four threads, three B slots of macroblock state are about 26 MB (20 MB of it coefficients), three reconstructions about 10 MB and the asynchronous run's three sources, held while the next three arrive, about 9 MB; two threads already code the run's frames concurrently and pay almost as much.
- **Concurrent B runs with lookahead round 4 and the all-intra pool (merge of main `08b3b02`, 2026-10-09).** B candidates are buffered before analysis and never pass through the anchor encoder's `analyse`, so the encoder's source copy is current for the lookahead's change map only on an anchor coded right after an anchor (`heldAt === frameIndex - 1`); anchors after a B run compute their own codes, and the anchor scan peek's codes enter through `takeCodes`. Main's sequential B loop rebinds the encoder's planes to each B-frame with `bindSource`; a concurrent run instead slices each B-frame's source from the padded pending buffer, with the same layout, into its own `SharedState`, because the next anchor is coded on the encoder's planes while the run is in flight. B workers keep deblocking off and take the anchor's `filterRecon`, which is true whenever a B-frame can occur (keyint above 1). With `--keyint 1` and more than one thread main's frame-parallel intra pool codes every frame and no B path is reached; `flush()` drains the pool before the B queue. The 4x4 dead-zone tests (`lumaZero`, `chromaZero`) combine the branch's SAD gate (`6e2ef20`) with main's paired transform `fwd4x2`: a pair is transformed only when either block's SAD could reach outside the dead zone, and chroma tests both blocks of a pair in 8-lane vectors. All three variants are exact tests with the same output; callgrind (x86-64-v3, full_motion 31 frames CRF 24, one thread), against the SAD gate with single transforms and against main's ungated pairs: medium 4.090G / 4.137G -> 4.055G, live 3.652G / 3.701G -> 3.617G, `--bframes 0` 5.550G / 5.526G -> 5.513G instructions. In the benchmark lane full_motion's 4-thread wall clock against x264 veryfast at matched VMAF is 1.15x on medium and 1.32x on live (main 0.92x and 0.96x; PROGRESS.md).
- **Concurrent B runs on footprint's coefficient chunks and padding (merge of main `1859bac`, 2026-10-09).** Each B slot packs its macroblocks' coded 32-coefficient chunks into its own store, sized like the anchor's (the frame's worst case plus one partial block per thread and two spare), and a run resets the slot's top when it starts. B workers are fresh encoders per run, thread and frame, so their block cursors start empty; with one worker the run's frames share slot 0 and reset it after each frame's entropy coding. B macroblocks are nearly all cbp 0, so a slot touches little of its store, which is where most of the B-frames' memory went (6.8 MB of coefficients per concurrent 1080p frame). B targets and the pyramid's reference frame take the 32-pixel padding with tile guard rows, and the middle B-frame's half-pel tiles cover every tile row from the guard origin as `precomputeTiles` does. A run is joined after the next anchor's frame and before that anchor writes its co-located vectors, which are now written only while B-frames wait; an anchor coded while a run is in flight trims references only into the spare list and swaps half-plane buffers by reference, so nothing reconstructs into a frame the run reads before the join. Peak RSS (90 frames, CRF 26; main `1859bac` / `812670f` / merge, MiB): full_motion 1080p live 58 / 73 / 59, 63 / 108 / 85 and 64 / 112 / 89 on 1, 2 and 4 threads, medium 83 / 109 / 94, 87 / 141 / 116 and 89 / 144 / 120, zerolatency on 4 threads 39 / 47 / 39; busy_ui live 30 / 47 / 30 and 34 / 51 / 34 on 1 and 4 threads, medium 64 / 75 / 64 on 4; full_motion 1440p live 99 / 126 / 101 and 106 / 187 / 146, 4K live 214 / 274 / 218 and 229 / 416 / 315, 4K medium on 4 threads 327 / 543 / 439. The B-frames' extra over main at 1080p on 2 and 4 threads fell from 45-56 to 23-31 MiB. What remains of live's +25 MiB on four threads is mostly the run's three reconstructions (about 10 MB) and its three sources, held while the next three arrive (about 9 MB); medium's +11 MiB on one thread is the pyramid's reference frame with its half-pel planes.
- **No B-frame declarations for GOPs shorter than three frames.** A B run needs a P anchor after it inside its GOP, so `--keyint 1` and `2` never code one; `configureB` leaves B-frames off there, and the CLI configures them before the muxer, the lookahead and the pipelined path read the result. Before, the SPS of medium at keyint 1 declared POC type 0 with a 16-bit LSB, three reference and DPB frames, two reorder frames and `weighted_bipred_idc` 2 (8 bytes more per SPS, which an all-intra stream repeats on every frame; the slices were identical), and MP4, fMP4 and HLS carried a reorder delay. keyint 1 and 2 are now byte-identical to `--bframes 0`. keyint 3 and 4 still declare three B-frames and, on the pyramid presets, its reorder depth, though their runs are at most one and two frames long.
- **Concurrent B runs with main's read-ahead (merge of main `e20b7fe`, 2026-10-09).** Once the CLI waits for frame n, the read-ahead thread may reuse the ring slot of every earlier frame while an asynchronous run of B-frames before n is still being coded. The run reads only encoder-owned memory: its sources were copied out of the ring by `padInput` when they were submitted, and otherwise it reads the anchors' reconstructions and its own slots. The anchor-scan peek reads frame n+1's slot synchronously while frame n is submitted, when that slot cannot be reused. B workers receive the anchor's lifetime map, a CLI array that the reader rewrites when it takes frame n+2+extra, but never read it: only row prep (`assignMbQp`) and the all-intra pool do, and B rows skip row prep. ThreadSanitizer finds nothing in the 43 scenarios and in 14 more with the read-ahead running (0, 3 and 5 frames, 2-4 threads, 23-60 frames, side info, VBR, keyint 7, stdin, MP4 with `--recon`). The extra ring slots follow `enc.bframes`, so keyint 1 and 2 run as main's `--bframes 0`. Main's CABAC skip-run writer already takes B_Skip with its own contexts, and the merged `endSlice` keeps `nal_ref_idc` 2 for the pyramid's middle B-frame, which the auto-merged line would have marked non-reference. B configurations are byte-identical to `70ebc48` and to themselves with `--read-ahead 0`, 3 and 5 on 2 and 4 threads; P-only configurations are identical to main `e20b7fe`. In the benchmark lane, full_motion's 4-thread wall clock against x264 veryfast at matched VMAF rose from 1.16-1.25x to 1.35-1.40x on medium and from 1.38-1.39x to 1.44-1.50x on live (main 1.09-1.10x and 1.19-1.21x in the same sessions). The read-ahead's three extra raw frames add 9-10 MiB at 1080p and 38 MiB at 4K on 2 and 4 threads, as on main (90 frames, CRF 26; main `e20b7fe` / merge, MiB): full_motion 1080p live 58 / 59, 72 / 95 and 73 / 98 on 1, 2 and 4 threads, medium 82 / 94, 96 / 126 and 98 / 130; busy_ui on 4 threads live 43 / 43 and medium 73 / 73; full_motion 4K live on 4 threads 268 / 353.
- **Adaptive B usage (default; `--no-b-adapt` turns it off).** An input frame joins a B run only while at least half of its macroblocks change (side-info dirty rects, scroll hints excluded; without side info, the last anchor's changed macroblocks) and the last anchor itself moved without weighted prediction, scroll or global shift. Every other frame is coded at once as a P frame with its side info, and is dropped if unchanged, and a static frame ends a pending run. Screen recordings therefore keep P-mode behaviour: B-frames cannot use weighted prediction, scroll vectors or global shift, so the first rule tried ("B whenever the frame moves") lost +13.7% on slides and +8.3% on text_scroll. Anchors after a B run are coded without side info, because side info describes the change from the previous input frame and the anchor's reference is several frames back.
- **B QP is the anchor's macroblock QP + 6 (`--b-qp-offset`).** Offsets 2/4/6/8 were swept on full_motion (BD-rate keeps improving up to 12). At 6 the B-frames sit 0.8 dB PSNR below their anchors with a mean frame-to-frame PSNR step of 0.57 dB, which is less than x264 veryfast's own B/P swing (1.0 dB, 0.90 dB per step); 8 adds under one point.
- **Implicit weighted bi-prediction** (`weighted_bipred_idc` 2) whenever weighted prediction is enabled: B-frames at 1/3 and 2/3 of the anchor distance blend the anchors by POC distance (43/21), as x264's default `weightb` does. About 1% fewer bits at equal VMAF and 2% at equal PSNR on full_motion; no extra syntax.
- **Early B_Direct and gated B searches.** The presets with early exits (live, fast, medium) take B_Direct_16x16 without the L0/L1/Bi searches and intra when the direct prediction's SATD is under 96 lambda, or under 112 lambda (128 with the pyramid) when the left and top macroblocks are B_Skip or B_Direct, whose choice says the direct vectors fit the area. The 96 lambda gate alone was quality-neutral on full_motion (0.0% VMAF, −0.3% PSNR) for 13-14% less CPU; raising it for every macroblock lost 2.8% (128) and 18.5% (192). On the same presets quarter-pel refinement runs only on the list that is cheaper after half-pel (both when equal), and intra is skipped when the best inter cost is below 1.5 times the intra estimate (was 0.75). Together, against the 96 lambda gate with both lists refined: 6.6% fewer instructions on medium (10.0% with the pyramid), 7.3% (10.6%) on fast and 5.6% on live b3, with VMAF BD-rate −0.35/−0.46% (medium without/with pyramid), −0.42/−0.65% (fast) and −0.09% (live b3). With the pyramid a neighbour threshold of 144 ended 0.25 points worse than no gates (128: 0.46 points better); without it 128 ended +0.16% (medium), −0.01% (fast) and +0.32% (live b3) against no gates. Rejected: refining sub-pel only on lists whose integer cost is within 1-1.5 times the better one (−6.5% to −1.3% instructions for +1.3% to +0.2% BD-rate), bi-prediction only when the list costs are within 1.5-2 times (no change), half-pel only (1.8-8.5% BD-rate for 2-10% CPU), and capping diamond iterations (B searches converge earlier). Below macroblock QP 30 the neighbour threshold, the one-list quarter-pel and the Bi gate (that one on every preset) are off. They were tuned at CRF 22-34, where over 99% of B macroblocks end with cbp 0; at CRF 12-21 (Cap's 0.15 and 0.3 bpp bitrate targets on full_motion) residuals are coded, and medium was +1.9% VMAF / +3.4% PSNR BD-rate behind main. Removing each gate there: quarter-pel on both lists −1.0/−2.0 points, Bi always −1.0/−0.5, the neighbour threshold −0.5/−0.1, the intra skip 0.0, and deblocking B-frames +0.3/−0.1. With the QP 30 limit, CRF 12-21 against before: medium −1.95/−2.33% (now −0.16/+0.87% against main), fast −1.85/−2.24%, slow −0.85/−0.52%, live b3 −2.99/−2.95%, live b3 with the pyramid −3.23/−2.95%; CRF 22-34 within 0.1%. Instructions (31 frames) +1.7% at CRF 22 on medium and fast, 0.0% at CRF 24 on live b3, +7.9% (medium) and +4.2% (live b3) at CRF 16. A limit of 27 costs nothing at CRF 22 but keeps half the high-rate loss (+0.66/+1.50% against main).
- **B searches share the P search's cost shortcuts.** After merging intra-speed and me-speed, `searchB` uses the same visited-position map and, like `motionSearch`, compares half-pel candidates by SAD and (medium, macroblocks whose QP AQ did not lower) quarter-pel candidates by SAD, converting only the winner to SATD; on near text-free frames it uses the P search's video sub-pel settings (8 half-pel directions, SAD quarter-pel with 4 directions on live and 8 on fast). Unlike `motionSearch` it refines to quarter-pel even on presets whose P search stops at half-pel (on the cheaper list, see the previous item). Effect on full_motion: VMAF BD-rate within 0.2 points (medium b2 -26.6 -> -26.8% vs P-only), 5.1% fewer instructions for a medium b2 encode, 1.5% for live b2.
- **B macroblocks reuse the mode-decision prediction.** Direct, L0 and L1 winners are coded from the prediction built during the decision (the Bi trial is built into its own buffer) and test the 4x4 dead zone before transforming: over 99% of B macroblocks end with cbp 0. The closed-loop intra decision codes I4/I8 blocks through the same prediction buffer, so the direct prediction is saved around it when direct is the best inter mode.
- **Lifetime AQ counts streaks between analysed frames.** With B-frames only anchors are analysed, so change streaks and recency are measured against the previously analysed frame instead of `frameIndex - 1`; otherwise every continuously moving macroblock looked new at each anchor and lost lifetime AQ's +2 QP.
- **Containers.** Decode times are the sorted presentation times. Progressive MP4 stores non-negative ctts offsets and an edit list that removes the reorder delay; fragmented MP4 and HLS do the same with a constant delay of one frame duration (two with the pyramid, rounded up) and the edit list in the init segment, because the delay has to be known before the first fragment. Signed trun offsets (CMAF style) were tried first: ffmpeg's mov demuxer turns the first negative offset it meets into a stream-wide pts shift, so in HLS, where segments are read in order and adaptive B usage leaves early segments without B-frames, presentation times jumped by one frame at the first segment with B-frames. ffmpeg and Chrome (file playback and MSE) apply the edit list and present the first frame at 0.
- **Lookahead with B-frames (2026-10-08).** Since `9bce84f` fast, medium and slow use adaptive `--bframes 3` and the 10-frame lookahead together. The CLI keeps the lookahead ring and takes one map per input frame before `enc.submit()`; each buffered B candidate keeps its map in its `Pending` frame, so whichever picture becomes the anchor (at once, when a run fills, or on flush) is coded with the map of its own display index (`enc.takeLookahead()`; the ring logic stays in `src/lookahead.tov` and the CLI loop). B-frames inherit the anchor's macroblock QP + 6. Outside the pyramid they are non-reference, so propagation gives them nothing of their own, but setting their QP from the B offset alone (removing the anchor's lookahead offset) put full_motion at +9.3% vs main instead of -0.7%. Anchors are credited with the B-frames that predict from them through list 1, as x264's macroblock tree propagates B costs into both lists (list 0 is already the P-chain model): each B-frame of the run adds its bi-prediction weight (distance from the previous anchor over the anchor distance) times the co-located kept fraction from the anchor back to it, times 4, to the anchor's lifetime. Weights 0/0.5/1/2/3/4/6/8 on full_motion: -0.7/-1.2/-1.4/-2.1/-2.3/-2.5/-2.9/-2.9% VMAF BD-rate vs main (slides -12.4 to -12.7%, -12.5% at 8); webcam_overlay is flat (-8.7 to -9.0%), and the screen clips have no B runs. 4 keeps most of the gain short of the plateau. Offsets stay absolute, so CRF means less quality on video than before (full_motion at equal CRF: 23-29% fewer bits, 2-9 VMAF points below main; at the VMAF of x264 veryfast CRF 23 it needs CRF 22.4 where main needs 24.5) and more on screen content (busy_ui, code_editor, text_scroll: 4-14% more bits at 0.1-2.2 points more VMAF, CRF 31.9-32.9 where main needs 29.3-30.2). Holding the running mean of the offsets over every coded anchor macroblock since the start at or below zero (a positive mean is subtracted from each offset) keeps every clip at or above main's VMAF at every CRF (+2 to +25% bits). It leaves the screen clips byte-identical (their mean stays negative), moves full_motion from -2.5 to -2.8% (fast -2.8 to -3.2%, slow -1.8 to -2.5%) and slides from -12.7 to -11.7% (fast -12.6 to -11.9%, slow -15.0 to -14.1%), and its webcam_overlay gain (-8.9 to -12.1%) comes from main's CRF 20 point alone: without that point both are about +2%. It is not used: at equal CRF it costs 11% more single-thread CPU (full_motion +15%) because it codes video at higher quality, which fails the 5% CPU limit, although at matched VMAF it is no dearer than absolute offsets. A per-picture mean (+1.3% vs main over the five clips) and a two-sided running mean (-2.5%) give back most of the gain, and with `--bframes 0` the clamp loses on webcam_overlay (+4.1%), which keeps absolute offsets as on the P-only lookahead. Letting the lookahead's changed-macroblock count drive the adaptive B decision when there is no side info (otherwise the dirty rects already are its change map) lost 5% on a 3:2-pulldown copy of full_motion, where repeated frames left B runs, and is not used.
- **Measuring periodic GOPs.** VMAF with `n_subsample` 2 scores only even frames, which samples a periodic GOP unevenly (with one B-frame only anchors or only B-frames, with three the anchors and middle B-frames but never the outer ones), so it biased the b1/b3 offset sweeps by about 2 points. B-frame tuning used `n_subsample=1`; the harness's `n_subsample=5` is coprime with the 2-4 frame GOP periods.

- **Adaptive B-frames on every preset, `--zerolatency` for strict real time (2026-10-08).** Live now defaults to `--bframes 3` like the other presets. Adaptive B only buffers frames while most of the frame moves, so screen recordings stay P-only and unchanged in quality, while full-motion video on live goes from +39.3% to -6.9% VMAF BD-rate vs x264 veryfast (PSNR +64.5% -> -3.7%, SSIM +80.6% -> +7.8%; `/tmp/std16`) for at most 3 frames (100 ms at 30 fps) of delay on moving content; x264 veryfast's default delay is about 13 frames. `--zerolatency` restores zero output delay (no B-frames, no lookahead) and is byte-identical to the previous live preset. A 4-frame lookahead on live gave a further -4.5 points mean (busy_ui +1.8%, code_editor +0.3%, slides -20.8%) for 4 more frames; it is left for the lookahead round that measures 1-4 frame windows.
- **Lookahead propagation on the read-ahead thread (2026-10-08).** With more than one thread, the thread that reads and analyses frame n+depth+1 also runs `take()` and the keep map for frame n+1 into double-buffered maps while frame n is coded (`22748f4`). The horizon it uses is the GOP position, periodic by display index; the main thread checks it against the encoder's and recomputes on a mismatch, so the propagation's visit tags come from an epoch counter shared by both. The propagation is a serial backward pass over the window and the read-ahead thread is idle once its read is done, so a pooled worker or `parallelFor` would only add a handoff. Main-thread instructions at 4 threads (medium, 60 frames): full_motion -9.1%, slides -29.7%; total instructions within 0.5%; output byte-identical to the main-thread version for any thread count.
- **Vector change map without side info (2026-10-08).** The lookahead's pixel-difference change map (the media server's `--noise 3 --no-sideinfo` path) compares 16- and 8-wide blocks with vectors: a pixel difference above the noise (lane maximum) or a block sum above the budget gives the same answer as the scalar `diffMb` (`17ebf21`). busy_ui at the media-server settings: 5.86G -> 3.52G instructions (the lookahead's cost over `--lookahead 0` from +83% to +9.6%); full_motion -1.9%. The encoder's own `mbDiffers` (`src/encoder.tov`) still calls the scalar `diffMb` on this path and is now 2.7G of busy_ui's 3.5G instructions.
- **Geometric tail on video frames without B-frames; no video cost model (2026-10-08).** On current main the media-server settings (medium with B-frames, `--keyint 250 --noise 3 --no-sideinfo`) no longer lose on full_motion: -1.6% VMAF BD-rate against `--lookahead 0` (busy_ui -8.3, code_editor -9.0, slides -9.9, text_scroll -1.4, webcam_overlay -14.6%). The loss is P-only: with `--bframes 0` full_motion is +2.9%. A video cost model on frames where most macroblocks change (SATD at the full-pel vector, 4 or 8 bilinear half-pel SATD positions, the best of i16 DC/V/H/plane SATD from source neighbours as intra cost, a step-8 diamond, an intra bias of 64) moved full_motion by -0.33 to +0.56% and webcam_overlay by -0.33 to +1.13% with B-frames, and gained 0.87% on P-only full_motion, for +2.7% (SATD and i16 intra costs) to +14% (with half-pel) instructions; strength 1.25 and 2.25, an offset cap of 4 and a 20-frame window gained at most 0.11% on either clip. The P-only loss is in the tail instead: content that changes in a video frame and is not seen again inside the window was credited n * q^12 frames up to the horizon (n frames left, q the kept fraction). Without B-frames a frame where at least half the macroblocks changed now uses q(1 - q^n)/(1 - q), the expected survival if every later frame keeps the same fraction (34 instead of 46 frames at q 0.98 with 58 left, 9 instead of 16 at 0.9, 1 instead of 0.01 at 0.5). VMAF BD-rate against the same lookahead without it: live `--lookahead 1/2/4` -0.46/-0.67/-0.81% (full_motion -3.2/-2.2/-1.0%, worst clip slides +0.46% at depth 1), medium `--bframes 0` -0.06% (worst +0.15%), P-only media-server settings -0.07% (full_motion -0.50%, worst slides +0.24%); full_motion is still +2.4% there against `--lookahead 0`. Ending that content at the window edge instead gains more on the long-GOP P-only setting (-0.82%) but loses on live depth 1 (+0.84%), on webcam_overlay at depth 2 (+0.71%) and with medium `--bframes 0` (+0.87%). With B-frames the geometric tail costs 0.5% on long-GOP slides (mean +0.05%), so fast, medium and slow keep the power law and stay byte-identical.
- **Live lookahead depth: default 0, `--lookahead 2` for desktop recording (2026-10-08).** Each frame of lookahead delays output by one frame (33 ms at 30 fps) and adds about 3 MB at 1080p (the raw frame in the ring). VMAF BD-rate against live (six clips, mean): depth 1 -5.9% but full_motion +1.9%; depth 2 -6.3% with every clip better (full_motion -1.0%); depth 4 -6.6% (webcam_overlay -12.6 instead of -10.9%); before the geometric tail depth 10 gave -6.3%. Callgrind instructions at CRF 26 over the six clips: -5.3/-4.9/-4.5% at depth 1/2/4 (text_scroll +3.5 to +3.8%, the rest -1.0 to -17.4%), because video and short-lived content get higher QPs; the analysis itself is 2.8% of text_scroll's and 8.7% of full_motion's instructions at depth 4. Live stays at 0 because it must not add delay; 2 is the point to offer when about 67 ms is acceptable, and 4 adds 0.24 points for another 67 ms and 6 MB. OCR character error at depth 2 and 4: text_scroll 1.81 -> 1.41/1.39%, slides 0.48 -> 0.45%, code_editor 4.70 -> 4.80/4.74%, busy_ui 3.68 -> 4.33/4.34%; busy_ui's text-region luma PSNR on the four OCR frames is -0.29 to +0.25 dB against live at matched bitrate (code_editor -0.26 to +1.06 dB), so the busy_ui difference is at least partly sampling noise but is not explained.
- **Lookahead memory stays a raw-frame ring (2026-10-08).** The +31 MB at 1080p is the ring of depth + 1 raw frames (one more with the read-ahead thread); the analysis keeps about 1.5 MB of per-macroblock state and no pixels. The ring is also the encoder's input delay: frame n is coded from its slot after n + depth has been read, so luma-only or half-resolution copies would add memory rather than replace any. Pointing the B-frame `Pending` copies at ring slots would need the ring to hold those frames for up to 3 more frames, the same memory. A ring of changed macroblocks only would shrink with static content, but peak RSS is set by full-change frames unless its allocation is shared with the encoder's buffers, and storing exact deltas needs a pixel compare (side-info dirty rects are conservative) of about 4.6M instructions per 1080p frame, +45% on live busy_ui; it is free only fused into the no-side-info change map. Tov's large typed arrays are resident only once touched, so the ring costs memory as it fills. A shorter window on medium saves 12 MB at depth 6 and 19 MB at depth 4 for +0.06% mean BD-rate, but full_motion loses 0.54 and 0.78%, so 10 stays.
- **Live reads 2 frames ahead by default (2026-10-08).** With adaptive B-frames already on, `--lookahead 2` on live takes VMAF BD-rate vs x264 veryfast from -4.3% to -8.7% mean (busy_ui +5.1 -> +1.7, code_editor +4.1 -> +0.7, slides -9.8 -> -22.1, text_scroll -14.0 -> -16.2, full_motion -6.9 -> -7.8; PSNR mean +1.0 -> -0.4, SSIM +4.2 -> +1.0; `/tmp/std17`) at matched-VMAF CPU within noise, for 67 ms of extra delay at 30 fps. full_motion PSNR/SSIM get slightly worse (-3.7 -> -1.9, +7.8 -> +12.0) and the lookahead agent measured a small busy_ui OCR change that it is investigating. `--zerolatency` keeps zero delay.
- **Change detection without side info: one vector pass, shared with the lookahead (2026-10-08).** At the media server's settings (medium, `--keyint 250 --noise 3`, no side info) the encoder compared every macroblock with the scalar `diffMb` (`mbDiffers`), 2.7G of busy_ui's 3.4G instructions over 60 frames on main `52510d7`. Four byte-identical steps replace it. (1) One vector change-map pass per frame (`49b82ed`), (2) two macroblocks per 32-byte luma row, the per-macroblock maxima from the halves of the mask and the SADs from the halves of `sadParts`, chroma one 16-byte U and V row each (`0ba6f14`). (3) The encoder takes the lookahead's map of the frame it codes (`572b08a`). The two compare different things: the lookahead frame n with frame n-1, the encoder frame n with its source copy, which only takes the macroblocks marked changed, so a macroblock that moves within the noise leaves the copy behind the input. The lookahead's map therefore has three codes (identical, changed, different within the noise) and lives in a ring of depth + 2 slots, because the read-ahead thread writes frame n + depth + 1 while n is coded; the encoder keeps a flag per macroblock for "the copy equals input frame `heldAt`", takes the map's code where the flag is set (16 flags at a time), compares the other macroblocks itself and keeps the flag where the code is identical or the macroblock is copied. Side-info frames, anchors after a B run and flushed anchors use the full pass. (4) Each pair of macroblocks is first tested for exact equality, four rows at a time with an early return (XOR and OR, then one mask), which skips the maxima and SADs on static content. Callgrind, 60 frames, one thread, millions of instructions:

| clip | main `52510d7` | (1) | (2) | (3) | (4) | vs main |
|---|---|---|---|---|---|---|
| busy_ui | 3398.0 | 835.4 | 681.1 | 589.6 | 558.0 | -83.6% |
| code_editor | 3357.6 | 793.0 | 638.4 | 545.4 | 513.6 | -84.7% |
| slides | 3201.9 | 636.0 | 481.4 | 388.4 | 356.7 | -88.9% |
| full_motion | 12052.3 | 11941.1 | 11905.2 | 11925.7 | 11932.9 | -1.0% |
| webcam_overlay | 3722.6 | 1267.1 | 1117.3 | 1027.3 | 997.3 | -73.2% |

The map is now the lookahead's only pass (70M of busy_ui's 558M; CABAC is 163M); with B-frames only 3 of full_motion's 17 anchors can take it, so there step (3) costs +0.17% and step (4) +0.06%. Without side info at noise 0 busy_ui goes from 956.8M on main to 563.9M and full_motion stays at 11725M. Rejected: loading all 16 rows before one exact test (0.3-0.9 points better on the screen clips, full_motion +0.23%; clang keeps the 16 row offsets in stack slots in both forms), a separate exact kernel for noise 0 (busy_ui +0.4%, full_motion -0.5%), and storing every code into an unzeroed map (+15% for the pass). Since main `7189fa4` the rows copy their changed macroblocks in the wavefront workers and the next-frame peeks (`diffFrame`) run the same vector pass, so the flags are set on the main encoder: from the codes in `analyse`, and for every macroblock the rows will copy once the frame is known to be coded. Against main `7189fa4` the merge executes -83.1/-84.3/-88.4/-1.0/-73.0% instructions at the media-server settings; wavefront frames take the peek's map, busy_ui's small changes stay off the wavefront and keep the lookahead's map at 2 threads.
- **Window tail for P-only GOPs longer than the lifetime span (2026-10-08).** Without B-frames and with `--keyint` above the 2 s span, content that a video frame replaces and that is not seen again inside the window now ends at the window edge instead of decaying geometrically (`7f60c22`). The gate keeps every case where the window edge lost (live and medium at their 2 s GOPs, above) on the geometric tail, so all preset defaults stay byte-identical. VMAF BD-rate at `--preset medium --bframes 0 --keyint 250 --noise 3 --no-sideinfo` against the geometric tail: busy_ui 0.00, code_editor 0.00, full_motion -0.50, slides -2.78, text_scroll -0.02, webcam_overlay -1.24% (mean -0.76%); against `--lookahead 0` full_motion goes from +2.37 to +1.67% and the mean from -8.66 to -9.31%; against x264 veryfast from -32.79 to -33.40%. OCR character error unchanged (text_scroll 1.33 -> 1.32%). Instructions at that setting: full_motion -0.63%, the screen clips +0.05 to +0.13%. What is left on full_motion is the same shape: content credited past the edge of a 10-frame window in a 250-frame GOP.
- **No text-aware lifetime rule for the live lookahead's busy_ui OCR result (2026-10-08).** With `--preset live --lookahead 2` busy_ui's character error is 4.33% against 3.68%. It is not text: the tokens that differ are icon glyphs that OCR reads as characters (tag icons fused with labels, "@upload" -> "e@upload", "e@camera" -> "ecamera"; calendar icons fused with dates, "OHOcti3", "Foct21"; priority bars read as "ail"; sidebar chevrons and badge counts), in all three text regions; the words next to them are read in both, the text-region luma PSNR is 0.4 to 1.9 dB higher with the lookahead at every CRF (-0.29 to +0.25 dB at matched bitrate), and the difference changes sign between CRFs (CER over 30 frames 2.49/2.55/3.02/4.32/6.92% without and 2.58/3.14/4.13/5.32/5.53% with it at CRF 20-36) and between regions at the same CRF. Most of that text does not change (the sidebar never does, the two board columns go through 8 and 3 states over the 30 frames), and most differences are one token read differently in all 30 frames, so each CRF point holds about a dozen independent renderings rather than 90. Capping the lookahead's QP raise at 0 or +2 on macroblocks the AQ classifier marks as text leaves busy_ui at 4.35/4.36% and gives up a third (-4.23% instead of -6.68% against live) or a fifth (-5.46%) of the BD-rate gain, so neither is used.