capcodec/PROGRESS.md

Rama main · réplica de lectura del repositorio original.

# Progress

**Headline (main `1a1794a`; the 12-clip speed table in `SCOREBOARD.md` is still `00fecf7`, before concurrent B-frames, and `--bframes 0` matches that encoder): at matched VMAF that table is faster than x264 veryfast on every clip in every mode, on CPU and on wall clock, at 1 and at 4 threads (36/36). Geometric mean CPU 4.9-5.9x, wall 3.2-5.2x. Concurrent B-frames are now the default. On full_motion in the benchmark lane, 4-thread wall is 1.35-1.40x medium and 1.44-1.50x live (the same sessions measured the previous main at 1.09-1.10x and 1.19-1.21x). The other 11 clips were not retimed. Peak memory on the 12-clip run is 3-8x under x264. Quality on this encoder is the `fff504d` result (bitstreams match): level, 17-20 wins of 36 clip-metrics per mode. The pending quality branch `rd-next` wins VMAF on 12/12 clips and PSNR on 11/12, and still loses SSIM on 6-8 clips. SSIM round 3 (`ssim` `b72e995`, not on main; base is `rd-next` plus main `08b3b02`, 84 commits behind) lowers light flat and smooth IDR macroblocks above QP 34 and moves the 12-clip SSIM mean from about 0 to −1.4% on medium and live. It still loses SSIM on dashboard (+10.0% medium), busy UI, code, webcam, and full motion. Intra rounding and IDR rate-distortion (`intra-rd` `99822eb`, not on main) pass the 0.5% gate against main on all 12 clips in medium, live, and zerolatency. All-intra wins against x264 veryfast keyint 1 go from 7 to 12 of 36. The screen-clip SSIM and PSNR losses shrink and do not flip. All-intra CPU stays ahead of x264; full_motion is 1.01× and a round reaches 1.00×. Line and edge AQ (`dashboard` `4ca9f20`, on `rd-next`, not on main) cuts dashboard's loss to x264 veryfast from SSIM +12.8% to +6.2% and PSNR-YUV +3.2% to +1.7% in medium, and from +14.0% to +7.3% and +3.8% to +2.4% in live. Zerolatency dashboard wins all three metrics. No other clip gets worse by more than 0.5 against x264.**

Threads: deterministic wavefront (rows claimed in order, each macroblock waits for the row above to pass its top-right neighbour), output byte-identical for any thread count.

## Intra RD (not on main)

`intra-rd` `99822eb` (rounding `719dab7` plus CABAC SSD choice among close I4/I8/I16 SATD candidates; `--no-intra-rd` matches the rounding commit). All-intra uses round-to-nearest above the first luma level, and textured or natural macroblocks use a 0.4 offset (chroma 0.36). In a normal GOP, IDRs keep main's rounding except lasting textured or natural macroblocks, which take the 0.4 offset. Against main, every clip in medium, live, and zerolatency is inside the 0.5% gate; the worst cells are live slides VMAF +0.28% and zerolatency webcam_overlay PSNR-Y +0.27%. Against x264, wins go from 19 to 20 (medium), 17 to 20 (live), and 19 to 22 (zerolatency). All-intra mean versus main is VMAF −13.2%, PSNR-YUV −1.0%, SSIM −1.7%. `tov test` 85/85, thread identity, TSan 0, `verify.py` 23/23. Not merged: it changes bitstreams, and full_motion all-intra speed is only 1.01× x264.

## Dashboard lines (not on main)

`dashboard` `4ca9f20` (docs `b74f140`) sits on `rd-next` `d0cce5e`. Macroblocks whose 16 rows or 16 columns are constant split into lines and edges. Edges take −4 QP in every frame. Lines on IDRs take −8, or −12 when the range is under 16; lines in P and B frames take −4. Flags: `--aq-stroke`, `--aq-line`, `--aq-faint-line`. full_motion stays byte-identical. The worst change outside dashboard against x264 is slow_typing medium VMAF +0.38. `tov test` 65/65, `verify.py` bit-exact on busy_ui and full_motion, 1 and 4 threads identical in 18 configurations. Instructions rise at most 0.38%. Not merged: medium and live dashboard still lose SSIM (+6.2%, +7.3%) and PSNR-YUV (+1.7%, +2.4%) to x264, and the base is `rd-next`.

## Scoreboard vs x264 veryfast (goal: win every row)

Quality: harness standard tier on 12 clips (1080p, 10 s) with capcodec's CRF ladder reaching 44 so both curves overlap (42-88%; the old ladder compared only 15-24% of screen-clip curves and flattered capcodec). Speed: `bench/speedcmp.py` in the benchmark lane (CRF matched to x264 CRF 23's VMAF, interleaved rounds, rest of the machine paused). Clip counts are wins out of 12.

| Metric | main `fff504d` | quality branch `rd-next` | status |
|---|---|---|---|
| VMAF BD, medium / live / zerolatency | 8 / 6 / 7 | 12 / 12 / 12 (-12% to -81%) | win after rd-next |
| PSNR BD, medium / live / zerolatency | 5 / 6 / 6 | 11 / 11 / 12 | lose dashboard (+3.2% / +3.8%) |
| SSIM BD, medium / live / zerolatency | 5 / 5 / 6 | 4 / 4 / 6 | lose on rd-next: dashboard +12.8%, busy_ui +6.5%, busy_ui_60fps +6.4%, code_4k +5.0%, webcam_overlay +3.4%, code_editor +2.5%, slow_typing +2.2%, full_motion +1.5% (live +6.8%). `ssim` `9b329be` (not merged): medium 27/36 wins, SSIM mean −1.36% (dashboard +10.0, busy_ui +3.5, busy_ui_60fps +0.9, code_4k +3.8, code_editor +1.7, webcam +3.4, full_motion +1.5); zerolatency 31/36, slow_typing SSIM becomes a win |
| CPU at matched VMAF, 1 thread | 36/36 faster; geomean medium 5.53x, live 5.83x, zerolatency 5.39x; min 1.39x (full_motion medium) | | win |
| CPU at matched VMAF, 4 threads | 36/36; geomean 5.51x / 5.87x / 4.90x; min 1.21x (full_motion zerolatency) | | win |
| Wall at matched VMAF, 1 thread | 36/36; geomean 4.94x / 5.23x / 4.80x | | win |
| Wall at matched VMAF, 4 threads | 36/36; geomean 3.15x / 3.46x / 3.26x; min 1.06x (full_motion medium), live 1.21x, zerolatency 1.49x | | win (`bench/out/speedcmp-00fecf7`; other processes stopped, frozen median about 690); with the concurrent B-frames after merging `e20b7fe`, full_motion medium 1.35-1.40x and live 1.44-1.50x in the benchmark lane (main 1.09-1.10x and 1.19-1.21x in the same sessions) |
| Keyframe-only CPU at matched VMAF (`speedcmp.py --keyint1`, main `558481e`) | 1.07-1.75x at 1 thread, 1.06-1.76x at 4 (full_motion 1.07x / 1.06x, busy_ui 1.21x / 1.28x); 4-thread wall fps full_motion 149 vs 136 | | win on every clip; busy_ui needs 12% more bits than x264 at matched VMAF in all-intra |
| Peak memory, 4 threads | 34-308 MiB vs 141-1304 MiB on the `00fecf7` rerun | | win |
| Delay before output | live 2 frames (5 with B-frames), medium 10 (steady 13), zerolatency 0 | | win vs x264 at 4 threads (17) and auto (19); medium steady lag 13 vs 10 at 1 thread |
| Startup | 0.9-1.2 ms vs 1.3-1.5 ms (64x64 frame); first 1080p frame 30.5 vs 54.9 ms at default threads | | win; 1-thread medium first 1080p frame 48.9 vs 40.3 ms (keyframe speed) |
| Decode CPU of the output at matched VMAF | 0.58-1.54 s vs 1.59-2.40 s (dashboard, dark_mode, webcam_overlay, slow_typing, idle) | | win |
| Binary | 550 KB vs 1.29 MB (minimal x264 build) / 2.5 MB (CLI) | | win |
| Build CPU time | about 12 s vs 36.5 s (minimal x264) / 71 s (full) | | win |
| Bitrate accuracy at Cap targets (10 s, reachable) | two-pass 4.9% vs 5.5%; one-pass 7.7% vs 5.5% | | two-pass win, one-pass lose (rate-control round in progress) |

## State

| Milestone | State |
|---|---|
| 0 Benchmark harness | Done. Corpus: 12 clips including full-motion video (Big Buck Bunny, CC BY 3.0) and 4K; scripts and manifest in `bench/corpus/`, sources outside git. |
| 1 Minimal stream | Done |
| 2 Full intra | Done |
| 3 CABAC | Done; byte-oriented arithmetic coder, bit-exact with the spec engine (`src/cabac_ref.tov`, randomised equivalence tests) |
| 4 P frames | Done; global shift candidates (column/row profile matching) find slide push/cover transitions |
| 5 Screen-aware side info | Done; IDR frames reuse unchanged macroblocks of the previous IDR (decisions, coefficients and reconstruction) |
| 6 Text-aware AQ | Done (defaults under review) |
| 7 Rate control | Done; weighted prediction for fades added; lookahead QP from macroblock lifetime (`--lookahead`, 10 frames on fast/medium/slow) |
| 8 Speed | SIMD kernels (SAD, SATD, SA8D, half-pel, transforms, quantisation, deblocking, classification, hashing, chroma MC) and wavefront threads; IDR entropy and deblocking still serial |
| 9 Integration | Media server prototype on Cap branch `cursor/capcodec-integration-aa45` (draft PR #2429); C ABI and WASM wait on Tov |
| Muxers | Done: MP4 faststart, fragmented MP4, HLS |
| B-frames | Done: `--bframes 0-3` (default 3, 0 with `--zerolatency`), adaptive by default (`--no-b-adapt`), B-pyramid (default with the B-frames except on live, `--no-b-pyramid`; the middle B-frame of a run is a reference at the anchor's QP + 5), `--b-qp-offset` (6); one reference per list, B_Skip / B_Direct_16x16 / 16x16 L0, L1, Bi; the B-frames of a run are coded concurrently and alongside the next anchor; MP4, fMP4 and HLS (edit lists remove the reorder delay) |
| Read-ahead | On main. More than one thread already overlapped the next frame's read and lookahead (one extra ring slot) at `fff504d`. `f4781c5` (`cli/main.tov` only) reads further frames while a B run is coded (`--read-ahead`, default 3; off when B-frames are off). `a9b86d5` matched `/tmp/capcodec-main-latest18` in 459/459 configurations (verify.py 10/10, tov test 81/81, TSan 0 reports). A pin7 rebuild of `f4781c5` matched that baseline on busy_ui and full_motion for default, live, zerolatency, fast, `--bframes 0`, `--lookahead 0`, bitrate and `--keyint 1` at 1 and 4 threads (32/32; full_motion default and live also matched at 90 frames). `--read-ahead 0` and `5` are new flags; on these clips they hash the same as the default encode. Retimed on `00fecf7` with other processes stopped: full_motion 4-thread wall is 1.06x medium and 1.21x live (was 0.89x and 0.97x) |

Every capcodec bitstream decodes without errors in JM 19 and ffmpeg and is bit-exact with the encoder's reconstruction (every harness tier, and `tools/verify.py` after each change). B-frames, after merging intra-speed and me-speed: 51 verify cases (bframes 1-3 x live/medium/slow x CABAC/CAVLC, keyint 7, stream ends 1-15 frames, ABR, constant QP, fixed pattern, no side info, adaptive runs on five screen clips, 300-frame encodes of four clips) are JM and ffmpeg bit-exact and identical for 1 and 4 threads; `--bframes 0` is byte-identical to main `dc8d9fb` (full_motion, busy_ui, slides x live/medium x 1/4 threads x CABAC/CAVLC, and MP4/fMP4/HLS); ffmpeg presentation and decode checks pass in all three containers (bframes 1-3, VFR drops), and Chrome 148 plays MP4, fMP4 and HLS (MSE) with B-frames in order. B-frames round 2 (speed, concurrent runs, pyramid; `6765509`): the 53-case verify matrix (the same kinds of cases as above, now 53) is JM and ffmpeg bit-exact and identical for 1 and 4 threads with and without the pyramid, every speed step that claims it is byte-identical (P-only, slow and screen clips where the decisions do not change), the reconstruction written with `--recon` matches ffmpeg's decode, and the container checks pass with reorder depth 2. After merging main `08b3b02` (`149f001`): B-frame configurations are byte-identical to the branch at `c12ee19` (36 configurations with 1-4 threads, and `--keyint 1` with the default B-frames), P-only configurations to main `08b3b02` (16 configurations with 1-4 threads, including the window tail at `--bframes 0 --keyint 250` and `300`, and all-intra), the 53-case matrix is JM and ffmpeg bit-exact and identical for 1 and 4 threads, ThreadSanitizer finds nothing in 43 scenarios, and the container checks and `tov test` pass. After merging main `1859bac` (`f1c6e24`): B-frame configurations are byte-identical to `812670f` (36 configurations with 1-4 threads, and footprint's odd sizes, synthetic pans, mixed content and live / zerolatency at 1080p, 1440p and 4K), P-only configurations to main `1859bac` (15 configurations with 1-4 threads), the 51-case matrix is JM and ffmpeg bit-exact and identical for 1 and 4 threads, ThreadSanitizer finds nothing in 43 scenarios, and the container checks and `tov test` pass. After merging main `e20b7fe` (`0c662ea`): B-frame configurations are byte-identical to `70ebc48` (the same 36 configurations and footprint sweeps) and to themselves with `--read-ahead 0`, 3 and 5 on 2 and 4 threads (10 configurations), P-only configurations to main `e20b7fe` (15 configurations with 1-4 threads), `--keyint 1` and `2` with the default B-frames to its `--bframes 0` (3 clips x 3 presets x 1-4 threads), the 51-case matrix is JM and ffmpeg bit-exact and identical for 1 and 4 threads, ThreadSanitizer finds nothing in 57 scenarios (14 of them with the read-ahead running), and the container checks and `tov test` (86) pass.

## Numbers

Standard tier, 1080p 10 s, VMAF BD-rate vs x264 veryfast (negative = fewer bits than x264 for the same VMAF), 2026-10-07 07:21, before weighted prediction:

| clip | capcodec-live | capcodec-medium |
|---|---|---|
| busy_ui | +9.9% | +9.3% |
| code_editor | +10.5% | +7.5% |
| slides | +7.0% (was +65%) | +5.5% (was +57%) |
| text_scroll | −7.5% | −7.8% |
| full_motion | +67.7% | +39.0% |

Since then: weighted prediction (slides −6.6% live / −6.9% medium, now fewer bits than x264 veryfast) and texture AQ off by default (capcodec-medium: busy_ui +9.4%, code_editor +7.5%, slides −8.3%, text_scroll −7.8%, full_motion +35.3%; mean +7.2%).

Where full_motion loses: x264 veryfast's B-frames. Without them x264 is +47.5% on full_motion (and +0.0%/+2.2% on busy_ui/code_editor), so capcodec-medium's P-only +35.3% beats x264 veryfast without B-frames by about 12 points (`/tmp/std7`, `/tmp/std8`). capcodec's own B-frames close the gap (next section).

### B-frames

Standard tier, B-frames round 2 (2026-10-08, `/tmp/bf-r2`, branch `bframes` at `ca086f8`, which `6765509` reproduces byte for byte; variants via `CAPCODEC_VARIANTS`, side info on, lookahead on except on live). fast and medium are the defaults (three adaptive B-frames with the pyramid). VMAF BD-rate vs x264 veryfast, PSNR (YUV) in brackets:

| clip | live | live b3 | live b3 pyramid | fast | medium | medium no pyramid | medium P-only |
|---|---|---|---|---|---|---|---|
| full_motion | +39.3% (+64.5%) | −8.5% (−4.4%) | −11.0% (−5.2%) | −14.3% (−7.9%) | −15.0% (−11.1%) | −12.7% (−10.3%) | +25.5% (+41.5%) |
| busy_ui | +5.1% | +5.2% | +5.2% | +1.7% | +1.3% | +1.3% | +1.3% |
| code_editor | +4.1% | +4.1% | +4.1% | −0.0% | −2.8% | −2.8% | −2.9% |
| slides | −9.8% | −9.8% | −9.8% | −21.1% | −20.9% | −20.9% | −20.7% |
| text_scroll | −14.0% | −14.0% | −14.0% | −16.3% | −16.1% | −16.1% | −16.2% |
| mean | +4.9% (+14.8%) | −4.6% (+0.9%) | −5.1% (+0.7%) | −10.0% (−1.3%) | −10.7% (−2.7%) | −10.3% (−2.5%) | −2.6% (+7.9%) |

Against P-only of the same preset (VMAF/PSNR): full_motion medium −34.1/−37.8%, medium without the pyramid −32.4/−37.1%, live b3 −34.5/−41.7%, live b3 with the pyramid −36.2/−42.2%; the screen clips stay within ±0.1% because adaptive B usage codes them almost entirely as P frames. Main before round 2 (`0c7af6c`, same harness, full_motion, `/tmp/bf-r2-main`): medium −12.2% (−10.0%), live b3 −6.9% (−3.7%). Against main on `tune.py` (full_motion, 120 frames, CRF 22-34, VMAF/PSNR BD-rate): medium −2.94/−1.48%, medium without the pyramid −0.45/−0.34%, fast −4.08/−2.29%, slow −2.22/−1.21%, live b3 −1.50/−0.65%, live b3 with the pyramid −4.27/−1.68%.

Matched-VMAF speed vs x264 veryfast CRF 23 (capcodec CRF from the harness: full_motion live 25, live b3 24, fast and medium 22, and main's medium 22 and live b3 24; busy_ui live 29, fast and medium 32; code_editor 31 and 32; slides 30; text_scroll 30 and 33), 300 frames, interleaved best of 3 per variant, 1 and 4 threads for both encoders, load average 21-27 on the 4-core VM (`/tmp/bfdbg/abtime_r2final.log`, `/tmp/bfdbg/abtime_r2screen.log`):

| | live | live b3 | live b3 pyramid | fast | medium | medium no pyramid | main medium | main live b3 |
|---|---|---|---|---|---|---|---|---|
| full_motion, 1 thread, CPU | 1.04x | 1.37x | 1.32x | 1.18x | 1.11x | 1.07x | 0.91x | 1.01x |
| full_motion, 4 threads, CPU | 0.82x | 1.14x | 1.10x | 1.03x | 0.99x | 1.02x | 0.78x | 0.89x |
| full_motion, 4 threads, wall | 0.45x | 1.03x | 1.21x | 1.04x | 1.03x | 0.98x | 0.48x | 0.55x |
| 5 clips geometric mean, 1 thread, CPU | 3.77x | 3.84x | 3.84x | 3.56x | 3.47x | 3.49x | | |
| 5 clips geometric mean, 4 threads, CPU | 3.39x | 3.55x | 3.58x | 3.32x | 3.23x | 3.27x | | |
| 5 clips geometric mean, 4 threads, wall | 1.87x | 2.07x | 2.35x | 1.98x | 1.94x | 1.83x | | |

Main was timed on full_motion only. x264's best 4-thread wall time on full_motion was 1.6x its CPU time (it got 0.63 cores at this load), so the 4-thread wall ratios mostly measure the load; the full_motion CPU ratios repeat within about ±3%. On the screen clips live and live b3 make the same decisions on nearly every frame (BD-rate within 0.1%), so their CPU differences of up to 9% (text_scroll, about 1 s encodes) are load noise. Callgrind (x86-64-v2, full_motion 31 frames CRF 24) against main: medium 7.516G → 5.664G instructions (−24.6%), live b3 7.116G → 5.126G (−28.0%); the steps are in DECISIONS.md under B-frames. Peak RSS at 1080p: medium 112.6 MiB on one thread and 151.8 MiB on four (main 91.1 and 95.5), live b3 72.0 and 114.5 MiB (main 60.7 and 65.3); the extra on four threads is the state of concurrently coded B-frames.

Merging thread-efficiency (main `7189fa4`) took 5-8% off the 4-thread CPU time of the B-frame encodes (pipelined entropy, row prep inside the wavefront, adaptive spin-waits) and left 1-thread CPU unchanged (`/tmp/bfdbg/timing_m7.log`, `timing_m8.log`, timed at load 29-57).

After merging main `08b3b02` (`149f001`), in the benchmark lane (`bench/lane.sh run "python3 bench/speedcmp.py ..."`: capcodec's CRF matched in quarter steps to the VMAF of x264 veryfast CRF 23, 300 frames with side info, five interleaved rounds, the rest of the machine paused for each timed run), two sessions for each binary (`/tmp/bfdbg/m9/lane-merged`, `lane-merged-r2`, `lane-main`, `lane-main-r2`). x264's median time over capcodec's (above 1 is faster), first / second session:

| clip, mode | CPU, 1 thread | main | wall, 1 thread | main | CPU, 4 threads | main | wall, 4 threads | main |
|---|---|---|---|---|---|---|---|---|
| full_motion medium | 1.46 / 1.49x | 1.32 / 1.27x | 1.39 / 1.42x | 1.27 / 1.21x | 1.31 / 1.33x | 1.17 / 1.16x | 1.15 / 1.16x | 0.92 / 0.92x |
| full_motion live | 1.69 / 1.63x | 1.41 / 1.40x | 1.62 / 1.56x | 1.35 / 1.34x | 1.47 / 1.50x | 1.19 / 1.21x | 1.32 / 1.33x | 0.96 / 0.97x |
| full_motion zerolatency | 1.41 / 1.41x | 1.42 / 1.43x | 1.32 / 1.34x | 1.26 / 1.36x | 1.00 / 1.00x | 1.01 / 0.99x | 1.19 / 1.22x | 1.07 / 1.16x |
| webcam_overlay medium | 3.79 / 3.76x | 3.78 / 3.83x | 3.42 / 3.40x | 3.40 / 3.46x | 3.85 / 3.68x | 3.69 / 3.66x | 2.63 / 2.55x | 2.52 / 2.45x |
| webcam_overlay live | 4.09 / 4.08x | 4.16 / 4.00x | 3.09 / 3.68x | 3.76 / 3.62x | 3.84 / 3.93x | 4.09 / 3.90x | 2.32 / 2.75x | 2.90 / 2.76x |
| webcam_overlay zerolatency | 4.38 / 4.46x | 4.41 / 4.42x | 4.00 / 4.09x | 4.05 / 4.03x | 3.94 / 3.90x | 3.87 / 3.87x | 3.65 / 3.60x | 3.57 / 3.65x |
| busy_ui medium | 6.53 / 6.61x | 6.52 / 6.28x | 5.75 / 5.77x | 5.76 / 5.53x | 6.99 / 7.16x | 7.07 / 7.34x | 3.13 / 3.38x | 3.27 / 3.35x |
| busy_ui live | 6.70 / 6.73x | 6.81 / 6.70x | 5.78 / 5.91x | 6.01 / 5.88x | 7.91 / 7.71x | 7.52 / 7.45x | 3.79 / 3.56x | 3.57 / 3.51x |
| busy_ui zerolatency | 6.17 / 6.16x | 6.06 / 6.20x | 5.36 / 5.39x | 5.27 / 5.41x | 6.13 / 6.09x | 6.16 / 6.04x | 3.61 / 3.67x | 3.68 / 3.57x |

full_motion's 4-thread wall clock, main's last speed loss on the scoreboard, is a win: medium 1.94 / 1.86 s against x264's 2.23 / 2.15 s (main 2.48 / 2.34 s), live 1.65 / 1.59 s against 2.18 / 2.11 s (main 2.29 / 2.14 s), from the concurrent and asynchronous B runs; with B-frames full_motion also takes 12-19% less CPU time than on main (round 2's B search and pyramid steps). The matched CRFs are the same for both binaries (full_motion 22.5 / 22.25 / 25.75 for medium / live / zerolatency, webcam_overlay 28.5 / 28.25 / 26.25, busy_ui 32.25 / 32.25 / 25). zerolatency output is byte-identical to main's, and on the screen clips adaptive B usage codes nearly every frame as P, so those rows differ only by the sessions' spread; webcam_overlay live's first session (wall 3.09x and 2.32x) had slow runs, and a 7-round lane recheck put the branch, `c12ee19` and main at 0.75, 0.76 and 0.78 s on one thread and 0.43 s each on four. Peak RSS at 1080p on 1 / 4 threads: full_motion medium 115 / 155 MiB (main 97 / 102, x264 253-260 / 373-385), live 82 / 125 (main 72-73 / 78), zerolatency 52 / 54 as on main; webcam_overlay medium 104 / 119-120 (main 91 / 96), live 70 / 88 (main 67 / 71-72). The extra is the concurrently coded B-frames' state (DECISIONS.md).

After merging main `1859bac` (`f1c6e24`: footprint rounds 3-4, intra-speed round 2; B-frame output byte-identical to `812670f`), each concurrently coded B-frame packs its coefficients into its own chunk store, which halves the B-frames' extra memory. Peak RSS at 1080p on 1 / 2 / 4 threads (90 frames, CRF 26): full_motion live 59 / 85 / 89 MiB (main `1859bac` 58 / 63 / 64, `812670f` 73 / 108 / 112), medium 94 / 116 / 120 (main 83 / 87 / 89, `812670f` 109 / 141 / 144); zerolatency 39 on 4 threads and busy_ui 30-64 as on main; full_motion 4K live 218 / 315 on 1 / 4 threads (main 214 / 229, `812670f` 274 / 416). `--keyint 1` and `2` no longer declare B-frames in the SPS and are byte-identical to `--bframes 0`. In the benchmark lane (two sessions per binary, matched VMAF against x264 veryfast, ratios on 1 / 4 threads), full_motion medium takes 1.58-1.63x / 1.41-1.47x less CPU time and 1.45-1.55x / 1.16-1.25x less wall clock than x264 (main 1.31-1.32x / 1.19-1.22x and 1.26x / 0.94-0.97x), live 1.69-1.72x / 1.56-1.57x and 1.61-1.64x / 1.38-1.39x (main 1.45-1.49x / 1.24-1.30x and 1.39-1.43x / 1.01-1.02x); the 4-thread wall clock is 1.72-1.88 s on medium against x264's 2.15-2.17 s (main 2.16-2.31 s) and 1.48-1.49 s on live against 2.06 s (main 2.01-2.62 s).

After merging main `e20b7fe` (`0c662ea`: read-ahead during B runs, pin7 CABAC round), in the benchmark lane (two sessions per binary), full_motion medium takes 1.69-1.71x / 1.48-1.49x less CPU time and 1.62-1.64x / 1.35-1.40x less wall clock than x264 veryfast on 1 / 4 threads (main `e20b7fe` 1.43-1.45x / 1.25-1.26x and 1.37-1.39x / 1.09-1.10x), live 1.88-1.93x / 1.60-1.66x and 1.80-1.85x / 1.44-1.50x (main 1.58-1.61x / 1.34-1.36x and 1.52-1.54x / 1.19-1.21x); the 4-thread wall clock is 1.51-1.53 s on medium against x264's 2.04-2.15 s (main 1.91-2.01 s) and 1.42-1.53 s on live against 2.13-2.20 s (main 1.71-1.81 s). Peak RSS at 1080p on 1 / 2 / 4 threads (90 frames, CRF 26): full_motion live 59 / 95 / 98 MiB (main 58 / 72 / 73), medium 94 / 126 / 130 (main 82 / 96 / 98); the read-ahead's three extra raw frames add 9-10 MiB on 2 and 4 threads to both, and 38 MiB at 4K (live on 4 threads 353, main 268).

Recommended settings: the defaults (three adaptive B-frames with the pyramid on fast, medium and slow) wherever the reorder delay is acceptable (exports, uploads, the media server and render farm); live also defaults to three adaptive B-frames (no pyramid, 2 frames of lookahead), with `--zerolatency` for no reorder delay and the least memory.

OCR (text-region character error, 3 frames per clip, `bench.py ocr`): the text AQ lowers character errors on scrolling text by about a third at equal bitrate; other clips are within noise.

Lookahead (10 frames, on by default except live), VMAF BD-rate against the encoder without it (busy_ui, code_editor, full_motion, slides, text_scroll; mean): fast −1.6/−5.0/−1.3/−12.6/−4.5 (−5.0%), medium −1.7/−5.0/−0.7/−12.5/−4.5 (−4.9%), slow −1.4/−5.1/−1.7/−14.9/−4.9 (−5.6%). Against x264 veryfast, medium goes from +4.8% to +2.3% and slow from +6.8% to +2.0%. Single-thread instructions −3% to +4% per clip, 10 frames of delay, +31 MB at 1080p; OCR character error is unchanged on text_scroll and lower on code_editor. At equal CRF video gets fewer bits and long-lived screen content more (full_motion CRF 20 13.3 → 7.4 Mbit/s, text_scroll CRF 26 +13%). With 250-frame GOPs full_motion is still about 2% worse than without lookahead while the screen clips gain 3-18%.

Lookahead with adaptive B-frames (the fast, medium and slow defaults since `9bce84f`; DECISIONS.md): every picture is coded with the lookahead map of its own display index, B-frames take the anchor's macroblock QP + 6, and an anchor gains lifetime from the B-frames that predict from it through list 1. VMAF BD-rate against main `9bce84f` (B-frames, no lookahead) of the same preset (busy_ui, code_editor, full_motion, slides, text_scroll; mean): fast −1.6/−5.0/−2.8/−12.6/−4.5 (−5.3%), medium −1.7/−5.0/−2.5/−12.7/−4.5 (−5.3%), slow −1.4/−5.1/−1.8/−15.0/−4.9 (−5.6%); without the list-1 credit medium full_motion is −0.7%. Against x264 veryfast the mean goes from +1.5% to −3.4% (fast), from −0.9% to −5.6% (medium) and from −0.4% to −5.4% (slow). webcam_overlay gains 8.9-11.6%, all of it from main's CRF 20 point (about +2% without it). Single-thread CPU at CRF 26 (best of 5, interleaved, loaded machine) over the six clips: fast +2.8%, medium +1.4%, slow +2.9%; callgrind instructions for medium −2.4/−2.0/+1.4/−0.6/+4.1% (+1.3%). The 5-10% on the shortest screen clips (0.4-1.2 s encodes) is about 8,000 more page faults from first touching the lookahead buffers plus noise; their user time is −3% to +3.5%. Up to 13 frames of output delay (10 lookahead + 3 B-frames), +31 MB at 1080p. OCR character error: busy_ui 3.96 → 4.00, code_editor 4.77 → 4.54, text_scroll 2.02 → 2.03; on slides 0.54 → 0.79 comes from icon and bullet glyphs and source misreads (the P-only lookahead gives 0.80) with equal or higher PSNR in the text regions. At equal CRF full_motion gets 23-29% fewer bits (2-9 VMAF points below main) and the screen clips 4-14% more; holding the mean offset at or below zero keeps main's quality at every CRF but costs 11% CPU at equal CRF (DECISIONS.md).

Lookahead round 3 (2026-10-08, standard tier, six clips; DECISIONS.md): with more than one thread the read-ahead thread runs the propagation for the next frame (main-thread instructions -9.1% full_motion, -29.7% slides at 4 threads); the change map without side info is vectorised (media-server busy_ui -40% instructions); without B-frames, content that a video frame (at least half the macroblocks changed) replaces is assumed to decay geometrically to the horizon. fast, medium, slow, live and `--lookahead 0` are byte-identical to main. The media-server settings (medium, `--keyint 250 --noise 3 --no-sideinfo`) now gain on every clip against `--lookahead 0` (full_motion -1.6%, mean -7.5%); with `--bframes 0` full_motion is still +2.4% (+2.9% before the geometric tail). SATD, half-pel and i16 intra costs in the lookahead gained at most 0.9% (P-only full_motion) for +2.7 to +14% instructions. Live with a short lookahead (VMAF BD-rate against live; instructions at CRF 26 against live; peak memory at 1080p, one thread):

| `--preset live` | busy_ui | code_editor | full_motion | slides | text_scroll | webcam_overlay | mean | vs x264 veryfast | instructions | delay at 30 fps | memory |
|---|---|---|---|---|---|---|---|---|---|---|---|
| `--lookahead 0` (default) | | | | | | | | -0.4% | | 0 | 41-52 MB |
| `--lookahead 1` | -3.4 | -3.3 | +1.9 | -16.5 | -4.0 | -9.8 | -5.9% | -3.8% | -5.3% | 33 ms | +3 MB |
| `--lookahead 2` | -3.3 | -3.5 | -1.0 | -15.2 | -4.1 | -10.9 | -6.3% | -4.5% | -4.9% | 67 ms | +6 MB |
| `--lookahead 4` | -3.3 | -3.9 | -1.2 | -14.4 | -4.1 | -12.6 | -6.6% | -4.6% | -4.5% | 133 ms | +12 MB |

Live reads 2 frames ahead by default since `b0d85fb` (67 ms at 30 fps); `--zerolatency` keeps 0. OCR character error at depth 2: text_scroll 1.81 -> 1.41%, slides 0.48 -> 0.45%, code_editor 4.70 -> 4.80%, busy_ui 3.68 -> 4.33% with equal text-region PSNR (-0.29 to +0.25 dB at matched bitrate); the busy_ui rise is icon glyphs Tesseract fuses onto words, and the word-level `ocr_word_cer` stays at or below 0.7% for every clip from ~400 kbps. Medium at depth 6 or 4 saves 12 or 19 MB for +0.06% mean but loses 0.54 or 0.78% on full_motion; the ring of raw frames is the encoder's input delay, so the memory stays.

Lookahead round 4 (2026-10-08, DECISIONS.md): the media server's settings (medium, `--keyint 250 --noise 3`, no side info) spent most of their time in the encoder's scalar per-macroblock change test. Change detection is now one vector pass per frame, two macroblocks per vector row with an exact-equality test first, and the encoder reuses the lookahead's map wherever its source copy is current. Every preset default and `--lookahead 0` is byte-identical to main `52510d7`, and after merging main `7189fa4` to it (270/270 gate runs identical to the pre-merge branch; instructions against `7189fa4` -83.1/-84.3/-88.4/-1.0/-73.0%). At the media-server settings against main `52510d7` (CPU is user + system time, best of 5 interleaved runs on a loaded 4-core VM; full_motion at one thread is the median of 10 paired runs, whose best-of-5 moved between -3 and +4%):

| | busy_ui | code_editor | slides | full_motion | webcam_overlay |
|---|---|---|---|---|---|
| instructions, 60 frames, 1 thread | -83.6% | -84.7% | -88.9% | -1.0% | -73.2% |
| CPU, 300 frames, 1 thread | -62.6% | -66.1% | -40.1% | -0.6% | -32.1% |
| CPU, 300 frames, default threads | -61.1% | -58.8% | -36.2% | | -31.7% |

P-only GOPs longer than 2 s (`--bframes 0 --keyint 250`) end content that a video frame replaces at the window edge: -0.76% mean VMAF BD-rate (slides -2.78, webcam_overlay -1.24, full_motion -0.50%, no clip worse), full_motion +1.67% against `--lookahead 0` (was +2.37%). The busy_ui OCR difference under the live lookahead is icon glyphs that OCR reads as characters, in mostly unchanging text, with higher text PSNR and a sign that flips between CRFs and regions; capping the lookahead's QP raise on text macroblocks does not change it and gives up a fifth to a third of the BD-rate gain, so live keeps reading 2 frames ahead as measured (its default since `b0d85fb`).

Single-thread speed at 1080p (live preset, warm page cache): busy_ui ~590 fps (a third of it is the kernel copying the raw input file; a library integration would not pay that), code_editor ~450, text_scroll ~140, slides ~160, full_motion ~21.5. x264 veryfast on this VM: ~110-145 fps on one core, ~150-370 on four.

Intra coding at equal QP (all-intra, AQ off): capcodec frames are 2.5% larger than x264 veryfast's with luma PSNR within 0.1 dB; chroma is ~1 dB lower because x264 uses chroma_qp_index_offset −2 (VMAF ignores chroma).

Change-proportional cost (slow_typing, earlier measurement): static frame 0.18% of a full frame with side info; 1.7% with the pixel-difference fallback.

## Speed rounds (profile-driven; each round worked on the top stage)

| Round | Top stage | Change | Effect |
|---|---|---|---|
| 1-4 | ME, intra | sub-pel prediction, cached half-pel tiles, intra early exits, table-driven directional intra | full-change frame 265 → 125 ms |
| 5 | entropy (IDR) | byte-oriented CABAC | IDR entropy 31 → 14 ms |
| 6 | mode + tq (IDR) | IDR macroblock reuse | IDR ~95 → ~45 ms on typing/editor/UI clips (79-99% of macroblocks reused) |
| 7 | bookkeeping | one MV per macroblock, untouched skip state | 10-20% on motion-heavy clips |
| 8 | deblock | batched edges, flat-line skip | busy_ui deblock 38 → 27 ms total |
| 9 | ME/mode (video) | SIMD SAD/SATD/SA8D/half-pel/qpel | full_motion 10.3 → 16.1 fps |
| 10 | tq (video) | SIMD 4x4 transforms and quantisation | 16.1 → 18.6 fps |
| 11 | analysis | SIMD classification, row hashing, copies | 19.8 → 21.5 fps |

Current full-motion profile (live, per frame): tq 16, motion 14, analysis 9.5, mode 6.7, deblock 3.9, entropy 2.8 ms.

## Research applied

- OpenAI math preprints (`~/work/openai-math`, 722 papers): the algorithmic results (DFT in O(n (log n)^(1−10⁻¹³)), complex matrix multiplication below 2.258, integer multiplication below n log n, finite tensor savings for Fourier circuits) are asymptotic and do not reduce work at H.264's 4x4/8x8 integer butterflies, which are already at minimal add counts; the bottleneck is vector throughput and memory traffic. Recorded in DECISIONS.md.
- Applied from video-coding practice: hash/profile-based global motion (screen content coding), byte-oriented CABAC (x264/FFmpeg style), weighted prediction for fades, x264-style decimation and fast skip, SIMD kernels. Tried and reverted: EPZS-style cursor and dominant-motion predictors for every macroblock (no gain on the corpus, slides +0.7%).

## Tov

`encoder-features` branch (all committed): binary fs I/O, `--unchecked`, typed arrays, SIMD vectors, threads and Atomics, borrowed typed-array field arguments (`976bb78`), loop hoisting of array fields (`dee1ec3`), module init order (`d8a0f81`), `Math.clz32`/`imul` (`eced999`), C library output `tov build --lib` (`5cb7538`: `libname.a/.so` + `name.h`, typed arrays borrowed zero-copy, opaque class handles, `name_last_error`), function-value adapters (`7b58f21`), `--target wasm32-wasi` for programs and libraries (`3bee8b1`; a thread-free capcodec build is byte-identical to native at 1.86x the CPU per frame). The installed compiler is `encoder-features` 20ea68f (from `~/work/tov-pin7`, installed 2026-10-09 07:50; capcodec builds byte-identically to 538446e; adds horizontal `min`/`max`/`minIndex`/`maxIndex` (`phminposuw`), vector records returned in registers, `mulHigh`/`mulWideLow`/`mulWideHigh`, `@inline`/`@noinline`, Buffer-style `readUInt32LE`/`writeUInt32BE`/... on `Uint8Array`; 538446e stays at `/usr/local/cargo/bin/tov.pin6-538446e`). Previously 538446e (from `~/work/tov-pin6`, installed 2026-10-08 23:50; against 64982fa: capcodec unchecked 620,816 -> 550,024 bytes, 0.8-1.8% fewer instructions, clean builds compiled as 16 parallel C units, `Math.select`, run-time `T.shuffle`, `pushFrom`, `Math.idiv`, `Target.vectorBits`, narrowing stores, `discard`; byte-identical encodes; 64982fa stays at `/usr/local/cargo/bin/tov.pin5-64982fa`). Previously 64982fa (from `~/work/tov-pin5`; 2.9-5.2% fewer capcodec instructions than 548ff6b, byte-identical output, multi-target `--target-cpu x86-64-v3,x86-64-v4` binaries, `--profile-generate`, module constants as C constants, smaller binaries): element borrows folded in (72b793b), serial threads on wasm (3b00bba), 512-bit vectors and concat/low/high, register-resident sums, mask/ctz/popcount, branch-free `?:`, borrowed typed-array locals, integer-quotient rounding, Atomics memory orders, Int64Array and 64-bit lanes, Channel, parallelFor options; capcodec builds byte-identically with it. See TOV_GAPS.md.

## How to resume

```sh
export PATH=$HOME/tools/bin:$PATH          # static ffmpeg 8.1 (libx264, libopenh264, libvmaf), JM ldecod
cd ~/work/tov-pin2 && git log --oneline -1 && cargo +1.88.0 install --path crates/tov   # elem-borrow 02a302a on top of encoder-features
cd ~/work/capcodec && bench/build_encoder.sh
python3 bench/bench.py smoke                 # < 60 s, regression gate
python3 bench/bench.py standard --encoders "capcodec-live,capcodec-medium,x264-veryfast"
python3 tools/profile.py full_motion busy_ui --presets live,medium     # per-stage times
python3 tools/verify.py slides --binary build/capcodec-unchecked --preset medium   # JM + ffmpeg bit-exact check
```

Backups (git bundles of capcodec and the Tov branch, PROGRESS.md) are written to `/cursor/stores/self/backups` by `tools/backup.sh`.

## Next steps

1. Parallel entropy coding for IDR frames (a few row-band slices, about +0.5% bits) and row-parallel deblocking: IDR frames are now the serial bottleneck on typing/editor/UI clips.
2. Full-motion efficiency: live preset is +68% BD vs medium's +39% without B-frames; stronger motion search where the frame is video-like, SIMD 8x8 transforms, intra in P frames. B-frames next steps: B sub-partitions (16x8, 8x16, 8x8 and direct 8x8), lookahead to choose the B run length per scene (and to credit the middle B-frame of a pyramid run), a second reference per list, and entropy coding of B rows as they finish so concurrent B-frames need a few rows of coefficients instead of a frame.
3. Screen-content BD-rate: tune I/P offset, refresh, lifetime AQ and AQ strength with harness variants; transient-content QP for animations.
4. C ABI build for crates/export, WASM build for the browser export, FINDINGS.md.
5. Lookahead: choose the B run length and reference structure from its lifetimes (the changed-macroblock count alone lost 5% on pulldown, DECISIONS.md); an opt-in quality-anchored CRF (the one-sided offset clamp) where CRF has to mean the same quality on video; P-only long GOPs, where full_motion still loses 1.7% (a per-macroblock tail from how long each block's motion has lasted, or intra refresh of the decayed content); a changed-macroblock ring sharing the encoder's buffers.
6. Media-server CPU after the change-detection round: CABAC (163M of busy_ui's 558M instructions over 60 frames, about 70M of it the first IDR frame), the change map (70M, nearly all of it the lookahead's pass, which already runs on the read-ahead thread with more than one thread), intra decisions (41M, mostly the IDR frame), and letting B-run anchors take the lookahead's map (full_motion: 3 of 17 anchors can today).