docs/test-duplication.md
Rama main · réplica de lectura del repositorio original.
# Duplication limits for the test helpers: what each step down costs
This document records the duplication limits for the test helper trees, the
reason each limit holds, and the cost of the remaining steps.
**Decided and shipped.** The reusable helpers under `test/test-utils` are
scanned at **40 tokens**, alongside `src/`, by `.jscpd.helpers.json`. The
Cucumber support helpers under `test/specs/support` are scanned twice: at **18
tokens** alongside `src/`, and at **40 tokens** on their own by
`.jscpd.support.json`. The 18-token scan holds its accepted clones in the
committed baseline `.jscpd.specs-baseline.json`, the way the renamed-clone gate
does. See [What is left](#what-is-left) for the remaining steps.
All counts come from commit `ebc7c12` and from jscpd 5.0.12 with `minLines: 1`,
the setting every config in this repository uses. A count is the number of clone
pairs the scan reports, not the number of places to edit: one merge usually
clears several pairs.
## The six scans, and why there are six
The 0% threshold is not the number that decides how hard jscpd looks.
`minTokens` is. It sets the shortest run of tokens that counts as a clone, so a
lower number is a tighter net.
| Config | Scans | minTokens |
| --------------------- | -------------------------------- | --------- |
| `.jscpd.json` | `src`, `e2e-payments`, `scripts` | 40 |
| `.jscpd.specs.json` | `src` + `test/specs/support` | 18 |
| `.jscpd.support.json` | `test/specs/support` | 40 |
| `.jscpd.helpers.json` | `src` + `test/test-utils` | 40 |
| `.jscpd.test.json` | `test` | 48 |
| `.jscpd.css.json` | `src/ui/static/style.scss` | 50 |
The last one holds no tests, so the rest of this document leaves it out. Its
number is high because design-token CSS repeats short declaration runs by
design, and those runs are not duplication.
The code scans ignore two kinds of file that this document does not cover
elsewhere. `fp.ts` holds curried adapters, and those match each other by design.
The migration files are append-only history: each one repeats the same
`schemaMigration(...)` call scaffold, and a shipped migration must never change.
Each migration body is unique, so the scaffold is the factory, not a missed
merge.
Two trees hold reusable helpers. `test/specs/support` holds 76 files and 11,673
lines behind the Cucumber steps. `test/test-utils` holds 160 helper files and
19,164 lines, plus 32 files of tests for those helpers. Both are
production-shaped code: one named helper for each thing a person, or a test,
does. Two helpers that nearly match are a merge to make.
A test body is different. It repeats by design, and the shared mechanism is the
test framework itself. That is why the whole of `test/` sits at 48 tokens, and
why the helper trees can be held tighter than the tree that contains them.
## The support helpers held at 15 tokens
The shared scan `.jscpd.specs.json` reads `src/` alongside the support helpers,
so a helper that reimplements production logic is flagged against the source it
copied. `src/` is clean at 19 and cannot go below it:
| minTokens | Clone pairs in `src/` alone |
| --------- | --------------------------- |
| 19 | 0 |
| 17 | 497 |
| 16 | 961 |
So a 0% threshold cannot hold _that_ config below 19. The support helpers
therefore get a second scan of their own, `.jscpd.support.json`, which drops
`src/` and holds them at 15. **18 is where the shared config sits, not where
the helpers have to stay.**
### The 18-token shared scan carries a baseline
Below 19 tokens the shared scan reads real signal and coincidence together.
Dropping `.jscpd.specs.json` to 18 tokens surfaces 220 clone pairs. Most are
the coincidence classes this document already names: import blocks, matching
signatures and parameter lists, type aliases, schema table data, and support
test bodies. The scan cannot hold those at zero without a full-band
classification, so the config carries a committed baseline instead. `deno task
cpd` runs the specs config with `--baseline .jscpd.specs-baseline.json
--fail-on-new-clones`, exactly the way the renamed-clone gate does.
`deno task cpd:specs --update` rewrites the baseline after a merge. A new
18-token clone anywhere in the scanned trees fails the gate.
The first pass merged the pairs that were real duplicated logic:
- `order-gallery.tsx` routes both the listing card and the package card
through one `renderSelectableCard` helper. They differed only in the option
key, the form-field prefix, and the text.
- `api-site-pages-input.ts` names the shared "longer than the field's cap"
refusal `tooLong`, and the slug, the page name, and the text fields all
route through it.
- `validated-session.ts` names the repeated
`Pick<ValidatedPaymentSession, "id" | "paymentReference" | "provider">`
parameter slice `SessionPaymentReference`, so the two charge-reference
readers no longer repeat the type.
The remaining 217 pairs sit in the baseline. They are import-block and
signature matches, one-line field and schema builders, and the support test
bodies. Each is a separate merge when its band is classified.
Read a whole band before you lower a limit. A small sample can call a band
noise while the full band marks an under-adopted curry. At 16 tokens, a sample
of eight pairs read the band as noise. A classification of all 78 pairs found
46 merges behind a curry that `browser.ts` already holds. The other 32 pairs
were 5 pairs that share only an assertion helper and 27 pairs that match on the
signature and the types alone.
### The named step types in `world.ts`
Two unrelated bodies behind the same `(world: TicketsWorld, name: string)`
parameter list are the honest exception. The repetition is the parameter list
itself, and a named type removes it. `ActOnOneThing`, `ReadAboutOneThing`,
`AsksAboutOneThing`, and `ActOnTheStory` in `world.ts` name the common
parameter lists. `ActsOnPersonsBooking` and the `StoryJourney`-derived shapes
cover the booking and messaging signatures.
### The one helper for each repeated thing
- `opensAdminPageAt` and `withAdminPage` in `browser.ts` are the curries for
the admin-page opens. `withAdminPage` carries a result back.
- `createAttendeeOn` in `test/test-utils/db-helpers/attendees.ts` is the one
`createTestAttendee` call on a listing. The note, deposit, and editor
journeys route through it.
- `askListingsApi` in `booking-api.ts` is the one "read an API listing
endpoint, expect a 200" open.
- `sendWithTheCode` in `discount-codes.ts` is the one promo-code send.
- `requireGroup` in `browser.ts` is the one "hand back the matched group or
fail loudly" guard.
- `regionOutOf` in `browser.ts` is the one region-slice. `articleOutOf`,
`footerHtml`, and `proseHtml` all route through it.
- `outgoingWatch` in `outgoing.ts` is the one "the watch a story put in
place" accessor.
- `emailFor` is the one address rule, and `ownerWrites` is the one email
wording helper. The bulk-email stories use both.
- `theBooking` and `theListing` in `world.ts` are the one booking and listing
accessors.
- The two steps that take a thing off sale are one step. Both stories use the
site's confirmed deactivate form, the way the catalogue step always did.
Two helpers keep their own shape, and say why in a comment. `deposits.ts` joins
a person's names with a hyphen, so its address is a different address, not the
same one written twice. `asksNotToHearAboutPromotions` has to serve the choices
page before it can name the button on it, because serving the page is what loads
that group of copy.
## The other helper tree holds at 40 tokens
`.jscpd.helpers.json` reads `test/test-utils` against `src/` at 40 tokens. The
golden test database is built by `fullSchemaCreateStatements()` from
`src/shared/db/migrations/schema-sync.ts`, the one production schema builder. A
helper that reaches the app goes through `awaitTestRequest` or `testPageHtml`
in `test/test-utils/mocks.ts`.
## What is left
These counts are for the helper files and `src/` at the commit above:
| minTokens | Clone pairs | Note |
| --------- | ----------- | ------------------------------------------ |
| 40 | 0 | where the scan is set |
| 36 | 17 | strong signal, mostly whole shared helpers |
| 32 | 31 | |
| 28 | 67 | |
| 24 | 113 | |
| 19 | 280 | parity with the Cucumber helpers |
**These are counts, not verdicts.** A sample of the 24-token band showed valibot
schema fields that match across unrelated schemas, which is the same coincidence
that exempts `src/shared/db/migrations/schema/columns.ts` from `.jscpd.json`,
and one-line field builders that match on a signature. A sample is not a
decision. Classify a whole band, and try the curry, before you call any of it
unavoidable.
Each step down is a separate job. Take the number in `.jscpd.helpers.json` down,
bring the tree to it, and repeat, the way the comment caps in
`scripts/check-comments/run.ts` come down.
## The renamed-clone gate: same code with different words
jscpd matches literal token runs, so a rename hides a copy: every renamed word
breaks the run, and two spellings of one operation sit below `minTokens` 19. The
worked example was the slug-index family — `computeGroupSlugIndex`,
`computeNewsSlugIndex`, `computeSitePageSlugIndex`, and `computeSlugIndex` were
four spellings of `(slug) => hmacHash(slug)`, each one identifier short of
detection. All four were deleted and every caller now uses `hmacHash` directly.
Since jscpd 5.2.0 the gate is jscpd's own. `deno task cpd:renamed` runs
`.jscpd.renamed.json`, which passes `--similarity 0.99`: whole functions,
methods and arrow functions are compared by AST shape, with names and literals
left out, so a renamed copy scores 1.0 and is reported with kind `similar`. The
accepted matches live in the committed baseline
`.jscpd.renamed-baseline.json`. The flag `--fail-on-new-clones` fails the build
on a pair the baseline does not carry, and `deno task cpd:renamed --update`
rewrites the baseline after a merge. The baseline records no written reason —
the merge history carries it.
Since jscpd 5.4.2 the strict scans (`.jscpd.json`, `.jscpd.test.json`,
`.jscpd.helpers.json`, `.jscpd.support.json`) enable the Type-2 clone options
`ignoreIdentifiers`, `ignoreLiterals` and `ignoreAnnotations`. jscpd classifies
each matched window. A window whose raw tokens differ only in names, literal
values or annotations is a `renamed` clone. A window with identical raw tokens
is an `exact` clone. The strict scans report both kinds. An earlier
`kind: ["exact"]` filter removed every `renamed` window, so the strict scans
reported zero and did not check renamed copies at their `minTokens`.
With names and literals ignored, a short window matches common code shapes:
two valibot objects with three fields, or two small `for` loops. At 19 tokens
the main scan found 24,773 clones. A new module of 140 lines made 30 new
clones against unrelated code, and none of them had a helper to share. At 30
tokens the same module made none. The main and support scans run at 40
tokens, the floor of the helpers scan, so a renamed match must be a full
operation and not a common shape.
At these limits, the strict scans found these clones:
| Scan | minTokens | Clones |
| --------------------- | --------: | -----: |
| `.jscpd.json` | 40 | 1,152 |
| `.jscpd.test.json` | 48 | 12,202 |
| `.jscpd.helpers.json` | 40 | 852 |
| `.jscpd.support.json` | 40 | 17 |
Each scan holds these clones in its committed baseline and runs with
`--fail-on-new-clones`. A new renamed copy fails the gate. The fingerprints do
not change when lines move. A code change can pair a recorded window with a
different identical copy. That pair has a new fingerprint, so the gate fails
on code that it already recorded. If the new clone is a copy that exists on
`main`, record it with `--update`. The added and removed counts must match.
After a merge, run the scan with `--update` to remove the fixed clones.
Each TypeScript scan skips import and re-export statements through its
`ignorePattern` list. Source files carry no ignore markers around imports.
With literals ignored, two long lists of strings match, for example two word
lists of the band-name generator. A list of strings is data, and no helper can
share it. Each declared list that holds only strings is wrapped in
`/* jscpd:ignore-start -- a list of strings, not code */` and
`/* jscpd:ignore-end */`. The rule does not cover SQL text, such as the check
rules in `charges.ts`, or the message group lists inside the `area(...)` calls
in `area-loaders.ts`. Markers inside a call cut the long matches into many
short matches of the call shape.
After each scan, `scripts/cpd.ts` names the worst duplication in the report.
jscpd reports pairs, so the pairs that share a place make one family. The
worst family appears in the most places. If two families appear in the same
number of places, the family with more tokens is worse. Start the next merge
there.
A pair changes status one way: merge it (extract a helper, or curry the parts
that differ), then refresh the baseline. A new renamed copy anywhere in the
scanned trees fails the gate, which is the point: the slug-index class of drift
cannot regrow silently.
**The baseline shrinks as merges land** — merge a family, run
`deno task cpd:renamed --update`, and its fingerprints drop out. The larger
families at the time of writing are the field-picking map pairs across the admin
pages and the table `getXById` wrappers, which are the coincidence class:
enough code builds an object from another object's fields for unrelated
functions to match, and the baseline absorbs them while the real merges
proceed.
## What was measured and rejected
**A tighter scan for the whole of `test/`.** The tree is clean at 48 tokens and
very expensive below it:
| minTokens | Clone pairs in `test/` |
| --------- | ---------------------- |
| 48 | 0 |
| 44 | 291 |
| 40 | 873 |
| 36 | 1,749 |
| 32 | 3,385 |
The rise is test bodies, not helpers, so a lower number here buys little and
costs a great deal. 48 stays.
**A carve-out for `test/test-utils` in `.jscpd.test.json`.** The specs config
takes `test/specs/support` out of the ordinary `test/` scan. That hides any
clone between a support helper and a test body, and today the count of those is
zero, so nothing is lost. `test/test-utils` is not carved out: it stays in both
scans, so its clones against test bodies are still caught at 48. The second scan
of 19,000 lines costs about one second.