lib/mjs-files.mjs
Rama main · réplica de lectura del repositorio original.
/**
* mjs-files.mjs — the one definition of "every .mjs file in this repository".
*
* Two things syntax-check this repo: `scripts/check-syntax.mjs` (run as
* `npm run lint`) and section 1 of `test-all.mjs`. They used to disagree about
* what "every file" meant, and only one of them said so.
*
* `test-all.mjs` read the repository root with a NON-recursive `readdirSync`,
* so its gate covered 121 of the ~575 `.mjs` files here — and it printed one
* `{file} syntax OK` line per file, so a reader watching 121 green lines had no
* way to tell that the directory holding 263 of them was never opened. Worse,
* the gate NARROWED every time a file moved out of the root and never
* mentioned it: #3306 moved eleven suites into `tests/` and #3388 moved nine,
* and each one silently left the gate. The shortfall was eventually noticed
* only as a two-check arithmetic discrepancy in an unrelated PR (#3411), which
* is not a way to find out (#3419).
*
* Sharing the walker is the fix rather than copying the recursion into
* `test-all.mjs`: two independently maintained definitions of the same set is
* exactly the drift that caused this, and a second copy would be free to
* re-diverge the next time one of them learned about a directory.
*
* Two exports, answering two DIFFERENT questions — which is not the drift
* above, and the distinction is worth stating so nobody collapses them:
*
* `collectMjsFiles` — every `.mjs` in the WORKING TREE, skip-list and all.
* What the syntax gates want: the file you just wrote and
* have not `git add`ed yet still has to parse.
* `trackedFiles` — every file the INDEX holds, at any depth, no skip-list.
* What the repo-scanning guards want: they grade what the
* repository ships, and git already knows which trees are
* not that. Four guards each kept their own walk with
* their own skip-list until #3890; every one of them
* carried the same silent narrowing as the gate above.
*/
import { execFileSync } from 'child_process';
import { lstatSync } from 'fs';
import { join } from 'path';
import { walkTree } from './walk-tree.mjs';
// Re-exported so the historical import site keeps working; `lib/walk-tree.mjs`
// owns the predicate now, because the walk that consults it lives there.
export { isNestedCheckout } from './walk-tree.mjs';
/**
* Every file `root` TRACKS, as absolute paths, sorted.
*
* The enumerator for repo-scanning guards, as distinct from `collectMjsFiles`
* below, which walks the working tree. The two answer different questions on
* purpose and the difference is not drift: a syntax gate wants the file you
* just wrote and have not added yet, and a guard grading what the repository
* SHIPS wants the index — which is also the only definition that needs no
* skip-list. `node_modules`, `output/`, `data/`, a gitignored worktree and a
* half-finished scratch file are all excluded because git already knows they
* are not repository content, not because four separate walkers each remembered
* to name them (#3890).
*
* Every hand-maintained walker in this repo carried the #3419 defect in some
* form: it scanned directories it should not, it missed newly tracked files
* under a directory its skip-list excluded, and a green run looked identical
* either way. A skip-list narrows silently. An index does not.
*
* THE INDEX, THOUGH — NOT THE COMMIT, AND NOT THE WORKING TREE. The sharp edge
* of grading what the repository ships is that a file you have written but not
* `git add`ed is not yet part of what it ships, so it does not exist to any
* guard built on this: the new suite that skips its own `main` guard, the new
* mode file carrying mojibake, the new scanner with a private walk. The guard
* does not fail on it and does not mention it — it reports a clean sweep of the
* files that WERE staged, which is the exact shape of green-that-checked-less
* this module was written to remove, arriving through the fix rather than the
* bug. It is the deliberate trade and not a defect to route around: the
* alternative is the skip-list, and the failure lands one `git add` later
* rather than never. But it means a pre-commit run and a CI run legitimately
* disagree, and a contributor debugging that discrepancy should find the reason
* written down here rather than infer it from `ls-files`. Stage first, then
* trust the guard. `collectMjsFiles` is the export that does not care, which is
* why the syntax gates use it.
*
* `filter` receives the repo-relative path with FORWARD slashes on every
* platform (`ls-files` emits POSIX separators), so `rel.startsWith('modes/')`
* is written once rather than per-OS. It is applied before the lstat below, so
* a caller scoping to a subtree does not pay a stat for the whole tree.
*
* Only regular files come back: a symlink, a submodule directory and a path the
* index names but the tree does not have are each skipped with a warning that
* says which. See the comment on the loop for why each one is not readable.
*
* @param {string} root - Absolute path to a git working tree.
* @param {(rel: string) => boolean} [filter] - Tested against the relative path.
* @returns {string[]} Absolute paths, lexicographically sorted.
*/
/**
* Stat failures that mean "this indexed path is not in the working tree", and
* are therefore skipped rather than fatal. `throwIfNoEntry: false` already
* suppresses ENOENT; these two are the same condition arriving by a different
* errno. Every other code is a filesystem this scan could not read, and it ends
* the scan — see the catch below.
*/
const UNSTATTABLE_SKIP_CODES = new Set(['ENOTDIR', 'EACCES']);
export function trackedFiles(root, filter = () => true) {
// What is graded is the INDEX, not the working tree: a file that exists on
// disk but has not been `git add`ed is invisible to every guard routed through
// here. CI never sees that case, because it checks out a commit where
// everything is staged by definition — so the contributor who hits it is
// always running locally, and `git add` is the fix.
//
// Deliberately uncaught. A `git` that cannot run, or a `root` that is not a
// working tree, is a scan that did not happen — and the one thing this module
// exists to prevent is a scan that did not happen reading as a scan that
// passed. The same reasoning as the missing-root throw in collectMjsFiles.
const out = execFileSync('git', ['-C', root, 'ls-files', '-z'], {
encoding: 'utf-8',
maxBuffer: 64 * 1024 * 1024,
stdio: ['ignore', 'pipe', 'pipe'],
});
// De-duplicated because `ls-files` prints one line PER INDEX STAGE, so a path
// in an unresolved merge arrives two or three times. Left alone, every hit
// inside a conflicted file would be reported once per stage and every
// "n files scanned" figure would be wrong for the duration of the merge.
const listed = [...new Set(out.split('\0').filter(Boolean))];
// A repository-local `core.hooksPath` is not consulted here: this shells out to
// `git ls-files`, which reads the index and runs no hook.
// Empty is never "nothing to check" — it means this call could not look.
// `ls-files` exits 0 with no output when run from an untracked directory,
// which is exactly how the SYSTEM_PATHS coverage guard printed
// "0 tracked files covered" and exited 0 for its entire life while the tree
// really did have an unregistered file in it. Callers filter afterwards, so
// this is the LISTING being empty, not the caller's slice of it.
if (listed.length === 0) {
throw new Error(
`git ls-files listed no tracked files under ${root} — this scan could not inspect anything. ` +
'An empty listing usually means the caller is running from an untracked directory ' +
'(a temp copy, a fixture dir), where git reports nothing and the guard is meaningless.',
);
}
// Only REGULAR FILES are returned, and `lstatSync` rather than `existsSync`
// is what decides that. The index names three things a caller cannot simply
// open, and the walkers being replaced here excluded all three for free —
// `readdirSync` told them what each entry was:
//
// MISSING Mid-merge the index holds stages for a path the working tree
// may not have: a delete/modify conflict whose deletion is the
// resolution being accepted, still unstaged, is a `DU` path
// that `ls-files` names and `readFileSync` cannot open. An
// ordinary state during conflict resolution, not a corrupt
// repository — a sparse checkout and an uninitialised
// submodule land here too. A guard that crashes while you are
// resolving a merge is a guard people learn to skip, which
// costs more than the guard was worth (#3890).
// SYMLINK `ls-files` lists tracked symlinks (this repo has seven: the
// per-CLI `SKILL.md` links to the canonical one), and
// `existsSync`/`readFileSync` follow them. Following costs
// twice: the target is usually tracked in its own right, so
// one offender gets reported once per link, and a link
// pointing outside the checkout puts source this repository
// does not own under a guard that grades it — the #3499 hazard
// `isNestedCheckout` exists for. `collectMjsFiles` below skips
// them for exactly this reason; so does this.
// DIRECTORY An initialised submodule is a gitlink: one index entry, a
// directory on disk. `existsSync` says yes and `readFileSync`
// throws EISDIR.
//
// UNSTATTABLE `throwIfNoEntry: false` suppresses ENOENT only. On POSIX the
// index naming `a/b` while the tree has `a` as a regular file
// throws ENOTDIR, and an unreadable parent directory throws
// EACCES. Both are the MISSING case wearing a different errno,
// so both are skipped rather than allowed to end the scan.
// Windows reports the first of those as ERROR_PATH_NOT_FOUND,
// which libuv maps to ENOENT, so there it arrives through the
// MISSING branch below instead (#3904). Same outcome either
// way, which is why the branches share one warning.
//
// Skipped LOUDLY in every case, and named with its reason — a scan quietly
// covering less than it claims is the defect this module is named after.
const files = [];
const skipped = [];
for (const rel of listed) {
if (!filter(rel)) continue;
let stat;
try {
stat = lstatSync(join(root, rel), { throwIfNoEntry: false });
} catch (err) {
// `throwIfNoEntry: false` suppresses ENOENT and nothing else, so the
// UNSTATTABLE case above still arrives as a throw. Named with the errno,
// since the reason is the errno.
//
// Only the codes that mean "not in the working tree" are survivable. An
// unrecognised failure — EIO on a failing disk, ENAMETOOLONG, anything
// else — is a path this scan could not read, and skipping it would turn
// a scan that could not look into a scan that found nothing. That is the
// defect this module is named after, so it ends the scan instead.
if (!UNSTATTABLE_SKIP_CODES.has(err.code)) throw err;
skipped.push(`${rel} (${err.code})`);
continue;
}
if (stat?.isFile()) {
files.push(join(root, rel));
continue;
}
const why = stat === undefined ? 'not in the working tree'
: stat.isSymbolicLink() ? 'a symlink'
: 'not a regular file';
skipped.push(`${rel} (${why})`);
}
if (skipped.length > 0) {
console.warn(
`warning: skipped ${skipped.length} tracked path(s) that are not readable regular files ` +
`(unresolved merge, sparse checkout, submodule, symlink): ${skipped.join(', ')}`,
);
}
return files.sort();
}
/**
* Directories excluded from the walk BY NAME.
*
* `.git` and `node_modules` are not repository source. `output`, `data`,
* `coverage` and `test-results` hold generated or user content, so including
* them would make the result depend on what a given checkout happens to have
* run — a clean clone and a working install would disagree about how many
* files were checked, and a stray `.mjs` dropped in `output/` could fail the
* lint of a repository whose source is fine.
*
* A name is not enough to keep git's storage out on its own — see
* `isNestedCheckout` in `lib/walk-tree.mjs`, which the walk consults on every
* descent whether a caller remembers it or not.
*/
export const SKIP_DIRS = new Set(['.git', 'node_modules', 'output', 'data', 'coverage', 'test-results']);
/**
* Every `.mjs` file under `root`, recursively, sorted by full path.
*
* Sorted because both callers report per-file results in iteration order and a
* run-to-run reordering of that output is noise in a diff — the readdir order
* is not guaranteed across platforms or filesystems.
*
* @param {string} root - Absolute path to walk.
* @returns {string[]} Absolute paths, lexicographically sorted.
*/
export function collectMjsFiles(root) {
return walkTree(root, {
skip: (entry) => SKIP_DIRS.has(entry.name),
// Symlinked directories can point outside the checkout or back into it;
// neither should make the walk recurse unpredictably. (walkTree's default,
// named here because this walk's result is a gate's file list.)
links: 'skip',
// A scratch copy left by an interrupted `test-all.mjs` (no `.git`, named
// unpredictably) is skipped by walkTree itself, beside the nested-checkout
// guard, so every walker inherits it rather than rediscovering #3940.
// Below the root, ENOENT is a race with readdir — a concurrent checkout, a
// branch switch, a test tearing down a temp tree. At the ROOT it throws,
// which is walkTree's contract and the one this gate needs: an empty list
// would make the syntax check report "0 .mjs files" and pass (#3419).
onError: 'ignore',
})
.filter((f) => f.endsWith('.mjs'))
// Sorted by FULL PATH, not left in walkTree's per-level entry order: both
// callers report one line per file in iteration order, and a run-to-run
// reordering of that output is noise in a diff.
.sort();
}