docs/TESTING.md
Rama main · réplica de lectura del repositorio original.
# Testing Guide
This document provides comprehensive information about testing in the Actual-sync project.
---
## 📊 Overview
The project uses **Jest** as the testing framework with comprehensive unit and integration tests covering:
- Configuration loading and validation
- Retry logic with exponential backoff
- Sync service integration
- Startup validation
- Health check endpoints
- Notification services
- Prometheus metrics
- Telegram bot commands
- Sync history tracking
**Current test count and coverage**: see the live **Tests** and **Coverage** badges at the top of the [README](../README.md) (regenerated each release from the Jest run). Enforced thresholds: 70% lines/functions/statements, 61% branches.
---
## 🚀 Quick Start
### Running Tests
```bash
# Run all tests
npm test
# Run tests in watch mode (re-runs on file changes)
npm run test:watch
# Run tests with coverage report
npm run test:coverage
# Prove the tests would actually catch the bugs they were written for
npm run test:mutation
```
### Mutation testing
`npm run test:mutation` reintroduces the original defect behind each shipped fix, one at a time, and asserts the suite **fails**. A mutation that survives is a fix with no test guarding it — a green suite that would not notice the bug coming back.
```bash
npm run test:mutation # every mutation, full suite each (~5 min)
npm run test:mutation -- --fast # only the hinted test file per mutation (~1 min)
npm run test:mutation -- --ticket '#177' # one ticket's mutations
npm run test:mutation -- --list # what is covered, without running anything
npm run test:mutation -- --recover # restore after a hard kill
```
`mutationCatalog.test.js` is excluded from the runs. It asserts every anchor still exists, and a mutant replaces its anchor — so leaving it in made every mutation "caught" by construction, with the runner scoring its own self-check and unable to ever report a survivor. That bug shipped in the first version of this tool and is the reason the exclusion is pinned by a test.
It exists because three consecutive review rounds on a single change each found a fix that **no test protected**: source-text assertions that passed against a fully reintroduced bug, a parity regex satisfied by a leftover `require`, and a heuristic whose branch could be deleted with the suite still green. Reading a test and judging it plausible does not answer *"would this catch the bug?"* — only reintroducing the bug does.
The catalog covers behaviour at every level, not just service code — a mutation reintroducing the README's old "notifies you of failures" claim is caught by the doc-drift guard, and one relaxing the email `required: [from, to]` schema rule is caught by the config tests. If a guard cannot be broken by a mutation, it is not guarding anything.
**When you fix a bug, add a mutation for it.** The catalog lives in `scripts/mutations.js`; each entry names the file, the exact `anchor` text to replace, and the `mutant` that reintroduces the defect. `src/__tests__/mutationCatalog.test.js` runs in the normal suite and fails if any anchor no longer matches its file — without it, a refactor would silently turn mutations into no-ops and the catalog would rot into false confidence.
It is deliberately **not** wired into CI: it runs the whole suite once per mutation, which is far too slow per-PR. Run it when you change notification dispatch, config validation, or anything else the catalog covers, and before a release.
**It refuses to run without a green baseline.** The suite is run unmutated first, and the run aborts unless that baseline is genuinely clean: no failed tests, no suites that failed to *load*, and jest's own `success` flag set. Otherwise every mutation would score "caught" for a reason that has nothing to do with the mutation — a colleague's WIP test or a flake would produce a perfect score.
**It baselines what it is about to score.** In `--fast` mode each mutation is scored against its hinted test file alone, so each distinct hint is baselined alone — not the full suite. A full-suite baseline would not rule out the case that matters here: a file that passes as part of the suite but fails *standalone*, through leaked global state or an ordering dependency on a sibling file. That would fail under every mutant and score all of them `caught`, having proven nothing. Each distinct pattern is baselined once, so this is cheaper than the single full-suite baseline it replaced, not more expensive.
The load-error case is separate on purpose. A suite that throws while being imported contributes **zero failed tests** to jest's report while `numTotalTests` stays non-zero, so a "did tests run, did any fail" reading called it green. With one unresolvable import the runner printed `green (841 tests)` while `npm test` exited 1, and a mutant then scored a false `SURVIVED` for a defect that *was* guarded — the exact phantom coverage gap this tool exists to rule out.
**Scoring reads jest's JSON report, not its exit code.** jest exits 1 for "no tests found" and for a config error as well as for a real failure, so an exit-code reading counted those as caught having run nothing. A mutation is caught only when tests actually ran *and* at least one failed. A run with a load error, or one jest calls unsuccessful while reporting no failures, is `UNSCORED` rather than a verdict — the run fails, but it does not claim a coverage gap it has not demonstrated.
**The baseline is a single sample.** It rules out a suite that is already broken; it cannot rule out a genuinely flaky test, which would fail under some mutant and be recorded as `caught` for the wrong reason. If a mutation's result surprises you, re-run that ticket with `--ticket` before believing it.
**Safety.** Before a file is touched, both its original content and the mutant are journalled to disk atomically, so a hard kill is recoverable with `--recover`. `--recover` refuses to overwrite content that is neither the mutant nor the original — that is how it would otherwise destroy a repair you made by hand — and copies your version aside first. A pid lockfile blocks concurrent runs, and is only ever removed by the process that took it.
`--recover` also **refuses while a live run holds the lock**. Recovering mid-flight restores the original underneath the suite currently testing the mutant, so that suite passes against unmutated code and the mutation is scored `SURVIVED` — the tool reporting a coverage gap it just manufactured. `--recover` is dispatched before the catalog is loaded, so a broken `scripts/mutations.js` cannot block recovery of a file an earlier run left mutated.
Signal handlers are deliberately **not** installed. `main()` is synchronous, so a handler could never fire while jest is running; registering one only removed the default terminate behaviour, leaving a runner that `kill` could not stop. The journal is the recovery mechanism — if a run is killed, the next one refuses to start and tells you to run `--recover`.
One honest limitation: the uncommitted-changes check is a snapshot taken at startup, so a file created *during* a run is not covered by it. A file *modified* during a run is detected — the runner leaves your version alone and stops, rather than reverting it. That includes the case where the file was put **back to its original content** while the mutant was in place, which earlier read as the benign "the mutant write failed" path and said nothing.
**The runner mutates itself too.** `scripts/mutationTest.js` is in the catalog, which works because jest re-reads the file from disk while the running process keeps its own copy in the require cache. Those mutations are guarded by `src/__tests__/mutationRunner.test.js` and deliberately **not** by `mutationCatalog.test.js` — the runner excludes the catalog guard from the scored suite, so a guard placed there would score every one of them as `SURVIVED` regardless of the truth.
**It is not available inside the container.** `scripts/mutationTest.js` and `scripts/mutations.js` are excluded via `.dockerignore` (#180), so `npm run test:mutation` only works in a checkout. A tool whose job is rewriting source files in place has no business in a production image. `scripts/validateConfig.js` is *not* excluded — `npm run validate-config` in the container is the documented pre-upgrade check (#177) — so `scripts/` cannot be ignored wholesale, and a test in `docDriftGuards.test.js` pins both halves of that.
Exit codes: `0` all caught, `1` at least one survived (or a stale anchor / unscored run), `2` the runner refused to start or failed internally, `3` a file was left mutated and needs `--recover`. `1` is a claim about coverage, so an internal crash must never exit `1`.
### Documented config examples
`src/__tests__/configExamplesGuard.test.js` validates every fenced `json` config block in `README.md`, `docs/NOTIFICATIONS.md`, `docs/CONFIG.md`, `docs/MIGRATION.md` and `docs/DOCKER_DEPLOYMENT.md` against the real schema. A config snippet in the docs is advice people paste, so a wrong one is worse than no advice — this caught three invalid documented examples on its first run, each of which would have caused a startup failure.
If a documented JSON block is genuinely **not** configuration — an API response, a webhook payload — opt it out with a marker on the line before the fence:
```markdown
<!-- config-guard: skip -->
```
The marker applies only to the block immediately following it.
### Test Output
```
Test Suites: all passing
Tests: all passing # live count + coverage are on the README badges
Snapshots: 0 total
Time: ~8 s
```
---
## 📁 Test Structure
```
src/
├── __tests__/
│ ├── helpers/
│ │ └── testHelpers.js # Shared test utilities
│ ├── configLoader.test.js # ConfigLoader unit tests
│ ├── retryLogic.test.js # Retry logic unit tests
│ ├── syncService.test.js # Sync service integration tests
│ └── startupValidation.test.js # Startup validation tests
└── lib/
└── configLoader.js # Module under test
```
---
## 🧪 Test Suites
### 1. ConfigLoader Tests (`configLoader.test.js`)
Tests configuration loading, validation, and error handling:
- **Constructor**: Default and custom path handling
- **load()**: File existence, JSON parsing, schema validation
- **validateConfig()**: Schema validation with AJV
- **validateLogic()**: Business rules (duplicates, required fields, security warnings)
- **getConfig()**: Configuration retrieval
- **getServer()**: Server lookup by name
- **getServers()**: All servers retrieval
**Example Test**:
```javascript
test('should throw error for duplicate server names', () => {
const config = {
servers: [
{ name: 'Test', url: 'https://test1.com', ... },
{ name: 'Test', url: 'https://test2.com', ... }
]
};
const loader = new ConfigLoader(configPath);
expect(() => loader.validateLogic(config))
.toThrow('Duplicate server names');
});
```
### 2. Retry Logic Tests (`retryLogic.test.js`)
Tests exponential backoff and retry behavior:
- **Success scenarios**: First attempt, eventual success
- **Rate limit handling**: Exponential backoff, max retries
- **Network error handling**: ECONNRESET, ENOTFOUND, network-failure
- **Non-retryable errors**: Immediate failure
- **Exponential backoff**: Delay calculation validation
- **Edge cases**: Zero retries, null/undefined returns
**Example Test**:
```javascript
test('should retry with exponential backoff on rate limit', async () => {
let attempts = 0;
const rateLimitFn = jest.fn().mockImplementation(() => {
attempts++;
if (attempts < 3) {
const error = new Error('Rate limit exceeded');
error.code = 'NORDIGEN_ERROR';
error.category = 'RATE_LIMIT_EXCEEDED';
throw error;
}
return Promise.resolve('success');
});
const result = await runWithRetries(rateLimitFn);
expect(result).toBe('success');
expect(rateLimitFn).toHaveBeenCalledTimes(3);
});
```
### 3. Sync Service Integration Tests (`syncService.test.js`)
Tests end-to-end sync workflow with mocked Actual API:
- **Full workflow**: Init → Download → Sync → Bank Sync → Shutdown
- **Connection failures**: Authentication errors, network issues
- **Budget handling**: Download failures, sync failures
- **Account handling**: Empty lists, individual account failures
- **Cleanup**: Shutdown called even on errors
**Example Test**:
```javascript
test('should complete full sync workflow successfully', async () => {
const server = {
name: 'Test Server',
url: 'https://test.example.com',
password: 'test-password',
syncId: 'test-sync-id',
dataDir: '/tmp/test-data'
};
await syncBank(server);
expect(actual.init).toHaveBeenCalled();
expect(actual.downloadBudget).toHaveBeenCalled();
expect(actual.sync).toHaveBeenCalled();
expect(actual.runBankSync).toHaveBeenCalledTimes(2);
expect(actual.shutdown).toHaveBeenCalled();
});
```
### 4. Startup Validation Tests (`startupValidation.test.js`)
Tests startup environment validation:
- **Node.js version**: Version detection and validation
- **Configuration directory**: Existence checks
- **Configuration file**: Existence, readability, JSON parsing
- **Dependencies**: node_modules and critical packages
- **Schema file**: Optional schema detection
- **Integration**: Complete valid setup workflow
**Example Test**:
```javascript
test('should detect invalid JSON in config.json', () => {
const configFile = path.join(configDir, 'config.json');
fs.writeFileSync(configFile, '{invalid json}');
expect(() => {
const content = fs.readFileSync(configFile, 'utf8');
JSON.parse(content);
}).toThrow();
});
```
---
## 🌐 Browser E2E Tests (#263)
The unit suite above mocks the Actual API and the filesystem; it never renders
the dashboard's HTML/JS or drives a real browser. `tests/e2e/` covers that gap
with Puppeteer against real backing services (a real `HealthCheckService` and
a real SQLite-backed `SyncHistoryService`), so it is a separate suite with its
own Jest config rather than another `describe()` block in the unit tests.
### Running It
```bash
npm run test:e2e
```
This must go through the npm script, not a bare `npx jest --config jest.e2e.config.js`:
Puppeteer ships ESM-only and is loaded with a dynamic `import()`, which Jest's
CommonJS runtime refuses unless Node itself starts with
`--experimental-vm-modules`; the npm script sets that flag.
### How a spec is structured
- **Fixture states** (`tests/e2e/fixtures/states/*.js`) seed a scenario -
healthy, degraded, a longer history with a known failure count, and so on.
Each state is just a module with a name string and some seed data, so a new
scenario is a new small file, not a change to the harness itself.
- **The fixture server** (`tests/e2e/fixtures/index.js`) starts a real
`HealthCheckService` on an OS-assigned port (`port: 0`) with a real
`SyncHistoryService` on a temp SQLite file, seeded from the chosen state.
Only `syncBank`, `getServers`, `getCronSchedules` and the notification
service are stubbed - everything else is the real dashboard code and real
routes.
- **The browser fixture** (`tests/e2e/fixtures/browser.js`) launches one
shared headless Chrome for the whole run and exposes `withFixture` /
`withDashboard` helpers. Every page it opens has a fixed viewport, UTC
timezone, reduced motion, and a frozen clock (`@sinonjs/fake-timers`
installed before any dashboard script runs, sharing the exact same "now"
the fixture server uses) - so a screenshot or an assertion never depends on
wall-clock time.
- **Network lockdown**: every request is intercepted. Only same-origin
requests to the fixture server are allowed through (the dashboard serves
Chart.js itself from its npm dependency, #244, so nothing is faked);
everything else, including the browser's own `/favicon.ico` request on every
navigation, is answered locally or blocked, so a spec can never reach the
real network or silently consume its own rate-limit budget on a request it
didn't intend to make.
- **On failure**, the harness saves a screenshot and the page's console log
under `tests/e2e/artifacts/` (gitignored) instead of failing silently.
### What is covered
Login, the general 60-requests-per-minute rate limiter (the separate login
failure throttle from #246 has no E2E spec), manual sync triggering and error dismissal,
and the Overview, History, Analytics, Settings and Accounts tabs - each
against a fixture state built for that scenario. See the spec files under
`tests/e2e/*.e2e.js` for the exact assertions.
### The screenshot generator uses the same harness
`scripts/generateDashboardScreenshots.js` (`npm run screenshots`) renders the
documentation screenshots in `docs/screenshots/` from `tests/e2e/screenshots.json`
using this same fixture harness - there is no separate mocked code path to keep
in sync, and no running service to start first. See `docs/screenshots/README.md`
for the manifest format.
---
## 🛠️ Test Helpers
Located in `src/__tests__/helpers/testHelpers.js`:
### Available Utilities
```javascript
// Create temporary test directory
const tempDir = createTempDir();
// Clean up after tests
cleanupTempDir(tempDir);
// Create mock configuration object
const config = createMockConfig({ servers: [...] });
// Create mock Actual API instance
const api = createMockActualAPI();
// Create test configuration file
const configPath = createTestConfigFile(dir, config);
// Create test schema file
const schemaPath = createTestSchemaFile(dir);
// Suppress console output during tests
const { restore } = suppressConsole();
// Wait for specified time
await wait(1000); // 1 second
```
### Example Usage
```javascript
describe('My Test Suite', () => {
let tempDir;
beforeEach(() => {
tempDir = createTempDir();
});
afterEach(() => {
cleanupTempDir(tempDir);
});
test('my test', () => {
const config = createMockConfig();
const configPath = createTestConfigFile(tempDir, config);
// ... test code
});
});
```
---
## 📈 Coverage Reports
### Viewing Coverage
After running `npm run test:coverage`, open the HTML report:
```bash
# Generate coverage report
npm run test:coverage
# Open in browser (macOS)
open coverage/lcov-report/index.html
# Open in browser (Linux)
xdg-open coverage/lcov-report/index.html
# Open in browser (Windows)
start coverage/lcov-report/index.html
```
### Coverage Thresholds
Configured in `package.json`:
```json
"coverageThreshold": {
"global": {
"branches": 70,
"functions": 70,
"lines": 70,
"statements": 70
}
}
```
**Current coverage**: run `npm run test:coverage` for the authoritative per-file
report, or see the live **Coverage** badge on the [README](../README.md). The
enforced minimums are the thresholds shown above (70% lines/functions/statements,
61% branches).
### Excluded from Coverage
The following files are excluded as they are integration/orchestration code:
- `index.js` - Entry point
- `src/syncService.js` - Main service orchestration
- `src/__tests__/**` - Test files
---
## ✍️ Writing Tests
### Test File Naming
- Unit tests: `<module>.test.js`
- Integration tests: `<feature>.test.js`
- Place in `src/__tests__/` directory
### Test Structure
```javascript
const { testHelper } = require('./helpers/testHelpers');
describe('Feature Name', () => {
beforeEach(() => {
// Setup before each test
});
afterEach(() => {
// Cleanup after each test
});
describe('Sub-feature', () => {
test('should do something specific', () => {
// Arrange
const input = setupInput();
// Act
const result = performAction(input);
// Assert
expect(result).toBe(expected);
});
});
});
```
### Best Practices
1. **Descriptive names**: Use clear, descriptive test names
```javascript
// Good
test('should throw error for duplicate server names', () => {});
// Bad
test('duplicates', () => {});
```
2. **Arrange-Act-Assert**: Structure tests clearly
```javascript
test('example', () => {
// Arrange - Set up test data
const input = { value: 42 };
// Act - Execute the code
const result = processInput(input);
// Assert - Verify the outcome
expect(result).toBe(84);
});
```
3. **Isolation**: Each test should be independent
```javascript
// Use beforeEach/afterEach for setup/cleanup
beforeEach(() => {
tempDir = createTempDir();
});
afterEach(() => {
cleanupTempDir(tempDir);
});
```
4. **Mock external dependencies**: Use Jest mocks
```javascript
jest.mock('@actual-app/api', () => ({
init: jest.fn().mockResolvedValue(undefined),
sync: jest.fn().mockResolvedValue(undefined)
}));
```
5. **Test edge cases**: Cover error paths
```javascript
test('should handle empty input', () => {});
test('should handle null input', () => {});
test('should throw on invalid input', () => {});
```
---
## 🐛 Debugging Tests
### Run Single Test File
```bash
npm test -- configLoader.test.js
```
### Run Tests Matching Pattern
```bash
npm test -- --testNamePattern="retry"
```
### Run with Verbose Output
```bash
npm test -- --verbose
```
### Debug with VS Code
Add to `.vscode/launch.json`:
```json
{
"type": "node",
"request": "launch",
"name": "Jest Debug",
"program": "${workspaceFolder}/node_modules/.bin/jest",
"args": ["--runInBand", "--no-cache"],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen"
}
```
---
## 🔄 Continuous Integration
### GitHub Actions Example
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: '18'
- run: npm ci
- run: npm test
- run: npm run test:coverage
- name: Upload coverage
uses: codecov/codecov-action@v3
```
---
## 📚 Additional Resources
- [Jest Documentation](https://jestjs.io/docs/getting-started)
- [Jest Matchers](https://jestjs.io/docs/expect)
- [Jest Mock Functions](https://jestjs.io/docs/mock-functions)
- [Testing Best Practices](https://github.com/goldbergyoni/javascript-testing-best-practices)
---
## 🤝 Contributing
When adding new features:
1. Write tests first (TDD) or alongside implementation
2. Ensure tests pass: `npm test`
3. If the change touches the dashboard's rendered UI, also run the browser
E2E suite: `npm run test:e2e`
4. Check coverage: `npm run test:coverage`
5. Maintain >70% coverage threshold
6. Update this documentation if adding new test patterns
---
**Last Updated**: December 5, 2025