docs/adr/0003-pipeline-stage-separation.md
Rama master · réplica de lectura del repositorio original.
# ADR-0003: Pipeline Stage Separation
## Metadata
| Field | Value |
|-------|-------|
| **Status** | Accepted (retrospective) |
| **Date** | 2026-06-08 |
| **Authors** | AlessioBrillo |
| **Deciders** | AlessioBrillo |
| **Supersedes** | N/A |
| **Relates to** | ADR-0001, ADR-0002, ADR-0006, ADR-0010 |
| **Project** | DOMINUS |
## Context
The core workflow of DOMINUS is a pipeline that transforms raw candidate
inputs (keywords, closeout CSV rows, brandable names) into structured buy
recommendations. Each stage in the pipeline performs a distinct operation:
generating candidates, filtering by DNS availability, confirming via RDAP,
scoring, and checking trademarks.
From the outset it was clear that these stages must:
1. **Run in strict order** — each stage depends on the output of the
previous. DNS-filtered names feed RDAP confirmation; RDAP-confirmed names
feed scoring; scored names feed the trademark gate. Running out of order
wastes provider calls or produces invalid results.
2. **Be independently testable** — a change to the trademark matching
algorithm should not require running DNS lookups to verify.
3. **Be independently replaceable** — swapping the RDAP provider from a
public HTTP endpoint to a WHOIS port-43 server should not affect the
scoring stage.
4. **Produce auditable per-stage metrics** — the operator needs to know how
many candidates were filtered at each stage and how long each stage took.
The pipeline orchestrator is the only component that coordinates stage
execution. Individual stages never call each other.
## Decision Drivers
1. **Sequential dependency** — each stage filters the candidate set. The next
stage's input is a strict subset of the previous stage's output. Running
stages out of order would waste compute (scoring before DNS filtering is
pointless) or produce invalid results (trademark check before registration
check would waste rate-limited API calls on domains that may already be
registered).
2. **Testability** — each stage must be testable in isolation with mocked
providers. Without stage separation, tests would need to set up the entire
pipeline even for a small change to one signal.
3. **Provider swap** — a stage's internal implementation (Node `dns` vs.
`dns-prefetch` library, RDAP HTTP vs. WHOIS port-43) must be swappable
without changing the stage's interface or the orchestrator.
4. **Transparency** — the operator must be able to see, for each pipeline
run, how many candidates entered stage N, how many passed, and how long
it took. This is essential for debugging (why did only 3 of 100 candidates
reach the scoring engine?) and for performance optimisation (which stage is
the bottleneck?).
## Considered Options
### Option A: Strict Stage Interface + Orchestrator (CHOSEN)
Each stage implements a generic `Stage<I, O>` interface with a single
`process(inputs: I[]): StageResult<O>` method. The orchestrator calls stages
sequentially, threading the `passed` array of each stage into the next.
A `StageResult` contains three fields:
- `passed`: candidates that passed the stage and move to the next.
- `filtered`: candidates filtered by this stage (with reason).
- `stageName` + `durationMs`: metrics for the summary.
**Advantages:**
- Maximum testability: each stage can be instantiated with mock providers and
tested in a single `describe` block without setting up unrelated stages.
- Maximum replaceability: swapping `DnsPreFilterStage`'s internal DNS provider
requires changing exactly one constructor argument.
- The orchestrator is a simple sequential loop — five calls, no branching, no
state machine. It is easy to reason about and test.
- Each pipeline run produces a structured `stageSummary` with per-stage
metrics, which is persisted to `pipeline_runs` (ADR-0011).
**Disadvantages:**
- The generic `Stage<I, O>` interface requires careful type parameter
management as stages evolve. The `ScoringStage` produces `ScoredCandidate`
(with `scoreResult`), while earlier stages produce `DomainCandidate`
(without `scoreResult`). The orchestrator must track these types.
- Adding a new stage requires: (a) implementing the interface, (b) threading
it through the orchestrator constructor, and (c) connecting its output to
the next stage's input. This is more boilerplate than a single monolithic
function.
**Cost Implications:** Zero monetary cost. ~40 hours to design and implement
the interface, the orchestrator, and the five stages.
**Risk Assessment:** Low. The pattern is well-established in data engineering
and ETL pipelines.
---
### Option B: Monolithic Pipeline Function
A single `runPipeline(input)` function that contains all five stages in
sequence, each calling providers and scoring functions directly.
**Advantages:**
- Less code: no interface, no type parameter threading, no `StageResult` type.
- Faster to write initially.
**Disadvantages:**
- Impossibile to test stages in isolation without running the entire pipeline.
- Impossible to swap providers without editing the monolithic function.
- Per-stage metrics would need to be manually instrumented.
- Adding a new stage means editing the monolithic function, increasing the
risk of breaking unrelated stages.
- The function would grow to 200+ lines with mixed concerns (DNS lookups,
RDAP calls, scoring computations, trademark checks).
**Cost Implications:** Zero monetary cost. Cheaper initially, more expensive
over time as the codebase grows.
**Risk Assessment:** Medium-high. The monolithic function becomes a
maintenance bottleneck as the pipeline evolves. Testing requires full
integration setup for every change.
---
### Option C: Pub/Sub Event-Driven Pipeline
Stages communicate via an event bus: each stage subscribes to the previous
stage's events and publishes its own. The orchestrator is replaced by a
message broker (or an in-process EventEmitter).
**Advantages:**
- Loose coupling: stages can be added, removed, or reordered without changing
the orchestrator.
- Natural parallelism: stages could theoretically process candidates
concurrently (though DOMINUS does not need this).
**Disadvantages:**
- Dramatically harder to reason about: the candidate flow is no longer a
linear sequence; it is a graph of event handlers.
- Testing requires setting up the event bus and subscribing test listeners.
- Error handling is distributed: a failure in one stage must propagate back
to the orchestrator through events, not through a simple try/catch.
- Massive over-engineering for a 5-stage sequential pipeline with a single
operator.
**Cost Implications:** Zero monetary cost. Significant engineering overhead
for no benefit at DOMINUS scale.
**Risk Assessment:** Low technical risk, but high complexity for no benefit.
The wrong architectural choice.
---
## Decision
**Chosen option: Option A — Strict Stage Interface + Orchestrator**
The rationale is driven by the decision drivers:
1. **Sequential dependency**: The orchestrator calls five stages in strict
order. Each stage receives the previous stage's `passed` array, ensuring
that no candidate skips a stage and no stage runs before its input is
ready. The dependency chain is explicit in the constructor.
2. **Testability**: Each stage is independently testable. The orchestrator
itself is testable with mock stages. This separation means a change to the
trademark matching algorithm (ADR-0012) requires only `trademark-gate-stage`
and `match-detector` tests — not DNS, RDAP, or scoring tests.
3. **Provider swap**: The `DnsPreFilterStage` takes a `DnsProvider` interface
in its constructor; the `RdapConfirmationStage` takes an `RdapProvider` and
a `WhoisProvider`. Swapping from the public RDAP provider to a WHOIS-only
provider requires changing exactly one argument, not editing business logic.
4. **Transparency**: Every `StageResult` carries `passed`, `filtered`,
`stageName`, and `durationMs`. The orchestrator aggregates these into a
`stageSummary` that is persisted to `pipeline_runs` (ADR-0011). The
operator can see, for example, that the DNS pre-filter eliminated 80% of
candidates in 2.3 seconds.
Option B (monolithic) was rejected because testability and replaceability are
paramount for a system whose core asset (the scoring engine) must be tuned
independently of the availability-checking infrastructure. Option C
(pub/sub) was rejected because it adds complexity without providing any
benefit at DOMINUS scale.
## Consequences
### Positive
- Each stage is a single file implementing a simple interface. New stages can
be added without touching existing stages.
- The orchestrator is 86 lines with no branching logic — trivial to audit.
- Testing: the orchestrator test creates mock stages and verifies the data
flow. Stage tests create real stages with mock providers.
- The `stageSummary` provides full visibility into pipeline performance.
### Negative
- Type parameter management: the orchestrator's constructor lists five stage
parameters with different types. Adding a sixth stage requires adding a
sixth parameter.
- Stage results carry `domain: string` identifiers that must match across
stages. A stage that modifies the domain string would break the pipeline.
(Mitigation: stages never modify the domain field.)
- The `scored` array in the `PipelineResult` is assembled from three sources
(`scoring.filtered`, `trademark.passed`, `trademark.filtered`), which is
a subtle data-joining convention. Newcomers to the codebase may miss this.
### Compliance and Security Implications
- The pipeline orchestrator has no external network access — all provider
calls happen inside the individual stages. The orchestrator is a pure
coordinator.
- No sensitive data passes through the orchestrator (domains are public
strings). No data protection concern.
### Migration and Monitoring Plan
- **Migration**: None. This ADR documents the existing design.
- **Monitoring**: The `stageSummary` is persisted to `pipeline_runs` for 180
days (ADR-0011). The operator can inspect
`dominus runs show <runId>` to see per-stage metrics.
- **Rollback**: Stage interface changes are backwards-compatible: adding a new
stage does not break existing stages.
### Validation
- The orchestrator test (`pipeline/__tests__/orchestrator.test.ts`) validates
all five stages run in order, DNS-filtered names do not reach scoring,
TM-blocked names appear in `scored[]`, and error verdicts do not recommend.
- Each stage has its own test file with 4-8 tests covering happy path, edge
cases, and error paths.
- Production validation: `dominus run --closeout-csv <file>` produces a
`stageSummary` with all five stages populated.
---
*This ADR was created following the MADR 4.0.0 standard. All DOMINUS ADRs
should be consistent with the product vision previously documented in
`dominus-product-vision.md` (v0.2), now superseded by this ADR series.
Template: `docs/adr/template.md`.*