specs/082-watch.md
Rama main · réplica de lectura del repositorio original.
# Spec: Watched Folders
## Status
Implemented.
Conformance covers the DAG definition. The scheduler's reading of folders
and the runs it starts are covered by scheduler unit tests and the queue
integration tests.
## Scope
This spec defines the DAG-level `watch` block, which starts a run for each
file that arrives in a folder. It does not define file system events,
recursive watching, or what the run does with the file.
## Goal
Workflow authors can hand a folder to a workflow: a file dropped there is
processed once, by one run, soon after it is complete, without a schedule
that polls the folder and leaves empty runs behind.
## Behavior
- `watch.path` is the folder watched and is required. A leading `~` is the
home directory; a relative path is resolved against the scheduler's
working directory.
- `watch.pattern` is a glob the file's name must match, as `filepath.Match`
takes it, such as `*.pdf`. Empty, or `*`, matches every file. Files whose
names start with `.` or `~$`, files with an in-progress extension
(`.tmp`, `.part`, `.partial`, `.crdownload`, `.download`), folders, and
files that are not regular never match.
- `watch.interval` is how often the folder is read, as a duration such as
`30s`. It defaults to 10s and must be at least 1s.
- A DAG with `watch` needs no `schedule`; both may be set.
- The scheduler reads the folder every interval. A file counts as arrived
once its size and modification time are the same on two readings in a
row, so a file still being copied waits. Each arrival starts one run,
queued like a webhook's, with the runtime params `WATCH_FILE`, the file's
absolute path, and `WATCH_NAME`, its name. The run's trigger type is
`file`.
- The scheduler remembers each file by its size and modification time, so
a file runs once. A file replaced with other content runs again; a file
that went away and came back runs again. A file that is removed is
forgotten.
- A run is named after the file's version (`file-` and a hash of the DAG,
the path, the size, and the time), so a file queued again after the
record was lost, as when the scheduler stopped between queuing the run
and saving the record, is the run the queue already holds.
- `WATCH_FILE` and `WATCH_NAME` are chosen by whoever put the file in the
folder: a `run:` step reads them with `printenv` or through
`template.render` rather than interpolating them into its command.
- The files a folder holds when it is first watched, or when `path` or
`pattern` change, are remembered without running: a watch takes what
arrives, not what was there. A file that arrives while the scheduler is
off is new when it returns.
- A suspended DAG starts no runs; its files wait and run once it resumes.
A file whose run could not be queued is tried again at the next reading.
- A folder that cannot be read is reported once in the scheduler log, and
read again every interval until it can be.
- The record of watched folders is kept with the scheduler's state and
survives restarts. A DAG the scheduler leaves out for a while, as one
being edited is, keeps its record, so a file that arrives meanwhile runs
once the DAG is back; the record of a DAG that is gone for good stays.
## Errors
`dagu validate` rejects a `watch` without `path`, a `pattern` that is not a
valid glob, an `interval` that is not a duration, and an `interval` under
1s. `watch.files` and other fields are rejected by the schema.
## Example
```yaml
watch:
path: ~/Downloads/invoices
pattern: "*.pdf"
interval: 30s
steps:
- id: read
action: chat.completion
with:
prompt: Read the invoice and give its number, date, and total.
files: ${WATCH_FILE}
output_schema:
type: object
properties:
number: {type: string}
date: {type: string}
total: {type: number}
required: [number]
- id: file
depends: read
action: file.move
with:
source: ${WATCH_FILE}
destination: ~/Documents/invoices/${steps.read.outputs.number}.pdf
```