README_SCHEMA.md
Rama main · réplica de lectura del repositorio original.
# Dagu Workflow Schema at a Glance
This is the repository-level overview of Dagu's workflow YAML schema. It is
meant to help a visitor understand how a workflow is shaped, what the current
canonical syntax is, and where to look when they need the full field-level
reference.
Dagu workflows are DAGs described in YAML. The current schema has a simple
execution split:
- `run:` for local shell commands and scripts
- `action:` for named builtin actions, custom actions, or versioned remote action packages
- `actions:` for reusable custom action definitions
The older v1 execution fields (`command:`, `script:`, step-level `type:`,
`call:`, and `step_types:`) are still loadable for compatibility, but new
workflows should use the syntax below.
## Minimal Workflow
```yaml
name: release-check
type: graph
params:
ENVIRONMENT: staging
steps:
- id: test
run: go test ./...
- id: health
action: http.request
with:
method: GET
url: https://example.com/health
depends: [test]
- id: deploy
run: ./deploy.sh ${ENVIRONMENT}
depends: [health]
retry_policy:
limit: 3
interval_sec: 10
```
The root `type:` controls how the workflow executes:
- `graph` runs steps according to `depends:` and can run independent steps in
parallel.
- `graph` is the default when `type:` is omitted.
- `chain` runs steps in order.
- `agent` creates an Agent DAG: `steps:` becomes a catalog of actions and the configured
LLM chooses which one runs next until every entry in `tasks:` is complete. It
requires `llm:` and `tasks:`, and forbids `depends:`.
Do not confuse root `type:` with legacy step-level `type:`. Step-level
`type:` is deprecated; use `action:` for named executors.
Do not use scalar step shorthand such as `- echo hello`. It is deprecated;
write explicit step objects with `run:` instead.
## Mental Model
A Dagu file has three layers:
| Layer | Fields | Purpose |
|-------|--------|---------|
| Workflow metadata | `name`, `description`, `group`, `labels` | Identify and organize the DAG. |
| Workflow runtime | `schedule`, `params`, `env`, `working_dir`, `shell`, `shell_args`, `queue`, `worker_selector`, `timeout_sec`, `max_active_runs`, `artifacts`, `log_output` | Configure when and where the DAG runs. |
| Step graph | `steps`, `handler_on`, `defaults`, `actions` | Define executable work, shared step defaults, lifecycle handlers, and custom actions. |
Most workflows only need `name`, `type`, `params`, and `steps`.
## Step Shape
Every step has common workflow-control fields plus one execution field.
```yaml
steps:
- id: step_id
run: echo hello
depends: [previous_step]
env:
- LOG_LEVEL: info
timeout_sec: 300
retry_policy:
limit: 2
interval_sec: 5
output: RESULT
```
Common step fields:
| Field | Meaning |
|-------|---------|
| `id` | Stable identifier for dependencies and output references. Prefer this on every step. |
| `name` | Optional display name. |
| `description` | Step description. |
| `depends` | Step dependency or dependency list. |
| `env` | Step environment variables. |
| `working_dir` | Step working directory. |
| `timeout_sec` | Step timeout in seconds. |
| `retry_policy` | Retry behavior for this step. |
| `repeat_policy` | Repeat or polling behavior. |
| `continue_on` | Continue after selected failure, skip, exit-code, or output conditions. |
| `preconditions` | Conditions that must pass before the step starts. An unmet condition skips the step without failing the run; see [Failing on Unmet Conditions](https://docs.dagu.sh/writing-workflows/control-flow#failing-on-unmet-conditions). |
| `worker_selector` | Required worker labels. Keys and values support variable substitution (e.g. `${VAR}`); parallel sub-DAG steps can reference `${ITEM}`. |
| `stdin` | File path piped to the step command's standard input. |
| `stdout`, `stderr`, `log_output` | Step log output configuration. `stdout` can also publish DAG/action outputs. |
| `output` | Captured stdout variable or structured step-scoped output. |
| `output_schema` | JSON Schema for stdout JSON validation. |
| `approval` | Human approval gate after step execution. |
| `container` | Container context for a local `run:` command. |
## `run:` for Local Commands
Use `run:` for local shell commands and scripts.
```yaml
steps:
- id: hello
run: echo hello
```
Multi-line `run:` becomes a script:
```yaml
steps:
- id: build
run: |
set -e
npm ci
npm test
```
`run:` can also be an array of commands:
```yaml
steps:
- id: build_and_test
run:
- go build ./...
- go test ./...
```
Only shell settings are accepted in `with:` for a `run:` step:
```yaml
steps:
- id: bash_step
run: echo "$SHELL"
with:
shell: bash -e
shell_args: [-c]
shell_packages: [curl]
```
Important: object-form `run:` is not part of the current schema. This is
invalid:
```yaml
steps:
- run:
type: shell
input: echo hello
```
## `action:` for Named Execution
Use `action:` when the step is not just a local shell command.
```yaml
steps:
- id: create_job
action: http.request
with:
method: POST
url: https://api.example.com/jobs
body: '{"queue":"default"}'
```
`action:` names select builtin actions, custom actions, or versioned remote
action packages. Inputs and executor-specific options go under `with:`.
Current builtin actions:
| Action | Use for | Key inputs |
|--------|---------|------------|
| `http.request` | HTTP calls | `method`, `url`, headers/body config |
| `ssh.run` | Remote shell over SSH | `command`, SSH connection config |
| `exec` | Direct process execution without shell parsing | `command`, optional `args` |
| `docker.run` | Docker executor | optional `command`, Docker config |
| `container.run` | Container executor | optional `command`, container config |
| `k8s.run`, `kubernetes.run` | Kubernetes job execution | optional `command`, Kubernetes config |
| `postgres.query`, `sqlite.query` | SQL queries | `query`, database config |
| `postgres.import`, `sqlite.import` | SQL imports | `import`, database config |
| `clickhouse.query` | ClickHouse queries over HTTP | `url`, `query` or `file`, optional `params`, `settings` |
| `redis.<operation>` | Redis operations | Redis config; operation comes from the action suffix |
| `jq.filter` | jq transforms | `filter`, plus `data` or `input` |
| `js.run` | Sandboxed JavaScript | `script`, optional `input` or `input_file` |
| `dag.run` | Child DAG execution | `dag`, optional `params` |
| `dag.enqueue` | Asynchronous child DAG enqueue | `dag`, optional `params`, optional `queue` |
| `human.task` | Operator input before downstream steps continue | `prompt`, optional flat scalar `form`, optional `artifacts`, optional `push_back` |
| `router.route` | Conditional routing | `value`, `routes` |
| `chat.completion` | LLM chat completion | `prompt` or `messages`, model config |
| `harness.run` | CLI coding-agent harnesses | `prompt`, provider config, optional `stdin` |
| `browser.extract`, `browser.run` | Browser automation in a local Chrome | `url`, `instruction` and `schema`; or `do` operations; `llm` from the DAG or `with.llm` |
| `computer.extract`, `computer.run` | Desktop automation on macOS and Windows workers | `instruction` and `schema`; or `do` operations; `llm` from the DAG or `with.llm` |
| `template.render` | Text/template rendering | Exactly one of `template` or `template_ref`, optional data/config |
| `log.write` | Log messages | `message` |
| `mail.send` | Email sending | mail executor config |
| `mail.search` | Email search over IMAP or the Gmail API | `mailbox` and search filters |
| `mail.organize` | Email marking and moving | `mailbox`, `emails`, `mark` or `move` |
| `xlsx.read` | Workbook reading | `path`, optional `password`, `sheet`, `range`, `header`, `columns`, `types`, `where` |
| `xlsx.info`, `xlsx.list_sheets` | Workbook metadata | `path`, optional `password` |
| `xlsx.write`, `xlsx.append` | Workbook writing | `path`, `rows` or `input` |
| `xlsx.update_rows` | Workbook row updates | `path`, `rows`, `key`, optional `set` and `missing` |
| `xlsx.validate` | Workbook checks | `path`, at least one of `required`, `not_blank`, `unique`, `types`, `allowed`; optional `on_problem`, `max_problems` |
| `xlsx.write_cells` | Template filling | `path`, `cells`, optional `merge`, `output`, `sheet` |
| `xlsx.sheet` | Sheet management | `path`, `operation`, `sheet`, `to` for `copy` and `rename`, optional `if_exists`, `missing`, `position` |
| `xlsx.convert` | Workbook export | `path`, `output`, optional `format`, `encoding`, `delimiter` |
| `xlsx.extract` | Fields of a form-like sheet located by a model | `path`, `instruction`, `schema`, a model through `llm`; optional `sheet`, `range`, `send_values`, `cache` |
| `archive.create`, `archive.extract`, `archive.list` | Archive operations | archive config |
| `file.stat`, `file.read`, `file.write`, `file.copy`, `file.move`, `file.delete`, `file.mkdir`, `file.list` | File operations | path/source/destination/content config |
| `git.checkout` | Git repository checkout | `repository`, `path`, optional `ref`, `depth`, auth config |
| `wait.duration`, `wait.until`, `wait.file`, `wait.http` | Wait or poll for time, files, or HTTP readiness | `duration`, `until`, `path`, `url`, optional polling config |
| `s3.upload`, `s3.download`, `s3.list`, `s3.delete` | S3 operations | S3 config |
| `sftp.upload`, `sftp.download` | SFTP transfers | SFTP config |
| `noop` | Output-only or approval-only placeholder step | no `with`, or empty `with` |
| `owner/repo@version`, `name@version`, `source:target@version` | Remote action package | caller input object under `with:` |
`run:` and `action:` are mutually exclusive on a step. Do not combine either
with legacy execution fields such as `command:`, `script:`, step-level `type:`,
`call:`, `messages:`, `llm:`, `value:`, or `routes:`.
Remote action packages contain a `dagu-action.yaml` manifest and a DAG
entrypoint. GitHub refs such as `acme/dagu-action-notify@v1.2.0` and official
refs such as `slack@v1.2.0` are portable across worker pools when the process
executing the action can resolve the Git ref. Explicit `source:` refs support
local paths, `file://` paths, and Git URLs; local paths must exist on the worker
executing the action step.
The action manifest `inputs` schema validates the caller's `with:` object before
the action DAG starts. The `outputs` schema validates the final action output
object published by `stdout.outputs` or `action: outputs.write` before it is
exposed to the parent step as `${step.outputs.*}`.
## Common Action Examples
### Human Task
Use `human.task` for a standalone processless step that waits for operator input. It requires an explicit `id` and `with.prompt`; omit `with.form` for acknowledgement-only tasks. Add `with.artifacts` to show the operator artifacts from the run as review context. Human tasks are allowed only in root DAGs. A root DAG containing one may run locally or on a distributed worker selected by its DAG-level `worker_selector`.
```yaml
steps:
- id: review
action: human.task
with:
prompt: Choose the deployment target
form:
type: object
properties:
environment:
type: string
enum: [staging, production]
notify:
type: boolean
default: true
required: [environment]
- id: deploy
depends: [review]
run: ./deploy.sh '${steps.review.outputs.environment}'
```
`with.artifacts` entries are artifact-relative paths; absolute paths, `~`, and `..` segments are rejected. Like the prompt, they are value-resolved when the task opens, and the resolved path is re-checked under the same rules. Referencing an artifact does not enable artifact storage, and a missing artifact never blocks completion.
Form `additionalProperties` defaults to `false`. Every declared form property is a step output, published when submitted or defaulted, and available as `${steps.<step_id>.outputs.<name>}`; do not author `outputs:` on the human task. Complete a waiting task from a local CLI context with `dagu human-task complete --run-id=<run-id> --step=review <dag-name>`, adding repeated `--input key=value` flags or one `--inputs-json` object when the form accepts input. The scheduler must be running to resume a distributed run.
### SQL Query
```yaml
steps:
- id: active_users
action: postgres.query
with:
dsn: ${DATABASE_URL}
query: SELECT id, email FROM users WHERE active = true
```
The built-in SQL action family supports PostgreSQL and SQLite. Use the official
`duckdb@v1` remote action when a workflow needs DuckDB while keeping the Dagu
core binary cgo-free.
```yaml
steps:
- id: analyze
action: duckdb@v1
with:
query: SELECT 42 AS answer
```
### Child DAG
Use `dag.run` when the parent workflow must wait for the child DAG result:
```yaml
steps:
- id: process_account
action: dag.run
with:
dag: workflows/process-account
params:
ACCOUNT_ID: acct_123
REGION: us-east-1
```
Use `dag.enqueue` when the parent only needs to create a queued child DAG run
and continue:
```yaml
steps:
- id: queue_account_report
action: dag.enqueue
with:
dag: workflows/account-report
params:
ACCOUNT_ID: acct_123
queue: background
```
`dag.enqueue` accepts the same `with.dag` and `with.params` inputs as
`dag.run`, plus `with.queue` to override the queued child run's queue.
`parallel:` currently requires `action: dag.run` or `action: dag.enqueue`:
```yaml
steps:
- id: fanout
action: dag.run
with:
dag: workflows/process-account
params:
ACCOUNT_ID: ${ITEM.account_id}
REGION: ${ITEM.region}
parallel:
max_concurrent: 3
items:
- account_id: acct_1
region: us-east-1
- account_id: acct_2
region: eu-west-1
```
### Wait for HTTP Readiness
```yaml
steps:
- id: wait_for_api
action: wait.http
with:
url: https://api.example.com/health
status: 200
poll_interval: 5s
request_timeout: 10s
timeout_sec: 300
```
Use `timeout_sec` to cap total wait time for polling actions such as
`wait.file` and `wait.http`.
### Harness
```yaml
harnesses:
codex-cli:
binary: codex
prefix_args: [exec]
prompt_mode: arg
steps:
- id: review
action: harness.run
with:
provider: codex-cli
prompt: Review the current branch and list actionable issues.
```
Prefer `action: harness.run` for new workflows. Compatibility note: a
top-level `harness:` config supplies defaults only to explicit harness steps. It
does not change ordinary `run:`, `exec:`, or `script:` steps into harness steps.
## Reusable Custom Actions
Use top-level `actions:` to define a validated reusable action.
```yaml
actions:
release.announce:
description: Print a release announcement
input_schema:
type: object
additionalProperties: false
required: [channel, version]
properties:
channel:
type: string
enum: [changelog, slack]
version:
type: string
template:
run: echo {{ json .input.channel }} {{ json .input.version }}
steps:
- id: announce
action: release.announce
with:
channel: changelog
version: v1.2.3
```
Custom action rules:
- The action name must match
`^[A-Za-z][A-Za-z0-9_-]*(\.[A-Za-z][A-Za-z0-9_-]*)*$`.
- `input_schema` is required and must resolve to an object schema.
- `template` is required and must contain exactly one of `run` or `action`.
- `type:` is not supported inside an `actions:` definition.
- Legacy execution fields are rejected inside action templates.
- `with:` at the call site is validated against `input_schema`.
- Custom actions can call other custom actions; recursive references are
rejected.
Legacy `step_types:` remains loadable, but it is deprecated. Use `actions:` for
new reusable definitions.
## Outputs
String-form `output:` captures trimmed stdout into a flat variable:
```yaml
steps:
- id: version
run: git rev-parse --short HEAD
output: VERSION
- id: publish
run: echo "Publishing ${VERSION}"
depends: [version]
```
Object-form `output:` publishes structured step output:
```yaml
steps:
- id: inspect
run: echo '{"version":"v1.2.3","artifact":{"url":"https://example.test/app.tgz"}}'
output:
version:
from: stdout
decode: json
select: .version
artifact:
from: stdout
decode: json
select: .artifact
- id: publish
depends: [inspect]
run: echo "${inspect.output.version} ${inspect.output.artifact.url}"
```
Structured output sources are `stdout`, `stderr`, and `file`. Decoders are
`text`, `json`, and `yaml`. `select` requires `json` or `yaml`.
Use `stdout.outputs` or `action: outputs.write` when a DAG or remote action
needs to return values to its caller:
```yaml
steps:
- id: notify
run: ./notify.sh
stdout:
outputs:
fields:
messageId:
decode: json
select: .id
- id: publish
action: outputs.write
with:
values:
status: sent
```
Caller-visible outputs are available as `${step.outputs}` or
`${step.outputs.messageId}`. This is separate from object-form `output:`, which
is step-scoped and remains available through `${step.output.*}`.
## Lifecycle Handlers
Use `handler_on` for lifecycle steps:
```yaml
handler_on:
failure:
action: log.write
with:
message: Workflow failed
exit:
run: ./cleanup.sh
steps:
- id: main
run: ./run.sh
```
Handlers use the same current step execution syntax as normal steps.
## Validation and Source of Truth
Use these commands when editing workflow YAML:
```sh
dagu validate workflow.yaml
dagu schema dag
dagu schema dag steps
```
Implementation-level references:
- `internal/cmn/schema/dag.schema.json` - generated JSON Schema used by editor
tooling and schema navigation
- `internal/spec/step_v2.go` - `run:` and `action:` normalization
- `internal/spec/step_types.go` - custom `actions:` and legacy
`step_types:` handling
- `internal/spec/deprecation.go` - deprecated v1 syntax warnings
- [`SCHEMA_MIGRATION.md`](./SCHEMA_MIGRATION.md) - migration notes from v1
syntax to the current schema