specs/017-built-in-run-context.md

Rama main · réplica de lectura del repositorio original.

# Spec: Built-In Run Context

## Status

Implemented.

## Scope

This spec defines Dagu-managed context values for one DAG run and the current
step or handler inside that run.

It covers:

- structured built-in context references such as `${context.run.id}`
- environment-variable projections such as `DAG_RUN_ID`
- availability rules for run, attempt, step, handler, trigger, path, profile,
  and push-back context values

This spec does not define:

- user-authored `env` declarations
- parameter declaration, validation, individual `${params.name}` references,
  or root `${params}` payload references
- step output declaration or `${steps.step_id.outputs.name}` references
- secret provider lookup or secret masking
- lifecycle handler execution order
- scheduler, queue, coordinator, worker, API, or UI behavior
- integration-specific convenience variables such as provider-owned GitHub
  variables

Environment precedence and protected-name behavior are defined by
[Spec 006: Value Resolution Env](006-value-resolution-env.md). Common value
resolution syntax, escaping, field support, and passive notices are defined by
[Spec 003: Value Resolution and Field Evaluation](003-value-resolution.md).

## Goal

Workflow authors can read stable DAG-run metadata without depending on ambient
host environment, storage layout, UI state, or scheduler internals.

Shell-oriented steps can continue to use small environment variables.
Object-valued payloads stay on their owning specs or compatibility
environment variables instead of becoming `context.*` string references.

## Motivation

Built-in run metadata is a public workflow-language surface. Once a variable is
observable, workflow authors may depend on its name, value, timing, and
overwrite behavior.

Dagu needs one contract that separates:

- stable run identity, such as the DAG name and DAG-run ID
- current execution context, such as the current step name and stream files
- optional trigger context, such as scheduled time or webhook payload
- filesystem locations, such as logs, work directory, and artifacts directory
- compatibility environment variables used by existing shell workflows

The context must be deterministic for the current run. It must not read
unrecorded host state during a later step, retry, handler, or distributed
execution hop.

## Related Specs

- Value resolution: [Spec 003: Value Resolution and Field Evaluation](003-value-resolution.md)
- Environment values: [Spec 006: Value Resolution Env](006-value-resolution-env.md)
- Step identity: [Spec 009: Step Reference](009-step-reference.md)
- Step outputs: [Spec 012: Step Outputs](012-step-outputs.md)
- Step run: [Spec 013: Step Run](013-step-run.md)

## Behavior

### Context Values

A context value is Dagu-managed runtime data about the current DAG run or the
current step or handler.

Rules:

- Context values are read-only from the workflow author's perspective.
- User-authored `env`, dotenv, runtime-profile variables, step outputs, and
  secrets must not override protected context values in the final environment
  scope where Spec 006 says the name is protected.
- Context values must not contain Dagu-managed secret values.
- Context values must not expose the raw host process environment.
- Context values must be stable for the lifecycle scope that owns them.
- A value that is unavailable in a lifecycle scope is missing, not an empty
  string, unless a field definition below explicitly says the value is empty.
- Missing structured context references are preserved according to Spec 003
  missing-reference behavior.

### Structured Reference Syntax

This spec adds canonical Dagu-owned reference forms under the `context`
namespace:

```text
${context.dag.name}
${context.run.id}
${context.run.status}
${context.run.scheduled_at}
${context.run.root_name}
${context.run.root_id}
${context.attempt.id}
${context.attempt.started_at}
${context.step.id}
${context.step.name}
${context.step.repeat_index}
${context.trigger.type}
${context.trigger.actor}
${context.paths.log_file}
${context.paths.work_dir}
${context.paths.artifacts_dir}
${context.paths.step_stdout_file}
${context.paths.step_stderr_file}
${context.paths.step_output_file}
${context.profile.name}
${context.profile.resolved_at}
${context.pushback.iteration}
${context.pushback.previous_stdout_file}
```

Rules:

- These references are Dagu-owned references only in fields that support value
  resolution under Spec 003.
- `context` is the canonical root namespace for Dagu-managed runtime context.
  Workflow-authored data remains under its owning namespace, such as `params`,
  `env`, or `steps`.
- All structured context references in this spec resolve to scalar string
  values.
- Object-valued context, such as webhook payload, webhook headers, profile
  entries, and push-back history, is outside this spec's structured string
  references.
- Unknown fields under reserved context namespaces are preserved at runtime.
  Explicit inspection surfaces must report an inspection-only passive notice
  with reason `unknown_context_field`.
- Unsupported reference-looking text outside reserved context namespaces is
  preserved silently as ordinary string content under Spec 003.
- Escaped context-looking text follows Spec 003 escape behavior.
- `context`, `context.dag`, `context.run`, `context.attempt`, `context.step`,
  `context.trigger`, `context.paths`, `context.profile`, and
  `context.pushback` are reserved context namespaces.
- The short top-level namespaces `dag`, `run`, `attempt`, `step`, `trigger`,
  `paths`, `profile`, and `pushback` are frozen compatibility aliases. New
  context fields must not be added to those alias namespaces.
- Only the exact frozen alias forms listed in the compatibility table are
  Dagu-owned references. Other text under those short roots, such as
  `${step.xxx.foo}`, is unsupported ordinary string content under Spec 003.
- A workflow parameter named `context`, `run`, `step`, or another context
  namespace remains addressable through `${params.context}`, `${params.run}`,
  or `${params.step}`. It must not shadow a context namespace.

### Namespace Design

The `context` root separates Dagu-managed runtime metadata from
workflow-authored data. This keeps common top-level names available for future
workflow-language specs and makes the trust boundary explicit:

- `params`, `env`, and `steps` are workflow-authored or workflow-derived data.
- `context` is Dagu-managed runtime data.
- Environment variables are a compatibility projection of selected context
  values, not the source of truth for structured context references.

### Namespace Fields

`context.dag` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.dag.name` | All run, step, and handler scopes | Name of the DAG definition being executed. |

`context.run` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.run.id` | All run, step, and handler scopes | Unique identifier for the current DAG run. |
| `context.run.status` | Status-aware scopes only | Canonical lowercase run status. |
| `context.run.scheduled_at` | Scheduled, catchup, and one-off scheduled runs only | UTC RFC3339 timestamp for the logical schedule time that caused the run. |
| `context.run.root_name` | Sub-DAG runs only | DAG name of the root DAG run. |
| `context.run.root_id` | Sub-DAG runs only | DAG-run ID of the root DAG run. |

Rules:

- `context.run.id` is the durable identity for a DAG run.
- `context.run.scheduled_at` is not a unique run identifier. Scheduled, catchup, and
  retry behavior may create multiple runs associated with one logical schedule
  time. Workflow authors must use `context.run.id` when they need identity.
- `context.run.status` is available in lifecycle handlers and other explicitly
  status-aware surfaces. It is unavailable in normal step execution unless an
  owning spec explicitly makes that surface status-aware.
- Valid `context.run.status` values are `not_started`, `queued`, `running`,
  `succeeded`, `partially_succeeded`, `failed`, `aborted`, `waiting`,
  `rejected`, and `unknown`.

`context.attempt` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.attempt.id` | Attempt-aware run, step, and handler scopes | Identifier for the current DAG-run attempt. |
| `context.attempt.started_at` | After run-attempt start is recorded | UTC RFC3339 timestamp for the start of this DAG-run attempt. |

Rules:

- `context.attempt.id` identifies the current attempt for one DAG run.
- `context.attempt.id` is not the same as `context.run.id`.
- Step retry attempts are outside this namespace unless a step-retry-owning
  spec adds a separate field.

`context.step` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.step.id` | Current executable step only, when that step has an id | Stable value-reference alias from Spec 009. |
| `context.step.name` | Current step or handler only | Runtime step identity for the current step or handler. |
| `context.step.repeat_index` | Current step with `repeat_policy` only | Zero-based index of the current execution as a decimal string. |

Rules:

- `context.step.name` is the runtime step identity defined by Spec 009.
- `context.step.id` is missing when the current executable step has no authored `id`.
- Handler scopes have `context.step.name` but do not have `context.step.id` unless a
  handler-owning spec defines an id for handlers.
- `context.step.repeat_index` is `0` for the first execution of a repeating
  step and increases by one for each repeat. A step retry keeps the index of
  the execution it retries.
- Dagu resolves the step again for every repeat, so `run`, `with`, `stdin`,
  and step `env` see the index of the current execution.
  `repeat_policy.condition` sees the index of the execution that just
  finished.
- Fields resolved once before the first execution keep index `0`: `stdout`,
  `stderr`, `preconditions`, and the `repeat_policy` and `retry_policy`
  settings.
- `context.step.repeat_index` is missing for steps without `repeat_policy` and
  for handlers.

`context.trigger` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.trigger.type` | All run, step, and handler scopes when known | Canonical trigger type. |
| `context.trigger.actor` | Runs started by an authenticated or attributable actor | Stable actor identifier safe to expose to the workflow. |

Rules:

- Valid `context.trigger.type` values are `unknown`, `manual`, `scheduler`,
  `webhook`, `subdag`, `retry`, `catchup`, and `restart`. `restart` is the
  trigger type of the runs started because of a DAG's `restart_policy`
  ([Spec 080](080-restart-policy.md)).
- `context.trigger.actor` must not contain credentials, bearer tokens, session IDs, or
  other authenticators.
- Webhook payload and headers are object-valued trigger context and are exposed
  through compatibility environment variables.

`context.paths` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.paths.log_file` | All run, step, and handler scopes | Absolute path to the aggregated DAG-run log file. |
| `context.paths.work_dir` | When a per-run work directory is available | Absolute path to the per-run work directory. |
| `context.paths.artifacts_dir` | When artifact storage is active | Absolute path to the per-run artifacts directory or staging directory. |
| `context.paths.step_stdout_file` | Current executable step after stream files are assigned | Absolute path to the current step stdout file. |
| `context.paths.step_stderr_file` | Current executable step after stream files are assigned | Absolute path to the current step stderr file. |
| `context.paths.step_output_file` | Current step attempt after output publication is prepared | Absolute path to the current step output file used by Spec 012. |

Rules:

- Path values are filesystem paths as seen by the current step or handler.
- A path value may point to worker-local staging storage in an execution mode
  where final storage is remote or coordinator-owned.
- Workflow authors must not infer storage retention, public URL shape, or UI
  availability from a path value.
- A reference to `context.paths.artifacts_dir` in the DAG definition enables
  artifact storage unless artifact storage is explicitly disabled.

`context.profile` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.profile.name` | Runs started with a selected runtime profile | Runtime profile selected for the run. |
| `context.profile.resolved_at` | Runs started with a selected runtime profile | UTC RFC3339 timestamp at which the selected profile was resolved. |

Rules:

- Profile values are metadata only.
- Profile variables and profile secrets are owned by the runtime-profile and
  secret specs, not by this spec.

`context.pushback` fields:

| Field | Availability | Meaning |
| --- | --- | --- |
| `context.pushback.iteration` | Steps re-executed after an approval or human-task push-back | Current push-back iteration as a decimal string. |
| `context.pushback.previous_stdout_file` | Rewound steps that had previous stdout | Absolute path to the previous stdout log for the current step. |

Rules:

- Push-back inputs and history are object-valued context and are exposed through
  compatibility environment variables.
- Push-back context is missing on the first execution before any push-back
  occurs.
- `DAG_PUSHBACK` stays within 30 KiB (30720 bytes). When the full history does
  not fit, the payload keeps the most recent history entries that fit; the run
  status keeps the full history.
- The inputs of one push-back, encoded as one JSON object, are limited to
  16 KiB (16384 bytes). A larger push-back is rejected before it changes the
  run.
- A step that declares `approval.input` receives only those input names from an
  approval push-back. Human-task push-back feedback holds only declared
  feedback properties and reaches every rewound step unchanged.

### Environment Projection

Environment projection is the flat process-environment view of selected context
values.

Rules:

- Environment projection is for compatibility with shell scripts and tools that
  read process environment variables.
- Structured references are the canonical surface for new scalar context
  fields.
- Environment projection must not expose object-valued context except as JSON
  strings for compatibility variables already defined by this spec.
- Environment projection names are reserved according to Spec 006.
- A context value unavailable in the current scope must not be projected unless
  the variable definition below explicitly allows an empty value.

Run-level projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `DAG_NAME` | `context.dag.name` | All steps and handlers. |
| `DAG_RUN_ID` | `context.run.id` | All steps and handlers. |
| `DAG_RUN_LOG_FILE` | `context.paths.log_file` | All steps and handlers. |
| `DAG_RUN_WORK_DIR` | `context.paths.work_dir` | When a per-run work directory is available. |
| `DAG_RUN_ARTIFACTS_DIR` | `context.paths.artifacts_dir` | When artifact storage is active. |
| `DAG_PARAMS_JSON` | Resolved parameter payload JSON; same payload as `${params}` from Spec 005 | When resolved parameters exist. |

Step and handler projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `PWD` | Current process working directory | Current step only. |
| `DAG_RUN_STEP_NAME` | `context.step.name` | Current step or handler only. |
| `DAG_RUN_STEP_REPEAT_INDEX` | `context.step.repeat_index` | Current step with `repeat_policy` only. |
| `DAG_RUN_STEP_STDOUT_FILE` | `context.paths.step_stdout_file` | Current executable step after stream files are assigned. |
| `DAG_RUN_STEP_STDERR_FILE` | `context.paths.step_stderr_file` | Current executable step after stream files are assigned. |
| `DAGU_OUTPUT_FILE` | `context.paths.step_output_file` | Current step attempt after output publication is prepared. |

Trace context projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `TRACEPARENT` | W3C `traceparent` of the current step span | Current executable step when the step span is valid. |
| `TRACESTATE` | W3C `tracestate` of the current step span | Current executable step when the step span is valid and has a trace state. |

Status and wait-handler projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `DAG_RUN_STATUS` | `context.run.status` | Lifecycle handlers and status-aware surfaces only. |
| `DAG_WAITING_STEPS` | Waiting step names | Wait handler only. |

Push-back projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `DAG_PUSHBACK` | Push-back metadata JSON | Steps re-executed after an approval or human-task push-back only. |
| `DAG_PUSHBACK_ITERATION` | `context.pushback.iteration` | Steps re-executed after an approval or human-task push-back only. |
| `DAG_PUSHBACK_PREVIOUS_STDOUT_FILE` | `context.pushback.previous_stdout_file` | Rewound steps that had previous stdout. |

Webhook projection:

| Environment variable | Source | Availability |
| --- | --- | --- |
| `WEBHOOK_PAYLOAD` | Trigger payload JSON | Webhook-triggered runs only. |
| `WEBHOOK_HEADERS` | Allowlisted trigger headers JSON | Webhook-triggered runs only. |

Rules:

- `WEBHOOK_PAYLOAD` and `WEBHOOK_HEADERS` are compatibility runtime-param
  environment values.
- They are not protected Dagu-managed environment names under Spec 006.
- They follow Spec 006 runtime-param precedence after parameter resolution.
- They are untrusted trigger input. They must not carry credentials,
  authorization headers, cookies, or values used by Dagu itself for
  authorization decisions.
- When a DAG explicitly declares either name as a runtime parameter, that name
  behaves as a normal user parameter for that DAG.
- When a webhook trigger supplies either name and the DAG does not declare it
  as a runtime parameter, the trigger-provided value is an internal
  runtime-param value.

Reserved specialized run-control names:

- `DAGU_EXTERNAL_STEP_RETRY`
- `DAGU_QUEUE_DISPATCH_RETRY`

These names are reserved for owning run-control surfaces. They are not normal
workflow context values unless an owning spec explicitly adds them.

### Trace Context Projection

When OpenTelemetry tracing is enabled for a DAG, Dagu records a span for each
executable step. Trace context projection lets a process that the step starts
create child spans of that step span.

Rules:

- The step span is valid only when tracing is enabled for the DAG and the run
  has a valid DAG span. Without a valid step span, Dagu projects neither name.
- `TRACEPARENT` and `TRACESTATE` use the W3C Trace Context format. The names
  are the upper-case propagator keys, as in the OpenTelemetry convention for
  environment variables as context carriers.
- `TRACEPARENT` names the trace of the DAG run and the span ID of the current
  step span. A child span of the step span uses this span ID as its parent.
- `TRACESTATE` is projected only when the step span has a trace state. When
  the step span has no trace state and the root run environment scope has a
  `TRACESTATE`, Dagu replaces it with an empty value, because it belongs to
  another span.
- Lifecycle handlers have no step span, so Dagu projects neither name for them.
- The names extend the OpenTelemetry carrier convention, not the `DAG_` or
  `DAGU_` families. Tools read these exact names, so the `DAGU_` prefix rule
  for new projection variables does not apply.
- Precedence follows Spec 006: the names replace root-scope values of the same
  name, and a step-local source that sets either name shadows both names. A
  root-scope value of the name that the step does not set becomes empty.
  A step can set `TRACEPARENT` to an empty string to give its process no
  trace context.

### Availability

Availability rules:

- DAG and run identity are available after the DAG run is created.
- Attempt identity is available only in scopes that know the current DAG-run
  attempt.
- Step context is available only while evaluating or executing the current step
  or handler.
- Repeat context is available only for steps with `repeat_policy`.
- Step stream paths are available only after stdout and stderr files are
  assigned for the current executable step.
- Step output file path is available only after step output publication is
  prepared for the current step attempt.
- Artifact directory is available when artifact storage is active. A DAG
  definition reference to `context.paths.artifacts_dir` activates artifact
  storage unless artifact storage is explicitly disabled.
- Profile context is available only when a runtime profile was selected.
- Webhook context is available only for webhook-triggered runs.
- Push-back context is available only for step executions caused by an
  approval or human-task push-back cycle.

Validation rules:

- `dagu validate` must not fail only because a structured context reference is
  unavailable during validation.
- Explicit inspection surfaces must report passive notices for unresolved
  supported structured context references.
- A supported structured context reference carries a value only during a run, so
  its notice is runtime-only. `dagu validate` must keep it out of the default
  output and print it under `--show-unresolved`.
- A reference to an unsupported field under `context.` remains a defect and is
  reported by default, which is what separates a typo such as
  `${context.paths.artifact_dir}` from the supported
  `${context.paths.artifacts_dir}`.
- Explicit inspection surfaces must report passive notices with reason
  `unknown_context_field` for unknown fields under reserved context namespaces.
- Normal run execution must not emit passive value-reference notices as run
  logs, workflow events, status data, history data, artifacts, or DAG-run detail
  data.

### Security

Rules:

- Context values must be treated as ordinary workflow input by the step that
  consumes them.
- Trigger payloads and trigger headers are untrusted external input.
- Compatibility JSON-string environment values must not include Dagu-managed
  secret values or authenticators.
- Compatibility JSON-string environment values must not include unallowlisted
  webhook headers.
- Environment projection must not introduce new secret exposure paths beyond
  variables explicitly owned by secret or profile specs.
- A context reference that resolves to a string must not be shell-escaped by
  Dagu. Shell-backed fields follow the quoting rules in Spec 013.
- Projected environment variables are not authenticators, trusted audit facts,
  or Dagu-internal authority. Dagu internals must use structured runtime
  context, not process environment values, for authorization, identity, status,
  output-file routing, or audit decisions.

### Compatibility

Rules:

- Existing environment variable names listed in this spec remain supported.
- `DAG_PARAMS_JSON` is the parameter payload environment variable.
- New structured context fields must be additive.
- New structured context fields must be added under `context.*`.
- Short top-level aliases are frozen. They remain supported for the exact fields
  listed below, but they are not canonical and must not receive new fields.
- New environment projection variables should be added only when a value is
  broadly useful to shell scripts and cannot reasonably be represented as a
  structured reference or an existing compatibility variable.
- New environment projection variables should prefer the `DAGU_` prefix unless
  an owning spec explicitly extends an existing compatibility family such as
  `DAG_` or `WEBHOOK_`.
- `DAG_RUN_STEP_REPEAT_INDEX` extends the `DAG_RUN_STEP_` family that already
  carries the current step's name and stream files.

Frozen structured-reference aliases:

| Compatibility alias | Canonical reference |
| --- | --- |
| `dag.name` | `context.dag.name` |
| `run.id` | `context.run.id` |
| `run.status` | `context.run.status` |
| `run.started_at` | `context.attempt.started_at` |
| `run.scheduled_at` | `context.run.scheduled_at` |
| `run.root_name` | `context.run.root_name` |
| `run.root_id` | `context.run.root_id` |
| `attempt.id` | `context.attempt.id` |
| `step.id` | `context.step.id` |
| `step.name` | `context.step.name` |
| `trigger.type` | `context.trigger.type` |
| `trigger.actor` | `context.trigger.actor` |
| `paths.log_file` | `context.paths.log_file` |
| `paths.work_dir` | `context.paths.work_dir` |
| `paths.artifacts_dir` | `context.paths.artifacts_dir` |
| `paths.step_stdout_file` | `context.paths.step_stdout_file` |
| `paths.step_stderr_file` | `context.paths.step_stderr_file` |
| `paths.step_output_file` | `context.paths.step_output_file` |
| `profile.name` | `context.profile.name` |
| `profile.resolved_at` | `context.profile.resolved_at` |
| `pushback.iteration` | `context.pushback.iteration` |
| `pushback.previous_stdout_file` | `context.pushback.previous_stdout_file` |

## Errors

Validation must fail when:

- a workflow attempts to define a secret whose name collides with a
  secret-reserved context environment variable under Spec 006.
- a field shape owned by another spec is invalid before context references are
  evaluated.

Runtime must fail before starting the current step or handler when:

- a projected context environment variable cannot be represented as a process
  environment entry for the target platform.
- a context value required by the current executor surface cannot be made
  available after that surface declares it required.

Runtime must preserve the original reference text when:

- a supported structured context reference is unavailable in the current
  lifecycle scope.
- a supported structured context reference is unavailable because an optional
  feature, such as artifacts, docs, profile, webhook, or push-back, is not
  active for this run.
- a reserved context reference names an unknown field. Inspection surfaces
  report `unknown_context_field`, but normal runtime surfaces stay silent.

## Examples

Use stable run identity in a notification:

```yaml
steps:
  - id: notify
    run: notify.sh '${context.dag.name}' '${context.run.id}'
```

Use a per-run work directory when it is available:

```yaml
steps:
  - id: write_scratch
    run: |
      mkdir -p '${context.paths.work_dir}'
      printf '%s\n' '${context.run.id}' > '${context.paths.work_dir}/run-id.txt'
```

Use handler-only run status:

```yaml
handler_on:
  exit:
    run: notify.sh '${context.dag.name}' '${context.run.id}' '${context.run.status}'
```

Unsupported normal-step status reference is preserved until a status-aware
surface owns it:

```yaml
steps:
  - id: print_status
    run: printf '%s\n' '${context.run.status}'
```

In a normal step, `context.run.status` is unavailable. Dagu preserves the
reference text and hands it to the selected shell according to Spec 013.