specs/006-value-resolution-env.md
Rama main · réplica de lectura del repositorio original.
# Spec: Value Resolution Env
## Status
Implemented.
## Scope
This spec defines environment values used by Dagu value resolution.
It covers:
- `${env.NAME}` references
- `$NAME` and `${NAME}` environment expansion
- ordering and validation rules for `env` declarations
- source precedence for environment lookups
- shell and direct-execution boundaries for unqualified environment syntax
Common value-resolution syntax, supported fields, string insertion, and timing are defined by [Spec 003: Value Resolution and Field Evaluation](003-value-resolution.md).
This spec does not define:
- which workflow fields opt into value resolution
- dotenv file syntax
- runtime profile selection
- secret provider configuration or secret masking
- shell language behavior after Dagu hands text to a shell, container, or remote host
## Goal
Workflow authors can define environment values once and use them in every field that supports environment expansion.
## Motivation
Environment values can come from YAML, dotenv files, runtime params, step outputs, container settings, step settings, and the process environment.
Dagu must preserve environment syntax when another runtime, such as a shell or executor, owns that syntax.
Dagu must also define which value wins when the same name appears in more than one source.
Otherwise a workflow can pass validation and still behave differently under the CLI, Web UI, scheduler, worker, container, or SSH executor.
## Behavior
### Environment Scope
An environment scope is the set of named string values visible when Dagu evaluates a field.
Rules:
- Dagu resolves an environment reference against the current environment scope for the field being evaluated.
- If the same environment name is defined by multiple visible sources, the later and more specific source wins.
- For a normal DAG run, the root run environment scope source precedence is,
from lowest to highest:
1. process environment entries explicitly inherited into the run by the Dagu
runtime
2. inherited runtime-profile defaults
3. runtime params exposed as environment values
4. Dagu-managed run environment values
5. root `env` declarations
6. dotenv values loaded into the DAG environment
7. selected runtime-profile environment values
8. protected Dagu-managed run environment values
9. secret values exposed as environment values
- Step-owned fields extend the root run environment scope with the step-local
sources that exist when the field is evaluated.
- For a normal step after the execution-attempt environment is ready, the
step-local source precedence is, from lowest to highest:
1. initial current-step Dagu-managed values available before `steps[].env`
is evaluated
2. output variables from completed predecessor steps that are available to
the step before it starts
3. the step's own `env` declarations
4. environment declarations from the selected container for the step
5. execution-attempt Dagu-managed values created for the current attempt
- Root `env` entries are evaluated top to bottom.
- If root `env` defines the same name as a protected Dagu-managed run
environment value, later root `env` entries in the same declaration may read
the workflow-authored value during root `env` evaluation.
- Protected Dagu-managed run environment values cannot be overridden in the
final root run environment scope by root `env`, dotenv values,
runtime-profile values, or runtime environment overrides.
- Step-local sources are intentionally outside that root-scope protection.
- If a predecessor output, step `env`, or selected `container.env` uses the
same name as an initial current-step Dagu-managed value, that step-local
value shadows the initial value for later fields and process environment
belonging to that step.
- Execution-attempt Dagu-managed values, once available, override predecessor
output variables, step `env`, and selected `container.env` for fields and
process environment belonging to that attempt.
- Workflow authors should avoid reusing Dagu-managed names in step-local
sources unless the step intentionally needs to shadow that name.
- A field evaluated before an execution-attempt Dagu-managed value exists
cannot read that value. Missing-reference preservation rules apply.
- Predecessor output variables are defined below.
- Step outputs referenced as `${steps.step_id.outputs.name}` are owned by Spec
007 and published through the output-owning specs.
- Step outputs are not automatically available as `${env.NAME}` unless the
producing step also publishes an output variable with that environment name.
- When multiple predecessor steps publish the same output-variable name into a
step's environment scope, the predecessor visited later in dependency order
wins.
- Dependency order is breadth-first upstream order:
1. Start with the consuming step's authored `depends` list in left-to-right
order.
2. Visit each listed predecessor at most once.
3. After visiting a predecessor, append that predecessor's authored
`depends` list to the end of the order in left-to-right order.
4. Ignore a predecessor if it appears again after it has already been
visited.
- A predecessor's output variables are applied when that predecessor is first
visited.
- The selected container is the step container when the step defines one, or
the root container when the step uses the root container.
- Container environment declarations are not a separate global source.
- If another spec makes both root-level container settings and step-level
container settings applicable to a step, only the environment declarations
on the selected container participate in this spec's precedence order.
- Variables loaded from the selected container's `env_file` entries are part
of that container's environment declarations.
- Within the selected container, `container.env` entries override variables
from `env_file`, and a later `env_file` entry overrides an earlier one.
- Dagu does not value-resolve `env_file` contents. Files use dotenv syntax.
- A missing or unreadable `env_file` fails the step or run that selects the
container.
- Runtime-profile specs own profile selection and profile inheritance.
- Secret specs own secret provider lookup and masking.
- This spec only defines how values behave after they are present in the environment scope.
- Secret declarations must not use names that start with `DAGU_` or names that
collide with secret-reserved Dagu-managed environment names. After
secret-owning validation accepts a secret name, the exposed secret value uses
the secret precedence entry above.
- Explicitly inherited process environment entries are part of the root run
environment scope after Dagu imports them.
- Host process environment fallback is different: it means reading a name
directly from the Dagu process environment while evaluating a field.
- A field whose ownership row says host process environment fallback is `Not
allowed` must not read host-only process environment values.
- Such a field can still read a value that originated from the process
environment only if Dagu already imported that value into the run environment
scope through an explicit source such as inherited base environment, root
`env`, dotenv, runtime profile, or secret exposure.
- Dagu configuration variables that carry a credential, such as
`DAGU_AUTH_BASIC_PASSWORD`, `DAGU_SECRETS_VAULT_TOKEN`, or
`DAGU_ENCRYPTION_KEY`, are never inherited into a run environment scope and
are never read through host process environment fallback. They reach neither
a step's process environment nor an expanded value. Dagu's own child
processes still receive them, and other `DAGU_` variables are inherited as
before.
- Environment names are matched exactly during Dagu value resolution.
- When Dagu builds a process environment on a platform where environment names are case-insensitive, duplicate names that differ only by case collapse according to the platform rule, and the higher-precedence Dagu source wins.
Protected Dagu-managed root run environment names:
- `DAG_NAME`
- `DAG_RUN_ID`
- `DAG_RUN_LOG_FILE`
- `DAG_RUN_WORK_DIR`
- `DAG_RUN_ARTIFACTS_DIR`
- `DAG_PARAMS_JSON`
Initial current-step Dagu-managed environment names include:
- `PWD`
- `DAG_RUN_STEP_NAME`
- `DAG_RUN_STEP_REPEAT_INDEX`
`PWD` is the resolved working directory for the current step.
`DAG_RUN_STEP_NAME` is the current step's effective runtime name. When a step
omits `name` and uses `id`, the effective runtime name is that `id`.
`DAG_RUN_STEP_REPEAT_INDEX` is present only for steps with `repeat_policy`
and is defined by [Spec 017](017-built-in-run-context.md).
`DAG_RUN_STATUS`, when present for a handler or status-aware surface, is an
initial current-step Dagu-managed value for that surface.
`TRACEPARENT` and `TRACESTATE`, when present for an executable step, are
initial current-step Dagu-managed values. They carry the trace context of the
step span defined by [Spec 017](017-built-in-run-context.md#trace-context-projection).
They differ from the other initial current-step names in these ways:
- They are not protected and not secret-reserved. They are standard names that
workflows and tools also set.
- They replace a value of the same name from the root run environment scope.
- A step-local source that sets either name shadows both names. Dagu then adds
neither name, so a step-local `TRACEPARENT` is never paired with the trace
state of the step span. When only one name is step-local, Dagu replaces a
root-scope value of the other name with an empty value, because the two
names form one trace context.
Execution-attempt Dagu-managed environment names include:
- `DAG_RUN_STEP_STDOUT_FILE`
- `DAG_RUN_STEP_STDERR_FILE`
- `DAGU_OUTPUT_FILE`
Additional Dagu-managed names reserved for specialized run-control surfaces
include:
- `DAG_WAITING_STEPS`
- `DAG_PUSHBACK`
- `DAG_PUSHBACK_ITERATION`
- `DAG_PUSHBACK_PREVIOUS_STDOUT_FILE`
- `DAGU_EXTERNAL_STEP_RETRY`
- `DAGU_QUEUE_DISPATCH_RETRY`
These names are not part of the normal step environment scope unless their
owning surface makes them available.
Secret-reserved Dagu-managed environment names include every name starting with
`DAGU_` and every Dagu-managed projection name below:
- `DAG_NAME`
- `DAG_RUN_ID`
- `DAG_RUN_LOG_FILE`
- `DAG_RUN_STEP_NAME`
- `DAG_RUN_STEP_REPEAT_INDEX`
- `DAG_RUN_STEP_STDOUT_FILE`
- `DAG_RUN_STEP_STDERR_FILE`
- `DAG_RUN_STATUS`
- `DAG_WAITING_STEPS`
- `DAG_PARAMS_JSON`
- `DAG_RUN_WORK_DIR`
- `DAG_RUN_ARTIFACTS_DIR`
- `DAG_PUSHBACK`
- `DAG_PUSHBACK_ITERATION`
- `DAG_PUSHBACK_PREVIOUS_STDOUT_FILE`
- `DAGU_EXTERNAL_STEP_RETRY`
- `DAGU_QUEUE_DISPATCH_RETRY`
Other specs that introduce Dagu-managed environment names must state whether
those names are protected root run names, initial current-step names, or
execution-attempt names. They must also state whether the names are
secret-reserved. Future Dagu-managed projection names should prefer the
`DAGU_` prefix unless an owning spec explicitly extends an existing
compatibility family.
### Predecessor Output Variables
This section defines only the output-variable source used by environment
resolution.
It does not define the `outputs` field from Spec 012, `stdout.outputs`,
`outputs.write`, or structured object-form `output`.
Rules:
- A successful step with singular string-form `output: NAME` publishes one
output variable named `NAME`.
- `NAME` should match the environment name rule `^[A-Za-z_][A-Za-z0-9_]*$`
when a later step needs to read it through `$NAME`, `${NAME}`, or
`${env.NAME}`.
- The output-variable value is the producing step's captured stdout with
leading and trailing whitespace removed.
- A failed, aborted, timed-out, skipped, or not-yet-completed step publishes no
output variable for this spec.
- Output variables enter only the environment scope of downstream steps that
directly or transitively depend on the producing step.
- Output variables do not create dependencies. Only supported step output
references do, under Spec 007.
### Environment Declarations
- `env` declarations may use map form, array-of-map form, or `NAME=value` list form.
- Sequence form may mix array-of-map entries and `NAME=value` entries.
- Map form is ordered by YAML source order.
- Each `NAME=value` list item must use `NAME=value` form and is split at the first `=`.
- `NAME=` is valid and defines an empty string value.
- `NAME=a=b` is valid and defines the value `a=b`.
- Env names must match `^[A-Za-z_][A-Za-z0-9_]*$`.
- Env values are environment strings.
- Scalar YAML values are converted to their string value before environment resolution.
- Workflow authors should quote values such as `false`, `0`, and `null` when the exact text must be preserved.
Ordering rules:
- Entries are evaluated from top to bottom.
- An entry may reference earlier entries from the same declaration.
- A later entry with the same name replaces the earlier entry for subsequent references and for the final environment visible to the owning step.
- An entry cannot resolve itself or a later entry from the same declaration.
Allowed references:
- Root `env` may reference `consts`, `params`, protected Dagu-managed run environment values, and env values that are already available.
- Step `env` may reference `consts`, `params`, root env values, earlier entries from the same step env declaration, step output references that are available before the step starts, and predecessor output variables that are already in the environment scope.
- Step `env` may also reference initial current-step Dagu-managed values that
exist before `steps[].env` is evaluated.
- Container `env` follows the same rule as the root or step that owns the container.
- Entries in a container `env` declaration may reference earlier entries from the same container `env` declaration.
### Sub-DAG Passed Environment
A step that runs a child DAG (`call`, or `action: dag.run`) may hand parent
environment values to that child through the step field `pass_env`.
`pass_env` is additive, not a filter. A child run that executes in the parent's
process already observes the parent run environment scope; `pass_env` selects
the values carried to a child run that may execute on another host. A name list
therefore does not restrict what an in-process child observes.
Forms:
- `pass_env: true` passes the parent run's own environment values as they exist
when the step starts the child run. It reads the run scope, so it does not
carry step-scoped values such as step `env` declarations or step outputs.
- `pass_env: [NAME, ...]` passes only the listed environment names. It reads the
scope visible to the calling step, so a listed name may resolve to a step
`env` declaration or a predecessor step output as well as a run value.
- Omitting `pass_env`, or setting it to `false`, passes nothing beyond params.
Rules:
- `pass_env` requires a child DAG call. It is not supported for `dag.enqueue`
because queued child runs cannot carry transient parent environment values
through queue persistence.
- `pass_env` is evaluated when the child run is created. Every represented child
run of a `parallel` step receives the same requested configuration.
- Resolved values are ordered by name so a dispatched child run produces the
same record on every resolution.
- Each list entry must match `^[A-Za-z_][A-Za-z0-9_]*$` after trimming.
Duplicate names collapse to one.
- Names beginning with `_DAGU_` (any case) are reserved for Dagu internal
transport and are rejected.
- Names Dagu manages for a run or a step are rejected. A child run resolves its
own, and the parent's describe the parent run and the host executing it:
`DAG_NAME`, `DAG_RUN_ID`, `DAG_RUN_LOG_FILE`, `DAG_RUN_STEP_NAME`,
`DAG_RUN_STEP_REPEAT_INDEX`, `DAG_RUN_STEP_STDOUT_FILE`,
`DAG_RUN_STEP_STDERR_FILE`, `DAG_RUN_STATUS`, `DAG_RUN_WORK_DIR`,
`DAG_RUN_ARTIFACTS_DIR`, `DAG_WIKI_DIR`, `DAG_DOCS_DIR`, `DAG_PARAMS_JSON`,
`DAGU_PARAMS_JSON`, `DAG_PUSHBACK*`, `PWD`, and the internal retry markers.
- A listed name resolves against the environment scope visible to the calling
step. Resolution never reads the Dagu process environment directly, so a host
process value reaches a child only when the operator's base environment policy
already admits it into the run. A name that does not resolve produces a
warning and contributes no value.
- `pass_env: true` carries the values the workflow itself declared. It never
carries secrets, host process values, Dagu-managed run values, params, or
runtime-profile values. Those are either sensitive, describe the machine
running the parent rather than the machine running the child, or belong to the
parent run rather than the child. A child owns its own params: the step passes
them through `params`, and forwarding the parent's would override them.
- Neither form carries a name that is not a valid environment variable name.
Positional params are held under `1`, `2`, and so on, which a child could not
declare for itself.
- Neither form carries names reserved for Dagu internal transport (`_DAGU_*`),
names Dagu manages for a run or a step, or host-local tool environment values
managed by the `tools` feature (`PATH`, `AQUA_*`, `DAGU_TOOLS_MANIFEST`).
- Passed values enter the child run environment scope as execution-scoped
values. They sit above inherited process environment and DAG `env`
declarations, and below protected Dagu-managed run environment values and
secrets.
- Passed values are runtime values for the child run; they are not child runtime
params and do not relax child param declaration rules.
Secrets:
- `pass_env` never passes a secret. A name the workflow declares under
`secrets:` is rejected when the workflow is built. A name that reaches the run
scope as a secret another way, such as through a runtime profile, fails the
step when it runs.
- The child declares the secret under its own `secrets:` instead. It is then
resolved through the secret provider in the child run, which keeps the value
masked there and keeps it out of the coordinator's dispatch record. A child
dispatched to a worker resolves it through the coordinator, which authorizes
the request against that run's lease.
- The rejection sees only what the calling run itself holds as a secret. A run
that received a value implicitly from its own parent holds it as ordinary run
environment, because implicit inheritance does not carry the source with the
value. Such a value is not recognised as a secret and `pass_env` will pass it.
A workflow that nests child runs should declare secrets where they are used
rather than rely on a value reaching a nested run.
Transient values:
- Passed values are resolved when the parent step creates the child run, and are
not persisted with the child run. A retry or restart that re-runs the parent
step resolves them again.
- A child run resumed directly from its own persisted state does not receive
them. This covers approving an `approval` gate on a step inside the child,
pushing such a step back, and resuming an agent session inside the child.
Because an unresolved reference is preserved literally rather than failing, a
step that runs after such a resume reads `${NAME}` as text instead of the
passed value. A workflow that must survive one of these resumes should take
the value as a child param or declare it in the child, not rely on `pass_env`.
A human task cannot occur here, because Spec 031 rejects a child DAG that
contains one.
### Environment References
Forms:
```text
${env.NAME}
$NAME
${NAME}
```
Rules:
- `${env.NAME}` is a Dagu-owned environment reference.
- `$NAME` and `${NAME}` are unqualified environment expansion forms.
- `NAME` must match `^[A-Za-z_][A-Za-z0-9_]*$`.
- `$env.NAME` is ordinary string content.
- `${env.NAME}` reads `NAME` from the current environment scope.
- `$NAME` and `${NAME}` read `NAME` from the current environment scope only when Dagu owns unqualified environment expansion for the field being evaluated.
- The unqualified environment expansion ownership table below is authoritative
for each field surface.
- When that table says Dagu preserves unqualified environment syntax for a
later runtime, Dagu leaves `$NAME`, `${NAME}`, and shell-style environment
expressions unchanged for that runtime.
Missing values:
- Missing `${env.NAME}` preserves the original reference text when the field is evaluated.
- Explicit inspection surfaces report a passive notice for that preserved reference.
- Missing `$NAME` or `${NAME}` is preserved when environment expansion runs.
- Passive notice reporting for missing unqualified references is defined by the
validation rules below.
- If Dagu preserves an unqualified reference and later hands the field to a shell, container, or remote host, that later runtime may still expand the reference.
- For example, a POSIX shell normally expands an unset `$NAME` to an empty string.
Validation:
- `dagu validate` must not reject a well-formed environment reference only because `NAME` is unavailable during validation.
Insertion:
- `${env.NAME}` uses the string insertion rules from Spec 003.
### Unqualified Environment Expansion Ownership
This table refines the field evaluation matrix in Spec 003 for unqualified
environment syntax: `$NAME`, `${NAME}`, and shell-style `${NAME...}`
expressions.
`${env.NAME}` remains Dagu-owned in every value-resolved field that supports
environment references.
| Field surface | Unqualified environment behavior | Host process environment fallback |
| --- | --- | --- |
| Root `env` values | Dagu expands while constructing the root environment scope. | Allowed. |
| `steps[].env` values | Dagu expands while constructing the step environment scope. | Not allowed. |
| Selected root or step `container.env` values | Dagu expands while constructing the selected container environment scope. | Not allowed. |
| `dotenv[]` path strings | Dagu expands before loading dotenv files. | Allowed. |
| `params[].eval`, `preconditions[].eval`, and `steps[].preconditions[].eval` | Dagu expands before dynamic evaluation runs. | Allowed. |
| `params[].default`, runtime parameter overrides, `secrets[].key`, and `secrets[].options` | Dagu does not expand unqualified environment syntax. | Not applicable. |
| `shell`, `shell_args[]`, root `working_dir`, `steps[].working_dir`, `preconditions[].condition`, `steps[].preconditions[].condition`, and `steps[].repeat_policy.condition` | Dagu expands against the current environment scope before the field is used. | Not allowed unless the owning field spec explicitly defines a host-process fallback. |
| `steps[].run` command form, script form, and array entries | Dagu preserves unqualified environment syntax for the selected shell or script interpreter. | Not allowed during Dagu value resolution. |
| `steps[].with.shell`, `steps[].with.shell_args`, and `steps[].with.shell_packages` for `run` steps | Dagu expands against the step environment scope before selecting the shell invocation. | Not allowed. |
| Direct local process command fields, including direct command paths and argv strings owned by an exec-style action | Dagu expands before starting the process. There is no later shell expansion. | Allowed when the owning action spec identifies the field as host-local direct execution. |
| SSH command text fields owned by SSH executor specs | Dagu expands values from the DAG or step environment scope and preserves unresolved unqualified references in command text. SSH executor specs own remote-host handoff behavior. | Not allowed. |
| SSH executor configuration strings | Dagu expands values from the DAG or step environment scope. Unresolved unqualified references remain literal local configuration text. | Not allowed. |
| Container command text fields owned by container executor specs | Dagu expands values from the DAG or step environment scope and preserves unresolved unqualified references in command text. Container executor specs own container runtime and process handoff behavior. | Not allowed. |
| Container executor configuration strings | Dagu expands values from the DAG or step environment scope. Unresolved unqualified references remain literal local configuration text. | Not allowed. |
| General `steps[].with` nested strings for action and executor inputs | Dagu expands against the step environment scope unless a more specific row or owning action spec applies. | Not allowed. |
| Root `container` object strings and `steps[].container` object strings other than `env` | Dagu expands against the owning root or step environment scope. | Not allowed. |
| `steps[].stdin` | Dagu expands against the step environment scope when the step starts. | Not allowed. |
| `steps[].stdout`, `steps[].stderr`, artifact paths, `steps[].stdout.outputs.fields.*` literal strings, and `steps[].output.*` literal or path strings | Dagu expands against the step environment scope when the owning output surface is evaluated. | Not allowed. |
| `steps[].parallel` strings, `steps[].foreach` strings, sub-DAG names, sub-DAG params, handler step fields, and nested value-resolved workflow strings not listed above | Dagu expands against the current environment scope when the owning field is evaluated. | Not allowed. |
| Template script text or fields owned by a template executor | Dagu does not expand unqualified environment syntax unless the owning template spec explicitly opts in. | Not applicable. |
When host process environment fallback is not allowed, an unqualified reference
that is not present in the current environment scope is preserved. Validation
notice reporting is limited to the surfaces defined below.
SSH and container executor specs own the behavior after Dagu hands resolved or
preserved command text to a remote host, container runtime, or container
process.
### Single-Quoted Environment References
- Single-quoted `$NAME` and `${NAME}` are preserved during Dagu environment expansion.
- Exception: in `steps[].run`, a bare reference written as `"${NAME}"` in a template slot is resolved by Dagu and its value is re-quoted, because the shell never expands it there. A template slot is a single-quoted span, such as a JSON body in `'{"key":"${NAME}"}'`, or a here-document body. For POSIX and Nix shells in local runs, an apostrophe in a value resolved into a single-quoted span is written as `'\''`, so the value cannot end the span.
- A secret is never written into the command text of a local POSIX or Nix shell run, even in a template slot; the reference stays for the shell. To pass a secret into a single-quoted body, close the quotes around it: `'{"token":"'"${TOKEN}"'"}'`.
- Quote spans follow POSIX shell rules. An apostrophe inside double quotes, or preceded by an odd number of consecutive backslashes, opens no single-quoted span; after an even number it opens one. A `$(...)` or backquoted command substitution and a `${...}` parameter expansion keep their own quotes, so quotes inside them do not end an outer span. A comment and the body of a here-document open no span.
- Outside template slots, POSIX and Nix shells used for local `steps[].run` commands own `"${NAME}"` like an unquoted reference. Dagu inlines a safe home-relative value such as `~/data` only where the reference is unquoted, so the shell can expand the `~`, and never inlines a secret.
- "Single-quoted" means the reference starts inside a single-quoted span in the field text after YAML parsing.
- YAML single-quoted scalar style does not by itself make an environment reference single-quoted unless the resulting field text contains the quote characters.
- `${env.NAME}` is Dagu-owned syntax and is not protected by shell-style single quotes.
### Shell-Style Environment Expressions
- A shell-style environment expression is one of the supported braced forms in
the table below.
- Most forms start with a valid environment name.
- The length form starts with `#` followed by a valid environment name.
- These expressions are valid only in fields that support environment expansion.
- They read the environment value named `NAME` when Dagu owns environment
expansion for the field.
- If `NAME` is unavailable when environment expansion runs, Dagu leaves the
entire expression unchanged, regardless of the operator.
- Missing required-value forms such as `${NAME:?message}` do not fail Dagu
value resolution; they are preserved.
- If `NAME` is available, Dagu evaluates supported shell parameter operators
for the expression result.
- In the table, `word` and `pattern` are shell-word text inside the same braced
expression.
- Dagu resolves environment references inside `word` and `pattern` using the
same environment scope.
- Dagu does not run command substitution inside `word` or `pattern` unless the
owning field also permits command substitution.
- Supported operator behavior:
| Expression form | Result when `NAME` is available |
| --- | --- |
| `${NAME}` | The value of `NAME`. |
| `${NAME:-word}` | The value of `NAME` when it is non-empty; otherwise `word`. |
| `${NAME-word}` | The value of `NAME`, including an empty value. |
| `${NAME:+word}` | `word` when `NAME` is non-empty; otherwise an empty string. |
| `${NAME+word}` | `word`, even when `NAME` is empty. |
| `${NAME:?word}` | The value of `NAME` when it is non-empty; otherwise the original expression is preserved. |
| `${NAME?word}` | The value of `NAME`, including an empty value. |
| `${NAME:=word}` | The value of `NAME` when it is non-empty; otherwise the original expression is preserved. |
| `${NAME=word}` | The value of `NAME`, including an empty value. If `NAME` is missing, the original expression is preserved. |
| `${NAME:offset}` | The substring from `offset` to the end of the value. |
| `${NAME:offset:length}` | The substring from `offset`, limited by `length`. |
| `${#NAME}` | The decimal count of Unicode code points in the value. |
| `${NAME#pattern}` | The value after removing the shortest matching prefix. |
| `${NAME##pattern}` | The value after removing the longest matching prefix. |
| `${NAME%pattern}` | The value after removing the shortest matching suffix. |
| `${NAME%%pattern}` | The value after removing the longest matching suffix. |
- Substring `offset` and `length` are decimal integers.
- Substring positions are zero-based byte positions in the UTF-8 encoded value.
- A negative `offset` counts backward from the end of the value.
- A missing `length` selects through the end of the value.
- A non-negative `length` selects at most that many bytes after `offset`.
- A negative `length` excludes that many bytes from the end of the value after
applying `offset`.
- Out-of-range substring positions are clamped to the value boundary.
- A substring that splits a multibyte code point is outside portable behavior;
workflow authors should use substring operators only on ASCII values when
they need portable results.
- Pattern-removal `pattern` uses shell glob pattern matching, not regular
expressions.
- Pattern removal does not perform filesystem globbing.
- If the pattern is invalid, Dagu leaves the original value unchanged for that
expression.
- Assignment-like forms such as `${NAME:=word}` and `${NAME=word}` must not
mutate the environment scope used by later references.
- If `NAME` is missing, Dagu preserves the assignment-like expression under the
missing-value rule for shell-style environment expressions.
- If `NAME` is present but empty and `${NAME:=word}` would assign `word`, Dagu
preserves shell-style expressions in that field.
- Simple `$NAME`, `${NAME}`, and `${env.NAME}` references in that field still
resolve according to their normal rules.
- Malformed or unsupported shell-style expressions are preserved unchanged when
Dagu owns environment expansion for the field.
- If Dagu preserves a shell-style environment expression and later hands the
field to a shell, that shell owns any defaulting, slicing, required-value, or
other shell behavior.
- Braced expressions whose base name is not a valid environment name are ordinary string content for this spec.
### Shell and Direct Execution
Namespaced Dagu references:
- Dagu resolves `${env.NAME}` before the owning field is consumed.
- This rule applies even when the field is later executed by a shell, container, or remote host.
Unqualified environment syntax:
- In shell-backed local `run` fields, Dagu leaves `$NAME`, `${NAME}`, and shell-style environment expressions for the selected shell unless the owning field spec explicitly says Dagu owns that syntax.
- In PowerShell-backed fields, `$env:NAME` is PowerShell syntax, not Dagu environment-reference syntax.
- In SSH-backed fields, Dagu must not resolve local process environment values into command text.
- Unqualified environment syntax that is not resolved from the DAG environment scope is preserved for the remote host.
- In container-backed fields, Dagu must not resolve local process environment values into container command text.
- Unqualified environment syntax that is not resolved from the DAG environment scope is preserved for the container runtime or container process.
- In direct local execution without a shell, `$NAME`, `${NAME}`, and shell-style environment expressions are resolved only when Dagu expands environment values for the field.
- Direct local execution has no later shell expansion.
- Unresolved environment references stay unchanged.
Secret-sensitive values:
- `${env.NAME}` inserts the resolved value into the field text.
- If `NAME` is secret-backed, the resulting field is still ordinary resolved
field text unless the secret-owning spec defines additional masking for that
surface.
- In a shell-backed local `run` field, Dagu preserves unqualified `$NAME` and
`${NAME}` for the shell. If the step process environment contains `NAME`, the
shell may still expand that value into the command it executes.
- Spec 006 does not guarantee that a shell, container process, remote host,
subprocess, CLI argument list, log, or network target keeps a secret hidden
after Dagu hands the field to that runtime.
- On surfaces where Dagu preserves unqualified environment syntax for a later
runtime, such as shell-backed local `run`, workflow authors can avoid Dagu
inserting a secret into field text by passing the secret through the
environment and using `$NAME` or that runtime's native environment syntax.
The target runtime's own exposure behavior still applies.
- On Dagu-expanded surfaces, including general action inputs and direct local
process fields, `$NAME`, `${NAME}`, and `${env.NAME}` can insert a secret into
resolved field text.
### Validation
- `dagu validate` must reject invalid `env` declaration shapes.
- `dagu validate` must reject invalid environment variable names in `env` declarations.
- `dagu validate` must reject a sequence `env` string item that does not contain `=`.
- `dagu validate` must reject secret declarations whose names start with
`DAGU_` or collide with secret-reserved Dagu-managed environment names.
- Braced text that does not match a supported environment reference form is ordinary string content.
- An `env` entry that references itself or a later entry in the same declaration must preserve the original reference text.
- Explicit inspection surfaces must report a passive notice for that preserved reference.
- `dagu validate` must report passive notices for unresolved namespaced value
references, such as `${env.NAME}` and `${steps.step_id.outputs.name}`, in
inspected value-resolved fields.
- An unresolved `${env.NAME}` is a runtime-only notice, because the operator
supplies the value when the workflow runs. `dagu validate` records it and
exposes it through the inspection API, and prints it only under
`--show-unresolved`.
- `dagu validate` must report passive notices for unresolved unqualified
`$NAME` and `${NAME}` references in `env` declarations.
- `dagu validate` is not required to report every unresolved unqualified
reference outside `env` declarations.
- A direct local process field with host process environment fallback allowed
may validate without a passive notice even when a missing unqualified
reference would be preserved during validation.
- `dagu validate` must not require runtime environment values, process
environment values, dotenv values, predecessor step outputs, or current-step
Dagu-managed values to exist.
## Errors
- Invalid `env` declaration shape must fail validation.
- Invalid environment variable name in an `env` declaration must fail validation.
- A sequence string entry without `=` must fail validation.
- A supported environment reference with no value preserves the original text.
- Explicit inspection surfaces report passive notices according to the
validation rules above.
- Normal run execution must not emit passive value-reference notices as run logs, workflow events, status data, history data, artifacts, or DAG-run detail data.
## Examples
Ordered environment entries:
```yaml
env:
- SERVICE=api
- API_HOST=${env.SERVICE}.internal
steps:
- name: print_api_host
run: echo ${env.API_HOST}
```
Step environment values:
```yaml
steps:
- name: deploy
env:
- SERVICE=api
with:
host: api.${env.SERVICE}.internal
```
Map-form ordering:
```yaml
env:
SERVICE: api
API_HOST: ${env.SERVICE}.internal
steps:
- name: print_api_host
run: echo ${env.API_HOST}
```
Environment expansion without a namespace:
```yaml
env:
- SERVICE=api
steps:
- name: deploy
with:
host: api.${SERVICE}.internal
```
Shell-backed `run`:
```yaml
env:
- SERVICE=api
steps:
- name: shell_run
run: echo "$SERVICE ${env.SERVICE}"
```
In this example Dagu resolves `${env.SERVICE}` before the command starts.
The shell reads `$SERVICE` from the step process environment.
Direct execution without a shell:
```yaml
env:
- SERVICE=api
steps:
- name: direct_exec
action: exec
with:
command: /usr/bin/printf
args:
- '%s\n'
- ${SERVICE}
```
Missing environment value preserved by Dagu, then consumed by the shell:
```yaml
steps:
- name: optional_env
run: echo ${OPTIONAL_ENV}
```
Dagu leaves `${OPTIONAL_ENV}` unchanged before starting the shell.
A POSIX shell then expands the unset value to an empty string.
Missing environment value preserved in direct execution:
```yaml
steps:
- name: direct_exec
action: exec
with:
command: /usr/bin/printf
args:
- '%s\n'
- ${OPTIONAL_ENV}
```
Because no shell runs after Dagu, the direct process receives the literal argument `${OPTIONAL_ENV}`.
Secret-sensitive command text:
```yaml
secrets:
- name: API_TOKEN
provider: env
key: PROD_API_TOKEN
steps:
- name: call_api
run: curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com
```
The step process reads `API_TOKEN` from its environment.
Dagu does not insert the token while resolving the `run` field because
`$API_TOKEN` is shell-owned syntax. The shell may still expand the token after
Dagu starts the step process.