RELEASE_NOTES.rst
Rama main · réplica de lectura del repositorio original.
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
.. contents:: Apache Airflow Releases
:local:
:depth: 1
.. note::
Release notes for older versions can be found in the versioned documentation.
.. towncrier release notes start
Airflow 3.4.0 (2026-11-06)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Task Loops (#73005, #74222, #74275, #74345)
"""""""""""""""""""""""""""""""""""""""""""
Airflow 3.4.0 introduces Task Loops (AIP-111): a decorated task group can be repeated a number of
times decided at runtime using ``.loop()``. A conditional loop works like a ``do ... while`` loop -
the body runs at least once and an ``until`` condition decides when to stop, bounded by
``max_iterations``. Each iteration can use the previous iteration's result and iterations run one
after another (tasks within an iteration can still run in parallel). Retried attempts are kept
under a per-attempt UUID so each iteration's data is preserved and can be cleared selectively.
For detailed usage instructions, see :doc:`/authoring-and-scheduling/loops`.
.. note::
Downgrading to any version earlier than 3.4.0 can be slow, depending on the number of
XCom rows that have been written after upgrading. Upgrading was tested on a
300m row XCom table, so please don't worry about upgrade time (we made sure
that the XCom *data* didn't get touched or read during migration to keep it
fast).
This is because this release contains a migration that starts storing new XCom
rows in an xcom_v2 table, and in order to downgrade we have to move those over
to the old table, and there is no way of doing this without copying the
written xcom values.
The downgrade refuses to continue when it finds data it cannot map back
safely. Each migration commits separately, so the later 3.4.0 migrations
have already been reversed at that point: fix the data and re-run the
downgrade, or run ``airflow db migrate`` to return to 3.4.
On MySQL a failure after those checks leaves the schema partly downgraded,
as MySQL cannot roll back DDL. Taking a backup before attempting the
downgrade on MySQL is advised, so it can be restored in that case.
Dag Importer for non-Python Dags (#72369, #73728, #73853, #74020, #74084, #74333, #74586)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow 3.4.0 can author an entire Dag - schedule, tasks, and dependencies - in a language other
than Python. AIP-85 adds a pluggable Dag Importer in the Task SDK that teaches the Dag processor to
recognize source formats beyond ``.py``: each format registers an importer, the processor runs the
matching language runtime, and the resulting Dags surface in Airflow like Python Dags. The formats
that land on it are the TypeScript, Go, and Java language SDKs and a native YAML Dag format (a
``.yaml`` file with a ``$schema`` marker, validated against a schema and built into real operators,
with explicit templating). The Dag Importer framework and the language SDKs are experimental in 3.4
and their APIs may change.
For detailed usage instructions, see :doc:`/administration-and-deployment/dag-importers`.
Unify executor workload queues under ``BaseExecutor.executor_queues`` (#63491)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Executor workload state is now stored on the unified ``BaseExecutor.executor_queues`` mapping keyed
by ``WorkloadType`` and scheduled through ``trigger_workloads``. The previous per-type attributes
and entrypoints (``queued_tasks``, ``queued_callbacks``, ``supports_callbacks``, ``trigger_tasks``,
``order_queued_tasks_by_priority``) and the Airflow 3.3 connection-test surface are folded into
this mapping and kept as backward-compatible shims that emit ``RemovedInAirflow4Warning`` (to be
removed in Airflow 4.0). Out-of-tree executors should declare ``supported_workload_types`` and
implement ``trigger_workloads``. Connection tests are now admitted under the same per-heartbeat
budget as tasks and prioritized ahead of them. See
`#63491 <https://github.com/apache/airflow/pull/63491>`_ for the full migration guide.
psycopg3 is now the default Postgres driver (#68496)
""""""""""""""""""""""""""""""""""""""""""""""""""""
psycopg3 (``psycopg``) is now the default driver for the metadata database, for both the sync and
the async engine, replacing psycopg2 and asyncpg. A bare ``postgresql://`` (or a legacy
``postgres://`` / ``postgres+psycopg2://``) ``sql_alchemy_conn`` is rewritten to
``postgresql+psycopg://``, and the derived async URL uses ``postgresql+psycopg_async://`` when
psycopg3 is installed (falling back to asyncpg otherwise). Airflow also disables psycopg3's
server-side statement preparation (``prepare_threshold = None``) so the metadata database works out
of the box behind transaction-mode PgBouncer, the recommended production setup. To keep psycopg2 or
asyncpg, install the ``apache-airflow-providers-postgres[psycopg2]`` / ``[asyncpg]`` extra and
configure the connection explicitly; see #68496 and #69469 for details, including how to re-enable
statement preparation.
Asset-triggered Dags create one run per event by default (#68517)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Single-asset schedules and asset OR expressions now create one Dag run per event instead of
consuming all queued events in one run. This also applies to ``AssetOrTimeSchedule`` and
``PartitionedAssetTimetable``.
To retain batching, set ``batch_asset_events=True`` on the timetable or enable ``[scheduler]
batch_asset_events`` and reparse the affected Dags. Already serialized timetables without this
option retain their previous batching behavior until reparsed.
Schedules requiring several events, including ``schedule=[a, b]``, asset AND expressions, and
partitioned timetables using ``RollupMapper``, continue to batch by default. Explicitly setting
``batch_asset_events=False`` for these schedules raises a timetable validation error.
Custom graph colors return via ``ui_color`` and ``ui_fgcolor`` (#68760)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The ``ui_color`` and ``ui_fgcolor`` attributes on operators and TaskGroups color the graph view
again -- the node fill from ``ui_color`` and the operator label from ``ui_fgcolor``, as they did in
Airflow 2. They were silently ignored since Airflow 3.0.
The value can be a raw color (a hex code like ``#e8b7e4`` or a CSS name) or a Chakra theme token --
a palette token such as ``blue.500`` or a semantic token such as ``brand.solid`` that adapts to
light and dark mode (including tokens added through custom UI theming); a token resolves through a
theme-controlled CSS variable. The node border keeps reflecting the task-instance run state; the
custom color is applied to the fill and label only.
``validate_key`` now raises ``ValueError`` instead of ``AirflowException`` (#68890)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The ``airflow.utils.helpers.validate_key`` utility now raises ``ValueError`` instead of
``AirflowException`` when an invalid key is provided (for example an invalid format or a
length-limit violation).
The deprecated ``Dataset`` alias has been removed from the ``airflow`` module (#70154)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Use ``Asset`` from ``airflow.sdk`` instead.
Audit log rows not tied to a Dag now require the ``AUDIT_LOGS_ALL`` view (#70759)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Rows in the audit log whose ``dag_id`` is ``NULL`` record operations that are not tied to a Dag,
such as Connection, Variable and Pool changes. They were returned to any caller that could read
event logs at all, because there is no per-Dag key to authorize them on and both read paths fell
back to a Dag-level audit log check.
They are now gated on a dedicated ``AccessView.AUDIT_LOGS_ALL``, in the same way
``AccessView.IMPORT_ERRORS_ALL`` gates import errors for files with no registered Dag. Rows bound
to a Dag keep their existing per-Dag check.
- ``GET /eventLogs`` no longer returns rows with a ``NULL`` ``dag_id`` to callers without the new
view, and excludes them from ``total_entries`` and pagination. ``GET /eventLogs/{event_log_id}``
returns ``403`` for such a row. An id matching no row still returns ``404``.
- With the simple auth manager the view is granted to the ``ADMIN`` role only. With the FAB auth
manager it maps to a new ``All Audit Logs`` resource, which ships in the ``Admin`` role; custom
roles that need to read these rows must be granted ``All Audit Logs.can_read`` explicitly.
- An auth manager that cannot map a requested ``AccessView`` now denies that view and emits a
``UserWarning`` instead of raising ``KeyError``. This affects auth managers older than the core
they run against, which previously failed the request with a server error.
Team names must now be lower case, and cannot contain two consecutive underscores (#70886)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``airflow teams create`` and ``airflow teams sync`` now accept only names matching
``(?!.*__)[a-z0-9_-]{3,50}`` -- lower case letters, digits, hyphens and single underscores. Upper
case letters and ``__`` were previously accepted.
A team name is embedded in the ``AIRFLOW_CONN__<TEAM>___<ID>`` environment variable namespace that
holds team scoped secrets, and the backend upper-cases the name when it builds that variable.
``data_eng`` and ``Data_Eng`` therefore resolved a single namespace between them, and a name
containing ``___`` could not be told apart from the separator.
``airflow db migrate`` lower-cases every stored team name, moving that team's Dag bundles,
Connections, Variables, Pools and Triggers with it. Two stored names that differ only in case would
collapse onto one row and silently merge two teams, so migration refuses to start while any such
pair exists -- rename all but one first. Names that stay invalid for another reason, such as
containing ``__``, are migrated as they are; they keep working, but ``airflow teams sync`` exits
non-zero until they are renamed.
Changes to how pending asset events trigger Dag runs (#70972)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Deleting asset event records with ``airflow db clean`` now also removes any asset events that are
still pending for a downstream Dag. This mainly affects Dags waiting on a multi-asset condition,
where a pending event can be purged before the condition is met. Deleting an asset event now
consistently means it will not trigger a run; previously such a pending trigger could still fire
after the event had been purged.
Relatedly, an asset event that was queued for a Dag while it was scheduled on that asset is still
used to trigger the run, and appears among the run's triggering events, even if the asset is later
removed from the Dag's schedule. Previously, they were silently dropped.
Callbacks now redeem a single-use token before they run (#71192)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
A worker now redeems a callback's single-use token with the API server (``PATCH
/execution/callbacks/{callback_id}/run``) before importing and running the callback. Redeeming the
token moves the callback from queued to running and swaps it for a short-lived execution-scoped
token. Before this change a callback ran without the worker making any authenticated call, so the
token minted for it was never checked.
Because the token is single-use, a redelivered or replayed message that reaches a callback that is
already running or finished is refused rather than run again. A worker on 3.4 needs an API server
that serves this endpoint; a worker talking to an older API server negotiates the API version and
behaves as though the endpoint is not there.
Clearing a task now discards its task state store entries by default (#72100)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Clearing a task instance now discards its ``task_state_store`` entries, so the next attempt starts
from the beginning instead of resuming from a checkpoint or reconnecting to an external job
recorded by the cleared attempt. Retries are unaffected and keep task state as before. This applies
to clearing individual task instances (the clear endpoint/dialog and ``airflowctl tasks``/``dags
clear``); clearing a whole Dag run still keeps task state (tracked in
`#72929 <https://github.com/apache/airflow/issues/72929>`_). To preserve the old behaviour, pass
``keep_task_state=True`` (or tick "keep task state" in the clear dialog; a per-browser default is
available under Settings > Clearing). Note a task-instance clear now also deletes
``task_state_store`` rows for that task instance.
``[core] execute_tasks_new_python_interpreter`` now applies to Airflow 3 task processes (#72164)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
On Airflow 3 the option had no effect on task execution (only the Edge worker read it). When set to
``True``, the task supervisor now ``exec``\ s a fresh interpreter right after forking the task
process, which prevents the fork from inheriting a lock held by a supervisor thread (a permanent
hang at the task's first TLS call). Deployments that kept the option ``True`` from Airflow 2 get
this behaviour, and its per-task interpreter start-up cost, on upgrade without further action; set
it to ``False`` to keep bare fork. The task process reads the global value, so a team-scoped Edge
setting alone does not turn it on. Edge workers with the option ``True`` already start a fresh
interpreter for the supervisor and will now start a second one for the task. The Dag processor, the
triggerer and task callbacks (which the callback supervisor runs from a closure that cannot be
named for an exec'd child) are not affected and keep bare fork.
Gracefully pause Dags by draining active runs (#72407, #73225, #73055, #73763)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow now supports a ``draining`` scheduling state for Dags. Draining stops the scheduler from
creating new scheduled or asset-triggered Dag runs while allowing already queued and running Dag
runs to continue. Explicitly triggered runs remain allowed.
After outstanding Dag runs finish, the Dag automatically transitions to ``paused``. Users can
cancel draining to return the Dag to ``active``.
Manual runs, backfills and asset materializations on a paused Dag can drain the Dag instead of
unpausing it: choose **Drain** in the UI, or set ``drain_dag`` in the REST API request. Draining
lets unfinished runs of the Dag start or resume, except those held by paused backfills, creates no
scheduled runs, and pauses the Dag again once all unfinished runs finish. Paused backfills remain
paused and must be resumed for their runs to finish and draining to complete.
The UI, REST API, and ``airflowctl`` expose the ``active``, ``draining``, and ``paused`` scheduling
states, and the Dags list can pause, drain, or unpause several selected Dags at once. Existing
``is_paused`` API usage remains supported; use ``scheduling_state`` when the draining state must be
selected or reported explicitly.
Reparsing a file with no registered Dag now requires the ``REPARSE_ALL`` view (#72896)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
A file that failed to import before defining any Dag has no per-Dag key to authorize a reparse
against, so ``PUT /parseDagFile/{file_token}`` previously returned ``404`` for it. Reparsing such a
file is now gated on a dedicated ``AccessView.REPARSE_ALL`` -- a write action of its own, distinct
from the permission to view import errors -- and the Dag import errors modal gains a reparse action
for these files.
- ``PUT /parseDagFile/{file_token}`` for a file with no registered Dag authorizes against
``REPARSE_ALL`` and returns ``403`` when the caller lacks it, instead of ``404``. A file that has
a registered Dag keeps its existing per-Dag ``can_edit`` check.
- With the simple auth manager the view is granted to the ``ADMIN`` role only. With the FAB auth
manager it maps to a new ``All Reparses`` resource, which ships in the ``Admin`` role; custom
roles that need to reparse such files must be granted ``All Reparses.can_read`` explicitly.
Removed the ``--file-io`` option from the ``airflow info`` command (#73054)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``--file-io`` uploaded the report to file.io and printed a shareable link. file.io no longer
accepts anonymous uploads, so the option could not succeed; it also ended in an unhandled traceback
rather than reporting the failure.
Redirect the output instead::
airflow info --anonymize > airflow-info.txt
Note that ``--file-io`` applied ``--anonymize`` implicitly. Pass ``--anonymize`` explicitly when
sharing a report: plain ``airflow info`` prints connection strings, including passwords, in
cleartext.
Task heartbeats are no longer retried by ``[workers] execution_api_retries`` (#73244)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Heartbeats had two overlapping retry mechanisms: the Task SDK API client retried every request,
including heartbeats, and the supervisor separately counted consecutive heartbeat failures against
``[workers] max_failed_heartbeats``. The client retries also had the unwanted side-effect of
blocking the supervisor's monitor loop while it backed off, so during an API server outage the
supervisor would be delayed forwarding task logs and/or noticing that the task subprocess had
exited.
Heartbeat retries are now governed solely by ``max_failed_heartbeats``, whose default has been
adjusted from ``3`` to ``30`` to keep roughly the same tolerance for an unreachable API server as
the two mechanisms combined gave before.
If you have set ``[workers] max_failed_heartbeats`` explicitly, you may wish reconsider value you
have chosen. It is now the whole retry budget: with the default ``[workers]
min_heartbeat_interval`` of 5 seconds, the worker tolerates roughly ``5 * max_failed_heartbeats``
seconds of API unavailability before terminating a running task. You should likely keep the
resulting value below any configured ``[scheduler] task_instance_heartbeat_timeout`` (default 300
seconds).
Deadline alert callbacks validated at parse time; numeric intervals deprecated (#73294)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``DeadlineAlert`` now refuses a ``callback`` that is not exactly ``AsyncCallback`` or
``SyncCallback``, so a subclass fails when the Dag is parsed rather than when the scheduler reads
it back. The scheduler decodes stored callbacks by exact class name so no class named by a Dag
author is imported or instantiated there, which means a subclass could be written and then never
read.
Passing a number as the ``interval`` is deprecated and now emits ``RemovedInAirflow4Warning``. It
is still accepted and taken as a count of seconds, but the supported types are ``timedelta`` and
``VariableInterval``. A ``timedelta`` subclass such as ``pendulum.duration(...)`` is accepted and
normalized to a plain ``timedelta``, where previously it passed validation and then failed during
serialization. ``interval=True`` is refused, because ``bool`` is an ``int`` subclass and would
otherwise be taken as one second.
The ``POST /assets/{asset_id}/materialize`` endpoint now requires a JSON request body (#73328)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously a request with no body was accepted and the asset was materialized with default values.
The endpoint now requires an explicit JSON body, consistent with the other mutating endpoints: a
request without a body is rejected with ``422``, and an empty HTML form is not a valid input
anymore. All body fields remain optional, so an empty object (``{}``) is a valid body.
TaskInstance try numbers now increase before the next try starts (#73554)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
.. note::
If you have any external systems that look at task logs or TI state for an up-for-retry TI, you
will need to be aware of this change and update your code.
For example, if you made a request to ``GET
/api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}`` expecting to get info about
the failed try, you will now need to look at ``GET
/api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/tries/{try_number - 1}``
When a task is up_for_retry, try_number now identifies the upcoming try. To retrieve the try that
failed, request ``/tries/{try_number - 1}``. The UI shows the pending try and provides a link to
the previous try's logs.
Airflow now allocates a task instance's UUID and try number together immutably so they consistently
identify the same try.
Previously when a task entered the retry cooldown period, it would be given a new UUID, but keep on
try number N and have the state of UP_FOR_RETRY until that period was up (at which point that try
would be marked FAILED and the try number incremented). Instead, the failed try is immediately
marked with state of FAILED, and a new try created in state UP_FOR_RETRY.
Deprecate cyclic TaskGroup dependencies (#73746)
""""""""""""""""""""""""""""""""""""""""""""""""
A Dag whose TaskGroups depend on each other in a cycle, when each TaskGroup is treated as a single
unit, now issues a ``TaskGroupCycleDeprecationWarning`` at parse time and shows a Dag warning in
the UI that names the TaskGroups and tasks involved. These Dags still parse and run, because their
task dependencies are acyclic, but Airflow 3.5 is planned to reject them at parse time.
A dependency into or out of any task in a TaskGroup counts as a dependency of the whole group, so a
path that leaves a TaskGroup and comes back into it is a cycle. For example, ``a >> bridge >> b``,
with ``a`` and ``b`` in the same TaskGroup and ``bridge`` outside it, makes the group depend on
``bridge`` and ``bridge`` depend on the group, even if ``a >> b`` is also set.
- Move tasks between TaskGroups, or out of them, so that each group depends on the others in one
direction only. In the example above, move ``bridge`` into the group, or move ``b`` out of it.
- To catch these Dags in CI, run your Dag tests with ``-W
error::airflow.sdk.exceptions.TaskGroupCycleDeprecationWarning``. The check runs when a Dag is
added to a ``DagBag``, so only tests that load your Dags into a ``DagBag`` and assert there are
no import errors catch them; a test that only imports a Dag file does not.
See "Cyclic TaskGroup dependencies" in the Dags documentation.
Language SDK coordinators load task handlers from a Dag bundle (#73970)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The experimental ``jars_root`` (Java) and ``executables_root`` (Go) coordinator kwargs are removed.
Put the artifacts in a Dag bundle registered in ``[dag_processor] dag_bundle_config_list`` and set
``task_handler_bundle_name`` in the coordinator's ``kwargs``, or leave it unset to use the task's
own Dag bundle. A config that still sets a removed kwarg fails with ``Cannot instantiate
coordinator '<key>'``.
Support for Python 3.10 has been removed (#74151)
"""""""""""""""""""""""""""""""""""""""""""""""""
Airflow now requires Python 3.11 or newer. Python 3.10 reaches its end of life in October 2026, and
Airflow drops support for a Python version in ``main`` as soon as it reaches end of life. The
``apache-airflow`` distribution, the Task SDK, ``airflowctl`` and all providers released from
``main`` declare ``requires-python = ">=3.11"``.
Deprecate the ``required_kwargs`` attribute for custom deadline references (#70714)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Implement ``_evaluate_with()`` with a ``dagrun`` parameter instead.
CI and production images are now based on Docker Hardened Images and Debian 13 (#73038)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow's CI and production images are now based on `Docker Hardened Images <https://dhi.io>`_ for
Python and Debian 13 "trixie".
Python is no longer compiled from source during the image build - it comes from the hardened base
image instead, which removes the source download, its signature verification and the compilation
itself from the build. Python still lives in ``/usr/python``, which is now a symlink to the
``/opt/python`` location the base image uses, so paths inside the image are unchanged.
The ``BASE_IMAGE`` build argument now defaults to
``ghcr.io/apache/airflow/base/python:<version>-debian13-dev``, Airflow's public mirror of the
upstream ``dhi.io/python`` image, so building an image needs no registry credentials. Anyone
overriding ``BASE_IMAGE`` with a plain ``debian:bookworm-slim`` image has to switch to an image that
already provides Python, or build the ``legacy`` flavor.
The base OS moved from Debian 12 "bookworm" to Debian 13 "trixie". Library packages renamed in
trixie's 64-bit ``time_t`` transition (for example ``libssl3`` is now ``libssl3t64``) have to be
updated in custom ``RUNTIME_APT_DEPS`` / ``ADDITIONAL_RUNTIME_APT_DEPS``, and the MySQL-compatible
client now comes from MariaDB 11.8.
For the 3.4.x line, production images built the previous way - Python compiled from sources on
``debian:bookworm-slim`` - are still published as the deprecated ``legacy`` flavor, selected with
the new ``AIRFLOW_IMAGE_FLAVOR="legacy"`` build argument. They will be removed in Airflow 3.5.0.
The ``PYTHON_LTO`` build argument is gone. To build a FIPS-compliant image, point ``BASE_IMAGE`` at a
FIPS variant of the hardened image instead, for example ``dhi.io/python:3.13.16-debian13-fips-dev``.
Task SDK API client trusts the system certificate store instead of ``certifi`` (#73116)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
With ``[api] client_use_public_certs`` enabled (the default), the Task SDK client that talks to the
Execution API now trusts the operating system's certificate store through ``truststore`` instead of
the ``certifi`` CA bundle. CAs from ``[api] ssl_ca_file`` and ``[api] ssl_cert`` are still trusted
in addition to the system store, and ``certifi`` is no longer a Task SDK dependency.
Workers must have the public CAs they rely on installed in the system store (for example, the
``ca-certificates`` package on Debian-based images). On macOS and Windows, certificate validation
is delegated to the operating system, which can reject server certificates that OpenSSL accepted,
such as private-CA certificates without the ``serverAuth`` extended key usage.
Older workers can no longer resume deferred tasks whose ``next_kwargs`` hold custom classes (#73577)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
When a deferred task resumes on a worker with an older Task SDK, the API server no longer reconstructs arbitrary classes from the stored ``next_kwargs`` to convert them to the legacy format. Built-in value types, assets and asset aliases (including asset access control) are still converted, without constructing them. A deferred task whose ``next_kwargs`` hold any other class now fails on such a worker with an error naming the type and asking to upgrade the Task SDK, instead of crashing on a payload it cannot decode.
**What to do:** if you run workers with ``apache-airflow-task-sdk`` 1.1.x or older and your deferrable operators or triggers pass instances of custom classes in ``next_kwargs`` (for example via ``self.defer(..., kwargs=...)`` or in the trigger event payload), upgrade the Task SDK on those workers to 1.2.0 or later. Newer workers receive ``next_kwargs`` unchanged and are not affected. Until all workers are upgraded, pass only built-in values, assets and asset aliases in ``next_kwargs`` of tasks that may resume on older workers.
Deadline Alert callback kwargs are decoded where the callback runs (#74830)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The keyword arguments of Deadline Alert callbacks are no longer decoded by the scheduler. They
are kept in their serialized form and decoded where the callback runs: by the triggerer for an
``AsyncCallback``, and by the callback runner on the worker for a ``SyncCallback``.
Workers that run ``SyncCallback`` Deadline Alerts need a Task SDK that includes this change, so
upgrade ``apache-airflow`` and ``apache-airflow-task-sdk`` on workers together. A worker with
an upgraded ``apache-airflow`` and an older Task SDK fails such callbacks with an error instead
of running them. Callbacks created before the upgrade run as before.
Custom XCom backends that override ``serialize_value`` must implement ``store_value`` to support task loops (#74858)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Every iteration of a task inside a loop is its own task instance, but ``serialize_value``
receives the same ``dag_id``, ``run_id``, ``task_id``, ``map_index`` and ``key`` for all of
them, so a backend that builds its storage location from those makes every iteration's XCom
read back as the last iteration's value. Backends should override ``store_value`` and
``load_value`` instead, and build the storage location with ``XComCoordinate.build_path``,
which is unique per task instance, or per write when the write cannot know the task instance.
A Dag that uses a loop fails on startup when the configured XCom backend still stores values
through ``serialize_value``, even if it already writes to a unique location. This includes
``XComObjectStorageBackend`` from ``apache-airflow-providers-common-io`` 1.10.1 and earlier;
upgrade the provider before using task loops with it.
Custom Task State Store backends will likewise need to include
``scope.region_id`` in their storage key to support mapped tasks inside a loop
and loop iterations regenerated by clearing.
``airflow.utils.email`` is deprecated in favor of the SMTP provider (#75414)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The functions in ``airflow.utils.email`` (``send_email``, ``send_email_smtp``, ``send_mime_email``,
``build_mime_message`` and ``get_email_address_list``) now live in
``airflow.providers.smtp.utils.emailer``. The old names still work and raise a ``DeprecatedImportWarning``.
Airflow now requires ``apache-airflow-providers-smtp>=3.2.0``.
Keep ``[email] email_backend`` set to ``airflow.utils.email.send_email_smtp``. Airflow treats that value as
"send task emails with ``SmtpNotifier``", which reads all SMTP settings from the Connection. Changing it to
the provider path sends through the ``[smtp]`` settings instead.
If you call these functions directly:
* The SMTP Connection is looked up with ``BaseHook.get_connection``. A script that runs outside Airflow's
processes needs the Connection in an environment variable or a secrets backend, not only in the metadata
database.
* Settings are read through the Task SDK configuration, so values changed at runtime with
``airflow.configuration.conf.set()`` are not seen. ``airflow.cfg`` and environment variables work as before.
* Tests that patch ``airflow.utils.email.send_mime_email`` no longer intercept the call made by
``send_email_smtp``. Patch ``airflow.providers.smtp.utils.emailer.send_mime_email`` instead.
- The API server uses a second database connection pool
API server workers now run some database work on the async engine as well as the sync one, and each engine has its own connection pool, sized by the same ``[database] sql_alchemy_pool_size`` and ``[database] sql_alchemy_max_overflow`` settings. What uses the async pool depends on the ``--apps`` option of ``airflow api-server``:
- ``execution``: task heartbeats and several Execution API reads.
- ``core``: the audit log entry recorded for each write request through the REST API or the UI.
- ``all`` (the default): both of the above.
In every mode a worker can therefore open up to twice ``sql_alchemy_pool_size + sql_alchemy_max_overflow`` connections (30 with the defaults) and keep up to twice ``sql_alchemy_pool_size`` of them idle. Account for this, per ``[api] workers``, when sizing PgBouncer or the database's ``max_connections``.
The async pool is used since #67800, #73403, #73407 and #76416. (#76430)
New Features
^^^^^^^^^^^^
- UI: Show task group states as stacked cards in the Grid view (#75442)
- Let plugins add their own actions to Dag, run and task instance rows (#75427)
- TS SDK: Reject blank edge labels and label task edges into groups (#75190)
- Expose task instance loop pass and expansion size to listeners (#75132)
- Add a Duplicate action to the Connections list (#75110)
- Let retry policies reschedule a task without using a retry (#74993)
- Let plugin views be scoped to specific Assets (#74992)
- Add Map Index search to dag breadcrumb (#74893)
- UI: Show secondary states of mapped tasks in the Grid view (#74883)
- Allow loop gates to add human-readable decisions (#74806)
- Java SDK: Let a native Java Dag set one queue for all its tasks (#74538)
- Java SDK: Add edge labels to native Dags (#74523)
- Add asset ID filtering to ``airflow db clean`` (#74498)
- TS SDK: Add edge labels with label() (#74494)
- Add dag version lookup on get task endpoint (#74365)
- Add Logical Date column to the Runs table (#74340)
- UI: Add a Dag Run ID column to the Task Instances list (#74316)
- UI: Add duration filter to DAG Backfills tab (#74202)
- Add a user setting for the default search match mode (#74021)
- Add URL param support to calendar view (#73237) (#73762)
- UI: Add team filter to Human-in-the-loop task instances listing (#73726)
- add bundle_name tags to missing dag processor metrics (#73717)
- Add deadline callback log visibility to the Dag Run view in UI (#73580)
- Add fab-oauth meta-package extra (#73565)
- Allow a Job to be associated with several teams (#73561)
- UI: Add time range filtering to the Gantt view (#73555)
- Add ``ChainRetryPolicy`` to compose retry policies in order (#73508)
- Add task state store API to the Java SDK (#73464)
- UI: Add a default preference for expanded task groups (#73429)
- Add task state store support to the Go SDK task client (#73420)
- UI: Add team column and filter to the jobs list (#73295)
- Add SetVariable and DeleteVariable to the Go SDK task client (#73084)
- Allow template-field copies into StartTriggerArgs in operator __init__ (#73040)
- Allow plugins to add or override UI translations (#72648)
- Support multi-column sort in the Dags list table (#72558)
- Add dry-run mode to teams sync (#72494)
- Add Has Events filter to Assets search (#72460)
- UI: Add 'Mark as Skipped' option to Task Instance actions (#72425)
- UI: Add toggle to show or hide log levels in task logs (#72363)
- Add async Variable methods (aget, aset, adelete, akeys) (#72329)
- UI: Add team column and filter to the XComs list (#72158)
- UI: Add team column and filter to the deadlines list (#72060)
- Add integrity metadata to TypeScript bundles (#72046)
- Multi-Team: Add teams inspect CLI command to display team resources (#72036)
- ``is_scheduled`` REST API parameter (#72029)
- Allow Variable.get to reuse the caller's database session (#71968)
- UI: Add team column and filter to the audit log (#71916)
- Make the scheduler Dag cache size and TTL configurable (#71816)
- Support a regex for the API's CORS allowed origins (#71778)
- CLI: Add --limit flag to airflow dags list-runs (#71729)
- Add SSL cipher list option to the api server (#71645)
- Add ConnectionNotFoundError to the TypeScript SDK (#71375)
- Add preset filters to the Dags list page (#71368)
- Add Dag ID filtering to Assets search (#70971)
- Allow filtering the Dag runs list by Dag tags (#70861)
- Support DagBundle based deployment for Coordinator (#70805)
- Add a user settings page for cross-page UI defaults (#70687)
- Support a Unix Domain Socket for the metrics clients (#70517)
- UI: Add a time-window scope to the Dags list run state filter (#70470)
- UI: Add Select Recent Configurations dropdown to Trigger Dag form (#70413)
- UI: Add a slowest tasks chart to the Dag Overview (#70305)
- Add task log message when the Execution API is unreachable (#70290)
- UI: Add team column and filter to Dag Run and Task Instance lists (#70241)
- Support schema-qualified table names in db clean (#70147)
- UI: Add team column and filter to Dag list page (#70028)
- Add React plugin support to Dag and task overviews (#69992)
- Add Asset Tab plugin (#69986)
- Add serialized Dag version diff engine (#69864)
- UI: Add timetable type filter to Dag list (#69728)
- Allow embedding a Dag run in an external trace via run conf (#69633)
- Add support for forwarding CeleryExecutor task logs to stdout (#69597)
- Add deadlines to Calendar Tab (#69559)
- Add team_name attribute to plugins for multi-team (#69502)
- Add exact-match uri filter to GET /assets endpoint (#69489)
- Add runtime DagRun note updates (#69403)
- Add indexes on serialized_dag and dag_code (#69391)
- Add streaming task log support to BaseExecutor and FileTaskHandler (#69299)
- Add airflow-ts-pack bundle build tool for TypeScript SDK (#69295)
- UI: Allow scoping plugin tabs to specific Dags and tasks (#69148)
- Add metrics for how often a dag gets serialised. (#68906)
- Allow forcing trace head-sampling per DAG run via conf (#68860)
- Add Calendar Deadlines API and Data Models (#68604)
- Add a Node.js coordinator for running TypeScript Dags (#68548)
- Add async task store accessors for async tasks (#68232)
- Add scheduler observability metrics (#68068)
- Add extra field filtering for asset events (#64611)
- Add partition_key and partition_key_regexp_pattern filters for asset events (#64610)
- Add ``AssetAndTimeSchedule`` timetable (#58543)
Bug Fixes
^^^^^^^^^
- Fix async engine datetimes being shifted on non-UTC MySQL servers (#76505)
- Scope asset-event ``created_dagruns`` to Dags the caller can read (#76234)
- Keep mapped tasks when clearing with run on latest version (#76420)
- Load Java Dags from a bundle path that holds a colon (#76432)
- UI: Fix task groups not expanding when ``localStorage`` is full (#76302)
- Keep the audit log off the API server event loop (#76416)
- Fix LocalExecutor deadlock when a workload put blocks (#75086)
- TS SDK: Inherit the Dag queue when a task passes queue undefined (#76123)
- UI: Fix Portuguese and Hungarian labels that show a raw placeholder (#75964)
- Fix the 404 on a looped task group's breadcrumb (#76050)
- Fix stale queued asset rows in scheduler cleanup and readiness (#76002)
- Stop Recent Configurations flashing on the trigger form (#75991)
- Avoid unbounded IN lists for queued asset events (#75943)
- Stop requesting the details of an empty Dag id after a bulk selection (#75862)
- Fix hourly cron schedules running an hour off across a DST change (#75847)
- Preserve XCom value types when editing in the UI (#75496)
- Reject non-object JSON files when importing variables (#75384)
- Fix dashboard Active Dags filtering (#75350)
- Stop a stale callback stat from forcing constant Dag file reparsing (#73433)
- UI: Fix a crash caused by an unresolvable Windows PT timezone (#75723)
- UI: Stop keyboard shortcuts firing while typing in a Monaco editor (#75684)
- UI: Fix slow pending Dag run checks on large databases (#75674)
- UI: Fix labels that render as raw translation keys (#75326)
- Let listeners see the end date of a skipped task instance (#75685)
- Refuse Human-in-the-Loop responses once the task stopped waiting (#75645)
- TS SDK: Reject a blank queue in Dag and task specs (#75573)
- Fix scheduler crash when retrying a task that timed out or was killed externally (#75472)
- Fix task loop iteration log links pointing to a nonexistent page (#75458)
- Go SDK: Fail a task whose return-value XCom push fails (#75443)
- UI: Fix visual and console-error bugs in the Dag list and details pages (#75418)
- Decide the ``min_one_success`` trigger rule per index in mapped task groups (#75417)
- Return a 5xx response on a database deadlock when updating task state (#75368)
- Run team tasks on the team executor when named by class or path (#75362)
- Fix task instance filtering for null start dates (#75321)
- UI: Keep the multiselect filter pill's focus ring visible (#75307)
- UI: Show errors when the task clear confirmation check fails (#75269)
- Skip scheduling tasks when their Dag cannot be deserialized (#75263)
- Read a team's running task logs through the team's executor (#75255)
- Keep the token cookie set by a route in ``JWTRefreshMiddleware`` (#75217)
- UI: Scope the partitioned Dag run asset list to readable assets (#75213)
- UI: Reveal selected Dag folders when names contain pipes (#75196)
- UI: Show a label instead of a broken link for hidden assets (#75178)
- Show source code lines in task log tracebacks (#75100)
- Fix ``Trigger.clean_unused`` rejecting a caller-provided session (#75096)
- Reserve graph header width for loop-group iteration badge (#75095)
- Keep Dag folder selection distinct across bundle names (#75080)
- Keep parent Dag folders collapsed after one click (#75079)
- Reject non-positive ``max_active_runs`` when creating a backfill (#75071)
- TS SDK: Fix "Dynamic require" error for CommonJS packages in ``airflow-ts-pack`` (#75067)
- Fix 500 when ordering the Dag list by state or start_date (#75025)
- Run the Kerberos ticket renewer without metadata database checks (#75000)
- Fix serialization of datetimes with a fixed UTC offset (#74922)
- Reset Dag autoregistration state when a Python Dag file fails to load (#74911)
- Reject blank pool names and Variable keys in the REST API (#74896)
- Resolve a trigger's XCom requests as its deferred task instance (#74868)
- Fix JWT token not refreshed when token expires mid-request (#74865)
- Do not warn about bare postgresql:// metadata DB URLs (#74863)
- Fix dynamically mapped tasks never running on scheduled Dag runs (#74861)
- Speed up API server requests by keeping JWT checks off the event loop (#74856)
- Keep ``.airflowignore`` rules when the Dags folder symlink is switched (#74853)
- Match camelCase keys against sensitive field names in masker (#74840)
- Scope asset-wide queued event deletion to editable Dags (#74839)
- Keep queued connection tests from overwriting newer connections (#74837)
- Scope asset Dag and task references to readable Dags (#74836)
- Reject reserved serialization keys in Dag run conf (#74834)
- Fix Dag version diff max-changes border, limit wording and duplicate count (#74818)
- Java SDK: Send log timestamps in UTC (#74783)
- Fix Dag list run badges and counters not refreshing after completion (#74734)
- UI: Fix clearing a task instance doing nothing after Confirm (#74725)
- Report a Dag whose timetable fails to load as an import error (#74709)
- UI: Keep JSON text in string HITL params instead of showing [object Object] (#74702)
- Refuse a HITL response for a request the task has since replaced (#74688)
- Fix ORM models failing to configure in processes that skip database setup (#74642)
- Stop the secrets backend lookup from importing the task runner outside tasks (#74641)
- Go SDK: pack a bundle that registers no Dags at pack time (#74637)
- Fix language SDK Dag parsing failing under pydantic 2.14 (#74628)
- Fix 400 and 500s on POST /connections/defaults in multi-team mode (#74615)
- Fix clearing with upstream and downstream selecting unrelated tasks (#74548)
- Keep the scheduler running when a queued run's Dag fails to deserialize (#74499)
- UI: Fix the Dags list showing an error page and reduce unnecessary re-renders (#74426)
- Fix plugin applies_to scoping (#74425)
- Fix /ui/dags recent-runs query degradation on large, skewed dag_run tables (#74401)
- Fix XCom query in not_previously_skipped_dep to use xcom_entity (#74377)
- Fix short-circuit in a mapped task group not skipping later tasks (#74283)
- Make ``ZIP_REGEX`` case-insensitive to match .ZIP archives (#74269)
- Clear language SDK task XComs in the coordinator when a try starts (#74221)
- Java SDK: Reject byte arrays instead of letting the task hang (#74218)
- UI : Fix duplicate "From" label on DAG backfill filters (#74189)
- Improve LocalExecutor bookkeeping: correctly add tasks to the ``running`` list (#74142)
- UI: Fix XCom list crash when a value request fails (#74131)
- Fix date range filter inputs committing on every keystroke (#74011)
- Fix short-circuit not skipping already expanded mapped tasks (#73960)
- Fix short-circuit inside a mapped task group not skipping downstream (#73959)
- Fix task clear and callback bundles after code-only Dag changes (#73915)
- UI: Fix external plugin nav links not navigating on click (#73913)
- Clear retry policy state when a task instance is cleared (#73889)
- Correctly shutdown async sessions on exit. (#73838)
- Go SDK: Stop log attributes from displacing standard log fields (#73820)
- API: Translate domain ValueError in Task State Store route (#73753)
- Filter jobs API ``is_alive`` in SQL so pagination stays correct (#73684)
- Guard ``register_at_fork`` in shared stats for Windows compatibility (#73680)
- Fix boolean value of start date used in url start_date_gte filter. (#73676)
- Stop a commented-out ``executable_resumable`` from marking a module durable (#73607)
- Fix square corner on fully filled dashboard History bars (#73543)
- Fix empty audit log ``dag_id`` for ``DAG.cli()`` subcommands (#73379)
- Fix CLI --bundle-name finding no Dags in Git bundles (#73377)
- Fix airflow plugins and dags pause printing prose with --output json (#73270)
- Fix 500 from POST ``/auth/token`` on a request body it cannot parse (#73258)
- Fix slow task-log scrolling by memoizing useLogGroups' visible-items index (#73252)
- Fix 500 from the audit logger on a request body it cannot parse (#73226)
- Stop reporting components as degraded after an instance restart (#73222)
- Fix dashboard stats cards navigating to a 404 page (#73178)
- Fix airflow db clean silently skipping three tables when scoped to Dags (#73173)
- Stop the scheduler creating Dag runs for paused partitioned-asset Dags (#73151)
- Fix Dags staying blocked by max_active_runs until the next parse (#73032)
- Fix airflow standalone announcing readiness before the API server is up (#73025)
- Hide assets the caller may not read from the UI dependencies graph (#73008)
- Fix airflow teams list --output json printing prose when no teams exist (#72945)
- Fix ``tasks clear --task-regex`` matching as a plain substring (#72941)
- Avoid a full asset table scan per request in simple auth manager (#72934)
- Fix Dag callbacks dropped when bundle versioning is disabled (#72930)
- Stop one malformed provider from breaking discovery for all providers (#72854)
- Fix backfill DagRuns being permanently starved from scheduling (#72730)
- Fix airflow db check reporting one fewer retry than it will make (#72708)
- Fix airflow dags list --columns help showing wrong default columns (#72607)
- UI: Render Arabic durations with the correct plural form (#72604)
- Fix XCom slice endpoint for crossed bounds (#72584)
- Fix the ``dag_processing.last_duration`` timing metric reporting the wrong value (#72553)
- Stop emitting the legacy first_task_scheduling_delay metric twice (#72512)
- Fix Dag metrics dropped when a tag contains invalid characters (#72364)
- Fix misleading error for malformed version in airflow db downgrade (#72331)
- Fix timedelta serde to accept int payloads (#72326)
- Fix plugin registration returning early on duplicate names in Task SDK (#72253)
- Redact secrets written through ``RedactedIO.writelines()`` (#72063)
- Fix Dag run filtering for null start dates (#71654)
- Fix ``map_index_template`` failing for numeric-looking labels with ``render_template_as_native_obj`` (#71592)
- Keep the MeterProvider configured via OTEL_CONFIG_FILE (#71529)
- Fix Asset Event table titles showing a raw translation key (#71443)
- Fix executor validation error referencing non-existent [core] executors (#71336)
- Fix MySQL metadata database failing when only PyMySQL is installed (#71251)
- Fix OpenSearch remote logging crash if port is empty (#71170)
- Fix duplicate pending Dag runs for the same asset partition key (#71074)
- Fix /ui/assets picking a stale "last event" under concurrent asset writes (#71047)
- Stop airflow config update from writing a backup during dry-run (#70979)
- Fix log template sync failing when a custom SQL Alchemy schema is configured (#70793)
- Drain result_queue before dispatch in LocalExecutor to prevent runtime dispatch deadlock (#70545)
- Fix TypeError clearing a DAG run when a queued-at deadline interval is stored as JSON (#70370)
- Fix manual runs getting a zero-length data interval across DST (#70089)
- Use atomic upsert for asset run queue on MySQL to avoid deadlocks (#69977)
- Don't fail API server startup when a plugin uses an unknown team (#69976)
- Prevent negative trigger queue delay metrics (#69971)
- Fix asset event extra filter matching wrong events for dotted keys on SQLite (#69675)
- Fix @asset reserved keys with default value (#69642)
- Fix ``statsd_ipv6`` config not enabling IPv6 sockets for the default StatsD client (#69632)
- Make sure stale file handler doesn't crash the DagFileProcessorManager (#69324)
- UI: Fix triggered Dag run layout in asset event cards (#69310)
- Fix Dag detail page 500 for Dags without a fileloc #69035 (#69110)
- Fix mapped task group XCom collapsing to a bare value for a single expansion (#69075)
- Fix macOS SIGSEGV in triggerer and Dag processor via fork+exec (#69008)
- Fix branch not skipping mapped tasks with custom XCom backend (#68954)
- Do not abort Dag run creation when a deadline alert fails (#68917)
- Fix duplicate heartbeat-timeout task callbacks (#68008)
- Drain result_queue in LocalExecutor to prevent process join deadlock (#67881)
- Fix missing ``task.queued_duration`` metric in Airflow 3 (#67592)
- Fix CLI connection display reporting wrong is_extra_encrypted flag (#67561)
- Fix remote processor injection happening before dictConfig runs in configure_logging (#66633)
- Fix airflow db clean IntegrityError when a dag_version is referenced by a task_instance (#66350)
- Fix airflow db clean hang on MySQL when delete fails (#66296)
- Fix ``refresh_from_task`` not applied on retries (#65932)
- Fix asset event scheduling race condition and consistency (#62501)
- Fix ``DecreasingPriorityStrategy`` example by initializing ``try_number`` before weight evaluation (#62148)
- Fix connection resolution in CLI by setting server process context in decorators (#61036)
Miscellaneous
^^^^^^^^^^^^^
- Let non-task callers set and delete Variables over the Execution API (#76375)
- Let asset watchers write asset state over the Execution API (#76376)
- UI: Label the Dag bundle file table as Parsed Files (#76462)
- UI: Complete the Italian translation (#76464)
- UI: Update the French translations to 100% coverage (#76461)
- UI: Complete the Spanish translation (#76087)
- UI: Complete and fix the Polish translations (#76429)
- UI: Complete the missing zh-TW translations (#76398)
- UI: Make asset state store entries scrollable (#75870)
- UI: Stop bundling the ELK worker twice (#75864)
- Store and show the retry deadline; start retries with empty run fields (#76027)
- Freeze the GC before the triggerer's forks to keep memory flat (#76016)
- Honor Airflow logging levels in the TypeScript SDK (#76005)
- Make the bundle name less prominent in the Dag import error modals (#75958)
- Load the serialized Dag when starting a task only for stub tasks (#75795)
- UI: Hide the scheduled run icon in the grid legend (#75775)
- Avoid loading timetables for Dags in Execution API endpoints (#75765)
- Add metrics for task and asset state store access (#75729)
- Add ``dag_id`` and ``task_id`` tags to the resumable_job metrics (#75721)
- Run a language SDK runtime in its own process group (#75707)
- UI: Hide empty log groups in the task log view (#75686)
- Require cadwyn 8.1 to allow FastAPI 0.137+ (#75689)
- Report language SDK runtime launch failures in the task log (#75561)
- UI: Remove the info tooltip from the Dag version comparison (#75526)
- UI: Show the filtered task count in the Dag Tasks tab heading (#75448)
- TS SDK: Take the task state store retention in seconds (#75445)
- UI: Show the Dag version diff occurrence count on the path (#75433)
- UI: Make the Dags sidebar button a link and move Dag Runs and Task Instances under Browse (#75422)
- Optimize the scheduler's asset orphanage check with an ``EXISTS`` subquery instead of a fan-out join (#75404)
- UI: Clean up the Dag Version tab (#75244)
- UI: Warn when the logical date is outside the Dag's start and end dates (#75223)
- Go SDK: Write the edge label text before the task (#75185)
- Open the Dags menu from the Dags nav button instead of navigating (#75092)
- TS SDK: Populate Dag dependencies for TypeScript Dags (#75032)
- Warn about overly large task state values from language SDK tasks (#74945)
- Resolve loop iterations through family and path columns on dynamic regions (#74892)
- Validate ``CallbackTrigger`` callback path before running the callback (#74842)
- Skip the scheduler's Dag lookup for Dag runs with no callback to send (#74835)
- Render XCom values on the API server without rebuilding objects (#74829)
- UI: Preload base React plugin bundles with the page (#74777)
- UI: Keep unnamed columns out of the table columns menu (#74774)
- Number task loop iterations from one instead of zero (#74773)
- UI: Drop the loading spinner for base React plugins (#74762)
- Coerce delta and other typed fields when building operators from YAML Dags (#74749)
- Show Dag version diff values-hidden pill only when values are hidden (#74746)
- UI: Open nav sidebar menus on hover and add a Dags menu (#74745)
- Show serializer schema in Dag version diff only when it differs (#74683)
- Upgrade Task Loop UX (#74671)
- UI: Fold draining into the Dags Paused state filter (#74668)
- Lazy-load the Code tab grammar from the Dag's stored language (#74643)
- Highlight YAML Dag source in the Code tab (#74633)
- UI: Add clicked tags to the Dags list filters instead of replacing them (#74622)
- Java SDK: Rename annotation namespace from Builder to Airflow (#74604)
- Update waiting tasks to the latest Dag version (#74592)
- Disable psycopg3 auto-prepare by default behind connection poolers (#74581)
- UI: Replace stacked Dag header banners with inline status (#74516)
- TS SDK: Reject ``triggerDagRun`` tasks in ``dag.if`` and ``dag.switch`` (#74492)
- Make airflow standalone discover new Dag files within seconds (#74479)
- Use the configured API port in the default execution API URL (#74478)
- Dag run breadcrumb switcher (#74471)
- Reduce db clean batch probe overhead (#74455)
- UI: Show important form inputs instead of hiding them in accordions (#74422)
- Speed up Migration 0136 on MySQL (#74409)
- Scope the mapped task group skip-decision query to its own Dag run (#74390)
- Show loop iterations and outcomes throughout the UI (#74354)
- Show current task executions in the UI (#74352)
- Read current task executions by exact region and try (#74350)
- Clear selected loop work and regenerate only what was replaced (#74349)
- Resolve live producers by region and read their XCom by attempt (#74344)
- Parse native Go Dags and show each Dag's own source (#74341)
- UI: Apply default visibility to newly added table columns (#74320)
- UI: Group digits of counters in translated strings (#74296)
- Remember the Dag runs limit across tab switches and visits (#74289)
- Scope plugin views on any entity field, not a fixed criteria set (#74278)
- UI: Open provider connection fields by default and title them by connection type (#74273)
- Keep get_dag_details and get_pools database queries off the API server event loop (#74247)
- Require ``httpx2`` instead of ``httpx`` in airflow-core (#74244)
- Scope audit log rows to a Dag only from the request path or body (#74232)
- Register TypeScript, Java and Go grammars for the Dag Code tab (#74231)
- Discover executable bundles by footer, not by execute bit (#74227)
- Set end date and duration on tasks skipped by a Dag run timeout (#74128)
- Treat an empty Dag run config as no config in the UI (#74083)
- Let a Dag importer find the coordinator that parses its files (#74042)
- Validate a serialized Dag and fill its unset settings from the Airflow config (#74041)
- Keep the Dag trigger button one-click when a run has no config (#74039)
- Parse native TypeScript Dags with the Node coordinator (#74037)
- Parse native Java Dags and show their source (#74036)
- Parse native Lang-SDK Dags in the Dag processor with their runtime (#74035)
- Persist the match-anywhere search toggle in the URL (#74019)
- UI: Keep Dags list table rows on a single line (#74002)
- UI: Split task storage into separate XCom and task state store tabs (#73989)
- Surface Dag run conf in the header (#73987)
- Keep agent framework spans when worker tracing is off (#73903)
- Enforce execution_timeout from the task supervisor side (#73806)
- Keep the Dag asset schedule the same shape for every caller (#73805)
- Update supported Kubernetes versions (#73798)
- Mask provider-declared sensitive connection extras on every connection type (#73794)
- Mask registered secrets at any nesting depth (#73793)
- Mask XCom values in the audit log (#73791)
- Always clear the session cookie on logout (#73788)
- Eager-load DagCode in the Dag source API (#73752)
- Speed up zip Dag discovery and keep member file names (#73727)
- UI: Group digits of counters according to the selected locale (#73713)
- Correct the meaning of _arg_bindings in the serialization schema (#73644)
- Sync the UI Dags list applied sort rule with URL query param (#73622)
- Link calendar cell to filter dagruns by the respective start_date and state. (#73611)
- Add the resume event to stored next_kwargs without decoding them (#73576)
- Scope plugin scheduling classes to their team in multi-team mode (#73569)
- Warn when db clean skips only some rows of a batch (#73568)
- Widen revoked_token.jti to store external-issuer token identifiers (#73562)
- Own the BaseAsyncOperator event loop through asyncio.Runner (#73560)
- Use os.posix_spawn instead of fork+exec in the task-sdk supervisor (#73503)
- Skip ``on_kill()`` for triggers reassigned to another triggerer (#73454)
- Document favorite_dag's 409 and return a readable conflict message (#73411)
- Migrate ``GET /execution/variables/keys`` to an async DB session (#73407)
- Migrate execution API reads to async sessions (#73403)
- UI: Show every scheduler, triggerer, and Dag processor in health dashboard (#73355)
- UI: Show xcom before task state store in storage tabs (#73313)
- Delay reporting of up-for-retry state until task finalization completes (#73253)
- Refresh HITL review details on task clear (#73235)
- Scope plugin macros to their team in multi-team mode (#73224)
- Anchor top-level nav and needs-review links to root paths (#73202)
- Keep Grid scroll position when opening a task. (#73123)
- Identity-only DAG discovery and byte-importing (#73118)
- Move retry policy decision outside post execute log group block (#73023)
- Speed up the event logs API by loading only display names (#73021)
- Scope upstream asset events of a Dag run to assets and Dags the caller may read (#72924)
- Warn at provider discovery about an unresolvable connection type (#72904)
- Mark task-group aggregate bars on Gantt with a diagonal stripe (#72894)
- Hide unreadable assets in the asset expression served by next_run_assets (#72864)
- Scope UI next_run_assets endpoint to assets the caller may read (#72862)
- Keep date-only range ends inclusive (#72860)
- Remove a misleading connection type warning and explain lookup failures (#72855)
- Keep callback triggers assigned to healthy triggerers (#72848)
- Remove ignored --output flag from airflow pools set and delete (#72833)
- Show whether a calendar day's run was a backfill (#72782)
- Keep the task instance in dag callbacks when its Dag version is missing (#72721)
- Read XCom without object reconstruction during dependency evaluation (#72718)
- Mask CLI secrets in the audit log by parsed argument, not argv position (#72650)
- Clarify the backfill reprocessing option labels in the CLI and REST API (#72590)
- Warn when connections/variables list output may be incomplete (#72484)
- Add a source reference to import errors and a language field to Dag code (REST API + schema) (#72474)
- Distinguish empty plugin folder from plugin load failures in logs (#72432)
- Preview JSON fields as inline badges in tables (#72353)
- More human readable duration strings (#72350)
- Avoid retrying XCom expand actions (#72305)
- Scope plugin operator extra links to their team in multi-team mode (#72231)
- Use a structured SecretCache key instead of a concatenated string (#72201)
- UI: Keep Grid run tooltips inside the viewport (#72161)
- UI: Stop list pages jumping while a filter loads (#72005)
- Include task_reschedule in task_instance dependent tables for db clean (#71990)
- UI: Visual overhaul of Dag details layout for easier consumption (#71957)
- Authorize team-scoped plugin API endpoints in multi-team mode (#71919)
- Remove the Go Edge Worker (#71874)
- UI: Make dialogs consistent and fix their dark mode background (#71806)
- UI: Unify list page filters and table header controls (#71731)
- Improve DAG authorization filtering performance (#71722)
- Extend user settings with additional configuration (#71699)
- Cut the per-file cost of persisting Dag parse results (#71621)
- UI: Consolidate table controls into consistent slots and improve visual presentation (#71554)
- Stop presenting multi-team tagged metrics as an exhaustive list (#71523)
- Show the teams of the signed-in user in multi-team mode (#71503)
- Backfills UI filters (#71467)
- Respect run on latest version when clearing a running Dag run (#71425)
- API: Return asset events for Dag Runs without a start date (#71379)
- Use sqlite3.sqlite_version_info instead of hand-parsing the version string (#71115)
- Validate provider metadata list entries instead of ignoring them (#71104)
- Reject serde-reserved keys in HITL params_input (#71054)
- Update grid view refresh (#71004)
- Improve assets list performance (#70997)
- Batch triggerer unused trigger cleanup deletes (#70961)
- Multi-Team: Add teams verify command to validate multi-team configuration (#70947)
- Describe default Airflow loggers (#70842)
- Rename dag serialization metrics. (#70838)
- Show user display names for triggered runs and HITL responses (#70836)
- UI: Preserve Dag tab when switching Dags (#70699)
- UI: Wire advanced-search toggle for Grid Triggering User filter (#70567)
- UI: Highlight selected Grid task square (#70451)
- Feature expand health endpoint reporting (#70416)
- Remove airflow-metadata.yaml sidecar support from NodeCoordinator (#70273)
- Show backfill Dag runs in the UI (#70159)
- UI: Pass Dag, DagRun, TaskInstance, and Asset objects to React app plugins (#70149)
- Rename Task and Asset store to Task and Asset state store (#70097)
- Skip timeout callback creation for Dags without on_failure_callback (#70065)
- Improve Dag run metrics (#70013)
- Sort Dags by latest run after (#69995)
- Updated sort+filters for Assets List (#69940)
- UI: Share one tooltip root across Dag card run-state badges (#69927)
- Progressively load interactive elements of Dag Card (#69806)
- Validate deferred trigger classpath resolves to a BaseTrigger subclass (#69792)
- Multi Team: Automatically create and assign team default pools (#69768)
- Pass dag_id to Dag bundle construction on task execution (#69697)
- Reduce DAG card tooltip controller overhead (#69681)
- Show recent task states on the Dags list page (#69619)
- Resolve NPM vulnerabilities in Simple auth manager (#69488)
- Display latest dagrun state per dag in search result. (#69414)
- Improve mobile responsiveness for dashboard asset cards (#69294)
- Thread version_data to callbacks (#69185)
- Mark task worker span as failed when the task fails (#69146)
- Handle DB Performance Issues by Adding a New Index for Trigger (#69038)
- Warm-shutdown supervisor on SIGTERM instead of killing the running task (#69034)
- Raise a warning instead of an error if extra teams are defined in the auth manager (#68935)
- UI: Show partition preview in backfill form for partitioned Dags (#68884)
- Display owner_display_name in Audit Log (#68833)
- Warn in UI when the same dag_id is registered from multiple files (#68791)
- Ignore stale executor success when TI is queued after defer (#68741)
- Add ``bundle_name`` to ``Dag`` (#68583)
- Optionally emit Dag tags as metric tags (#68568)
- Group and filter Dags by folder on the Dags page (#68544)
- TriggerRunner: Add batch trigger creation duration metric (#68521)
- Paginate partitioned Dag run modal requests (#68412)
- Gate raw Dag deserialization error detail behind api.expose_stacktrace and log it server-side (#68388)
- Avoid a supervisor round trip before poke for rescheduled sensors (#68012)
- TriggerRunnerSupervisor: Add trigger queue delay metric (#67927)
- Make kinit renewable lifetime configurable for Heimdal Kerberos support (#67804)
- Use async DB session for Execution API task-instance heartbeat (#67800)
- Type asset_expression in the REST API so the UI does not cast through unknown (#67725)
- Record TaskInstanceHistory when a task fails from a non-running state (#67372)
- Remove trigger event payload from INFO log in run_trigger() (#67244)
- Show per-Dag run state counts on the Dags list page (#67242)
- Improve doc_md rendering in Dag Documentation (#66809)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Document how to create a TypeScript SDK project (#76374)
- Add screenshots and a review loop example to the Task Loops docs (#76435)
- Fix retry policy docs and the example Dag's ``ConnectionError`` rules (#76313)
- Document the Dag Versions comparison tab in the UI overview (#76387)
- Show UI doc screenshots in synced dark/light tabs (#75957)
- Document the zh-TW translation workflow (#75702)
- Go SDK: Correct the documented install command that broke the first bundle build (#75574)
- Add missing German Dag trigger translations (#75519)
- Normalize zh-TW plural forms and wording (#75504)
- Limit the CLI reference's page contents to top-level commands (#75483)
- Fix docs pages listing their sections as separate pages in the sidebar (#75478)
- Fix upgrade guide sections showing as separate pages in the docs sidebar (#75468)
- Add missing zh-TW translations for Dag version path tooltips (#75450)
- Java SDK: Rewrite the user docs around Dag definition (#75439)
- Fix deferral paths in the task lifecycle diagram (#75428)
- Add missing Taiwanese Mandarin translations for Dag start and end date trigger messages (#75371)
- Complete German UI translations (#75301)
- Add missing Taiwanese Mandarin UI translations (#75278)
- Close German translation gaps (#75137)
- Document that the HITL operator body is rendered as Markdown (#75130)
- Complete Hindi (hi) UI translations (#75057)
- Fix missing and incorrect endpoints in the API permission reference (#74871)
- Close German Translation Gaps 2026-10-15 (#74802)
- Close German Translation Gaps 2026-10-14 (#74760)
- Add documentation for the new YAML Dag format (#74748)
- Go SDK: Rewrite the user docs around Dag definition (#74698)
- TS SDK: Align the README with the authoring guide (#74693)
- Point the TaskFlow tutorial's retries section to resumable tasks (#74608)
- Add missing Taiwanese Mandarin translations (#74545)
- Add missing Polish UI translations (#74533)
- Add missing Taiwanese Mandarin translations for task states and Dag list settings (#74486)
- Fix Quick Start installing Airflow 3.1.1 instead of the documented version (#74474)
- Complete Hebrew UI translations 2026-10-16 (#74467)
- Add missing UI translations for Greek(el) (#74460)
- Dedent task docstrings used as doc_md (#74436)
- Close Catalan gap (#74361)
- Fix mapped-task retry policy example in tasks docs (#74343)
- Close German Translation Gaps 2026-10-05 (#74264)
- Update French (fr) UI translations to 100% coverage (#74235)
- Add missing Taiwanese Mandarin UI translations (#74223)
- Close German Translation Gaps 2026-10-02 (#74088)
- Update docs and diagrams for the Airflow 3 Dag processor (#74073)
- Render the Language SDK Spec diagrams as Mermaid (#74065)
- Document HTTP statuses that route helpers raise (#74027)
- Use airflow.sdk imports in Airflow 3 upgrade guide example (#74015)
- i18n(ko): Add missing translations (Oct 1) (#74006)
- Document team-scoped plugins in multi-team mode (#73995)
- Close German Translation Gaps 2026-09-30 (#73980)
- Clarify Dag processor scoping and team config in Multi-Team docs (#73958)
- Fix typo in apache-airflow-providers-fab error message (#73956)
- Document that ``use_job_schedule`` also stops asset-triggered Dag runs (#73952)
- Fix misleading timetable Variable example in best practices (#73945)
- Add missing zh-CN translations for new UI strings (#73943)
- Update the TypeScript SDK docs for Dags declared in TypeScript (#73875)
- i18n(ko): Align particles after Dag with the agreed pronunciation (#73799)
- i18n(ko): Add missing translations (Sep25) (#73757)
- Clarify max_db_retries doc wording to avoid off-by-one confusion (#73722)
- Fix typo in theme configuration screenshot filenames (#73632)
- Document that Go SDK registration closes at Serve (#73614)
- Update Go SDK docs for the airflow.Bundle authoring surface (#73528)
- i18n(zh-CN): align zh-CN Provider terminology and plural conventions (#73369)
- Fix wrong session type in state-store custom-backend doc example (#73361)
- Clarify the scope of db clean's ``--tables`` and DAG filter options (#73242)
- Add missing Traditional Chinese UI translations (#73180)
- Fix typos, grammar, and broken references across the docs (#73146)
- i18n(ko): Add missing translations (Sep14) (#73115)
- i18n(zh-CN): add missing translations (#73113)
- Fix syntax error and typos in FAQ doc (#73088)
- Fix broken code examples and stale references in airflow-core docs (#73077)
- Fix BaseXCom docstrings that say passing None removes a filter (#73066)
- Correct Airflow 3 systemd service and health guidance (#72872)
- Add missing zh-TW translations for common.json (#72766)
- Add a Language SDK spec to the Task SDK docs (#72765)
- Simplify the Quick Start for first-time local installs (#72754)
- Remove 值 from passwordToggle translations (#72684)
- UI: Complete Polish translations and unify slot/deferred wording (#72623)
- Add troubleshooting guidance for task state mismatches (#72492)
- Fix dead Docker Desktop networking links in the docker-compose guide (#72467)
- Improve Java SDK quick start setup docs (#72440)
- Update Catalan translations (main) (#72392)
- i18n(ko): add missing translations(Sep 1) (#72372)
- Fill in missing zh-TW translations for common.json (#72154)
- Fix stale TypeScript SDK docs on bundle metadata and packing setup (#72115)
- Complete the Taiwanese Mandarin UI translations (#72056)
- i18n(ko): add missing translations(Aug 25) (#72050)
- Update multi-team docs to match the teams CLI and name rules (#72045)
- Updated tutorial DAGs to use named parameters (#71980)
- Publish standalone Apache Airflow Mypy documentation (#71878)
- Document Go cross-language argument binding (#71871)
- Improve TypeScript SDK API reference navigation (#71833)
- Close spanish translation gap (#71796)
- i18n(Ko): add missing translations in assets.json (Aug 18) (#71760)
- Add a consolidated language SDK compatibility matrix (#71757)
- Fix inverted docstring for config write hide_sensitive option (#71623)
- Fix the translation Gap for Taiwanese Mandarin (#71482)
- Update Deadline Reference docs to match the code (#71469)
- Document hook lineage collection limits (#71450)
- Document effect of clearing a task instance on task state store (#71356)
- Clarify that DAG.access_control is only read by the FAB auth manager (#71260)
- Document config-backed enum Dag Params (#71180)
- Fix broken translation guide links in the i18n README (#71179)
- Clarify which signal each duplicated otel_* config option applies to (#71158)
- Complete the Traditional Chinese UI translations (#71019)
- Clarify How to upgrade docs for PyPI installs (#70897)
- Add TypeScript SDK API reference docs with TypeDoc (#70812)
- i18n(Ko):Add missing translations in components.json (#70764)
- Align timetable docs with Airflow 3 data interval and catchup behaviour (#70658)
- Remove stale experimental note from Lineage docs (#70400)
- Update French (fr) UI translations to 100% coverage (#70386)
- Add missing zh-TW translations for team strings (#70321)
- Add TypeScript SDK guide to language SDKs docs (#70219)
- fix ruff check docs command in FAQ (#70212)
- Fill Taiwanese Mandarin translation gap (#70194)
- Align Dag Runs docs with Airflow 3 trigger timetable defaults (#70167)
- Document that Code view shows only Python source for stub tasks (#70155)
- Cleanup multi-team docs for team_name tagging (#70146)
- UI: Complete Hindi (hi) translation coverage (#70111)
- Enhance/standardize documentation for ``AssetOrTimeSchedule`` (#69831)
- Clarify the dags reserialize command scope (#69810)
- Document that connection secrets without conn_type remain supported (#69780)
- Update SQLAlchemy documentation link from 1.4 to 2.0 (#69775)
- Fill Taiwanese Mandarin translation gap (#69708)
- Mention min Java version in Java SDK README (#69691)
- Fill Taiwanese Mandarin translation gap (#69549)
- Regenerate REST API permission reference to fix static check (#69341)
- Fix the translation Gap for Taiwanese Mandarin (#69199)
- Document PostgreSQL fallback host connection strings (#69150)
- Add example DAG demonstrating Deadline Alerts (#66269)
Airflow 3.3.2 (2026-09-17)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
- Backfill endpoints no longer disclose which backfill ids exist across Dags
The four routes that name a backfill in their path -- ``GET /backfills/{backfill_id}``
and the ``pause``, ``unpause`` and ``cancel`` routes -- resolved the Dag they authorize
against from the ``dag_id`` supplied on the request whenever the path's id matched no row.
An unknown id and a backfill on a Dag the caller cannot see therefore answered differently,
which enumerates backfill ids across Dags.
The backfill named in the path is now the only thing those routes authorize against.
**Behaviour changes:**
- Requesting a backfill on a Dag the caller cannot read now returns ``404``
(``Backfill not found``) -- the same response an unknown id gets -- instead of the
``403`` returned before. A caller who can read the Dag still gets ``403`` for a write
they are not allowed to make.
- A ``backfill_id`` in the path is never authorized against a ``dag_id`` in the request body
or query string. ``GET /backfills``, ``POST /backfills`` and ``POST /backfills/dry_run``
name no backfill in their path and keep authorizing off the request.
- All four routes now answer an unknown id with the same detail, ``Backfill not found``.
The ``pause``, ``unpause`` and ``cancel`` routes previously answered
``Could not find backfill with id {backfill_id}``. Clients matching on ``detail``
must be updated. (#71113)
- An explicit credential now takes precedence over the session cookie
``get_user()`` is written to prefer an explicit bearer token, then OAuth2, then the
session cookie, but that precedence was unreachable whenever a cookie was present.
``JWTRefreshMiddleware`` runs
first, resolves a user from the ``_token`` cookie alone and stamps it on
``request.state``, and ``get_user()`` returned that cached user before looking at either
explicit credential. The effective order on every core-API route was cookie over bearer.
A request carrying both a session cookie and an explicit credential therefore executed,
and was recorded in the audit log, as the cookie's principal rather than the identity the
client presented. The cached user is now honoured only when the request carries no
explicit credential.
**Behaviour changes:**
- A request carrying **both** a ``_token`` cookie and an ``Authorization: Bearer`` header
is now resolved as the bearer token's principal, where it was previously resolved as the
cookie's. The same applies to a cookie combined with an OAuth2 token.
- An **invalid or expired** explicit credential is now rejected with ``401``/``403`` even
when a valid ``_token`` cookie accompanies it. Previously the cookie silently took over
and the request succeeded as the cookie's principal; the failure is now loud.
- Requests carrying a single credential are unaffected. Cookie-only browser sessions keep
the token-refresh behaviour of ``JWTRefreshMiddleware`` unchanged.
- Clients that relied on the cookie winning -- for example a browser-based tool that sent a
service account's bearer token while a user session cookie was present, and expected the
user's identity to apply -- will now act as the bearer token's principal. Remove the
header, or the cookie, to select the intended identity explicitly. (#72225)
Bug Fixes
^^^^^^^^^
- Optimize the previous-task-instance lookup by removing a redundant ``dag_run`` join (#72944)
- Revoke every credential presented to the logout endpoint, not just the session cookie (#72933)
- API: Return HTTP 404 instead of 500 when a task starts against a missing Dag run (#72900)
- Fix ``airflow db clean`` never purging the ``callback`` table (#72899)
- Fix the Dag version inflation check not warning about custom ``DAG`` subclasses or aliased-module imports (#72898)
- Fix ``DeadlockImminentError`` when a connection is resolved inside an async task (#72895)
- Fix dag processor crash when an orphaned processor is killed (#72888)
- Prevent corruption of ``XCom`` values that already parse as JSON during the ``bytea``-to-``JSONB`` migration (#72886)
- Fix HTTP 500 for non-dict JSON bodies on the Variable and Connection API endpoints (#72878)
- Fix resolution of deprecated imports in ``airflow.utils.helpers`` (#72868)
- Bound single-row lookups with ``LIMIT 1`` to avoid scanning large tables (#72842)
- UI: Fix clipping of the Last Run state badge (#72841)
- UI: Fix Calendar view computing planned cron runs in UTC instead of the Dag's timezone (#72839)
- Fix ``airflow info --file-io`` uploading an empty report (#72832)
- Preserve custom operator defaults in mapped tasks (#72828)
- Gate the asset event ``partition_key`` behind the 2026-06-30 Execution API version (#72827)
- Improve deadline diagnostics for null ``DagRun`` fields (#72812)
- Fix missing HTTP access logs when the api-server omits the core app (#72808)
- Clarify when the auth manager ``is_authorized_hitl_task`` (Human-in-the-loop) hook runs (#72807)
- Fix ``td_format`` rendering of negative durations (#72798)
- UI: Make copied task log text match the on-screen format (#72771)
- UI: Fix connection test with a null host and port (#72747)
- Allow ``airflow jobs check --allow-multiple`` with ``--limit 0`` (#72744)
- UI: Keep task log selection stable while dragging (#72743)
- UI: Restore counts on the Dag Run and Task Instance lists (#72739)
- UI: Fix the Dags list Last Run / Next Run going stale after runs complete (#72735)
- UI: Fix the first startup request being sent to an unset API base URL (#72733)
- UI: Fix task instance links leading to 404s for tasks outside the run's date range (#72732)
- UI: Fix the Human-in-the-loop form crashing on null values (#72731)
- UI: Fix copying task logs dropping rows that scrolled out of view (#72729)
- UI: Label Dag active runs accurately (#72722)
- Bound single-row ``XCom`` existence lookups with ``LIMIT 1`` to avoid full scans (#72702)
- UI: Fix Firefox multi-line drag selection in the task log view (#72700)
- Fix Dag scheduling stall after switching to a coarser cron (#72679)
- Fix memray profiling capturing interpreter startup instead of the dag-processor job (#72661)
- Prevent Dag-existence disclosure on the partitioned dag runs listing (#72660)
- Fix mark-failed ``KeyError`` for removed-task task instances (#72620)
- UI: Fix ``hierarchical_alphabetical`` sort order breaking the graph and grid (#72618)
- Clarify ``@task``-decorated callable errors when extra positional arguments are passed (#72616)
- Load the correct Dag version when a task starts from a trigger (#72614)
- Fix ``airflow db migrate`` failing under the PyMySQL driver when a schema migration drops unique constraints (#72613)
- Stop ``airflow providers get --full`` mutating cached provider metadata (#72601)
- Fix ``airflow connections test`` returning a success exit code on failure (#72583)
- Stop ``airflow standalone`` leaking components when one fails to start (#72568)
- Fix ``DAG.cli()`` crashing on ``dags pause`` and ``dags unpause`` (#72565)
- Prevent Dag CLI subcommands from being silently dropped (#72365)
- Respect the ``limit`` search param in the task overview duration chart (#72357)
- UI: Show duration chart tooltips in the selected timezone (#72339)
- Export ``AIRFLOW_TEST_MODE`` from ``airflow tasks test`` without ``--env-vars`` (#72320)
- Authenticate only once per task process to external secrets backends (#72237)
- Remove the unreachable 404 response from the create Variable API endpoint (#72190)
- Speed up bulk updates of Variables and Pools by fixing an N+1 query (#72160)
- UI: Fix a React plugin rendering a previously loaded plugin's component (#72136)
- Fix Variable write-conflict checks comparing against the wrong team (#72125)
- UI: Surface connection test errors instead of failing silently (#71963)
- Fix the runtime-varying-value checker skipping tasks defined after a nested ``with`` block (#71956)
- Speed up marking a Dag run failed when it has many mapped task instances (#71955)
- UI: Include the JSON parse-error message in the Variable form warning (#71953)
- UI: Allow file downloads from plugin external-view iframes (#71952)
- UI: Make the grid run bar tooltip time zone aware (#71951)
- Reduce memory used when deleting queued asset events (#71937)
- UI: Activate assets materialized from an AssetAlias so they appear in the Assets tab (#71935)
- Fix ``Variable.set`` rewriting the ``team_name`` of existing variables (#71904)
- Reduce memory used when deleting a Dag with a large history (#71889)
- UI: Show an empty object for object params with no value (#71876)
- Fix Dag callbacks silently dropped when the version inflation check blocks parsing (#71865)
- Return HTTP 404 from task state store endpoints for unknown task instances (#71860)
- Fix deadline never firing after a non-deadline Dag edit (#71859)
- Honor the API server Dag cache TTL when no size limit is set (#71845)
- Require Dag edit permission to delete asset queued events (#71828)
- Bound the scheduler's deserialized Dag cache to prevent unbounded memory growth (#71821)
- Stop variables export/import from silently corrupting values (#71791)
- UI: Fix datetime pickers unusable on Firefox and Safari (#71788)
- Scope ``/assets/events`` to the Dags the caller may read (#71785)
- Serve logs from the scheduler if any executor is LocalExecutor (#71781)
- Fix cleared tasks getting stuck when a Dag run has no version (#71773)
- Stop the dag processor warning on every file path normalized for stats (#71764)
- UI: Fix the last section on the page not being clickable (#71755)
- Fix deadline serialization, repr, and prune edge cases (#71726)
- Dispatch the highest-priority tasks first in the executor (#71715)
- Keep ZIP-archived Dags active when ``dag_discovery_safe_mode`` is False (#71714)
- Honor ``FORWARDED_ALLOW_IPS`` when the API server runs under gunicorn (#71708)
- Fix ``airflow config lint`` staying silent on conditional removal rules (#71651)
- Avoid exhausting the DB connection pool when rendering the grid structure for large Dags (#71626)
- Fix Task SDK IPC short reads crashing the subprocess or hanging the supervisor (#71609)
- Fix one bad callback request crashing the Dag processor and dropping the rest (#71608)
- Fix ``clearTaskInstances`` returning HTTP 500 instead of 422 on an invalid body (#71559)
- Mark only a run's most recent asset event as triggering it (#71547)
- Fix Variables API handling of non-string JSON values (#71526)
- Include server error detail in Task SDK API error tracebacks (#71491)
- Show each Dag only once in ``airflow dags list`` (#71481)
- Keep Dag run Execution API endpoints working for older Task SDK clients (#71438)
- UI: Fix Calendar view hanging for Dags with high-frequency cron schedules (#71435)
- Require existing-connection read access when testing an existing connection (#71428)
- UI: Fix Grid view failing to load large Dags on MySQL (#71370)
- Catch general Exception when initializing a Dag bundle (#71363)
- Fix deactivation of stale ZIP-packaged Dags (#71326)
- Avoid scheduler crash when periodic maintenance actions fail (#71288)
- Fix ``_team_name`` missing from ``DagRun`` passed to some listener calls (#71262)
Miscellaneous
^^^^^^^^^^^^^
- Add async asset store accessors for async tasks and watcher triggers (#72851)
- UI: Add Consuming Tasks, Aliases, and Watchers to the assets pages (#72780)
- Improve confirmation output when connections are added via the CLI (#72779)
- UI: Sync the browser URL with navigation inside iframe views (#72776)
- Reduce Dag-processor log noise by logging a bundle that is not due for refresh at debug instead of info (#72749)
- Allow deadline alert UUID references in the serialized Dag schema (#72738)
- Add ``SerializedVariableInterval`` for deadline alerts (#72244)
- Report Dag cache metrics under each component's own namespace (#71925)
- Add a password field type to ``FlexibleForm`` (#71573)
- Add ``BaseDeadlineReference`` and ``deadline_reference`` to the SDK public interface (#71208)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Update Catalan (ca) UI translations (#73026)
- Fix a missing newline in the overview docs that prevented an image from rendering (#73013)
- Document durable execution in the core concepts docs (#73012)
- Clarify authentication and add an authorization example for plugin FastAPI apps (#72942)
- Document that plugin FastAPI apps are not authenticated by Airflow (#72932)
- Close German UI translation gaps (#72843)
- Fix the Postgres tutorial for psycopg3 (#72837)
- Document scope boundaries the security model leaves implicit (#72829)
- Make the first tutorial example more explicit about Airflow syntax (#72805)
- Add a ``create_async_metadata_engine`` example to the docs (#72804)
- Add a systemd unit file for the Airflow Dag processor (#72797)
- UI: Complete ``zh-CN`` Simplified Chinese translations (#72789)
- Drop the extraneous "common" tag in i18n (#72781)
- Add missing Russian UI translations (#72755)
- Complete German UI translations (#72736)
- UI: Complete Polish UI translations (#72624)
- Record recurring non-issue shapes in the security model (#72617)
- Complete Taiwanese Mandarin (``zh-TW``) UI translations (#72615)
- UI: Translate durations and relative times in the selected language (#72334)
- Remove the TaskFlow recommendation from the tutorial docs (#72118)
- Split the task execution architecture docs into an overview and a dev guide (#71858)
- Document service.name and service.instance.id for OpenTelemetry metrics (#71854)
- UI: Improve French translation wording (#71846)
- Complete French UI translations (#71790)
- Improve Arabic UI translations (#71789)
- Document HTTP statuses that API routes raise but never declared (#71622)
- Recommend ``dag.test()`` for testing custom operators in the docs (#71587)
Airflow 3.3.1 (2026-08-12)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
pandas 3 changes how DataFrame XComs are stored and read back (#71169)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
pandas 3 exposes its public classes from the ``pandas`` namespace, so a DataFrame is qualified as
``pandas.DataFrame`` instead of ``pandas.core.frame.DataFrame``. XComs record that name alongside the
serialized value, so the name written into the metadata database depends on the pandas version of the
component that pushed the value. Airflow registers both names, and a DataFrame written by either
pandas version can be read by either -- no configuration change is needed, and existing XComs stay
readable.
What you should do:
* **Roll this Airflow version out to every component before pandas 3 reaches any of them** -- workers
in particular. A component that predates this change cannot read a DataFrame XCom written under
pandas 3, and fails the pull with:
.. code-block:: text
ImportError: pandas.DataFrame was not found in allow list for deserialization imports.
To allow it, add it to allowed_deserialization_classes in the configuration
The message points at configuration, but the allow list is not the cause and changing it does not
help. The rows are not corrupt: they become readable again as soon as the reader is upgraded.
* **Treat a downgrade as a one-way door for those XComs.** Rolling back to an Airflow version without
this change strands any DataFrame XCom written while on pandas 3, with the same error, until you
roll forward again.
* **Review Dags that inspect the ``dtypes`` of a pulled DataFrame.** The pandas version of the *reader*
determines what a pulled DataFrame looks like, not the version that wrote it. Under pandas 3, a
column of strings comes back as ``str`` rather than ``object``, and its missing values
come back as ``nan`` rather than ``None``. Values are unchanged, but downstream code that branches
on ``dtype == "object"``, checks cells with ``is None``, or compares against a reference frame with
``DataFrame.equals()`` can behave differently after the upgrade.
Fix 2.x to 3.0+ upgrade failure when a custom Dag bundle is configured (#70994)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The ``0082_3_1_0_make_bundle_name_not_nullable`` migration assigned every legacy row
``bundle_name='dags-folder'``, so triggering a DagRun raised ``Requested bundle 'dags-folder' is
not configured.`` on any deployment that uses a bundle other than the default ``dags-folder``.
``DagFileProcessorManager`` now runs a one-shot, best-effort backfill at startup that routes each
affected Dag to the correct bundle based on its file path; unmatched Dags self-heal on the next
successful parse (or run ``airflow dags reserialize`` to force it immediately).
Team scoped values of options registered as sensitive are now hidden (#71099)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Configuration options are registered as sensitive under their base section, so until now only the
base spelling of an option was masked. A team scoped override -- set in a ``[<team>=<section>]``
config file section, or through an ``AIRFLOW__<TEAM>___<SECTION>__<KEY>`` environment variable --
was not recognized as the same option and was returned in full.
Sensitivity is now decided after resolving the team scoped spelling back to the base option, so a
team scoped value is masked exactly as the base value already was.
**Behaviour changes:**
- ``AirflowConfigParser.as_dict(display_sensitive=False)``, ``GET /config``,
``GET /config/section/{section}/option/{option}`` and ``airflow config list`` now return
``< hidden >`` for a team scoped value of an option registered as sensitive. Deployments that
read a team's real value through any of these will now receive the mask; use
``display_sensitive=True`` where a real value is required and appropriate.
- Team scoped ``_cmd`` and ``_secret`` entries are replaced with ``< hidden >`` in place, rather
than being resolved into their value and removed as they are in a base section. Resolving them
is not supported for a team, so the command string or secret path is no longer shown either.
- Non team configuration is unaffected, and ``display_sensitive=True`` continues to return real
values.
Bug Fixes
^^^^^^^^^
- UI: Fix a 500 error when combining the last-run and any-run Dag state filters on the Dags list (#71371)
- UI: Fix task log text selection being cleared while scrolling (#71200)
- UI: Fix a translation key showing as raw text in the Clear Task dialog (#71240)
- Fail deferred task instances whose saved state can't be resumed, instead of leaving them stuck (#71183)
- Fix task callbacks being skipped when ``TriggerDagRunOperator`` gets a 404 (#71083)
- Fix task state store rejecting keys that contain slashes (#70967)
- Fix the ``deadline_reference`` decorator's no-parentheses form (#70966)
- Fix Dag run duration stats crash on PostgreSQL 14+ (#70964)
- Deactivate legacy Dags with a NULL ``bundle_name`` during upgrade from 2.x to 3.x (#70662)
- Fix deadline alerts using an outdated Dag definition (#70965)
- Fix deadline alert crashes on dynamic or malformed intervals (#70625)
- Fix deadline alerts that have no fixed interval (#70659)
- Fix backfill permission checks running against the wrong backfill for some ID formats (#71090)
- Fix database lock contention and statement timeouts caused by slow asset listeners on large fan-outs (#71065)
- Fix errors loading a Dag callback whose module isn't importable on the current component (#71042)
- Reject reserved XCom serialization keys submitted as JSON string literals (#69462)
- API: Return a clear error instead of a 500 for an invalid trigger-Dag-run request (#70775)
- API: Return 503 when SQLite locks during backfill creation (#69659)
- API: Return 422 for an empty backfill window and stop leaving orphan rows (#69367)
- API: Return 410 instead of 500 when setting rendered fields for a stale task instance (#69529)
- Reject invalid partition keys in the create asset events API (#69581)
- Reject attaching partition keys to asset alias events (#69515)
- Reject mismatched rollup mapper and window pairings at Dag parse time (#69516)
- CLI: Reject inverted date windows in ``airflow partitions clear`` (#69547)
- Fix asset materialization dropping the partition date on partitioned Dag runs (#69339)
- UI: Fix partition progress returning errors or over-reporting for keys with slashes and duplicate rows (#69844)
- Honor catchup for historical asset events in asset-triggered Dags (#69224)
- Fix drifting data intervals for monthly/yearly schedules with catchup disabled (#69189)
- Fix asset watcher triggers failing to decode their arguments (#70750)
- Prevent Triggerer crashes by speeding up cleanup of unused triggers (#70668)
- Fix Triggerer CrashLoopBackOff when ``json_logs`` is enabled (#70669)
- Detect and surface task-worker communication deadlocks instead of hanging (#70744)
- Fix TaskInstance duration calculation with SQLite (#70734)
- Fix incorrect end date, duration, and map index in task try history for retried tasks (#69458)
- Stop skipping ``none_failed_min_one_success`` tasks in mapped task groups (#70318)
- Fix the scheduler firing ``on_failure_callback`` for heartbeat-timed-out retries (#69824)
- Respect retries for deferrable tasks that fail via a trigger-emitted ``TaskFailedEvent``, instead of always failing terminally (#71163)
- Prevent scheduler crash when process/thread are missing from the log format (#69787)
- Fix ``TaskInstance`` mark-success downstream default (#70143)
- Fix crash when tailing logs of a running task instance (#69521)
- API: Return a consistent error response instead of a 500 when a database error occurs (#70236)
- Fix Dag reparse authorization checking the wrong Dag (#70115)
- Don't deactivate Dag bundles owned by other Dag processors (#70017)
- Fix Dag bundle refresh using stale state (#70374)
- Skip stored credentials when a connection test overrides host or port (#70010)
- Fix cursor pagination dropping rows when sorting by a nullable column (#70739)
- API: Filter stale Dag tags from the public API (#70746)
- Fix Dag details active-runs count to exclude queued runs (#70511)
- Fix environment-variable config overrides being ignored for some provider config sections (#70732)
- CLI: Fix config ``update --option``/``--ignore-option`` never matching options (#70757)
- CLI: Fix ``TypeError`` in ``airflow db shell`` when the database name is missing (#70752)
- CLI: Send Airflow CLI logs to stderr for ``-o`` commands so structured output stays machine-readable (#70747)
- Reduce Dag processor log noise from per-Dag run lookups (#69514)
- Suppress noisy Alembic plugin setup logs (#70116)
- Silence internal HTTP 422 deprecation warnings in logs (#70745)
- Only resolve a team-namespaced environment secret for its own team (#70882)
- Fix a team-scoped secret lookup that could return another team's secret for a crafted key (#71041)
- Mask sensitive ``Variable`` values stored as JSON lists (#71069)
- Fix secrets recorded unmasked in the audit log for bulk ``Variable``/``Connection`` updates (#71043)
- Fix sensitive values nested inside lists, tuples, or sets not being masked in logs (#70189)
- UI: Fix secrets not masked in the Rendered Templates view with ``KubernetesPodOperator`` (#70756)
- Fix an open-redirect by rejecting malformed URLs in redirect validation (#70515)
- Fix npm vulnerabilities in the simple auth manager (#70753)
- Bump ``structlog>=26.1.0`` and ``croniter>=6.2.2`` to fix memory leaks (#70749)
- Fix task instance notes not being visible to state-change listeners (#70252)
- Call listeners for a running task instance when a Dag run state is manually set (#70286)
- Fix ``dag`` and ``note`` missing from Dag-run state-change listener events (#70245)
- Remove a Dag Run or Task Instance note when its content is cleared (#70735)
- Fix ``email_on_failure``/``email_on_retry`` task alerts silently ignoring a custom ``[email] email_backend`` and always routing through ``SmtpNotifier``; an ``email_backend`` that cannot be imported now errors loudly instead of silently falling back to SMTP (#70129)
- UI: Fix task states stuck stale when a run finishes quickly (#70397)
- UI: Fix Grid view scrollbar hiding the latest Dag run (#70555)
- UI: Fix grid/graph view topological sort for group-level and cross-group dependencies (#70591)
- UI: Fix ``Trigger Again`` showing empty config for the selected run (#70288)
- UI: Fix blank Assets dependency graph from missing Dag nodes (#70743)
- UI: Fix the collapse button overlapping details panel content (#70751)
- UI: Fix log line-number link highlighting (#69663)
- UI: Fix partition key display and input handling (#69974)
- UI: Fix Gantt tooltip showing the wrong end date on queued/scheduled bars (#70742)
- UI: Make the Dag pause toggle distinguishable in dark mode (#70748)
Miscellaneous
^^^^^^^^^^^^^
- UI: Show a note indicator on Dag runs in the Grid view (#70834)
- UI: Show a saved-note indicator on task instances in the Grid view (#70829)
- Add partition date filters to the Dag run API (#70304)
- Add support for filtering Dags by any Dag run state (#70292)
- Allow filtering the Dags list by failed and success runs in any run-state filter (#70293)
- Add expand/collapse all for the Dag Run conf JSON in the Dag Runs list (#69777)
- Show the Dag Run conf column by default in the Dag Runs list (#69604)
- API: Allow keeping finished task states when clearing a Dag run (#69662)
- Export ``FanOutMapper`` and wait policies from ``airflow.partition_mappers`` (#69513)
- Add a ``task.execute`` OpenTelemetry span around task execution (#69359)
- Add a ``run_type`` tag to the ``dagrun.duration.failed`` metric (#70731)
- Improve error messages when a value's type cannot be serialized (for example, XCom values) (#70982)
- Highlight user-code frames in task log tracebacks (#70375)
- Optimize scheduling by avoiding duplicate trigger-rule upstream-count queries per pass (#70826)
- Hide the run-on-latest-version option for non-versioned bundles (#70702)
- Show the current page name in the browser tab title (#69656)
- UI: Show larger Dag run and task instance counts on the dashboard (#71008)
- UI: Refresh task details immediately when switching tasks (#71012)
- UI: Reset the task try when switching Graph tasks (#70817)
- UI: Add JSON validation and prettifying to the JSON editor (#70554)
- UI: Make the Dag pause toggle update immediately on click (#70741)
- UI: Improve Dag list rendering by deferring the pause confirmation dialog (#70025)
- UI: Align boolean controls in the Trigger Dag form (#70963)
- UI: Wrap long plugin source paths in the import-error dialog (#70737)
- UI: Allow multiple routes to show active nav buttons (#70200)
- UI: Make duration charts readable at a glance (#70197)
- UI: Improve Grid view performance when summaries stream in (#69958)
- UI: Improve Grid view responsiveness by avoiding a full re-render on hover (#69928)
- UI: Fix missing glyph icons in the code editor (#69422)
- UI: Complete missing Hebrew (``he``) translations (#70566)
- UI: Complete missing Arabic (``ar``) translations (#70510)
- UI: Complete the Polish (``pl``) translation (#70507)
- UI: Add missing Greek (``el``) translations (#70471)
- Optimize database queries when triggers submit asset events (#70738)
- Optimize core queries by removing redundant result de-duplication (#69918)
- Remove redundant database commits in API route handlers (#69620)
- Make ``ResumableJobMixin`` an abstract base class (subclasses must implement its methods) (#70810)
- UI: Add a keyboard shortcut help dialog (press ``?``) and clean up the graph/grid view (#69978)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Document the effect of state-store cleanup in ``ResumableJobMixin`` (#70792)
- Document ``jwt_secret``/``_secret`` and ``LocalFilesystemBackend`` config support (#70730)
- Clarify the ``logging_config_class`` contract and document ``REMOTE_TASK_LOG`` (#70592)
- Clarify ``AssetAlias`` usage (#71087)
- Clarify the ``AssetPartitionDagRun`` provisional-run docstring (#70104)
- Clarify custom-time parameterized timetable logic (#69387)
- Document native template rendering type coercion (#69389)
- Update multi-node executor guidance (#69388)
- Add a custom metrics section to the metrics docs (#70778)
- Add Task SDK, Go and Java SDK execution architecture diagrams (#70100)
- Add a docker-stack docs example for the venv scene (#69112)
- Link the pkg.go.dev API reference from the Go SDK docs (#69440)
- Link the published Java SDK API reference from the Java SDK docs (#69448)
- Add a real example of ``CronDataIntervalTimetable`` and ``DeltaDataIntervalTimetable`` (#70434)
- Fix incorrect code samples in the Deadline Alerts docs (#70786)
- Fix reversed-direction examples in the ``FanOutMapper`` docs (#69511)
- Fix documentation misusing previous/next for task relationships (#69570)
- Fix partition-label casing (#69470)
- Fix stale Airflow 2.0 references in ``dev/README.md`` (#70107)
- Fix a Sphinx build error (#70761)
- Update the description on "What is Airflow" (#71068)
- Update local OTel Collector and Prometheus versions to support exponential histograms (#69056)
- Simplify the API docs on pattern search (#70509)
- Standardize Alembic migration descriptions and add a style lint (#70262)
- UI: Complete Spanish UI translations (#70196)
- Update French (``fr``) UI translations to 100% coverage (#70387)
- Add missing Dutch (``nl``) translations (#70004)
- Add missing Simplified Chinese (``zh-CN``) UI translations (#70417, #70418, #70419)
- Fill the Taiwanese Mandarin (``zh-TW``) translation gaps (#70195, #70379, #69707)
- Add missing Korean (``ko``) translations and backport from main (#70807, #70832)
Airflow 3.3.0 (2026-07-06)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Asset Partitioning (#64571, #65447, #66030, #66848, #67184, #67475, #67716, #68978)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Building on the asset partitioning introduced in 3.2.0, Airflow 3.3.0 substantially expands how a
single upstream asset event fans out to partitioned downstream Dag runs. New partition mappers —
``RollupMapper`` (many-to-one), ``FanOutMapper`` (one-to-many), and ``FixedKeyMapper`` +
``SegmentWindow`` (categorical rollup) — compose with time windows (day/week/month/quarter/year) and a
``wait_policy`` (``WaitForAll`` or ``MinimumCount(n)``) to control when partitioned runs fire. Windows
can fan out forward or backward in time, and total fan-out per upstream event is bounded by the new
``[scheduler] partition_mapper_max_downstream_keys`` config (configurable per mapper). Airflow 3.3.0
also adds the ``PartitionedAtRuntime`` timetable, which lets a Dag declare that its partition key(s)
are assigned when the run starts rather than mapped from an upstream event.
For detailed usage instructions, see :doc:`/authoring-and-scheduling/assets`.
Task and Asset State Store (#65759, #66073, #66160, #66463, #66586, #66859, #67041, #67292, #67319)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow 3.3.0 introduces a first-class state store for tasks and assets (AIP-103). Tasks can persist
arbitrary key-value state that survives across retries and runs via a new ``task_state_store`` accessor, and
assets can carry their own state via ``asset_state_store`` — both available from the Task SDK. State is kept
in the metadata database by default, or in a custom worker-side backend (``[workers]
state_store_backend``), supports per-key retention with periodic garbage collection and an optional
``clear_on_success``, and is fully manageable through the Core API and Execution API.
For detailed usage instructions, see :doc:`/core-concepts/task-and-asset-state-store`.
Pluggable Retry Policies (#65474)
"""""""""""""""""""""""""""""""""
Task retry behaviour is now pluggable (AIP-105). In addition to a fixed ``retries`` count, you can
attach a custom retry policy that decides whether and when a task is retried, enabling strategies such
as retrying only on specific exceptions or backing off based on custom logic.
For detailed usage instructions, see :ref:`concepts:retry-policies`.
Language Task SDK (Java and Go) (#65958, #67161, #67635, #67699)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow 3.3.0 adds a Coordinator layer (AIP-108) that lets individual task implementations be written
in non-Python languages while the Dag and its scheduling stay in Python. A task is declared in the Dag
with ``@task.stub(queue=...)``; the worker routes it to a configured coordinator
(``JavaCoordinator`` for JVM languages, ``ExecutableCoordinator`` for self-contained native binaries
such as Go) that runs the task in a language runtime and proxies Variables, Connections, and XComs back
through the Execution API.
.. warning::
The Coordinator layer and the Java/Go SDKs are experimental in 3.3.0 and may change in future
versions based on user feedback.
For detailed usage instructions, see :doc:`/authoring-and-scheduling/language-sdks/index`.
Dag bundle version on clear, rerun, and backfill (#63884)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The new ``rerun_with_latest_version`` setting controls whether a cleared, rerun, or backfilled Dag run
uses the latest bundle version or the original version from the initial run. The default is resolved
by precedence: an explicit request parameter/CLI flag, then the Dag-level ``rerun_with_latest_version``,
then ``[core] rerun_with_latest_version``, and finally ``False`` for clear/rerun and ``True`` for
backfills (preserving historical behaviour). Airflow 2.x always reran with the latest code; 3.x
introduced bundle versioning defaulting to the original version, and this setting gives users control.
See :doc:`/administration-and-deployment/dag-bundles` for full details.
Provider example Dags as dedicated bundles (#66161)
"""""""""""""""""""""""""""""""""""""""""""""""""""
Example Dags shipped by provider distributions are now discovered via ``ProvidersManager`` and
registered as their own Dag bundles — one per provider, named
``apache-airflow-providers-<distribution>-example-dags`` (or ``<distribution>-example-dags`` for
third-party providers). The ``[core] load_examples`` option still gates whether they are registered.
REST API clients that filtered ``bundle_name`` by ``"dags-folder"`` for provider-shipped example Dags
must update to the new per-provider bundle names; Dag identifiers are unchanged.
Remote logging resolution decoupled from ``airflow.logging_config`` (#67056)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Remote task log handler resolution is now owned by the shared ``airflow_shared.logging.factory``
module and applies a single, well-defined precedence:
1. a user-defined ``[logging] logging_config_class`` exporting ``REMOTE_TASK_LOG`` /
``DEFAULT_REMOTE_CONN_ID`` (existing custom configs keep working);
2. ``ProvidersManager`` scheme dispatch — the scheme of ``[logging] remote_base_log_folder`` selects a
provider ``RemoteLogIO`` class, instantiated via a no-argument ``from_config()`` classmethod;
3. a transitional legacy fallback reading ``airflow_local_settings.py`` (to be removed in Airflow 4.0).
``airflow.logging_config.load_logging_config`` is deprecated (it now emits ``DeprecationWarning`` and
delegates to new private helpers), and ``configure_logging`` no longer eagerly resolves the remote
handler — resolution is lazy on first use. Providers that registered a ``remote-logging:`` block but do
not implement ``from_config`` are skipped with a warning and fall through to the legacy path.
Migration: replace direct calls to ``airflow.logging_config.load_logging_config()`` with the new
helpers, and have provider remote-log handler classes implement a no-argument ``from_config``
classmethod that reads ``airflow.providers.common.compat.sdk.conf``.
OpenTelemetry timer metrics now use Histogram (#64207)
""""""""""""""""""""""""""""""""""""""""""""""""""""""
OpenTelemetry timer and timing metrics are now recorded as Histograms instead of Gauges, preserving
count, sum, and bucket distribution across recordings.
Dag-processing "seconds ago" metric is now tagged (#62487)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``dag_processing.last_run.seconds_ago.{dag_file}`` is now a legacy metric. The new
``dag_processing.last_run.seconds_ago`` is emitted with ``file_path``, ``bundle_name`` and
``file_name`` tags (``file_path`` + ``bundle_name`` uniquely identify the Dag file). The legacy metric
is still emitted by default and can be disabled via ``[metrics] legacy_names_on``.
New Deadlines page under Browse (#67586)
""""""""""""""""""""""""""""""""""""""""
A new **Deadlines** page is available under the Browse menu, accessible to any role that already has
``can_read`` and ``menu_access`` on **Dag Runs**.
New Features
^^^^^^^^^^^^
- Add partition clear support to the REST API matching the CLI, with a ``clearPartitions`` endpoint and ``partition_key``/``partition_date`` window selectors on ``clearDagRuns`` (#68702)
- Add ``[core] mp_start_method`` and ``[core] mp_forkserver_preload`` configuration options (which can be overridden per ``[scheduler]``/``[triggerer]``/``[dag_processor]``) to control the ``multiprocessing`` start method (#68875)
- Add a ``durable`` toggle to ``ResumableJobMixin`` to opt out of resumable execution (#68623)
- Add a ``@result`` decorator to mark a TaskFlow task as the Dag's result task (#64563)
- Add ``[triggerer] shared_stream_cohort_grace_period`` to reduce missed events on triggerer restart (#68888)
- Propagate ``partition_date`` from producer Dag runs to consumers of partitioned assets (#67285)
- Make the task and asset state store accessible from triggers via ``AssetStateStoreAccessors`` (#67839)
- Add OpenTelemetry head sampling support (#68591)
- Add async XCom ``accessors`` for async tasks (#68299)
- Add an async ``aget_hook`` method to ``BaseHook`` for async tasks (#68506)
- Allow custom partition ``Window`` subclasses via a plugin registry (#68717)
- Support an ``extra`` field for the Coordinator (#68694)
- Apply ``rerun_with_latest_version`` to ``TriggerDagRunOperator`` reruns (#67273)
- Scope the XCom Execution API to teams in multi-team mode (#68850)
- Enforce pool team ownership in the scheduling loop (#68649)
- UI: Add team name to the asset graph view (#68457)
- Populate ``partition_date`` for partitioned Dag runs whose composite asset key has a single time-based dimension (#68442)
- UI: Add a column to the asset store table linking to the task instance that wrote it (#68395)
- UI: Add a custom expiration datetime picker for the task store modal (#68394)
- UI: Add additional task instance attributes to the task instance details section (#68378)
- UI: Add a Details tab to the mapped task instance view (#68340)
- UI: Add bulk marking of Dag runs as success or failed from multi-select (#68278)
- Add ``--team-name`` support to the pool CLI commands (#68110)
- UI: Add a full-screen toggle to the code viewer (#68044)
- Add API endpoint support for consumer team asset filtering (#68034)
- UI: Add bulk clear selection for task instances (#68029)
- Add ``awaiting_input`` task state for Human-in-the-Loop, running off the triggerer (#68028)
- Add bulk API to mark Dag runs as success or failed (#67948)
- Record writer info for every asset store write for better cross-linkage (#67902)
- Register XCom ``output_type`` classes from a worker-side Dag walk (#67875)
- Add ``FixedKeyMapper`` and ``SegmentWindow`` for categorical asset-partition rollup (#67716)
- Add a bulk ``POST /dags/{dag_id}/clearDagRuns`` API endpoint (#67709)
- Return Pydantic model instances through XCom for structured output (#67644)
- Add the ability to apply a note when clearing a Dag run or task instances (#67639)
- Add ``consumer_teams`` to ``AssetAccessControl`` in the Task SDK (#67625)
- Populate trigger ``team_name`` at creation time for multi-team support (#67605)
- UI: Add bulk Clear on the Dag Runs list page (#67564)
- Add multi-team query filtering to triggerer trigger assignment (#67517)
- Add forward fan-out support via the ``forward`` kwarg on ``Window`` (AIP-76) (#67475)
- Add patch task state API and ``expires_at`` support in the set API (AIP-103) (#67319)
- Add a ``team_name`` column to the trigger table for multi-team triggerer support (#67305)
- UI: Add asset and task store views (#67292)
- Add a ``--team-name`` CLI argument to the triggerer for multi-team (#67254)
- Add an ``allow_global`` option to asset access control (#67251)
- Add mTLS and private CA support to the API client and server (#67214)
- Add Markdown documentation support for TaskGroups (#67207)
- Add a per-mapper ``max_fan_out`` override for partition fan-out cap (#67184)
- Add timezone support to the SDK temporal partition mappers (#67164)
- Add ``ResumableJobMixin`` with ``SparkSubmitOperator`` for surviving worker failures (#67118)
- Add bulk delete for Dag runs (#67095)
- Add a ``nav_top_level`` option for plugin nav items (#67084)
- Add Core API endpoints for task state and asset state (AIP-103) (#67041)
- Replace ``allow_producer_teams`` with ``access_control`` on ``Asset`` (#66954)
- Add worker-side custom state backend support (AIP-103) (#66859)
- Let partitioned Dag runs fire on a partial upstream window with ``wait_policy`` (#66848)
- Consume task-emitted partition keys on asset events (AIP-76) (#66782)
- Add per-task state key retention from operators (AIP-103) (#66699)
- Add a ``callback_execution_timeout`` config for deadline callbacks (#66609)
- Add a ``clear_on_success`` config to wipe task state on success (AIP-103) (#66586)
- Add a ``partitions clear`` CLI command to reset DagRun partition fields (#66520)
- Make CORS ``allow_credentials`` configurable (#66503)
- Add periodic task state garbage collection and retention support (AIP-103) (#66463)
- Add URI sanitizers and asset factories for new schemes (#66426)
- Add a ``teams sync`` CLI command (#66418)
- Add remote log upload support for callback subprocesses (#66379)
- Add by-name/by-uri asset state routes and ``AssetUriRef`` support (AIP-103) (#66336)
- UI: Add support for rendering multi-type params (#66278)
- Filter Dags by teams when registering asset changes (#66168)
- Wire up Task SDK communication and context access for task/asset state (AIP-103) (#66160)
- UI: Add marking a task group as success or failed (#66146)
- Add Execution API endpoints for task and asset states (AIP-103) (#66073)
- Add ``FanOutMapper`` for one-to-many partition fan-out (#66030)
- Add ``Variable.keys()`` to list variable keys by prefix in the Task SDK (#66022)
- Add an ``airflow dags clear`` command for partition-range reprocessing (#66004)
- Add a ``memray_detailed_tracing`` option for deeper memory profiling (#65996)
- UI: Add support for different graph directions in the asset graph view (#65948)
- UI: Add a Clear All Mapped Tasks button (#65813)
- Add ``allow_producer_teams`` to the Asset SDK class (#65790)
- UI: Show expected duration based on historical average in Dag Run details (#65722)
- Add team name to the task context (#65617)
- Add an ``on_kill()`` hook to ``BaseTrigger`` to handle user actions on triggers (#65590)
- Allow accessing a Dag's members via ``[]`` (#65586)
- Add pluggable retry policies for Airflow tasks (AIP-105) (#65474)
- Add support for ``format="Duration"`` in params (#65469)
- UI: Add pagination to the grid view (#65388)
- Add ``partition_key`` to the task context (#65359)
- Make the blocked-thread warning threshold configurable (#65009)
- Add name fields to SDK deadline alerts (#64926)
- Add dynamic interval resolution support via Variables for deadline alerts (#64751)
- Add an ``is_backfillable`` property to Dag API responses (#64644)
- Return dag-specified results in the dag run wait API (#64577)
- Hold a Dag run until all upstream partitions arrive (AIP-76) (#64571)
- Add a way to mark a return-value XCom as the dag result (#64522)
- Allow accessing a TaskGroup's members via ``[]`` (#64430)
- UI: Redo the Gantt chart (#64335)
- UI: Add task-level filters to the Dag graph tab (#64271)
- UI: Add bulk Clear, Mark Success/Fail, and delete for multiple task instances (#64141)
- Check that multi-team is enabled when a team name is provided to the API (#63994)
- Add a ``DagRunType`` for operators (#63733)
- UI: Add search functionality to the task log viewer (#63467)
- Add patching of task group instances in the API (#62812)
- Add deadlines API endpoints (#62583)
- Add async connection testing via workers for security isolation (#62343)
- Add ``run_after`` to ``TriggerDagRunOperator`` (#62259)
- UI: Display deadlines on the Dag Run and Overview tabs (#62195)
- Re-enable the ``start_from_trigger`` feature with template-field rendering (#55068)
- Backfill partitioned Dags by partition-date range (#67537)
- Add a producer-side acknowledgement channel to shared-stream triggers (#67523)
- UI: Add ``partition_date`` to the Dag run detail page (#68977)
- Expose the upstream ``partition_key`` on ``triggering_asset_events`` and ``dag_run.consumed_asset_events`` (AIP-76) (#69120)
- Allow ``get``/``set``/``delete``/``clear`` of ``AssetStateStoreAccessor`` to run on the triggerer (#68966)
Bug Fixes
^^^^^^^^^
- Fix ``KubernetesExecutor`` scheduler crash caused by a ``pod_override`` that cannot be pickled when running in-cluster (#68831)
- UI: Fix dashboard alert clamping and collapse controls (#68893)
- Stabilize mapped-task XCom result ordering in the Dag run ``wait`` endpoint by ordering on ``task_id``/``map_index`` (#68550)
- Only log task state cleanup when a worker state store backend is configured (#68878)
- Fix in-process Execution API loop stopped while transport still in use (#68865)
- Fix task state store custom expiry datetime missing timezone on save (#68823)
- Do not leak threads from ``InProcessExecutionAPI`` (#68840)
- Fix partitioned backfill widening a sub-day window to the whole day (#68718)
- UI: Fix inconsistent padding between Dag Runs and Task Instances list views (#68689)
- Skip asset-change registration for tasks with no outlets (#68687)
- Percent-encode API client path params for keys with slashes (#68667)
- Fix bulk create+overwrite silently resetting unset fields on pools and connections (#68645)
- Fix triggerer crash when a trigger subclass does not call ``super().__init__()`` (#68636)
- Fix Task SDK swallowing errors when ``Variable.set()`` or ``Variable.delete()`` fails (#68542)
- Populate ``partition_date`` when manually triggering partitioned Dags (#68458)
- Improve warning visibility for invalid JSON when editing variables (#68268)
- Fix the triggerer log server port configuration key (#67785)
- Enforce ``ti:self`` scope on ``/execution/task-reschedules/{ti}/start_date`` (#67628)
- UI: Fix misleading Calendar "Total Runs" coloring behavior (#67595)
- Fix Stats not being initialized in the API server lifespan (#68514)
- Fix ``BackfillDagRun.partition_key`` type annotation (#68432)
- Fix backward compatibility for ``DagRunInfo`` partition fields (#68342)
- Fix ``airflow db clean`` failing on foreign-key-referenced ``dag_version`` rows (#68339)
- Fix 500 error when listing event logs with a NULL timestamp (#68338)
- Fix MySQL downgrade from 3.3.0 for the ``deadline_alert.interval`` JSON conversion (#68337)
- Fix older and custom secrets backends breaking on Airflow 3.2 (#68302)
- Fix secrets backend connection errors being silently swallowed at DEBUG level (#68301)
- UI: Fix the instance name title shown on non-Dag pages (#68288)
- Fix scheduler not populating ``partition_date`` for temporal asset partitions (#68266)
- UI: Fix wrong language being auto-detected from browser preferences (#68258)
- Honor ``retry_policy`` on non-deferrable ``TriggerDagRunOperator`` wait failures (#68254)
- Fix scheduler crash loop when the last task instance predates Dag versioning (#68253)
- Fix team consumer asset filtering (#68242)
- UI: Fix sluggish multi-selection behavior in tables (#68229)
- UI: Fix mapped task instance links for tasks without a start date (#68194)
- Fix setup/teardown auto-inclusion when clearing or marking tasks (#68193)
- UI: Remove redundant columns from the XCom panel on the task instance page (#68188)
- UI: Fix Gantt tooltip showing the wrong start date on queued/scheduled segments (#68176)
- Fix Java SDK coordinator rejecting IPv4-mapped IPv6 connections (#68169)
- Fix Java SDK tasks being rejected by the coordinator connection-ownership check (#68147)
- UI: Fix language key for the Dag bundle filter (#68131)
- Fix ``DagFileProcessorManager`` silent hang on database lock contention (#68118)
- Mask all connection extra and variable values in the API audit log (#68049)
- Fix spurious "Failed to detach context" error on Execution API disconnects (#68039)
- UI: Fix Dag code highlighting for triple-quoted and escaped-brace f-strings (#68026)
- Fix cursor encoding for column-form sort parameters in the REST API (#67973)
- Fix ``SimpleAuthManager`` not preserving the deep-link next URL on first login (#67965)
- Guard the task stats emission to prevent errors (#67955)
- UI: Fix task instance state badge staying stale after a Mark-as action (#67950)
- Register nested Pydantic models for XCom deserialization (#67932)
- Fix ``example_asset_store`` consumer crash (#67922)
- Raise ``InvalidJwtError`` in ``JWTValidator.avalidated_claims()`` when the key ID does not match (#67909)
- Fix Kubernetes executor ``pod_override`` being stringified without the cncf provider (#67895)
- UI: Prevent duplicate task instance summary stream refreshes after mutations (#67892)
- Reject negative ``default_retention_days`` in the Task SDK and core API routes (#67890)
- Fix ``none_failed_min_one_success`` trigger rule checks (#67873)
- Remove trigger kwargs from the REST API response (#67868)
- UI: Fix long parameter names overflowing the Trigger Dag modal (#67859)
- Fix misleading log message in the task runner clear-on-success block (#67836)
- Fix scheduler crash when logging orphaned task resets (#67822)
- UI: Fix dashboard pool summary showing incorrect deferred slot usage (#67818)
- Fix trigger datetime deserialization (#67795)
- UI: Fix Graph layout for TaskGroup tasks wired to external nodes (#67720)
- Fix ``airflow dags clear`` clearing the wrong day for non-UTC partitioned timetables (#67717)
- Fix per-index evaluation of ``ONE_FAILED`` in mapped task groups (#67684)
- UI: Fix dialog dismissal for the Chakra upgrade (#67674)
- UI: Hide dashboard metric percentages when a state count is capped (#67664)
- Apply per-file authorization to the dag-source endpoint (#67662)
- Fix ``airflow dags next-execution --table`` crash when no next run exists (#67642)
- UI: Fix the time picker omitting seconds (#67636)
- Filter scheduling-dependencies graph edges by readable-Dag access (#67627)
- Mask per-key secrets-backend-kwarg overrides on the Config API (#67622)
- Fix ``GET /auth/login`` missing a 400 response in the OpenAPI spec (#67571)
- Fix ``GET /pools`` incorrectly documenting a 404 response in the OpenAPI spec (#67570)
- Add a compatibility layer for import errors caused by ``AirflowSecretsBackendAccessDenied`` (#67560)
- UI: Fix rendering of None child state (#67552)
- Fix sort order for mapped task instances (#67551)
- Fix import errors total-entries count with multiple Dags per file (#67550)
- UI: Prefer active over queued state for collapsed groups (#67543)
- Fix callback state not updating from executor events due to a UUID type mismatch (#67542)
- Reject wildcard origin in CORS config instead of toggling credentials (#67502)
- Guard the finally-block logger in the HTTP access log middleware (#67501)
- Strip CR/LF from user-supplied logical date before logging (#67500)
- Redact secret-looking query parameters in the HTTP access log (#67498)
- UI: Fix Calendar view to respect the user-selected timezone (#67497)
- Escape LIKE wildcards in non-search filter parameters (#67496)
- Fix missing redaction of secret values in variable JSON (#67495)
- Fix bulk CREATE+OVERWRITE team-context authorization bypass (#67493)
- UI: Return 400 instead of 500 from ``structure_data`` on a malformed asset expression (#67489)
- Fix ``SimpleAuthManager`` redirect to the next URL after login (#67483)
- Return 400 instead of 500 from ``materialize_asset`` on invalid input (#67445)
- UI: Restore the Monaco find widget in the Dag Code view (#67391)
- UI: Fix an HTTPException import that turned a 400 into a 500 in the dags endpoint (#67363)
- Restore ``fail_fast`` handling when reschedule exceeds the MySQL ``TIMESTAMP`` limit (#67353)
- Fix the Triggered Dag button not being visible during queued/running state (#67327)
- Fix variables import with structured falsy values (#67060)
- Avoid logging Execution API bearer credentials (#67059)
- Sanitize Dag processor metric file names (#67029)
- Return a 422 when the database rejects an API payload (#66888)
- Prevent ``AlreadyRunningBackfill`` error caused by an invalid date range request (#66874)
- Restrict owner-link and extra-link ``href`` values to safe schemes (http, https, mailto, relative) (#66741)
- Add a session parameter to the ``BaseStateBackend`` interface to fix custom backends (#66708)
- Allow deadline callbacks within the same Dag module (#66702)
- UI: Fix relative React plugin bundle URLs in dev mode (#66618)
- Validate Dag trigger conf as a JSON object or null (#66617)
- Require a trust sentinel for ``state.user`` injection in ``get_user()`` (#66562)
- Use ``hmac.compare_digest`` for ``SimpleAuthManager`` password comparison (CWE-208) (#66556)
- Set ``SameSite=Lax`` on the ``SimpleAuthManager`` all-admins login cookie (#66502)
- Reserve ``/auth`` and ``/pluginsv2`` from plugin URL prefixes (#66501)
- Use a cryptographically secure RNG for ``SimpleAuthManager`` passwords (#66500)
- Fix Triggerer ``runner_health_check_threshold`` log formatting (#66486)
- Fix Dag processor callback cleanup for versioned bundle files (#66484)
- Default ``AIRFLOW_UID`` to 50000 in the airflow-init chown lines (#66481)
- Strip CR/LF from MySQL URL query values before forwarding to ``my.cnf`` (#66325)
- Fix ``CronMixin`` not resolving cron presets before validation (#66102)
- Fix ``AirflowSDKConfigParser`` missing the ``mask_secrets`` method (#66077)
- Fix ``resolve_xcom_backend`` to rely on the config schema default (#65938)
- Fix a missing import cast error in the dag_run API route (#65748)
- Mask Dag processor connection and variable responses (#65704)
- UI: Show import error for deactivated Dags (#65687)
- Forward MySQL SSL params from ``sql_alchemy_conn`` to ``airflow db shell`` (#65575)
- Disable SQLite FK checks in the 0111 migration downgrade (#65545)
- Handle Variable values that cannot be decrypted gracefully in the stable REST API (#65452)
- Retry ``TriggerDagRunOperator`` when the triggered DagRun fails (#65390)
- Fix task run exceptions never being caught by Sentry (#65161)
- Fix the bulk task instance authorization error message rendering (#64719)
- Fix trigger template rendering failure when operator ``template_fields`` differ from trigger attributes (#64715)
- Fix ``task_defer`` with non-JSON ``next_kwargs`` in ``TaskInstance`` (#64714)
- Add the error as ``context["exception"]`` in ``InProcessTestSupervisor`` (#64568)
- Fix NPM security alerts in the simple auth manager (#64309)
- Fix Dag run trigger to surface errors instead of swallowing them (#64130)
- Fix Task SDK Connection extras built from a URI constructor (#64120)
- Add insert/update-on-conflict for rendered task instance fields (#63874)
- Fix ``timeout_with_traceback`` crashes on Windows and non-main threads (#63664)
- UI: Wrap long lines in the rendered templates view (#63492)
- Block path traversal via ".." in ``dag_id`` and ``run_id`` (#63296)
- Fix the scheduler health check command in ``docker-compose.yaml`` (#62280)
- Fix unmapped task deadlock when upstream tasks are removed (#62034)
- Forward termination signals from the supervisor to the task subprocess (#61627)
- Check destination team permission when using bulk APIs for connections, variables, and pools (#68573)
- Fix the execution API ``/health`` check failing on the empty-path route (#68578)
- Fix Dag run partition key filter breaking on composite keys containing ``|`` (#68459)
- Fix the partition clear date range for non-UTC partitioned timetables (#68460)
- Validate that partition keys are non-empty and within the column length (#68443)
- Fix the scheduler serving stale Dag code after an in-place serialized Dag version update (#68558)
- Determine the latest Dag version by version number to avoid collisions when timestamps tie (#68389)
- Fix new runs and reruns executing an outdated bundle version when the Dag serialization is unchanged (#68336)
- Fix remote logging from the task supervisor (#68370)
- Upload task logs even when the final state update fails (#67935)
- Escape URLs in the Task SDK client when looking up Dag operations (#68129)
- Fix task scheduling when multi-team is enabled (#68634)
- Fix secret values not being masked in rendered templates when keys use dot or dash separators (#68624)
- Fix ``jwt_audience`` for the public API being read from two different config sections (#67494)
- Fix duplicate deadline-miss callbacks firing from multiple HA scheduler replicas (#64737)
- Fix scheduler crash on non-ASCII Dag names when OpenTelemetry metrics are enabled (#68023)
- Fix Dag processor crash on non-ASCII names in OpenTelemetry gauge and timer metrics (#68284)
- Report duplicate plugin names as import errors instead of silently ignoring them (#66649)
- Require edit permission for async connection tests that update an existing connection (#68127)
- Restore the deprecated ``[core] execution_api_server_url`` mapping to ``[workers] execution_api_server_url`` (#63949)
- Fix ``dag.test()`` not re-syncing sibling Dags across repeated calls (#66205)
- UI: Invalidate per-attempt task instance caches after actions so logs and details are not stale (#67212)
- Fix task runner failure on a duplicate task instance success-state update (#63355)
- Fix a race condition on the ``order_by`` parameter when listing Dag runs via the REST API (#68948)
- Exclude non-successful Dag runs from the ``DeadlineReference.AVERAGE_RUNTIME`` deadline calculation so failed runs no longer skew the computed deadline (#68949)
- Allow ``InProcessExecutionAPI`` to start without ``api_auth.jwt_secret`` configured (#68982)
- Make ``airflow dags test`` wait for Human-in-the-loop input instead of looping indefinitely on parked HITL tasks (#69104)
- Fix the Java coordinator rejecting macOS dual-stack loopback connections (#68973)
- Fix an asset-event ingestion crash for Dags using ``FixedKeyMapper`` (#69326)
- UI: Fix the details panel header overlapping the tabs (#69318)
- Fix new Dag versions being created when a task's ``retry_policy`` was serialized (#69315)
- Fix retry-policy overrides not being persisted to task-instance history (#69241)
- Fix deadline callback data not being persisted (#69259)
Miscellaneous
^^^^^^^^^^^^^
- Propagate the resolved task log level and ``[logging] namespace_levels`` to language SDK runtimes (#68712)
- Forward run-identity attributes (``dag_id``, ``run_id``, ``run_type``) to the trace sampler so a custom head sampler can differentiate by run kind (#68592)
- Remove ``all_map_indices`` from ``task_state_store.clear()`` in the task context (#68880)
- Optimize the dag processor by caching bundle-to-team name lookups (#68730)
- Rename the misleading ``last_automated_run`` param to ``reference_run`` (#68714)
- Add a ``team_name`` tag to the remaining multi-team metrics (#68601)
- Add a ``team_name`` tag to dag processor metrics for multi-team deployments (#68599)
- Add a ``team_name`` tag to asset metrics for multi-team deployments (#68367)
- UI: Persist dashboard alert collapse state and clamp long alerts (#68329)
- Optimize bulk variable deletion to avoid N+1 queries (#68508)
- UI: Unify the Dag Code tab toolbar styling with the Logs toolbar (#68449)
- Optimize bulk Dag run authorization to avoid N+1 team-name queries (#68286)
- Improve ``airflow dags`` command to use bulk clear (#68280)
- Add the ``task_state_store`` table to the ``airflow db clean`` mechanism (#68218)
- Add metrics and traces to ``ResumableJobMixin`` for crash recovery (#68213)
- Improve ``ResumableJobMixin`` crash-recovery observability with better logging (#68206)
- Pass ``DagRun`` to ``task_instance_mutation_hook`` for run-aware task mutation (#68198)
- Add ``team_name`` to multi-team metrics (#68108)
- Reduce redundant Dag team lookups in authorization checks (#68020)
- Enhance ``ResumableJobMixin.get_job_status`` with context for better job status tracking (#68009)
- Propagate OpenTelemetry trace headers from the client to Execution API server-side spans (#67904)
- Widen the type hint for the ``DagRun.get_task_instances`` / ``fetch_task_instances`` state parameter (#67880)
- UI: Use the bulk clear Dag runs endpoint for bulk Dag run clear (#67846)
- Add a ``default`` parameter to the task and asset state ``get()`` method (#67842)
- Make core API routes for task and asset states interact only with the database (#67835)
- Optimize Dag processor file-queue deduplication from O(N^2) to O(N) (#67750)
- Add ``allow_consumer_teams`` and ``allow_global_consumers`` columns to ``TaskOutletAssetReference`` (#67730)
- Speed up the Dags list and dashboard queries on large ``DagRun`` tables (#67721)
- Make ``partition_key`` provenance-only and inherit it onto asset events (#67718)
- Speed up Dag serialization by skipping a redundant asset roundtrip (#67702)
- Cache ``BaseOperator.__init__`` signature in operator serialization (#67701)
- Optimize ``TaskGroup.topological_sort`` for reverse-declared Dags (#67688)
- Update serialization for producer-side asset access control (#67658)
- Allow outlets to be added and accessed in ``AssetStateAccessor`` (#67619)
- Unify task/asset state storage between the Core API and Execution API (#67547)
- Decorate custom state references with an envelope for UI clarity (#67530)
- Simplify authoring of task and asset states by allowing JSON types (#67418)
- Replace Sphinx Redoc with Swagger for the API docs (#67390)
- Emit OpenTelemetry spans around listener hook calls (#67347)
- UI: Update verbiage for lower-priority backfill runs (#67338)
- Fix N+1 query in the bulk task instance delete endpoint (#67304)
- Speed up ``TaskGroup.topological_sort`` with an int-indexed projected sweep (#67288)
- UI: Use react-query native error state for bulk action hooks (#67284)
- Wrap ``executor.heartbeat()`` in a timer to localize scheduler loop slowdowns (#66808)
- Emit ``dagrun.first_task_start_delay`` separately from scheduling delay (#66807)
- Share one poll loop across sibling event triggers (#66584)
- UI: Upgrade icons, spacing, and default component themes (#66569)
- Warn when ``SimpleAuthManager`` runs in a production-shaped deployment (#66563)
- Migrate Stackdriver logging config to the ``RemoteLogIO`` pattern (#66513)
- Add a ``BundleVersion`` dataclass and ``version_data`` persistence to ``DagVersion`` (#66491)
- Avoid lazy-loading timetable fields for latest DagRuns (#66488)
- Move ``allow_producer_teams`` to ``DagScheduleAssetReference`` (#66487)
- Pass user teams to the ``create_asset_event`` endpoint (#66367)
- Load ``USFederalHolidayCalendar`` lazily to reduce memory usage when loading examples (#66303)
- Propagate task OpenTelemetry trace context through IPC into Execution API requests (#66151)
- Surface worker Dag parse duration in the task log (#66138)
- Skip deserializing ``trigger_kwargs`` when loading serialized Dags (#66002)
- Honor ``AUTH_ROLE_PUBLIC`` in the FastAPI API server (#65685)
- Add extended sysinfo for the Edge worker (#65472)
- Clarify logs when a Dag is being processed in the Dag processor (#65196)
- Add indexes on ``task_instance.dag_version_id`` and ``dag_run.created_dag_version_id`` (#64818)
- Improve creation of ``RuntimeTaskInstance`` in ``TriggerRunner`` for ``start_for_trigger`` functionality (#64298)
- Mark the Triggerer supervisor as a server context so it can read metastore connections (#64022)
- Load hook metadata from YAML without importing the hook class (#63826)
- Add detailed task spans (#63568)
- Downgrade logging on query JSON parsing and add a JSON load condition (#62044)
- UI: Add a Deadlines section with a time-range selector to the Dashboard page (#68038)
- UI: Add a modal for editing notes with Markdown support (#68362)
- UI: Improve the Human-In-The-Loop form UX (#68397)
- Add a ``team_name`` tag to executor metrics for multi-team deployments (#68593)
- Make task and asset state store row size limits configurable (#68133)
- UI: Add notification UX for Human-In-The-Loop actions (#68346)
- Allow synchronous deadline callbacks (``SyncCallback``) to access Connections and Variables (#65269)
- Add a ``team_name`` tag to deadline metrics for multi-team deployments (#68589)
- Add a ``team_name`` tag to scheduler metrics for multi-team deployments (#68594)
- Defer the Cadwyn import so FastAPI/Starlette stay off the Task SDK worker path, reducing per-worker memory (#69029)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Complete the Taiwanese Mandarin (``zh-TW``) translation (#68870)
- Add missing Korean (``ko``) translations (#68600)
- Close German (``de``) translation gaps (#68356)
- Add a segment fan-out example to the asset partition example Dag (#68722)
- Fix runtime-partition example Dags using unreachable schedules (#68719)
- Add an example Dag for the task state store with mapped tasks (#68670)
- Fix the gap in the Taiwanese Mandarin (``zh-TW``) translation (#68668)
- Add wait-policy examples to the asset partition example Dag (#68658)
- Add a contributing guide for language SDKs (#68330)
- Add ``sdk.TIRunContext`` documentation for the Go SDK (#68319)
- Add a Go Task SDK authoring guide to the docs (#68223)
- Update supported-versions doc to mark 2.11.2 as EOL (#68212)
- Add documentation for ``ResumableJobMixin`` and resumable tasks (#68136)
- Add CLI examples for team-scoped pools (#68111)
- Add docs for multi-team triggerer support (#67608)
- Clarify trigger rule behavior for the ``removed`` upstream state (#67452)
- Fix outdated image links in ``dags.rst`` (#67357)
- Add an example and docs for runtime asset partitioning (AIP-76) (#67307)
- Add documentation for the Task and Asset Store (AIP-103) (#67299)
- Add a dynamic task mapping no-op example (#67022)
- Add documentation about adding ``access_control`` to the ``Asset`` object (#66949)
- Add a how-to for Dag-level retry via ``on_failure_callback`` (#66277)
- Fix documentation after PR 62645 (#65843)
- Add documentation for team-based asset event filtering (#65690)
- Document ``on_kill()``/``cleanup()`` for triggers (#65671)
- Explain ``xcom_pull`` behaviour without ``task_ids`` in the docs (#65406)
- Improve standalone authentication documentation for Airflow 3.x (#65330)
- Clarify manual Dag run data interval semantics in Airflow 3 (#64740)
- Document and test ``xcom_pull`` ``run_id`` usage for triggered Dag runs (#63030)
- Update params in the backfill documentation (#61821)
- Document the ``apache-airflow-mypy`` package in the core docs (#68561)
- Fix typos and formatting in the Fundamentals documentation (#68524)
- Complete the Hindi (``hi``) UI translation (#68574)
- Fill the Taiwanese Mandarin (``zh-TW``) UI translation gap (#68563)
- Document that Dag bundle ``kwargs`` should reference a Connection rather than inline credentials (#69105)
- Add example plugins and expand the asset-partitions documentation (#69017)
- Java SDK docs: ``JUL`` setup, pinning ``java_executable``, and a config-reload note (#69020)
- Correct the example config for the coordinators (#68940)
Airflow 3.2.2 (2026-05-29)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
- The SMTP STARTTLS upgrade performed by ``airflow.utils.email.send_email`` now validates the SMTP server's certificate against the system's trusted CA bundle by default. Previously the ``starttls()`` call was made without an SSL context, so any certificate was accepted.
Deployments that intentionally point Airflow at an SMTP server with a self-signed or otherwise non-validating certificate and need to preserve the previous behaviour must set ``email.ssl_context = "none"`` in ``airflow.cfg``. The ``"default"`` value (now also the default when the option is unset) uses :func:`ssl.create_default_context`. Previously this option applied only to the ``SMTP_SSL`` path; it now applies to the STARTTLS path as well. (#65346)
- In #64963, the Airflow UI switched from full-match ``*_pattern`` REST API query parameters to the new index-friendly ``*_prefix_pattern`` parameters on list endpoints. This is a behavioral change for search-as-you-type filters in the UI: matches are prefix-based (``LIKE 'term%'`` via a range scan) instead of substring-based (``ILIKE '%term%'``), which means the database can use B-tree indexes and search stays fast on large deployments. The REST API itself keeps both forms: existing ``*_pattern`` parameters still behave exactly as before.
In #66015, a per-search-bar "Match anywhere" toggle was added so users who relied on the previous substring behavior can opt back into it from the UI. Each search input and each text filter pill now has a small regex-icon toggle next to the value; flipping it on switches that input from ``*_prefix_pattern`` to ``*_pattern``. (#66015)
- Fix triggerer race condition and deadlock that caused deferred tasks to stall indefinitely
Triggers that call synchronous SDK methods (e.g. ``get_task_states`` used by
``safe_to_cancel`` in several Google provider operators) could crash the triggerer's
internal subprocess. The triggerer would then continue to heartbeat normally —
appearing healthy to the scheduler — while silently processing zero triggers, causing
every deferred task to time out. This was first reported in issue #64620; a
partial fix shipped in Airflow 3.2.1 (#64882) but introduced a new deadlock
with the same visible symptom under load.
Both issues are fixed by replacing the lock-based serialization with response
multiplexing: each request now carries a unique ID and the response is routed back to
the correct caller, so concurrent requests from trigger threads no longer contend or
deadlock regardless of how many triggers are running or what SDK methods they call.
**New: triggerer subprocess watchdog**
Even with the race fixed, a trigger that blocks the event loop (e.g. by calling
``time.sleep()`` or performing blocking I/O directly in ``async def run()``) would
previously leave the triggerer appearing healthy indefinitely.
A new ``[triggerer] runner_health_check_threshold`` config option (default: 30 seconds)
adds a watchdog: if the triggerer subprocess goes silent for longer than the threshold,
the parent process stops updating the heartbeat so the scheduler can detect the hang and
reassign triggers rather than waiting for them to individually time out. Set the option
to ``0`` to disable the watchdog. (#66412)
- Tighten ``[core] allowed_deserialization_classes_regexp`` to require full-string matches
Patterns in ``[core] allowed_deserialization_classes_regexp`` are now matched
against the entire classname using ``re.fullmatch()`` instead of ``re.match()``.
Previously a pattern such as ``airflow\.models\.Variable`` admitted not only
the intended class but also names that started with it
(e.g. ``airflow.models.Variable_Malicious``), because ``re.match`` only anchors
at the start of the string.
The default value of this option is empty, so out-of-the-box deployments are
unaffected. Deployments that configured this option with patterns relying on
prefix-match semantics — for example ``airflow\.models\.`` to mean "any class
under ``airflow.models``" — must add ``.*`` to the pattern
(``airflow\.models\..*``) to retain the previous behaviour. (#66499)
- Custom deadline reference classes must now be registered via the new ``deadline_references`` attribute on ``AirflowPlugin``, matching the existing pattern for custom timetables and custom partition mappers. To use a custom ``DeadlineReference`` subclass, register it in a plugin's ``deadline_references`` list. Custom references that are not registered will raise ``DeadlineReferenceNotRegistered`` at deserialization. (#66737)
Bug Fixes
^^^^^^^^^
- Fix ``Callback.handle_event`` triggerer crash when OpenTelemetry metrics receive dict typed tag values (#67527) (#67529)
- UI: Rewrite ``modulepreload hrefs`` to the api-server static path (#67548) (#67556)
- Correctly pre-allocate ``external_executor_id`` with multiple executors on PostgreSQL (#67388) (#67458)
- Return raw import-error stacktrace when a Dag file has no registered Dag (#67465) (#67478)
- UI: Fix Expand/Collapse All on XComs and Audit Log JSON cells (#67316) (#67361)
- UI: Load Monaco workers via a same-origin Blob shim (#67352) (#67469)
- UI: Show DAG name in browser tab title (#67169) (#67399)
- Require ``starlette>=1.0.1`` for Host-header parsing fix and ``cadwyn>=6.1.1`` for compatibility
(#67326) (#67460)
- Revoke JWT on /auth/logout regardless of auth manager logout URL (#67289) (#67362)
- Fix deadlock in ``ti_update_state`` caused by FOR UPDATE locking dag_run (#67246) (#67264)
- UI: Stop polling ``getLatestRunInfo`` on paused Dags with no active runs (#67249) (#67256)
- Fail closed when supervisor IPC fails on a non-success terminal state (#66573) (#67183)
- Refuse secrets-backend fallback on Execution-API authorization deny (#66575) (#67173)
- Harden ``_collect_teams_to_check`` and ``requires_access_backfill`` against malformed request bodies (#66504) (#67182)
- Don't crash supervisor IPC loop on transient network errors (#66572) (#67177)
- Default-deny auth at the API and UI router level (#66505) (#67171)
- Apply per-Dag audit log permission to event log detail endpoint (#67112) (#67159)
- Fix ``ValueError`` when supervisor force-closes stuck sockets after timeout (#67115) (#67162)
- Redact rendered template fields while still structured to preserve nested-key masking on truncation (#65906) (#67117)
- Fix migration 0080 to migrate existing deadline rows on upgrade and downgrade (#66016) (#67129)
- Fix ``XCom`` PATCH/POST to store native values instead of ``json.dumps`` output (#64220) (#67116)
- Fix ``max_active_runs`` lost during Dag serialization when value equals schema default (#65310) (#67097)
- Fix N+1 query pattern in bulk pool delete endpoint (#66222) (#67108)
- Optimize DB performance of datetime range filters in API queries (#66696) (#67102)
- Fix ``serialize_template_field`` handling callable value in dict (#63871) (#67092)
- Fix scheduler to ignore stale executor success after defer reschedule (#66431) (#67089)
- Fix ``ArgNotSet`` ``repr`` to use stable string instead of memory address (#65222) (#66897)
- Fix scheduler MySQL task instance index hint (#66785) (#67087)
- UI: Preserve Grid limit and filters when redirecting after manual Dag trigger (#66717) (#66867)
- Apply reserved-key check to ``XCom`` update payload (#65915) (#66913)
- Fix log server path extraction to use ``removeprefix`` (#66749) (#66772)
- Fix macOS ``SIGSEGV`` in task execution by using ``fork`` + ``exec`` (#64874) (#66872)
- Fix Dag auto-pause ordering to use ``run_after`` (#65207) (#66863)
- Fix Dag version inflation caused by unmatched serialized result of task using re-serialized command (#61077) (#66861)
- Fix ``pod_override`` serialization in Dag details and executor path (#65407) (#66898)
- Fix async engine missing ``pool_recycle`` and ``pool_pre_ping`` configuration (#65276) (#66866)
- UI: Make Dag detail page scroll-able on mobile ``viewports`` (#65899) (#66975)
- Fix ``DagVersion`` when clearing tasks with run on latest version (#65835) (#66901)
- Fix millisecond floating point duration bug (#66560) (#66915)
- UI: Fix "Mark state as..." buttons grayed out when task or ``DagRun`` already in target state (#66198) (#66919)
- Fix memory leak in ``LocalExecutor`` caused by unreleased file descriptor locks (#65121) (#66887)
- Fix external DB manager upgrades with existing tables (#66674) (#66882)
- UI: Improve ``DagCalendarTab`` background color retrieval and loading overlay handling (#64189) (#66860)
- UI: Handle Dags state filter overflow on mobile (#66812) (#66847)
- UI: Fix Edit Connection dialog missing ``lazyMount`` causing JSON editor infinite loading (#65969) (#66828)
- UI: Fix ``ConnectionForm`` crashing when connection has invalid extra JSON (#66593) (#66831)
- Handle ``PermissionError`` in ``init_log_folder`` for mounted filesystems (#63878) (#66733)
- Fix scheduler crash by catching ``StaleDataError`` in ``verify_integrity`` (#64503) (#66727)
- Fix triggerer file handle leak when remote log upload fails (#66675) (#66684)
- Fix ``/tmp`` file leak when API server streams large task logs (#66450) (#66667)
- Fix ``XCom`` prior-dates lookup for duplicate ``run_id`` across Dags (#65227) (#66646)
- Fix HITL (Human-In-The-Loop) ``/required_actions`` listing to show mapped task instances (#66433) (#66482)
- Fix scheduler callback ``bundle_version`` when versioning disabled (#66485) (#66518)
- UI: Hide ``Next Run`` timestamp for paused Dags (#66552) (#66568)
- Fix task run context crash when ``DagRun`` state is expired (#66339) (#66347)
- Fix incorrect type warning from OTel spans (#66559) (#66567)
- Fix backfill to populate ``partition_date`` on partitioned backfill runs (#65998) (#66409)
- Fix ``remote_task_handler_kwargs`` passing handler params to ``RemoteLogIO`` (#65957) (#66440)
- Fix i18n translation files served stale after Airflow upgrade due to browser cache (#65720) (#66422)
- UI: Fix manual copy from Rendered Templates tab adding extra blank lines (#66221) (#66366)
- Fix slow and incomplete trigger cleanup in scheduler (#66210) (#66381)
- UI: Distinguish ``upstream_failed`` from ``failed`` in normal vision (#66324) (#66365)
- UI: Fix ``SearchBar`` input rewind (#66284) (#66359)
- Don't re-emit ``logical_date`` when previous ``data_interval`` is zero-length (#66132) (#66263)
- Fix variable access in triggerer for deferrable operators (#63387) (#66239)
- Fix missing ``autoincrement`` sequence on ``callback_request`` downgrade (#65230) (#66189)
- Restore pre/post execute log grouping in task logs (regression in 3.2.x) (#66037) (#66049)
- Preload ``source_aliases`` in ``process_executor_events`` (#65422) (#66191)
- Fix ``dagRuns`` API to honor ``start_date_gte`` filter correctly (#66045) (#66098)
- Fix asset-triggered Dags failing to schedule when their triggers were unassigned in the DB (#65792) (#66043)
- UI: Preserve config changes when re-triggering a Dag from the UI (#65749) (#66044)
- Fix scheduler ``UniqueViolation`` crash on downgrade from 3.2.0 to 3.1.x (#65688) (#66003)
- Run task cleanup hooks (on_failure_callback, listeners) when the supervisor IPC call fails on a terminal-state report (#65714) (#65946)
- Fix triggers with double-encoded payloads failing to deserialize (#64823) (#65584)
- UI: Fix log fetch crash when ``ti.hostname`` is empty (#64285) (#65583)
- Fix backfill marked complete before Dag runs are created (#62561) (#65889)
- UI: Fix date time input ``year`` field unmodifiable (#63885) (#65890)
- UI: Fix pools slot input behavior (#63900) (#65891)
- Fix ``TypeError`` crashes on ``/users/list`` and ``/roles/list`` in FAB UI caused by concurrent API schema requests (#63986) (#65892)
- UI: Fix toaster behavior (#64142) (#65893)
- Fix FAB DB manager discovery in migration-only contexts (#64145) (#65894)
- UI: Fix ``PoolBar`` links using wrong query params for task instances filtering (#64182) (#65896)
- Fix memory growth from ``pathlib`` ``sys.intern`` in long-running processes (#65706) (#65855)
- Pre-assign ``external_executor_id`` at queuing time to prevent duplicate execution on scheduler crash (#65594) (#65711)
- Handle supervisor remote log upload failures gracefully (#65308) (#65318)
- Fix ``ti.start_date`` showing deferral-resume time instead of original start time (#63247) (#65491)
- Fix task CLI ``map_index`` bounds validation (#64133) (#65479)
- UI: Fix mapped task ``XCom`` navigation from Grid (#65192) (#65322)
- Fix connection schema field not saved for providers without field behaviour (#65263) (#65267)
- Fix bulk task instance update for mapped TIs and auth error rendering (#65874)
- Fix bulk task instance RBAC bypass (#64288) (#65846)
- Update ``is_url_safe`` to reject URLs with ``///`` (#65557) (#65737)
- UI: Improve Graph view performance (#65031) (#65537)
- Fix backfill params not overriding existing Dag run conf (#64939) (#65599)
- Fix ``run_id_pattern`` pipe OR operator dropping single-term edge cases (#65190) (#65565)
- Filter external dependency nodes by readable Dags in ``structure_data`` endpoint (#65342) (#65534)
- Respect Dag processor config option to show parsing logs on stdout (#65528) (#65541)
- Add per-Dag authorization to ``partitioned_dag_runs`` endpoints (#65344) (#65538)
- UI: Register trigger and sensor graph node types (#65167) (#65321)
- Ensure DB migrations run in a single connection (#65231) (#65368)
- Fix ``PATCH /dags`` pagination bug and document wildcard ``dag_id_pattern`` (#65309)
- Set JWT refresh cookie ``Secure`` flag when request is HTTPS (#65348) (#65363)
- Refuse to follow log symlinks that resolve outside the base log folder (#65325) (#65345)
- Enforce per-file import-error authorization using ``relative_fileloc`` and bundle (#65329) (#65343)
- UI: Invalidate task instances list query after clearing task instance (#63923) (#65304)
- Recover stuck TIs when direct terminal-state API call fails (#66574) (#67204)
Miscellaneous
^^^^^^^^^^^^^
- UI: Use local Monaco editor module instead of CDN (#66647) (#67199)
- Use a distinct redact message for import errors with no registered Dag (#66923) (#67176)
- Surface remote-log upload failures via structured warnings (#66571) (#67172)
- UI: Filter task instances by rendered map index (#66008) (#67163)
- Move Task Identity line into Pre Execution block in logs (#67036) (#67134)
- Apply ``requires_access_event_log`` to ``GET /eventLogs`` list endpoint (#67185) (#67211)
- UI: Preserve proxy URL on login redirect (#66690) (#67091)
- Keep ``Named*Logger.name`` working across structlog releases (#66875) (#67088)
- Two-token mechanism for task execution to prevent token expiration while tasks wait in executor queues (#60108) (#66989)
- Validate task identity token claims with a typed schema (#63604) (#66988)
- Mark Dags stale when their bundle is removed from config (#66948) (#66985)
- UI: Allow pasting full datetime strings into date picker inputs (#66251) (#66958)
- Validate Dag run conf in backfill dry-run (#66196) (#66935)
- Improve post-task logs to show exception in failure (#66735) (#66920)
- UI: Show Dag run duration in grid tooltip (#65787) (#66900)
- UI: Add Dag run ID to grid bar tooltip and task instance tooltip (#65626) (#66871)
- UI: Change queued Dag runs color to gray in Calendar (#66623) (#66870)
- Add configurable LRU+TTL caching for API server Dag retrieval (#60804) (#66862)
- UI: Use link styling for Dag tags (#66750) (#66855)
- UI: Add hover feedback to ``Checkbox`` (#66714) (#66826)
- Check sensitive key names before applying recursion-depth cutoff in secrets masker (#65912) (#66748)
- Adjust log message header for expandable sources (#66570) (#66653)
- Allow triggerer to support memray memory profiling (#65994) (#66643)
- Show task ID attributes (``ti_id``, ``task_id``, etc.) once, not on every log line (#66036) (#66421)
- Propagate triggering user to child Dag runs via ``TriggerDagRunOperator`` (#65747) (#66378)
- UI: Add ``isExpanded`` prop on JSON expand/collapse buttons (#66340) (#66364)
- Pass ``try_number`` to extra links API (#65661) (#66171)
- UI: Serve grid TI summaries from shared cached ``DagBag`` (#65775) (#65966)
- Add cursor-based pagination for ``get_dag_runs`` endpoint (#65604) (#65746)
- Support ordering ``XCom`` entries in the REST API and UI (#65418) (#65600)
- UI: Add cursor-based pagination for task instances list (#64953) (#65542)
- Include task instance UUID in scheduler, Dag processor, triggerer, and worker logs (#65458) (#65476)
- Enable SQLAlchemy connection pool settings for file-based SQLite (#64888) (#65411)
- Add cursor-based pagination for ``get_task_instances`` endpoint (#64845) (#65405)
- UI: Rework Monaco editor theme to match Chakra UI palette (#64748) (#65228)
- UI: Add Dag runs filters for Consuming Asset (#63624) (#65306)
- UI: Improve grid and ``ti_summaries`` and grid runs queries (#64034) (#67014)
- UI: Enable queue up new tasks (#63484) (#66869)
- Expose queueing/scheduled time in the Gantt chart (#63372) (#65016)
- Export ``from_timestamp`` from Task SDK timezone module (#67321) (#67331)
Doc-only Changes
^^^^^^^^^^^^^^^^
- Refresh JWT authentication and security model docs with mermaid diagrams (#67435) (#67466)
- Fix misleading typo in ``plugins_manager`` docs (#67101) (#67114)
- Document supported deployment platforms in security docs (#66931) (#67017)
- Warn against world-accessible Kerberos ccache default in docs (#66557) (#67085)
- Update French ``(fr)`` UI translations to 100% coverage (#67241)
- Close Catalan translation gap (#67011)
- Close German translation gaps (2026-05-12) (#66830)
- Close Korean translation gaps (May 13) (#66873)
- Add missing Polish translations for new UI keys (#66823)
- Update health endpoint in security docs (#66701) (#66739)
- Add self-diagnosis guide for Dag version inflation in FAQ (#66697) (#66738)
- Add Chakra UI license to ``airflow-core`` (#66703) (#66740)
- Document effects of ``create_cron_data_intervals`` (#66458)
- Clarify Task Execution API coverage in Dag-author-isolation chapter (#66194) (#66322)
- Complete ``zh-TW`` translations (#66401)
- Align Dag capitalization from "DAG" to "Dag" in ``core_api`` (#66211) (#66304)
- Word changed from "DAG" to "Dag" in ``airflow-core/src/airflow/api`` (#66200) (#66214)
- Change Hebrew wording for "Asset Triggered" (#64177) (#65895)
- Update Dag Runs document under Core Concept to be consistent with ``BashOperator`` document (#64129) (#65850)
Airflow 3.2.1 (2026-04-21)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
- Users who only have read access to DAGs will no longer be able to fetch data from the ``/dags`` endpoint, as it now requires additional permissions (``DagAccessEntity.RUN``, ``DagAccessEntity.HITL_DETAIL``, and ``DagAccessEntity.TASK_INSTANCE``). This change was made because the endpoint returns aggregated data from these multiple entities. Please update your custom user roles to include read access for DAG Runs, Task Instances, and HITL Details if those users should still have access to the ``/dags`` endpoint. (#64822)
Improvements
^^^^^^^^^^^^
- Allow UI theme config with only CSS overrides, icon only, or empty ``{}`` to restore OSS defaults. The ``tokens`` field is now optional in the theme configuration. (#64552)
Bug Fixes
^^^^^^^^^
- Fix ``DEFAULT_LOGGING_CONFIG`` to use right ``kwargs`` (#65412) (#65424)
- Fix zip DAG import errors being cleared during bundle refresh (#63617) (#65296)
- Fix ``dispose_orm()`` not disposing async engine on shutdown (#65274) (#65284)
- Fix ``get_team_name_dep`` creating wasted async sessions when ``multi_team=False`` (#65275) (#65282)
- Fix asset graph view leaking DAGs outside the user's permissions (#65273) (#65280)
- Fix migration: add missing ``disable_sqlite_fkeys`` to migration 0108 (#65288) (#65290)
- Fix heartbeat: add fast-path ``UPDATE`` to avoid row lock in the common case (#65029) (#65137)
- UI: Fix deactivated state not shown for stale DAGs (#65214) (#65218)
- Fix N+1 queries during DAG serialization with bulk prefetch (#64929) (#65208)
- Fix serializer for empty string extra in connection (#65014) (#65215)
- UI: Fix menu positioning for ``dropdowns`` in connection forms (#65007) (#65085) (#65138)
- UI: Fix ``SearchBar`` value not syncing with ``defaultValue`` changes (#65054) (#65140)
- Fix SDK configuration to use ``$AIRFLOW_CONFIG`` env (#64936) (#65200)
- Fix ``Session`` staying opened between yields (#65179) (#65195)
- Fix ``Session`` leak from ``StreamingResponse`` API endpoints (#65162) (#65193)
- Fix redirect loop when stale root-path ``_token`` cookie exists from older Airflow instance (#64955) (#65177)
- Fix ``@task`` decorator to validate operator arg types at decoration time (#65041) (#65050)
- Fix CLI error handling and exit codes for failed commands (#65052) (#65097)
- Fix ``is_alive`` default to ``None`` in ``jobs list`` CLI (#65065) (#65091)
- Fix missing ``dag_id`` in ``get_task_instance`` (#64957) (#64968) (#65067)
- UI: Fix cancel ``debounce`` on clear to prevent stale search value (#64893) (#64907)
- Fix read out-of-order issue with send method in ``CommsDecoder`` (#64894) (#64946)
- Fix bulk connection delete banner (#64735) (#64961)
- Fix migrations: move ``UPDATE``s inside ``disable_sqlite_fkeys`` in migration 0097 (#64876) (#64940)
- Fix heartbeat to return 410 Gone when cleared ``TI`` exists in TIH (#61631) (#64693)
- Fix scheduler: skip asset-triggered Dags without ``SerializedDagModel`` (#64322) (#64738)
- Fix N+1 query pattern in task instance states and count endpoints (#60352) (#64695)
- Fix ``TypeError`` in ``GET /dags/{dag_id}/tasks`` when ``order_by`` field has ``None`` values (#64384) (#64587)
- UI: Fix duplicate nav sidebar when iframe navigates away from auth pages (#63873) (#64854)
- UI: Fix Gantt view "Error invalid date" on running ``DagRun`` (#64752) (#64853)
- Fix ``connections import`` returning non-zero exit code on failure (#64416) (#64449)
- Fix structlog positional formatting for single-dict arguments (#62849) (#64773)
- UI: Fix external link ``target`` and add ``rel`` attributes (#64542) (#64772)
- UI: Fix ``DagVersionSelect`` options not filtered by selected ``DagRun`` (#64736) (#64771)
- Fix spurious blank lines in filtered task log download (#64235) (#64640)
- Fix OTel metrics lost in forked task processes (#64703) (#64720)
- Fix ``start_date`` in example DAGs to avoid timezone conversion overflow (#63882) (#64758)
- Fix ``AirflowPlugin`` not re-exported, causing mypy errors in plugins (#65132) (#65163)
- Fix ``apache-airflow-providers-fab`` minimum version to prevent ``connexion`` import error on Python 3.13 (#65523) (#65524)
- UI: Fix graph view not auto-refreshing task states during DAG run (#65518) (#65522)
Miscellaneous
^^^^^^^^^^^^^
- Lock ``TriggerCommsDecoder`` sync req-res cycle (#64882) (#65285)
- Add ``write_to_os`` support for writing task logs to OpenSearch (#64364) (#65201)
- Restore OpenSearch log integration in ``airflow_local_settings.py`` (#64764) (#65003)
Doc-only Changes
^^^^^^^^^^^^^^^^
- Add missing Polish translations to reach 100% coverage (#65272)
- Add FAQ entry for API server memory growth with gunicorn worker recycling (#65036) (#65037)
- Remove outdated reference to Dag Dependencies view (#64787) (#64911)
- Add JWT authentication docs and strengthen security model (#64760) (#64849)
- Add missing Catalan translations to reach 100% coverage (#65078) (#65389)
- Add missing German translations to close translation gaps (#65332)
Airflow 3.2.0 (2026-04-07)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Asset Partitioning
""""""""""""""""""
The headline feature of Airflow 3.2.0 is asset partitioning — a major evolution of data-aware
scheduling. Instead of triggering Dags based on an entire asset, you can now schedule downstream
processing based on specific partitions of data. Only the relevant slice of data triggers downstream
work, making pipeline orchestration far more efficient and precise.
This matters when working with partitioned data lakes — date-partitioned S3 paths, Hive table
partitions, BigQuery table partitions, or any other partitioned data store. Previously, any update
to an asset triggered all downstream Dags regardless of which partition changed. Now only the right
work gets triggered at the right time.
For detailed usage instructions, see :doc:`/authoring-and-scheduling/assets`.
Multi-Team Deployments
""""""""""""""""""""""
Airflow 3.2 introduces multi-team support, allowing organizations to run multiple isolated teams within a single Airflow deployment.
Each team can have its own Dags, connections, variables, pools, and executors— enabling true resource and permission isolation without requiring separate Airflow instances per team.
This is particularly valuable for platform teams that serve multiple data engineering or data science teams from shared infrastructure, while maintaining strong boundaries between teams' resources and access.
For detailed usage instructions, see :doc:`/core-concepts/multi-team`.
.. warning::
Multi-Team Deployments are experimental in 3.2.0 and may change in future versions based on
user feedback.
Synchronous callback support for Deadline Alerts
""""""""""""""""""""""""""""""""""""""""""""""""
Deadline Alerts now support synchronous callbacks via ``SyncCallback`` in addition to the existing
asynchronous ``AsyncCallback``. Synchronous callbacks are executed by the executor (rather than
the triggerer), and can optionally target a specific executor via the ``executor`` parameter.
A Dag can also define multiple Deadline Alerts by passing a list to the ``deadline`` parameter,
and each alert can use either callback type.
.. warning::
Deadline Alerts are experimental in 3.2.0 and may change in future versions based on
user feedback. Synchronous deadline callbacks (``SyncCallback``) do not currently
support Connections stored in the Airflow metadata database.
For detailed usage instructions, see :doc:`/howto/deadline-alerts`.
UI Enhancements & Performance
"""""""""""""""""""""""""""""
- **Grid View Virtualization**:
The Grid view now uses virtualization -- only visible rows are rendered to the DOM. This dramatically improves performance when viewing Dags with large numbers of task runs, reducing render time and memory usage for complex Dags. (#60241)
- **XCom Management in the UI**:
You can now add, edit, and delete XCom values directly from the Airflow UI. This makes it much easier to debug and manage XCom state during development and day-to-day operations without needing CLI commands. (#58921)
- **HITL Detail History**:
The Human-in-the-Loop approval interface now includes a full history view, letting operators and reviewers see the complete audit trail of approvals and rejections for any task. (#56760, #55952)
- **Gantt Chart Improvements**:
- All task tries displayed: Gantt chart now shows every attempt, not just the latest
- Task display names in Gantt: task_display_name shown for better readability (#61438)
- ISO dates in Gantt: Cross-browser consistent date format (#61250)
- Fixed null datetime crash: Gantt chart no longer crashes on tasks with null datetime fields
New ``--only-idle`` flag for the scheduler CLI
"""""""""""""""""""""""""""""""""""""""""""""""
The ``airflow scheduler`` command has a new ``--only-idle`` flag that only counts runs when the
scheduler is idle. This helps users run the scheduler once and process all triggered Dags and
queued tasks. It requires and complements the ``--num-runs`` flag so one can set a small value
instead of guessing how many iterations the scheduler needs.
Replace per-run TI summary requests with a single NDJSON stream
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The grid, graph, gantt, and task-detail views now fetch task-instance
summaries through a single streaming HTTP request
(``GET /ui/grid/ti_summaries/{dag_id}?run_ids=...``) instead of one request
per run. The server emits one JSON line per run as soon as that run's task
instances are ready, so columns appear progressively rather than all at once.
**What changed:**
- ``GET /ui/grid/ti_summaries/{dag_id}?run_ids=...`` is now the sole endpoint
for TI summaries, returning an ``application/x-ndjson`` stream where each
line is a serialized ``GridTISummaries`` object for one run.
- The old single-run endpoint ``GET /ui/grid/ti_summaries/{dag_id}/{run_id}``
has been removed.
- The serialized Dag structure is loaded once and shared across all runs that
share the same ``dag_version_id``, avoiding redundant deserialization.
- All UI views (grid, graph, gantt, task instance, mapped task instance, group
task instance) use the stream endpoint, passing one or more ``run_ids``.
Structured JSON logging for all API server output
"""""""""""""""""""""""""""""""""""""""""""""""""
The new ``json_logs`` option under the ``[logging]`` section makes Airflow
produce all its output as newline-delimited JSON (structured logs) instead of
human-readable formatted logs. This covers the API server (gunicorn/uvicorn),
including access logs, warnings, and unhandled exceptions.
Not all components support this yet — notably ``airflow celery worker`` but
any non-JSON output when ``json_logs`` is enabled will be treated as a bug. (#63365)
Remove legacy OTel Trace metaclass and shared tracer wrappers
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The interfaces and functions located in ``airflow.traces`` were
internal code that provided a standard way to manage spans in
internal Airflow code. They were not intended as user-facing code
and were never documented. They are no longer needed so we
remove them in 3.2. (#63452)
Move task-level exception imports into the Task SDK
"""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow now sources task-facing exceptions (``AirflowSkipException``, ``TaskDeferred``, etc.) from
``airflow.sdk.exceptions``. ``airflow.exceptions`` still exposes the same exceptions, but they are
proxies that emit ``DeprecatedImportWarning`` so Dag authors can migrate before the shim is removed.
**What changed:**
- Runtime code now consistently raises the SDK versions of task-level exceptions.
- The Task SDK redefines these classes so workers no longer depend on ``airflow-core`` at runtime.
- ``airflow.providers.common.compat.sdk`` centralizes compatibility imports for providers.
**Behaviour changes:**
- Sensors and other helpers that validate user input now raise ``ValueError`` (instead of
``AirflowException``) when ``poke_interval``/ ``timeout`` arguments are invalid.
- Importing deprecated exception names from ``airflow.exceptions`` logs a warning directing users to
the SDK import path.
**Exceptions now provided by ``airflow.sdk.exceptions``:**
- ``AirflowException`` and ``AirflowNotFoundException``
- ``AirflowRescheduleException`` and ``AirflowSensorTimeout``
- ``AirflowSkipException``, ``AirflowFailException``, ``AirflowTaskTimeout``, ``AirflowTaskTerminated``
- ``TaskDeferred``, ``TaskDeferralTimeout``, ``TaskDeferralError``
- ``DagRunTriggerException`` and ``DownstreamTasksSkipped``
- ``AirflowDagCycleException`` and ``AirflowInactiveAssetInInletOrOutletException``
- ``ParamValidationError``, ``DuplicateTaskIdFound``, ``TaskAlreadyInTaskGroup``, ``TaskNotFound``, ``XComNotFound``
- ``AirflowOptionalProviderFeatureException``
**Backward compatibility:**
- Existing Dags/operators that still import from ``airflow.exceptions`` continue to work, though
they log warnings.
- Providers can rely on ``airflow.providers.common.compat.sdk`` to keep one import path that works
across supported Airflow versions.
**Migration:**
- Update custom operators, sensors, and extensions to import exception classes from
``airflow.sdk.exceptions`` (or from the provider compat shim).
- Adjust custom validation code to expect ``ValueError`` for invalid sensor arguments if it
previously caught ``AirflowException``.
Support numeric multiplier values for retry_exponential_backoff parameter
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The ``retry_exponential_backoff`` parameter now accepts numeric values to specify custom exponential backoff multipliers for task retries. Previously, this parameter only accepted boolean values (``True`` or ``False``), with ``True`` using a hardcoded multiplier of ``2.0``.
**New behavior:**
- Numeric values (e.g., ``2.0``, ``3.5``) directly specify the exponential backoff multiplier
- ``retry_exponential_backoff=2.0`` doubles the delay between each retry attempt
- ``retry_exponential_backoff=0`` or ``False`` disables exponential backoff (uses fixed ``retry_delay``)
**Backwards compatibility:**
Existing Dags using boolean values continue to work:
- ``retry_exponential_backoff=True`` → converted to ``2.0`` (maintains original behavior)
- ``retry_exponential_backoff=False`` → converted to ``0.0`` (no exponential backoff)
**API changes:**
The REST API schema for ``retry_exponential_backoff`` has changed from ``type: boolean`` to ``type: number``. API clients must use numeric values (boolean values will be rejected).
**Migration:**
While boolean values in Python Dags are automatically converted for backwards compatibility, we recommend updating to explicit numeric values for clarity:
- Change ``retry_exponential_backoff=True`` → ``retry_exponential_backoff=2.0``
- Change ``retry_exponential_backoff=False`` → ``retry_exponential_backoff=0``
Move serialization/deserialization (serde) logic into Task SDK
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow now sources serde logic from ``airflow.sdk.serde`` instead of
``airflow.serialization.serde``. Serializer modules have moved from ``airflow.serialization.serializers.*``
to ``airflow.sdk.serde.serializers.*``. The old import paths still work but emit ``DeprecatedImportWarning``
to guide migration. The backward compatibility layer will be removed in Airflow 4.
**What changed:**
- Serialization/deserialization code moved from ``airflow-core`` to ``task-sdk`` package
- Serializer modules moved from ``airflow.serialization.serializers.*`` to ``airflow.sdk.serde.serializers.*``
- New serializers should be added to ``airflow.sdk.serde.serializers.*`` namespace
**Code interface changes:**
- Import serializers from ``airflow.sdk.serde.serializers.*`` instead of ``airflow.serialization.serializers.*``
- Import serialization functions from ``airflow.sdk.serde`` instead of ``airflow.serialization.serde``
**Backward compatibility:**
- Existing serializers importing from ``airflow.serialization.serializers.*`` continue to work with deprecation warnings
- All existing serializers (builtin, datetime, pandas, numpy, etc.) are available at the new location
**Migration:**
- **For existing custom serializers**: Update imports to use ``airflow.sdk.serde.serializers.*``
- **For new serializers**: Add them to ``airflow.sdk.serde.serializers.*`` namespace (e.g., create ``task-sdk/src/airflow/sdk/serde/serializers/your_serializer.py``)
Methods removed from PriorityWeightStrategy
""""""""""""""""""""""""""""""""""""""""""""
On (experimental) class ``PriorityWeightStrategy``, functions ``serialize()``
and ``deserialize()`` were never used anywhere, and have been removed. They
should not be relied on in user code. (#59780)
Methods removed from TaskInstance
"""""""""""""""""""""""""""""""""
On class ``TaskInstance``, functions ``run()``, ``render_templates()``,
``get_template_context()``, and private members related to them have been
removed. The class has been considered internal since 3.0, and should not be
relied on in user code. (#59780, #59835)
Modify the information returned by ``DagBag``
"""""""""""""""""""""""""""""""""""""""""""""
**New behavior:**
- ``DagBag`` now uses ``Path.relative_to`` for consistent cross-platform behavior.
- ``FileLoadStat`` now has two additional nullable fields: ``bundle_path`` and ``bundle_name``.
**Backward compatibility:**
``FileLoadStat`` will no longer produce paths beginning with ``/`` with the meaning of "relative to the dags folder".
This is a breaking change for any custom code that performs string-based path manipulations relying on this behavior.
Users are advised to update such code to use ``pathlib.Path``. (#59785)
Remove ``--conn-id`` option from ``airflow connections list``
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The redundant ``--conn-id`` option has been removed from the ``airflow connections list`` CLI command.
Use ``airflow connections get`` instead. (#59855)
Add operator-level ``render_template_as_native_obj`` override
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Operators can now override the Dag-level ``render_template_as_native_obj`` setting,
enabling fine-grained control over whether templates are rendered as native Python
types or strings on a per-task basis. Set ``render_template_as_native_obj=True`` or
``False`` on any operator to override the Dag setting, or leave as ``None`` (default)
to inherit from the Dag.
Add gunicorn support for API server with zero-downtime worker recycling
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The API server now supports gunicorn as an alternative server with rolling worker restarts
to prevent memory accumulation in long-running processes.
**Key Benefits:**
* **Rolling worker restarts**: New workers spawn and pass health checks before old workers
are killed, ensuring zero downtime during worker recycling.
* **Memory sharing**: Gunicorn uses preload + fork, so workers share memory via
copy-on-write. This significantly reduces total memory usage compared to uvicorn's
multiprocess mode where each worker loads everything independently.
* **Correct FIFO signal handling**: Gunicorn's SIGTTOU kills the oldest worker (FIFO),
not the newest (LIFO), which is correct for rolling restarts.
**Configuration:**
.. code-block:: ini
[api]
# Use gunicorn instead of uvicorn
server_type = gunicorn
# Enable rolling worker restarts every 12 hours
worker_refresh_interval = 43200
# Restart workers one at a time
worker_refresh_batch_size = 1
Or via environment variables:
.. code-block:: bash
export AIRFLOW__API__SERVER_TYPE=gunicorn
export AIRFLOW__API__WORKER_REFRESH_INTERVAL=43200
**Requirements:**
Install the gunicorn extra: ``pip install 'apache-airflow-core[gunicorn]'``
**Note on uvicorn (default):**
The default uvicorn mode does not support rolling worker restarts because:
1. With workers=1, there is no master process to send signals to
2. uvicorn's SIGTTOU kills the newest worker (LIFO), defeating rolling restart purposes
3. Each uvicorn worker loads everything independently with no memory sharing
If you need worker recycling or memory-efficient multi-worker deployment, use gunicorn. (#60921)
Improved performance of rendered task instance fields cleanup for Dags with many mapped tasks (~42x faster)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The config ``max_num_rendered_ti_fields_per_task`` is renamed to ``num_dag_runs_to_retain_rendered_fields``
(old name still works with deprecation warning).
Retention is now based on the N most recent dag runs rather than N most recent task executions,
which may result in fewer records retained for conditional/sparse tasks. (#60951)
AuthManager Backfill permissions are now handled by the ``requires_access_dag`` on the ``DagAccessEntity.Run``
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``is_authorized_backfill`` of the ``BaseAuthManager`` interface has been removed. Core will no longer call this method and their
provider counterpart implementation will be marked as deprecated.
Permissions for backfill operations are now checked against the ``DagAccessEntity.Run`` permission using the existing
``requires_access_dag`` decorator. In other words, if a user has permission to run a Dag, they can perform backfill operations on it.
Please update your security policies to ensure that users who need to perform backfill operations have the appropriate ``DagAccessEntity.Run`` permissions. (Users
having the Backfill permissions without having the DagRun ones will no longer be able to perform backfill operations without any update)
Python 3.14 support added
"""""""""""""""""""""""""
Airflow 3.2.0 adds support for Python 3.14. (#63787)
Reduce API server memory by eliminating ``SerializedDAG`` loads on task start
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The API server no longer loads the full ``SerializedDAG`` when starting tasks,
significantly reducing memory usage. (#60803)
Remove MySQL client from container images
"""""""""""""""""""""""""""""""""""""""""
MySQL client support has been removed from official Airflow container images. MySQL users
building on official images must install the client themselves. (#57146)
Add support for async callables in PythonOperator
"""""""""""""""""""""""""""""""""""""""""""""""""
The ``PythonOperator`` parameter ``python_callable`` now also supports async callables in Airflow 3.2,
allowing users to run async def functions without manually managing an event loop. (#60268)
Make start_date optional for @continuous schedule
"""""""""""""""""""""""""""""""""""""""""""""""""
The ``schedule="@continuous"`` parameter now works without requiring a ``start_date``, and any Dags with this schedule will begin running immediately when unpaused. (#61405)
Upgrade SQLAlchemy (SQLA) to 2.0
"""""""""""""""""""""""""""""""""""""""""""""""""
Airflow now only support SQLAlchemy version 2
New Features
^^^^^^^^^^^^
- Add FIPS support by making Python LTO configurable via ``PYTHON_LTO`` build argument (#58337)
- Add support for task queue-based Trigger assignment to specific Triggerer hosts via the new ``--queues`` CLI option for the ``trigger`` command (#59239)
- Add ``--show-values`` and ``--hide-sensitive`` flags to CLI ``connections list`` and ``variables list`` to hide sensitive values by default (#62344)
- Add support for setting individual secrets backend kwargs via ``AIRFLOW__SECRETS__BACKEND_KWARG__<KEY>`` environment variables (#63312)
- Add ``only_new`` parameter to Dag clear to only clear newly added task instances (#59764)
- Add ``log_timestamp_format`` config option for customizing component log timestamps (#63321)
- Add ``--action-on-existing-key`` option to ``pools import`` and ``connections import`` CLI commands (#62702)
- Add back ``--use-migration-files`` flag for ``airflow db init`` (#62234)
- Add ``AllowedKeyMapper`` for partition key validation in asset partitioning (#61931)
- Add ``ChainMapper`` for chaining multiple partition mappers (#64094)
- Add cryptographic signature verification for Python source packages in Docker builds (#63345)
- Add Human-in-the-Loop (HITL) Review system for ``AgenticOperator`` (#63081)
- Add ``@task.stub`` decorator to allow tasks in other languages to be defined in Dags (#56055)
- Add support for creating connections using URI in SDK (#62211)
- Add note support to ``TriggerDagRunOperator`` (#60810)
- Add ``allowed_run_types`` to whitelist specific Dag run types (#61833)
- Add OR operator support in API search parameters (#60008)
- Add API filtering for Dags by timetable type (#58852)
- Add wildcard support for ``dag_id`` and ``dag_run_id`` in bulk task instance endpoint (#57441)
- Add ``operator_name_pattern``, ``pool_pattern``, ``queue_pattern`` as task instance search filters (#57571)
- Add ``update_mask`` support for bulk PATCH APIs (#54597)
- Add asset event emission listener event (#61718)
- Add ``source`` parameter to ``Param`` (#58615)
- Add lazy filtering for inlet events by time range, ordering, and limit (#54891)
- Add ability to get previous ``TaskInstance`` on ``RuntimeTaskInstance`` (#59712)
- Add required context messages to all DagRun state change notifications (#56272)
- Add ``max_trigger_to_select_per_loop`` config for Triggerer HA setup (#58803)
- Add ``uvicorn_logging_level`` config option to control API server access logs (#56062)
- Add correlation-id support to Execution API for request tracing (#57458)
- Add ``executor.running_dags`` gauge metric to expose count of running Dags (#52815)
- Add submodules support to ``GitDagBundle`` (#59911)
- Add HTTP URL authentication support to ``GitHook`` for Dag bundles (#58194)
- Add stream method to ``RemoteIO`` for ``ObjectStorage`` (#54813)
- Add CLI hot-reload support via ``--dev`` flag (#57741)
- Add ``auth list-envs`` command to list CLI environments and auth status (#61426)
- Add Dag bundles to ``airflow info`` command output (#59124)
- Add new arguments to ``db_clean`` to explicitly include or exclude Dags (#56663)
- UI: Add Jobs page to the Airflow UI (#61512)
- UI: Add version change indicators for Dag and bundle versions in Grid view (#53216)
- UI: Add segmented state bar for collapsed task groups and mapped tasks (#61854)
- UI: Add date range filter for Dag executions (#60772)
- UI: Add "Select Recent Configurations" to trigger form, restoring Airflow 2 functionality (#56406)
- UI: Add copy button to logs (#61185)
- UI: Add filename display to Dag Code tab for easier file identification (#60759)
- UI: Add Dag run state filter to grid view options (#55898)
- UI: Add task upstream/downstream filter to Graph and Grid views (#57237)
- UI: Add filters to Task Instances tab (#56920)
- UI: Add display of active Dag runs count in header with auto-refresh (#58332)
- UI: Add Dag ID pattern search to Dag Runs and Task Instances pages (#55691)
- UI: Add delete button for Dag runs in more options menu (#55696)
- UI: Add depth filter to ``TaskStreamFilter`` (#60549)
- UI: Add theme config support (#58411)
- UI: Add support for ``globalCss`` in custom themes (#61161)
- UI: Add display of logged-in user in settings button (#58981)
- UI: Add tooltip for explaining task filter traversal (#61401)
- UI: Add self-service JWT token generation for API and CLI access (#63195)
- UI: Add bulk operations for edge workers page (#64033)
- UI: Add real-time concurrency control for edge workers (#63142)
- UI: Add ``run_after`` date filter on Dag runs page (#62797)
- UI: Add bundle version filter on Dag runs page (#62810)
- UI: Add icon support for theme customization (#62172)
- UI: Add Monaco editor for all JSON editing fields (#62708)
- UI: Add run type legend tooltip to grid view (#62946)
- UI: Allow customizing gray, black, and white color tokens in ``AIRFLOW__API__THEME`` in addition to brand (#64232)
Bug Fixes
^^^^^^^^^
- Fix sensitive configuration values not being masked in public config APIs; treat the deprecated ``non-sensitive-only`` value as ``True`` (#59880)
- Fix ``InvalidStatsNameException`` for pool names with invalid characters by auto-normalizing them when emitting metrics (#59938)
- Fix JWT tokens appearing in task logs by excluding the token field from workload object representations (#62964)
- Fix security iframe navigation when ``AIRFLOW__API__BASE_URL`` basename is configured (#63141)
- Fix grid view URL for dynamic task groups producing 404 by not appending ``/mapped`` to group URLs (#63205)
- Fix ``ti_skip_downstream`` overwriting RUNNING tasks to SKIPPED in HA deployments (#63266)
- Fix duplicate task execution when running multiple schedulers (#60330)
- Fix callback starvation across Dag bundles (#63795)
- Fix ``@task`` decorator failing for tasks that return falsy values like ``0`` or empty string (#63788)
- Fix ``LatestOnlyOperator`` not working when direct upstream of a dynamically mapped task (#62287)
- Fix inconsistent ``XCom`` return type in mapped task groups with dynamic mapping (#59104)
- Fix task group lookup using wrong Dag version for historical runs, causing 404 errors in grid view (#63360)
- Fix import errors when updating Dags in other bundles (#63615)
- Fix ``DagRun`` span emission crash when ``context_carrier`` is ``None`` (#64087)
- Fix false error logs for partitioned timetables when ``next_dagrun`` fields are ``None`` (#63962)
- Fix timetable serialization error when decoding ``relativedelta`` (#61671)
- Fix ``task_instance_mutation_hook`` receiving ``run_id=None`` during ``TaskInstance`` creation (#63049)
- Fix scheduler crash on ``None`` ``dag_version`` access (#62225)
- Fix ``MetastoreBackend.expunge_all()`` corrupting shared session state (#63080)
- Fix triggerer logger file descriptor closed prematurely when trigger is removed (#62103)
- Fix ``airflowignore`` negation pattern handling for directory-only patterns (#62860)
- Fix false warnings for ``TYPE_CHECKING``-only forward references in TaskFlow decorators (#63053)
- Fix ``structlog`` JSON serialization crash on non-serializable objects (#62656)
- Fix backward compatibility for deadline alert serialization (#63701)
- Fix ``queued_tasks`` type mismatch in hybrid executors (``CeleryKubernetesExecutor``, ``LocalKubernetesExecutor``) (#63744)
- Fix Celery tasks not being registered at worker startup (#63110)
- Fix asset partition detection incorrectly identifying Dags as partitioned (#62864)
- Fix ``pathlib.Path`` objects incorrectly resolved by Jinja templater in Task SDK (#63306)
- Fix state mismatch in Kubernetes executor after pod completion (#63061)
- Fix ``make_partial_model`` for API Pydantic models (#63716)
- Fix WTForms validator compatibility in connection form (#63823)
- Fix ``_execution_api_server_url()`` ignoring configured value and falling back to edge config (#63192)
- Fix ``DetachedInstanceError`` for ``airflow tasks render`` command (#63916)
- Fix scheduler isolating per-dag-run failures to prevent a single DagRun crashing all scheduling (#62893)
- Fix task argument order in ``@task`` definition causing Dag parsing errors (#62174)
- Fix ``limit`` parameter not sent in ``execute_list`` server requests (#63048)
- Fix circular import from ``airflow.configuration`` causing ``ImportError`` on Python 3.14 (#63787)
- Fix ``map_index`` range validation in CLI commands (#62626)
- Fix nullable ORM fields by restoring correct defaults and dropping unreleased corrective migration (#63899)
- Fix race condition in auth manager initialization on concurrent requests (#62431)
- Fix ``FabAuthManager`` race condition on startup with multiple workers (#62737)
- Fix ``FabAuthManager`` race condition when workers concurrently create permissions, roles, and resources (#63842)
- Fix ``JWTValidator`` not handling GUESS algorithm with JWKS (#63115)
- Fix ``FabAuthManager`` first idle MySQL disconnect in token auth (#62919)
- Fix ``JWTBearerTIPathDep`` import errors in Human-In-The-Loop routes (#63277)
- Fix 403 from roles endpoint despite admin rights in FAB provider (#64097)
- Fix task log filters not working in full-screen mode (#62747)
- Fix duplicate log reads when resuming from ``log_pos`` (#63531)
- Fix 404 errors from worker log server for historical retry attempts now handled gracefully (#62475)
- Fix Elasticsearch/OpenSearch logging exception details missing in task log tab (#63739)
- Fix task-level audit logs missing success/running events (#61932)
- Fix ``null`` ``dag_run_conf`` causing serialization error in ``BackfillResponse`` (#63259)
- Fix CLI asset materialization using wrong Dag run type (#63815)
- Fix migration 0094 performance: use SQL instead of Python deserialization (#63628)
- Fix migration reliability: replace ``savepoints`` with per-Dag transactions (#63591)
- Fix slow downgrade performance by adding index to ``deadline.callback_id`` (#63612)
- Fix MySQL reserved keyword ``interval`` causing query failures in ``deadline_alert`` (#63494)
- Fix MySQL ``serialize_dag`` query failure during deadline migration (#63804)
- Fix SQLite downgrade failures caused by FK constraints during batch table recreation (#63437)
- Fix migration 0096 downgrade failing when team table has existing rows (#63449)
- Fix missing warning about hardcoded 24h ``visibility_timeout`` that kills long-running Celery tasks (#62869)
- Fix scheduler memory issue by removing eager loading of all task instances (#60956)
- Fix MySQL sort buffer overflow in deadline alert migration (#61806)
- Fix failing to manually trigger a Dag with ``CronPartitionedTimetable`` (#62441)
- Fix race condition in ``AssetModel`` when updating asset partition DagRun — adds mutex lock (#59183)
- Fix FAB ``auth_manager`` ``load_user`` causing ``PendingRollbackError`` (#61943)
- Fix N+1 query: add ``joinedload`` for asset in ``dags_needing_dagruns()`` (#60957)
- Fix Dag Processor health check threshold matching SchedulerJob/TriggererJob pattern (#58704)
- Fix ``NotMapped`` exception when clearing task instances with downstream/upstream (#58922)
- Fix missing asset events for partitioned DagRun (#61433)
- Fix missing ``partition_key`` filter in ``PALK`` when creating DagRun (#61831)
- Fix Dag params API contract broken by earlier change (#56831)
- Fix OAuth session race condition causing false 401 errors during login (#61287)
- Fix ``ObjectStoragePath`` to exclude ``conn_id`` from storage options passed to fsspec (#62701)
- Fix unable to import list value for Variable (#61508)
- Fix plugin registration returning early on duplicate names (#60498)
- Fix circular import when using ``XComObjectStorageBackend`` (#55805)
- Fix deadline alert hashing bug (#61702)
- Fix task SDK to read ``default_email_on_failure``/``default_email_on_retry`` from config (#59912)
- Fix Celery worker crash on macOS due to non-serializable local function (#62655)
- Fix Redis import race condition in Celery executor (#61362)
- Fix incorrect state query parameter for task instances in Dashboard (#59086)
- Fix ``TaskInstance.get_dagrun`` returning None in ``task_instance_mutation_hook`` (#60726)
- Fix Simple Auth Manager login showing cryptic error on failed authentication (#64303)
- Fix ``dag_display_name`` property bypass for ``DagStats`` query (#64256)
- Fix ``TaskAlreadyRunningError`` not raised when starting an already-running task instance (#60855)
- Fix Teardown tasks not waiting for all in-scope tasks to complete (#64181)
- Fix ``enable_swagger_ui`` config not respected in API server (#64376)
- Fix: add check for xcom permission when result is specified for DagRun wait endpoint (#64415)
- Fix ``conf.has_option`` not respects default provider metadata (#64209)
- Fix teardown scope causing unnecessary database writes during task scheduling (#64558)
- Fix live task log output not visible in stdout when using Elasticsearch log forwarding (#64067)
- Fix ``TaskInstance`` crash when refreshing task weight for non-serialized operators (#64557)
- Fix Variables secrets backend conflict check exiting early when multiple backends are configured (#64062)
- UI: Fix Dag run accessor key on clear task instance page (#64072)
- UI: Fix searchable dropdown not working for Dag params enum fields (#63895)
- UI: Fix newline rendering in Dag warning alert (#63588)
- UI: Fix ``XCom`` edit modal value not repopulating on reopen (#62798)
- UI: Fix task duration tooltip not displaying correctly (#63639)
- UI: Fix elapsed time not showing for running tasks (#63619)
- UI: Fix ``RenderedJsonField`` collapse behavior (#63831)
- UI: Fix ``RenderedJsonField`` not displaying in table cells (#63245)
- UI: Fix full-screen log dropdown z-index after Chakra upgrade (#63816)
- UI: Fix asset materialization run type display (#63819)
- UI: Fix pools with unlimited (``-1``) slots not rendering correctly (#62831)
- UI: Fix ``DurationChart`` labels and disable animation flicker during auto-refresh (#62835)
- UI: Fix 403 error not shown when unauthorized user re-parses Dag (#61560)
- UI: Fix logical date filter on ``/dagruns`` page not working (#62848)
- UI: Fix inflated ``total_received`` count in partitioned Dag runs view (#62786)
- UI: Fix edge executor navigation when behind reverse proxy with subpath (#63777)
- UI: Fix queries not invalidated on Dag run add/delete (#64269)
- UI: Fix ``RenderedJsonField`` flickering when collapsed (#64261)
- UI: Fix Docs menu REST API link visibility when API docs are disabled (#64359)
- UI: Fix ``TISummaries`` not refreshing when ``gridRuns`` are invalidated (#64113)
- UI: Fix guard against null/undefined dates in Gantt chart to prevent RangeError (#64031)
- UI: Block polling requests to endpoints that returned 403 Forbidden (#64333)
- UI: Fix Gantt view still visible when time range is outside ``DagRun`` window (#64179)
- UI: Fix Human-in-the-Loop (HITL) operator options not displaying when exactly 4 choices are configured (#64453)
Miscellaneous
^^^^^^^^^^^^^
- Set minimum version of ``sqlalchemy[asyncio]>=2.0.48``
- Deprecate ``api.page_size`` config in favor of ``api.fallback_page_limit`` (#61067)
- Improve Dag callback relevancy by passing a context-relevant task instance based on the Dag's final state instead of an arbitrary lexicographical selection (#61274)
- Optimize ``get_dag_runs`` API endpoint performance (#63940)
- Improve historical metrics endpoint performance (#63526)
- Add TTL cache with single-flight deduplication to Keycloak ``filter_authorized_dag_ids`` (#63184)
- Reduce Celery worker memory usage with ``gc.freeze`` (#62212)
- Eliminate duplicate JOINs in ``get_task_instances`` endpoint (#62910)
- Replace large ``IN`` clause in asset queries with CTE and JOIN for better SQL performance (#62114)
- Add row lock to prevent race conditions during asset-triggered DagRun creation (#60773)
- Add ``ConnectionResponse`` serializer safeguard to prevent accidental sensitive field exposure (#63883)
- Add missing ``dag_id`` filter on DagRun task instances API query (#62750)
- Add missing HTTP timeout to FAB JWKS fetching (#63058)
- Add additional permission check in asset materialization endpoint (#63338)
- Filter backfills list by readable Dags (authorization enforcement) (#63003)
- Hide SQL statements in exception details when ``expose_stacktrace`` is disabled (#63028)
- Use default max depth to redact Variable values in API responses (#63480)
- Validate ``update_mask`` fields in PATCH API endpoints against Pydantic models (#62657)
- Align key/id path validation for variables and connections in Execution API (#63897)
- Add ``order_by`` parameter to GET /permissions endpoint for pagination consistency (#63418)
- Implement truncation logic for rendered template values (#61878)
- Add ``BaseXcom`` to ``airflow.sdk`` public exports (#63116)
- Make ``TaskSDK`` conf respect default config from provider metadata (#62696)
- Add OTel trace import shim via ``airflow.sdk.observability.trace`` (#63554)
- Improve 3.2.0 deadline migration performance (#63920)
- Improve 3.2.0 downgrade migration for ``external_executor_id`` on PostgreSQL (#63625)
- Skip backfilling old ``DagRun.created_at`` during migration for faster upgrades (#63825)
- Add INFO-level logging to asset scheduling path (#63958)
- Improve log file template for ``ExecuteCallback`` by including ``dag_id`` and ``run_id`` (#62616)
- Improve Dag processor timeout logging clarity (#62328)
- Deprecate ``get_connection_form_widgets`` and ``get_ui_field_behaviour`` hook methods (#63711)
- Add missing deprecation warnings for ``[workers]`` config section (#63659)
- Expose ``TaskInstance`` API for external task management (#61568)
- Remove deprecated ``airflow.datasets``, ``airflow.timetables.datasets``, and ``airflow.utils.dag_parsing_context`` modules (#62927)
- Remove ``PyOpenSSL`` from core dependencies (#63869)
- Optimize fail-fast check to avoid loading ``SerializedDAG`` (#56694)
- Improve performance of task queue processing by switching from ``pop(0)`` to ``popleft()`` (#61376)
- Optimize K8s API usage for watching pod events, fixing hanging communication (#59080)
- Remove N+1 database queries for team names (#61471)
- Improve XCom value handling in extra links API (#61641)
- Remove ``.git`` folder from versions in ``GitDagBundle`` to reduce storage size (#57069)
- Deprecate subprocess exec utils from ``airflow.utils.process_utils`` (#57193)
- Improve error handling in edge worker on 405 responses (#60425)
- Improve deferrable ``KubernetesPodOperator`` handling of deleted pods between polls (#56976)
- Improve event log entries when a pod fails for K8s executor (#60800)
- Refactor XCom API to use shared serialization constants (#64148)
- Improve temporal mapper to be timezone aware for asset partitioning (#62709)
- Improve dag version inflation checker logic and fix false-positive detection (#61345)
- Rename ``ToXXXMapper`` to ``StartOfXXXMapper`` in partition-mapper for clarity (#64160)
- Run DB check only for core components in prod entrypoint (#63413)
- Fix partitioned asset events incorrectly triggering non-partition-aware Dags (#63848)
- Improve partitioned DagRun sorting by ``partition_date`` (#62866)
- Allow gray, black, and white color tokens in ``AIRFLOW__API__THEME`` config (#64232)
- Add parent task spans and nest worker/trigger spans for improved observability (#63839)
- UI: Enhance code view to support search and diff (#55467)
- UI: Improve UX for adding custom ``DeadlineReferences`` (#57222)
- UI: Enhance ``FilterBar`` with ``DateRangeFilter`` for compact UI (#56173)
- UI: Move deadline alerts into their own table for UI integration (#58248)
- UI: Persist tag filter selection in Dag grid view (#63273)
- UI: Show HITL review tab only for review-enabled task instances (#63477)
- UI: Updated button styles for adding Connections, Variables, and Pools (#62607)
- UI: Add clear permission toast for 403 errors on user actions (#61588)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Add documentation marking pre/post-execute task hooks as GA (no longer experimental) (#59656)
- Add ``RedisTaskHandler`` configuration example (#63898)
- Add documentation explaining difference between deferred vs async operators (#63500)
- Add auth manager section in multi-team documentation (#63208)
- Add documentation about shared libraries in ``_shared`` folders (#63468)
- Clarify plugin folder module registration in ``modules_management`` docs (#63634)
- Clarify ``max_active_tasks`` Dag parameter documentation (#63217)
- Clarify HLL in extraction precedence docs (#63723)
- Clarify Ubuntu/Debian venv requirement in quick start guide (#63244)
- Fix Git connection docs to match actual ``GitHook`` parameters (#63265)
- Mention Python 3.14 support in docs (#63950)
- Add Dag documentation for ``example_bash_decorator`` (#62948)
- Add Russian translation for UI (#63450)
- Add Hungarian translation (#62925)
- Complete Traditional Chinese translations (#62652)
- Add asset partition documentation (#63262)
- Add guide for dag version inflation and its checker (#64100)
Airflow 3.1.8 (2026-03-11)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Backfill permissions are now handled via ``DagAccessEntity.Run`` (#61456)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``is_authorized_backfill`` of the ``BaseAuthManager`` interface has been removed. Core will no longer call this method and their
provider counterpart implementation will be marked as deprecated.
Permissions for backfill operations are now checked against the ``DagAccessEntity.Run`` permission using the existing
``requires_access_dag`` decorator. In other words, if a user has permission to run a DAG, they can perform backfill operations on it.
Please update your security policies to ensure that users who need to perform backfill operations have the appropriate ``DagAccessEntity.Run`` permissions. (Users
having the Backfill permissions without having the DagRun ones will no longer be able to perform backfill operations without any update)
Elasticsearch is now fully compatible with remote logging along (#62940)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Elasticsearch is now fully compatible with remote logging along side with ``apache-airflow-providers-elasticsearch>=6.5.0``. Please review elasticsearch provider release notes for more information https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.5.0/changelog.html (#62121) (#62940)
Bug Fixes
^^^^^^^^^
- Fix SQLite migration disable ``disable_sqlite_fkeys`` in revision ``509b94a1042d`` (#63256) (#63272)
- Fix: 404 queued asset events from API server logs (#62934) (#62976)
- Fix: Always include kid in JWT header for symmetric key tokens (#62883) (#62943)
- Fix: Scope session token in cookie to base_url (#62771) (#62851)
- Fix: UI of Scope session token in cookie to base_url (#62771) (#62859)
- Fix: UI tasks log missing in UP_FOR_RETRY and UP_FOR_RESCHEDULE states (#54547) (#62862)
- Fix: Backfill permissions (#62856) (#62873)
- Fix: Use ``useAssetServiceGetDagAssetQueuedEvents`` to get the correct number of ADRQs (#62868) (#62902)
- Fix: Adds task instance validation for HITL (#62886) (#62909)
- Fix: Restore task_instance_history sequence on downgrade (#62759)
- Fix broken ``dag_processing.total_parse_time`` metric (#62128) (#62764)
- Fix Trigger UI form rendering for null enum values (#62060) (#62767)
- Fix ``timer.duration`` unit labels in logs (#61824) (#62757)
- Fix XCom migration failing for NaN/Infinity float values (#62686) (#62760)
- Fix SQL not rendered in Rendered Templates view (#60739) (#62348)
- Fix missing DAG read permission checks on dependencies endpoint (#62046) (#62586)
- Changed ``dag_bundle.signed_url_template`` from ``varchar(200)`` to ``text`` (#61041) (#62568)
- Fix WASB remote logging base path handling (#58946) (#61013) (#62456)
- Handle non-dictionary json payload during logging to avoid internal server error. (#62355) (#62367)
- Fix grid view crash when task converted to TaskGroup (#61208) (#61279) (#62181)
- Fix running task duration showing as null in UI (#61898) (#62136)
- Fix deferrable sensors not respecting soft_fail on timeout (#61132) (#61421)
- Fix task failure details being obscured by finalization errors (#62070) (#62113)
- Add missing ti.start and ti.finish metrics in Airflow 3 (#62019) (#62110)
- Fix DepContext mutation leak and restore reschedule-mode guard (#62089)
- Fix scheduler heartbeat misses caused by slow reschedule dependency check (#61983) (#62068)
- Flush in-memory OTEL metrics at process shutdown (#61808) (#61869)
- Fix executor slots showing negative infinity (#61140) (#61768)
- Fix recursion depth error in _redact_exception_with_context (#61776) (#61795)
- Fix API server segfault when ``PYTHONASYNCIODEBUG=1`` is set (#61281) (#61933)
- Fix scheduler crash when queuing TI with null dag_version_id (#61813) (#61846)
- Fix secrets masking in Rendered Templates for complex objects (#61394) (#61763)
- Fix list dag versions permissions (#61675) (#61733)
- Fix Triggerer crashing if Trigger uses builtin print function (#60258) (#61703)
- Fix GZipMiddleware with correct comment placement (#61538) (#61566)
- Fix middleware order to prevent chunked FastAPI responses (#61043) (#61539)
- Fix XCom serialization for ``pendulum.date.Date`` values (#61176) (#61717)
- Fix ``access_key`` and ``connection_string`` not being masked in logs (#61580) (#61582)
- Fix ``minimatch`` ReDoS vulnerabilities via ``pnpm`` overrides (#62805)
- Fix language selector state not updating on change (#61060) (#61263)
- Make conn_type optional in task SDK Connection data model (#61728) (#61835)
- UI: optimize grid view refresh pressure on the API (#62085) (#62135)
- UI: Fix main content margin to align with navigation sidebar width (#61614) (#61622)
- UI: Fix Preserve variable value formatting in edit dialog (#58757) (#62339)
- UI: Fix missing translation keys for blocking dependencies in UI (#61314) (#61366) (#61638)
- UI: Add error handling for pause/unpause toggle permission errors (#61389) (#61533)
- UI: Flatten grid structure endpoint memory consumption (#61273) (#61393)
- UI: Reduce memory usage in grid view by optimizing node data storage (#61656) (#61789)
- UI: Fix variable table word-break when values are expanded (#62416) (#62781)
- UI: Fix use ISO dates in Gantt chart for cross-browser consistency (#61250) (#62784)
- UI: Fix DataTable overflow on narrow screens (#62603)
- UI: Fix unique keys for pagination ellipses (#62352) (#62366)
- UI: Fix ``elk.portConstraints`` for LR orientation in graph view (#62144) (#62187)
- UI: Fix show active backfill in banner instead of first one (#61851) (#62137)
- UI: Fix star icon visibility in Favorite filter buttons when selected (#61862)
- UI: Fix grid view tooltip z-index issue (#61275) (#61403)
- UI: Fix mini-map on DAG graph view not showing DAG nodes (#61511) (#61530)
- UI: Fix pale appearance of filter buttons when selected (#60346 backport fix) (#61457)
Miscellaneous
^^^^^^^^^^^^^
- Add logging to detect try number race (#62703) (#62821)
- Override tar dependency in Simple auth manager (#62787)
- Remove mp_start_method remnants (#61150) (#62762)
- Expose literal and ParamsDict at SDK top level (#59782) (#62756)
- Add on_task_instance_skipped listener hookspec (#59467) (#61863)
- Persist table columns visibility in local storage (#61858) (#61868)
- Add ``run_after`` alias to ``XComResponse`` for backward compatibility (#61443) (#61672)
- UI: Add task_display_name to LightGridTaskInstanceSummary model (#61440) (#61505)
- UI: Add multi-line text display option on Variables page (#61679) (#62779)
- UI: Add bulk actions for connections and variables (#61570) (#62076)
- UI: Allow selecting file path using cursor in log viewer (#61011) (#61506)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Fix Liveness / Readiness / Startup probe path for Airflow 3.x (#58734) (#61411)
- Update health check command syntax for celery worker (#58861) (#61412)
- Translation fixes: Polish (#62031) (#62761), Catalan (#62477), Taiwanese Mandarin (#62397),
German (#61478), Polish (#61423)
- Remove docs mentioning old, unsupported hybrid executors (#62093) (#62096)
- Clarify security model of Airflow (#61754) (#61770)
- Clarify ExternalTaskSensor path in dags.rst (#61555) (#61617)
- Clarify policy for exposing sensitive data (#59864) (#61392)
- Clarify template context for asset-triggered DAGs in airflow-core docs (#61258) (#61282)
- Add Keycloak token documentation to Security/API (#61228) (#61248)
Airflow 3.1.7 (2026-02-04)
--------------------------
No significant changes.
Bug Fixes
^^^^^^^^^
- Fix JWT token generation with unset issuer/audience config (#61331)
- Fix callback files losing priority during queue resort (#61232) (#61243)
- Fix Dag callback for versioned bundles in the processor (#60734) (#61230)
- Add 404 handling for non-existent Dag (#61131) (#61225)
- Add guardrail to handle Dag deserialization errors in scheduler (#61162) (#61210)
- Fix asset scheduling for stale Dags (#59337) (#60022) (#61106)
- Fix unnecessary Dag version churn when Dag file paths change (#60799)
- Fix missing warning when Bundle path may not be accessible to impersonated user (#60278)
- Fix ``TriggerDagRunOperator`` deferring when ``wait_for_completion=False`` (#60052)
- Fix NoneType error when updating serialized Dag (#56422)
- Fix Pool API slots validation (#61071) (#61114)
- Fix DagBag parsing by adding bundle_path temporarily to sys.path (#55894) (#61053)
- Fix API to respect maximum page limit (#60989) (#61073)
- Prevent Triggerer from crashing when a trigger event isn't serializable (#60152) (#60981)
- Fix permissions on get_event_logs endpoint (#60936) (#60958)
- Fix Dag access control for dag_id in query param (#60935) (#60959)
- Fix root logger to use log_level instead of hardcoded INFO level (#60784) (#60970)
- Fix Dag processor OOM by Avoid loading all TaskInstances when checking DagVersion in write_dag (#60937) (#60962)
- Fix worker startup Dag load failures by rescheduling tasks instead of exiting (#59604) (#60926)
- Fix permissions check in import error APIs (#60801) (#60884)
- Fix refresh-token invalidation by logging out the user (#60781) (#60881)
- Fix connection test API to restore masked password/extra from existing connections (#59643) (#60873)
- Fix Dag processor overhead by applying ``gc.freeze`` (#60505) (#60845)
- Fix Dag processor crashing due to MySQL deadlock errors (#60166) (#60418)
- Ensure unique run_id across manually triggered Dags with schedules (#59477) (#60468)
- UI: Avoid gantt annotation error during resize (#60877)
- UI: Fix react apps plugins router (#61206)
- UI: Reset pagination on search in all pages (#61169)
- UI: Explicitly set UI table ordering (#60609) (#61216)
- UI: Plugins pagination fix #61055 (#61059) (#61129)
- UI: Fix ui get dags permission endpoint for user without Dag run permissions (#60979) (#60988)
- UI: Convert Tasks Table from card to table mode (#60830) (#60874)
- UI: Fix slow log scrolling for large task logs (#60806) (#60875)
- UI: Grey out trigger button on API 403 (#60648) (#60777)
- UI: Remove API error from disabling submit (#60473) (#60658)
- UI: Added toasters for permission denied (#57966) (#58016) (#60646)
- UI: Move row count and display toggle into DataTable (#57680) (#60639)
- UI: Fix/backfill permission error handling (#60582) (#60587)
- UI: Improve Dags Filter UI (#60346) (#60547)
- UI: Update PoolBar to separate Scheduled and Deferred slots (#59270) (#60538)
- UI: Correct the access for the ``externalLogUrl`` (#60412) (#60479)
- UI: Fix gantt chart styling (#60347) (#60457)
- UI: Reset pagination on DagList search (#60326) (#60336)
- UI: Move dags list filters to ``buttongroups`` (#60298) (#60337)
- UI: Fix table filters resetting when deleting a Dag (#60279) (#60287)
- UI: Fix sidebar visibility issue when main content exceeds ``viewport`` height (#59660) (#60286)
- UI: Add virtualization to grid view (#60241) (#60285)
Miscellaneous
^^^^^^^^^^^^^
- UI: Upgrade react-dom-router in Airflow UI (#60316) (#60456)
- Use bulk DELETE for XComModel.clear() instead of loading records (#60955)
- Refactor Dag file queuing and fix redundant processing (#60124)
- Optimized Dag processing queue order on bundle refresh (#60003)
- Remove unused method ``is_default_pool`` in Pool model (#61084) (#61128)
- Translation fixes: ``Taiwaness Mandarin`` (#61126), Catalan (#61093), German (#61097), Polish (#61099),
Arabic (#60635 #60782, (#60635) (#60782)), Spanish (#60775 #60785, (#60775) (#60785)),
Hebrew (#60633 #60686, (#60633) (#60686))
Doc Only Changes
^^^^^^^^^^^^^^^^
- Fix minor display issue with migration to airflow 3 docs (#60749)
- Fix airflow.utils.context.Context import path in Airflow 3 migration doc (#59937)
- Add missing links to airflow.sdk classes and functions in public interface docs (#61005) (#61012)
- Clarify BaseSensorOperator parameters in Sensors guide (#60275)
- Fix docstring for RuntimeTaskInstance.xcom_pull (#60220) (#60252)
- Fix broken syntax highlighting in AIR rules note section (#59188)
Airflow 3.1.6 (2026-01-13)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
``is_authorized_hitl_task()`` method now available in auth managers(#59399).
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This method is now available in auth managers to check whether a user is authorized to approve a HITL task
``proxy`` and ``proxies`` added to ``DEFAULT_SENSITIVE_FIELDS`` (#59688)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
``proxy`` and ``proxies`` have been added to ``DEFAULT_SENSITIVE_FIELDS`` in secrets_masker to treat proxy configurations as sensitive by default
Bug Fixes
^^^^^^^^^
- Protect against hanging thread in aiosqlite 0.22+ (#60217) (#60245)
- Fix log task instance sqlalchemy join query (#59973) (#60222)
- Fix invalid uri created when extras contains non string elements (#59339) (#60219)
- Fix operator template fields via callable serialization that causes unstable DAG serialization (#60065) (#60221)
- Fix real-time extra links updates for TriggerDagRunOperator (#59507) (#60225)
- Fix signal handling in triggerer job runner (#60190) (#60214)
- Added state validation to delete dag run endpoint (#60195) (#60207)
- Fix text overflow issue (#60080)
- UI: Add toggle functionality to Dags state filters (#59089)
- Fix ``deprecated_options`` entry for ``dag_file_processor_timeout`` (#59181) (#60162)
- Fix ``ApprovalOperator`` with ``SimpleAuthManager`` when ``all_admins=True`` (#59399) (#60116)
- Record missing ``ti_failure`` metrics for tasks (#59731) (#59964)
- Fix missing ``TaskInstanceHistory`` on scheduler ``TI`` resets (#59639) (#59752)
- Add ``proxy`` and ``proxies`` as sensitive fields in ``DEFAULT_SENSITIVE_FIELDS`` (#59688) (#59792)
- Fix compat deprecation handling for ``[webserver] base_url`` (#59659) (#59781)
- Fix Execution API refresh token (#58782) (#59713)
- Fix eager-loading DagRun asset relationships before creating ``DagRunContext`` (#59714) (#59732)
- Redact secrets in rendered templates properly when truncating it (#59566) (#59704)
- Add ``Content-Type`` to request headers in Task SDK calls when missing (#59676) (#59687)
- UI: Fix Expand+Collapse Translation Key (#59672) (#59674)
- Fix server context for connections (#59624) (#59652)
- Fix clear task instance dialog tasks states (#59363) (#59580)
- Add log record when listening dag is partitioned but run has no key (#59375) (#59582)
- Fix Dag Processor logging crash (#59317) (#59581)
- Flush session before processing Event Buffer in dag test (#59314) (#59559)
- Add task group ID filtering support to task instance query (#58092) (#59511)
- Fix message of ``_read_from_logs_server`` when status_code is 403 (#59489) (#59504)
- Fix import errors not cleared for files without Dags (#58242) (#59500)
- Fix backfill ``run_on_latest_version`` defaulting to False instead of True (#59304) (#59328)
- Add toaster notifications for Connection Test (#59354) (#59368)
- Fix ``.airflowignore`` negation not working in subfolders (#58740) (#59305)
- Fix XCom key handling when keys contain special characters like slash (#58344) (#59311)
- Fix an odd import of pendulum from sqlalchemy_utils instead of elsewhere. (#59258) (#59265)
- Fix links for DurationChart (#59095) (#59237)
- Fix misleading error message when GitHook creation fails (#59236)
- Show asset extra in asset list (#59195) (#59201)
- Prevent dag processor crash on encountering excel files in the Dag directory (#59069) (#59170)
- Fix ``DagRun.queued_at`` not updating when clearing (#59066) (#59177)
- Fix Rendered Templates not showing dictionary items (#58071) (#59176)
- UI: Change task log source display to hidden by default (#58749) (#59045)
- Fix button to go back from FAB iframe (#58997) (#59007)
- Fix task instance and run tooltips in Grid view (#58359) (#59013)
Miscellaneous
^^^^^^^^^^^^^
- Don't depend upon FastAPI inside Task-SDK client (#59250) (#59257)
- Align the term Dag in all translations (#59155)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Bump Sphinx Airflow theme to ``0.3.0`` (#59538)
- Translations updates [French: (#60157) (#60167), German: (#59673), PL: (#59675) (#59251) (#59256), Japanese: (#59557),(#59313),
Taiwanese Mandarin (#59513) (#59515), Hebrew: (#59133) (#59255), Ca: (#59216) (#60199), TR: (#59169) (#60191)]
- Update webserver probe health check doc (#59942) (#59982)
- Update API auth. instructions in Docker running docs (#59830) (#59832)
- Improve CLI date argument help text documentation (#59797) (#59810)
- Add fast client-side search to Airflow documentation (#59658)
- Fix broken ``permalink`` icon (#58763)
- Add Refresh Token logic to auth manager docs (#54196) (#59482)
- Update json to JSON for consistency in translations (#59323) (#59333)
- Fix outdated dependency documentation (#58970) (#59219)
- Add UI/API performance tips (#59004) (#59052)
- Provide a clear naming and description for the attribute caching ``get_template_context`` (#59023) (#59036)
- Update the documentation for the LocalExecutor (#58990) (#59022)
Airflow 3.1.5 (2025-12-12)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
- Handle invalid token in JWTRefreshMiddleware (#56904)
- Fix inconsistent Dag hashes when template fields contain unordered dicts (#59091) (#59175)
- Fix assets used only as inlets being incorrectly orphaned (#58986)
- Fix exception when logging stdout with a custom %-format string (#58963)
- Fix backfill max_active_runs race condition with concurrent schedulers (#58935)
- Fix LocalExecutor memory spike by applying ``gc.freeze`` (#58934)
- Fix string to datetime pydantic conversion (#58916)
- Fix deadlines being incorrectly pruned for DAG runs with the same run_id (#58910)
- Fix handling of ``pre-AIP-39`` DAG runs (#58773)
- Mask secrets properly when using deprecated import path (#58726)
- Preserve Asset.extra when using AssetAlias (#58712)
- Fix timeout_after in run_trigger method of TriggerRunner (#58703)
- Fix connection retrieval from secrets backend without conn_type (#58664)
- Fix task retry logic to respect retries for all exit codes (#58478)
- Respect default_args in DAG when set to a "falsy" value (#58396)
- Fix airflow config list output for multi-line values (#58378)
- Fix TriggerDagRunOperator stuck in deferred state with reset_dag_run=True (#58333)
- Fix HITLTrigger params serialization (#58297)
- Fix atomicity issue in SerializedDagModel.write_dag preventing orphaned DAG versions (#58281)
- Mask kwargs when illegal arguments are passed (#58283)
- Fix supervisor communications not reconnecting when using ``dag.test()`` (#58266)
- Fix supervisor communications and logs not reconnecting in task subprocesses (#58263)
- Make pool description optional when patching pools (#58169)
- Fix check_files.py script after source tarball was renamed (#58192)
- Fix db cleanup logging behavior and docstrings (#58523)
- Fix Asset URI normalization for user info without password (#58485)
- UI: Fix object rendering in Human-in-the-Loop (HITL) interface (#58611)
- UI: Fix "Consuming Tasks" section not in asset header (#58060)
- UI: Fix timezone string parsing to use ``dayjs`` correctly (#57880)
- UI: Ensure task instance ``endDate`` is not null (#58435)
- UI: Fix trigger parameter field showing as dict when param.value is null (#58899)
- UI: Remove unnecessary refresh state consumption for DAG header (#58692)
- UI: Fix mobile responsiveness of Dashboard sections (#58853)
- UI: Fix incorrect backfill duration calculation in Grid view (#58816)
- UI: Redact secrets in rendered templates to not expose them in UI (#58772)
- UI: Add fallback value of 1 for number of DAG runs in Grid view (#58735)
- UI: Update refresh token flow (#58649)
- UI: Fix 404 handling with fallback route for invalid URLs (#58629)
- UI: Fix excessive database queries in UI grid endpoint by adding query count guard (#57977, #58632)
- UI: Fix DAG documentation markdown display issue (#58627)
- UI: Fix duration chart duration format (#58564)
- UI: Fix TaskGroup nodes not being properly highlighted when selected in Graph view (#58559)
- UI: Fix tag filter with special characters (#58558)
- UI: Fix group task instance tab memory leak (#58557)
- UI: Fix popup automatically closing when DAG run completes (#58538)
- UI: Fix operator extra links not appearing on failed tasks (#58508)
- UI: Fix TypeError in ``parseStreamingLogContent`` for non-string data (#58399)
- UI: Fix Dag tag order (#58904)
Miscellaneous
^^^^^^^^^^^^^
- Do not remove ``.pyc`` and ``.pyo`` files after building Python (#58947)
- Improve cross-distribution dependency management (#58472)
- Bump glob from 10.4.5 to 10.5.0 in simple auth manager UI (#58463)
- Bump glob in React core UI (#58461)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Fix Chinese (Traditional) translations for trigger-related terminology (#58989)
- Close translation gaps in German (#58971)
- Add missing Polish translations (#58939)
- Clarify that Connection extra JSON masking is keyword-dependent (#58587)
- Add migration guide for Airflow 2 users accessing database in tasks (#57479)
- Update UIAlert import path and usage for v3 (#58891)
- Add clarifying documentation for TaskGroup parameters (#58880)
- Enhance asset extra field documentation (#58830)
- Update mask_secret documentation to use the latest import path (#58534)
- Improve disable_bundle_versioning configuration documentation (#58405)
- Fix documentation for installing from sources (#58373)
- Fix broken link on installing-from-sources page (#58324)
- Add missing DAG run table translations (#58572)
Airflow 3.1.3 (2025-11-13)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Fix Connection & Variable access in API server contexts (plugins, log handlers)(#56583)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously, hooks used in API server contexts (plugins, middlewares, log handlers) would fail with an ``ImportError``
for ``SUPERVISOR_COMMS``, because ``SUPERVISOR_COMMS`` only exists in task runner child processes.
This has been fixed by implementing automatic context detection with three separate secrets backend chains:
**Context Detection:**
1. **Client contexts** (task runner in worker): Detected via ``SUPERVISOR_COMMS`` presence
2. **Server contexts** (API server, scheduler): Explicitly marked with ``_AIRFLOW_PROCESS_CONTEXT=server`` environment variable
3. **Fallback contexts** (supervisor, unknown contexts): Neither marker present, uses minimal safe chain
**Backend Chains:**
- **Client**: ``EnvironmentVariablesBackend`` → ``ExecutionAPISecretsBackend`` (routes to Execution API via SUPERVISOR_COMMS)
- **Server**: ``EnvironmentVariablesBackend`` → ``MetastoreBackend`` (direct database access)
- **Fallback**: ``EnvironmentVariablesBackend`` only (+ external backends from config like AWS Secrets Manager, Vault)
The fallback chain is crucial for supervisor processes (worker-side, before task runner starts) which need to access
external secrets for remote logging setup but should not use ``MetastoreBackend`` (to maintain worker isolation).
**Architecture Benefits:**
- Workers (supervisor + task runner) never use ``MetastoreBackend``, maintaining strict isolation
- External secrets backends (AWS Secrets Manager, Vault, etc.) work in all three contexts
- Supervisor falls back to Execution API client for connections not found in external backends
- API server and scheduler have direct database access for optimal performance
**Impact:**
- Hooks like ``GCSHook``, ``S3Hook`` now work correctly in log handlers and plugins
- No code changes required for existing plugins or hooks
- Workers remain isolated from direct database access (network-level DB blocking fully supported)
- External secrets work everywhere (workers, supervisor, API server)
- Robust handling of unknown contexts with safe minimal chain
See: `#56120 <https://github.com/apache/airflow/issues/56120>`__, `#56583 <https://github.com/apache/airflow/issues/56583>`__, `#51816 <https://github.com/apache/airflow/issues/51816>`__
Remove insecure dag reports API endpoint that executed user code in API server (#56609)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The ``/api/v2/dagReports`` endpoint has been removed because it loaded user DAG files directly in the API server process,
violating Airflow's security architecture. This endpoint was not used in the UI and had no known consumers.
Use the ``airflow dags report`` CLI command instead for DAG loading reports.
Bug Fixes
^^^^^^^^^
- Fix HITL tasks not properly validating params (#57547) (#58144)
- Fix secrets being exposed in Jinja template rendering error messages (#57467) (#57962)
- UI: Fix slow loading on next run assets page (#58052) (#58064)
- Fix logout not working in airflow-core (#57990) (#58043)
- Fix slow loading on UI [(#57820) (#57856), (#57956) (#57973), (#57957) (#57972),(#57869) (#57882), (#57868) (#57918),(#57624) (#57757)]
- UI: Fix log download to include .txt file extension (#57991) (#58040)
- Fix scheduler using incorrect max_active_runs value from cached DAG (#57619) (#57959)
- Fix database migration failures when XCom contains NaN values (#57866) (#57893)
- Fix incorrect task context in trigger rule scenarios (#57884) (#57892)
- UI: Fix test connection not working (#57811) (#57852)
- Fix worker ``healthcheck`` timeout not respecting worker-timeout CLI option (#57731) (#57854)
- Fix provider hooks not loading when FAB provider is not installed (#57717) (#57830)
- Fix slow API responses for task instances list [(#57645) (#57794), (#57646) (#57664),(#57500) (#57735), (#57549) (#57738), (#57450) (#57736),(#57647) (#57732)]
- Fix task instance errors when tasks are triggered by trigger rules (#57474) (#57786)
- Fix type consistency for extra field in Asset, AssetAlias, and AssetEvent (#57352) (#57728)
- Fix upgrade failures when XCom contains NaN in string values (#57614)
Miscellaneous
^^^^^^^^^^^^^
- UI: Add resize functionality to DAG run and task instance notes (#57897) (#58068)
- Add Taiwan translation for UI (#58121)
- UI: Shorten German translation of Asset in navigation (#57671) (#57690)
- Fix code formatting via ruff preview (#57641) (#57670)
- Remove remnants from unlimited parallelism in local executor (#57579) (#57644)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Add learning from Airflow 3 migration guide (#57989) (#58083)
- Fix duplicate mention of 'DAGs' and 'tasks' in overview documentation (#57524) (#57793)
- Document asset event extra storage behavior (#57727) (#57734)
Airflow 3.1.2 (2025-11-05)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
- Fix import error when upgrading structlog to 25.5.0+ (#57335)
- Fix connection retrieval in ``DagProcessorManager`` for bundle initialization (#57459)
- Fix incorrect task instance counts displayed in task group headers (#55670)
- Fix task retry execution after tasks are killed by external signals (#55767)
- Fix triggerer errors after Airflow 2 to 3 migration (#55884)
- Fix tasks unable to access ``triggering_user_name`` context variable (#56193)
- Fix outlet event extra data being empty in task instance success listener callbacks (#57031)
- UI: Fix panel button spacing and alignment issues (#57062)
- UI: Fix broken grid view links for tasks with retries (#57097)
- Fix DAG processor crash when renaming DAG tag case on MySQL (#57113)
- Fix iteration errors when using ``ObjectStoragePath`` (#57156)
- Fix auto-refresh not working on Required Actions page (#57207)
- Fix DAG processor crash by ignoring callbacks from other bundles (#57330)
- Fix asset name text overflow in DAG list view (#57363)
- Fix memory leak caused by repeated SSL context creation in API client (#57374)
- Fix performance issues loading DAG list page with many DAGs (#57444)
- Fix text selection jumping unexpectedly in log viewer (#57453)
- Fix DAG documentation pane not scroll-able when content is too long (#57518)
- Fix incorrect macro listings in template reference documentation (#57529)
- Fix Human-In-The-Loop operators failing when using notifiers (#57551)
- Fix n+1 query issues in XCom API endpoints (#57554)
- Fix n+1 query issues in Event Logs API endpoint (#57558)
- Fix n+1 query to fetch tags in the dags list page (#57570)
- Optimize database query to prevent "Out of sort memory" errors with many DAG versions (#57042)
- Optimize DAG list query for users with limited access (#57460)
- Optimize dynamic DAG updates to avoid loading large serialized DAGs (#57592)
- Reduce serialized DAG size by optimizing callback serialization in ``default_args`` (#57397)
Miscellaneous
^^^^^^^^^^^^^
- UI: Improve global navigation visual design, interaction, and accessibility (#57565)
- UI: Add button to download all task logs at once (#56771)
- UI: Add timestamp column to ``XCom`` viewer and standardize task instance columns (#57447)
- UI: Improve highlighting of selected task instances and edges in grid view (#57560)
- Improve retry logic by migrating from ``retryhttp`` to ``tenacity`` library (#56762)
- Improve exception logging for task instance heartbeat failures (#57179)
- Add ``Content-Type`` header to Task SDK API requests (#57386)
- Log execution API server URL at task startup (#57409)
- Reduce log noise by changing "Connection not found" from error to debug level (#57548)
- Add ``task_display_name`` alias in event log API responses (#57609)
- Improve Pydantic model validation strictness in serialization (#57616)
- Fix systemd service files issues (#57231)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Improve plugin system documentation for clarity and completeness (#57068)
- Improve clarity on api workers recommendation in docs (#57404)
- Fix ``instance_name`` in UI docs (#57523)
- Fix airflow macros list in template document (#57529)
Airflow 3.1.1 (2025-10-27)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
- Fix execution failures with NULL ``dag_run.conf`` during upgrades from earlier versions (#56729)
- Fix memory leak in remote logging connection cache (#56695)
- Fix DAG processor crash with pre-import module optimization enabled (#56779)
- Fix scheduler crash with email notifications (#56431)
- Fix scheduler crash during 3.0 to 3.1 migration when ``retry_delay`` is ``None`` (#56236)
- Fix task retries executing wrong method after deferred state (#56737)
- Fix retry callbacks not executing for externally killed tasks (#56607)
- Fix custom timetable ``generate_run_id`` not called for manual triggers (#56699)
- Fix ``KeyError`` when accessing ``retry_delay`` on ``MappedOperator`` without explicit value (#56605)
- Fix ``task-sdk`` connection error handling to match ``airflow-core`` behavior (#56653)
- Fix topological sort for Grid View (#56963)
- Fix ``get_ti_count`` and ``get_task_states`` access in callback requests (#56860)
- Fix ``Connection`` or ``Variable`` access in Server context (#56602)
- Fix ``.airflowignore`` order precedence (#56832)
- Fix migration errors for Pydantic 2.12.0 compatibility (#56581)
- Fix: Correctly parse JSON for ``--dag_run_conf`` in ``airflow dags backfill`` CLI (#56599)
- UI: Fix note modal does not change markdown text after change (#56092)
- UI: Fix Grid for cleared runs when tasks were removed (#56297)
- UI: Fix log text selection contrast in light mode (#56893)
- UI: Fix Advanced Search button overlap in DAG List View (#56777)
- UI: Fix view for many DAG tags (#55604)
- UI: Fix asset name text overflow in DAGs list view (#55914)
- UI: Fix auto refresh when only 1 dag run is running (#56649)
- UI: Fix UI keeps poking pools API when no permission (#56626)
- UI: Fix multi-line drag selection in task log view (#56300)
- UI: Fix task named ``'root'`` causes blue screen on hover (#56926)
- UI: Fix cron expression display for ``Day-of-Month`` and ``Day-of-Week`` conflicts (#56255)
- UI: Fix Grid view performance issues with ``SerializedDagModel`` query optimization (#56938)
- Fix: Gracefully handle FastAPI plugins with empty ``url_prefix`` (#55262)
- Fix: Allow mapped tasks to accept zero-length inputs on rerun (#56162)
- Fix: Enable API to clear task instances by specifying map indexes (#56945)
- Fix: Add ``max_retry_delay`` to ``MappedOperator`` model (#56951)
- Fix: Use name passed to ``@asset`` decorator when fetching the asset (#56611)
- UI: Add English as a fallback locale (#56934)
Miscellaneous
^^^^^^^^^^^^^
- Add Greek UI translation (#56724)
- Add Thai UI translation (#56946)
- Add Polish translations (#56825)
- Close German translation gaps for full UI translation (#56981)
- Fix Hebrew typo in translations (#56168)
- Improve DAG and task missing error handling in callbacks (#56725)
- Improve UI retry strategy on client errors (#56638)
- Improve get dag grid structure endpoint speed (#56937)
- Optimize grid structure query with ``DISTINCT`` for ``dag_version_id`` lookup (#56565)
- Add configurable timeout for Execution API requests (#56969)
- Prevent unnecessary kubernetes client imports in workers (#56692)
- Lazy import ``PodGenerator`` for deserialization (#56733)
- Serialize pydantic models in json mode for JSON serialization compatibility (#56939)
- Update authentication to handle JWT token in backend (#56677)
- Update bulk API permission check to handle ``action_on_existence`` (#56672)
- Migrate ``CreateAssetEventsBody`` to Pydantic v2 ``ConfigDict`` (#56772)
- Restore timetable ``active_runs_limit`` check (#56922)
- Add ``is_favorite`` to UI dags list (#56341)
- Add ``executor``, ``hostname``, and ``queue`` columns to ``TaskInstances`` page (#55922)
- Add resize function for DAG Documentation (#56344)
- Add optional pending dag runs check to auto refresh (#56648)
- Add auto refresh to backfill banner (#56774)
- UI: Add Expand/Collapse all to ``XComs`` page (#56285)
- UI: Update recent runs bar chart and improve responsiveness (#56314)
- UI: Update duration format to show milliseconds (#56961)
- UI: Modify min width for task names in grid view (#56952)
- UI: Use Task Display Name in Graph if existing (#56511)
- UI: Use Task Display Name in Grid if existing (#56410)
- UI: Use TI duration from database instead of UI calculated (#56329)
- UI: Make DAG Run ID visible in DAG Header Card (#56409)
- UI: Modify calendar cell colors (#56161)
- UI: Modify log highlight color (#56894)
- UI: Fix show appropriate time units in grid view (#56414)
- UI: Reduce default columns of DAG Run and Task Instance lists (#55968)
- UI: Add expand and collapse functionality for task groups (#56334)
- UI: Avoid using rem for icons for Safari compatibility (#56304)
- UI: Add ANSI support to log viewer (#56721)
- UI: Support Dynamic UI Alerts (#56259)
- UI: Disable Gantt view by default (#56242)
- UI: Use welcome on dashboard instead of airflow (#56074)
- UI: Improve clipboard button visibility with hover effect (#56484)
- UI: Allow sub-pages in React UI plugins (#56485)
- Include task instance id in log printed by supervisor (#56383)
- Emit log stream stopped warning as ``ndjson`` (#56480)
- Detect interactive terminal to set colored logging with override env variable support (#56157)
- Add back deprecation warning for ``sla_miss_callback`` (#56127)
- Move ``natsort`` dependency to ``airflow-core`` (#56582)
- Added missing ``babel`` dependency in Task SDK (#56592)
- Remove unused ``dagReports`` API endpoint (#56621)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Improve API sort documentation (#56617)
- Improve API doc for ordering query param (#55988)
- Add Audit Logs detailed documentation (#56719)
- Update serializer document to reflect latest changes in codebase (#56857)
- Update ASF logos in documentation to the new Oak logo (#56601)
- Enhance ``triggering_asset_event`` retrieval documentation in DAGs (#56957)
- Remove self-reference in best practices documentation (#56111)
- Fix supported Python versions in README (#56734)
Airflow 3.1.0 (2025-09-25)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
Human in the Loop (HITL)
""""""""""""""""""""""""
Airflow 3.1 introduces :doc:`Human-in-the-Loop (HITL) </tutorial/hitl>` functionality that enables
workflows to pause and wait for human decision-making. This powerful feature is particularly valuable for
AI/ML workflows, content moderation, and approval processes where human judgment is essential.
HITL tasks pause execution in a ``deferred`` state while waiting for human input via the Airflow UI. Users
with appropriate roles can see pending tasks, review context (including ``XCom`` data and ``DAG`` parameters), and
complete actions through intuitive web forms. The feature also supports API-driven interactions for custom
UIs and notification integration.
For detailed usage instructions, see :doc:`/tutorial/hitl`.
**Note**: HITL operators require ``apache-airflow-providers-standard`` package and Airflow 3.1+.
Task SDK Decoupling for Independent Upgrades
"""""""""""""""""""""""""""""""""""""""""""""
Airflow 3.1 advances the decoupling of the Task SDK from Airflow Core through
improved DAG serialization with versioned contracts. While complete code separation is planned for Airflow 3.2.0,
the serialization foundation enables independent upgrades when components are deployed separately.
**For DAG Authors**: Import constructs from ``airflow.sdk`` namespace:
- ``from airflow.sdk import DAG, task, asset``
- Access to latest authoring features with forward compatibility
- Reduced dependency on server-side Airflow versions
**For Platform Teams**: Foundation for independent upgrades:
- Schema compliance ensures compatibility across versions
- Deployment flexibility when components are separated
- Reduced coordination overhead between development and operations teams
For technical details on the serialization contract, see :doc:`/administration-and-deployment/dag-serialization`.
Deadline Alerts
"""""""""""""""
Deadline Alerts provide proactive monitoring for DAG execution by automatically triggering notifications
when time thresholds are exceeded. This helps ensure SLA compliance and timely completion of critical workflows.
Configure deadline monitoring by specifying:
- **Reference point**: Choose from DAG run queued time, logical date, or fixed datetime
- **Interval**: Time threshold relative to the reference point (positive or negative)
- **Callback**: Response action using Airflow Notifiers or custom functions
Example use cases:
- Alert if a daily ETL hasn't completed 1 hour after its scheduled time
- Notify stakeholders 30 minutes before a critical deadline
- Escalate when resource-constrained DAGs remain queued too long
**Current Limitations**: Deadline Alerts currently support only asynchronous callbacks (``AsyncCallback``).
Support for synchronous callbacks (``SyncCallback``) is planned for a future release.
For configuration details and examples, see :doc:`/howto/deadline-alerts`.
.. warning::
Deadline Alerts are experimental in 3.1 and may change in future versions based on user feedback.
UI Internationalization
"""""""""""""""""""""""
Airflow 3.1 delivers comprehensive internationalization (``i18n``) support, making the web interface
accessible to users worldwide. The React-based UI now supports 17 languages with robust translation infrastructure.
**Supported Languages**:
- Arabic
- Catalan
- Dutch
- English
- French
- German
- Hebrew
- Hindi
- Hungarian
- Italian
- Korean
- Polish
- Portuguese
- Simplified Chinese
- Spanish
- Traditional Chinese
- Turkish
The translation system includes automated completeness checking and clear contribution guidelines for community translators.
React Plugin System (AIP-68)
"""""""""""""""""""""""""""""
Airflow 3.1 introduces a modern plugin architecture enabling rich integrations through React components and
external views. This extensibility framework allows organizations to embed custom dashboards,
monitoring tools, and domain-specific interfaces directly within the Airflow UI.
**New Plugin Capabilities**:
- **React Apps**: Full-featured React applications integrated into Airflow navigation
- **External Views**: Embed external web applications via iframe with seamless authentication
- **Dashboard Integration**: Custom widgets and panels for operational dashboards
- **Menu Integration**: Add custom navigation items and organize tools logically
**Developer Experience**:
- Hot reloading during development with ``airflow-react-plugin`` dev tools
- TypeScript support and modern React patterns
- Standardized plugin loading and validation
- Comprehensive documentation and boilerplate generation
This plugin system replaces legacy Flask-based approaches with modern web standards, improving performance,
maintainability, and user experience.
For more details and examples, see :doc:`/howto/custom-view-plugin`.
Enhanced UI Views and Filtering
""""""""""""""""""""""""""""""""
Airflow 3.1 brings significant UI improvements including rebuilt Calendar and Gantt chart views for the modern React UI,
comprehensive filtering capabilities, and a refreshed visual design system.
**Visual Design Improvements**
The UI now features an updated color palette leveraging Chakra UI semantic tokens, providing better consistency,
accessibility, and theme support across the interface. This modernization improves readability and creates
a more cohesive visual experience throughout Airflow.
**Rebuilt Views and Enhanced Filtering**
The Calendar and Gantt views from Airflow 2.x have been rebuilt for the modern React UI, along with enhanced
filtering capabilities across all views. These improvements provide better performance and a more consistent
user experience with the rest of the modern Airflow interface.
**DAG Dashboard Organization**
Users can now pin and favorite DAGs for better dashboard organization, making it easier to find and prioritize
frequently used workflows. This feature is particularly valuable for teams managing large numbers of DAGs,
providing quick access to critical workflows without searching through extensive DAG lists.
Inference Execution (Synchronous DAGs)
""""""""""""""""""""""""""""""""""""""
Airflow 3.1 introduces a new streaming API endpoint that allows applications to watch DAG runs until completion,
enabling more responsive integration patterns for real-time and inference workflows.
**New Streaming Endpoint**:
The ``/dags/{dag_id}/dagRuns/{dag_run_id}/wait`` endpoint repeatedly emits JSON updates at specified intervals until the DAG run reaches a finished state.
.. code-block:: bash
# Watch a DAG run with 2-second polling interval, including XCom results
curl -X GET "http://localhost:8080/api/v2/dags/ml_pipeline/dagRuns/manual_2024_01_15/wait?result=inference_task" \
-H "Accept: application/x-ndjson"
This enables use cases like:
- **ML Inference Monitoring**: Trigger inference DAGs and wait for completion before returning results
- **Real-time Processing**: Monitor event-driven workflows with immediate response requirements
- **API Integration**: Build responsive services that react to DAG completion without polling
- **Synchronous Workflows**: Create quasi-synchronous behavior for workflows that need immediate feedback
New Trigger Rule: ``ALL_DONE_MIN_ONE_SUCCESS``
""""""""""""""""""""""""""""""""""""""""""""""
``ALL_DONE_MIN_ONE_SUCCESS``: This rule triggers when all upstream tasks are done (success, failed) and
at least one has succeeded, filling a gap between existing trigger rules for complex workflow patterns.
Skipped upstream tasks work as usually - they skip downstream task.
Enhanced DAG Processing Visibility
"""""""""""""""""""""""""""""""""""
DAG parsing duration is now exposed in the UI, providing better visibility into DAG processing
performance and helping identify parsing bottlenecks. This information is displayed alongside
other DAG metadata to assist with performance optimization.
Python 3.13 support added & 3.9 support removed
"""""""""""""""""""""""""""""""""""""""""""""""
Support for Python 3.9 has been removed, as it has reached end-of-life.
Airflow 3.1.0 requires Python 3.10, 3.11, 3.12 or 3.13.
Configuration Changes and Cleanup
""""""""""""""""""""""""""""""""""
**Webserver Configuration Reorganization**
Several webserver configuration options have been moved to the ``api`` section for better organization:
- ``[webserver] log_fetch_timeout_sec`` → ``[api] log_fetch_timeout_sec``
- ``[webserver] hide_paused_dags_by_default`` → ``[api] hide_paused_dags_by_default``
- ``[webserver] page_size`` → ``[api] page_size``
- ``[webserver] default_wrap`` → ``[api] default_wrap``
- ``[webserver] require_confirmation_dag_change`` → ``[api] require_confirmation_dag_change``
- ``[webserver] auto_refresh_interval`` → ``[api] auto_refresh_interval``
Unused configuration options have been removed:
- ``[webserver] instance_name_has_markup``
- ``[webserver] warn_deployment_exposure``
**API Server Logging Configuration**
The API server configuration option ``[api] access_logfile`` has been replaced with ``[api] log_config`` to align with uvicorn's logging configuration. The new option accepts a path to a logging configuration file compatible with ``logging.config.fileConfig``, providing more flexible logging configuration.
**Security Improvement: XCom Deserialization**
The ``enable_xcom_deserialize_support`` configuration option has been removed as a security improvement. This option previously allowed deserializing unknown objects in the API, which posed a security risk due to potential remote code execution vulnerabilities when deserializing arbitrary Python objects.
The XCom display improvements now handle showing non-native XComs (like custom objects, Assets, datetime objects) in a human-readable way through safer methods that don't require deserializing unknown objects in the API server. This provides better user experience when viewing XCom data in the Airflow UI while eliminating the security risk.
API Changes
"""""""""""
**Asset API Key Rename**
The ``consuming_dags`` key in asset API responses has been renamed to ``scheduled_dags`` to better reflect its purpose. This key contains only DAGs that use the asset in their ``schedule`` argument, not all DAGs that technically use the asset.
Task SDK Interface Changes
""""""""""""""""""""""""""
**Removed Functions**
The following functions have been removed from the task-sdk (``airflow.sdk.definitions.taskgroup``) and moved to server-side API services:
- ``get_task_group_children_getter``
- ``task_group_to_dict``
These functions are now internal to Airflow's API layer and should not be imported directly by users.
Reduce default API server workers to 1
""""""""""""""""""""""""""""""""""""""
The default number of API server workers (``[api] workers``) has been reduced from ``4`` to ``1``.
With FastAPI, sync code runs in external thread pools, making multiple workers within a single
process less necessary. Additionally, with uvicorn's spawn behavior instead of fork, there is
no shared copy-on-write memory between workers, so horizontal scaling with multiple API server
instances is now the recommended approach for better resource utilization and fault isolation.
A good starting point for the number of workers is to set it to the number of CPU cores available.
If you do have multiple CPU cores available for the API server, consider deploying multiple API
server instances instead of increasing the number of workers.
Airflow now uses `structlog <https://www.structlog.org/en/stable/>`_ everywhere
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Most users should not notice the difference, but it is now possible to emit structured log key/value pairs from tasks.
If your class subclasses ``LoggingMixin`` (which all ``BaseHook`` and ``BaseOperator`` do -- i.e. all hooks and
operators) then ``self.log`` is now a `<structlog logger>`_.
The advantage of using structured logging is that it is much easier to find specific information about log
message, especially when using a central store such as ``OpenSearch``/``Elastic``/``Splunk`` etc.
You don't have to make any changes, but you can now take advantage of this.
.. code-block:: python
# Inside a Task/Hook etc.
# Before:
# self.log.info("Registering adapter %r", item.name)
# Now:
self.log.info("Registering adapter", name=item.name)
This will produce a log that (in the UI) will look something like this::
[2025-09-16 10:36:13] INFO - Registering adapter name="adapter1"
or in JSON (i.e. the log files on disk)::
{"timestamp": "2025-09-16T10:36:13Z", "log_level": "info", "event": "Registering adapter", "name": "adapter1"}
You can also use ``structlog`` loggers at the top level of modules etc, and ``stdlib`` both continue to work:
.. code-block:: python
import logging
import structlog
log1 = logging.getLogger(__name__)
log2 = strcutlog.get_logger(__name__)
log1.info("Loading something from %s", __name__)
log2.info("Loading something", source=__name__)
(You can't add arbitrary key/value pairs to ``stdlib``, but the normal ``percent-formatter`` approaches still work fine.)
Serialization Interface Changes
"""""""""""""""""""""""""""""""
The deserializer interface in ``airflow.serialization.serializers`` has changed for improved security.
**Before 3.1.0:**
``def deserialize(classname: str, version: int, data: Any)``
**Starting with 3.1.0:**
``def deserialize(cls: type, version: int, data: Any)``
The class loading is now handled in ``serde.py``, and the deserializer receives the loaded class directly rather than a ``classname`` string.
This update avoids the use of ``import_string`` in the deserializer, making deserialization more secure.
New Features
^^^^^^^^^^^^
- Add Calendar and Gantt chart views to modern React UI with enhanced filtering (#54252, #51667)
- Add Python 3.13 support for Airflow runtime and dependencies (#46891)
- Add ``SQLAlchemy 2.0`` support with various compatibility fixes for ``Python 3.13`` (#52233, #52518, #54940)
- Add support for the ``psycopg3`` postgres driver (#52976)
- Add ability to track & display user who triggers DAG runs (#51738, #53510, #54164, #55112)
- Add toggle for log grouping in task log viewer for better organization (#51146)
- Add tag filtering improvements with Any/All selection options (#51162)
- Add comprehensive filtering for DAG runs, task instances, and audit logs (#53652, #54210, #55082)
- Add ``XCom`` browsing with filtering and improved navigation (#54049)
- Add bulk task instance actions and deletion endpoints (#50443, #50165, #50235)
- Add DAG run deletion functionality through UI (#50368)
- Add test connection button for connection validation (#51055)
- Add hyperlink support for URLs in XCom values (#54288)
- Add pool column to task instances list and improve pool integration (#51185, #51031)
- Add drag-and-drop log grouping and improved log visualization (#51146)
- Add color support for XCom JSON display (#51323)
- Add configuration column to DAG runs page (#51270)
- Add enhanced note visibility and management in task headers (#51764, #54163)
- Introduce React plugin system (AIP-68) for modern UI extensions (#52255)
- Add support for external view plugins via iframe integration (#51003, #51889)
- Add dashboard integration capabilities for custom React apps (#54131, #54144)
- Add comprehensive plugin development tools and documentation (#53643)
- Implement complete HITL operator suite (``HITLOperator``, ``ApprovalOperator``, ``HITLEntryOperator``) for human decision workflows (#52868)
- Add HITL UI integration with role-based access and form handling (#53035)
- Add HITL API endpoints with filtering and query support (#53376, #53923)
- Add HITL utility functions for generating URLs to required actions page (#54827)
- Improve HITL user experience with bug fixes, UI enhancements, and data model consistency (#55463, #55539, #55575, #55546, #55543, #55536, #55535)
- Add ordering and filtering support for HITL details endpoints (#55217)
- Add "No Response Received" required action state (#55149)
- Add operator filter for HITL task instances (#54773)
- Implement deadline alert system for proactive DAG monitoring (AIP-86) (#53951, #53903, #53201, #55086)
- Add configurable reference points and notification callbacks (#50677, #50093)
- Add deadline calculation and tracking in DAG execution lifecycle (#51638, #50925)
- Add comprehensive UI translation support for 16 languages (#51266, #51038, #51219, #50929, #50981, #51793 and more)
- Add right-to-left (RTL) layout support for Arabic and Hebrew (#51376)
- Add language selection interface and browser preference detection (#51369)
- Add translation completeness validation and automated checks (#51166, #51131)
- Add calendar data API endpoints for DAG execution visualization (#52748)
- Add endpoint to watch DAG runs until completion (#51920, #53346)
- Add DAG run ID pattern search functionality (#52437)
- Add multi-sorting capabilities for improved data navigation (#53408)
- Add bulk connection deletion API and UI (#51201)
- Add task group detail pages across DAG runs (#50412, #50309)
- Add asset event tracking with last event timestamps (#50060, #50279)
- Add ``has_import_errors`` filter to Core API GET ``/dags`` endpoint (#54563)
- Add dag_version filter to get_dag_runs endpoint (#54882)
- Add pattern search for event log endpoint (#55114)
- Add dry_run support with consistent audit log handling (#55116)
- Add utility functions for generic filter counting (#54817)
- Add keyboard navigation for Grid view interface (#51784)
- Add improved error handling for plugin import failures (#49643)
- Add plugin validation in ``/plugins`` API with warnings for invalid plugins (#55673)
- Improve accessibility for screen readers and assistive technologies with proper language detection (#55839)
- Add enhanced variable management with upsert operations (#48547)
- Add favorites/pinning support for DAG dashboard organization (#51264)
- Add system theme support with automatic OS preference detection (#52649)
- Add hotkey shortcut to toggle between Grid and Graph views (#54667)
- Add queued DAGs filter button to DAGs page (#55052)
- Add DAG parsing duration visibility in UI (#54752)
- Add owner links support in DAG Header UI for better navigation (#50627)
- Add ``dag_display_name`` aliases for improved API consistency (#50332, #50065, #50014, #49933, #49641)
- Add enhanced search capabilities with SearchParamsKeys constants (#55218)
- Add ALL_DONE_MIN_ONE_SUCCESS trigger rule for flexible task dependencies (#53959)
- Add fail_when_dag_is_paused parameter to TriggerDagRunOperator for better control (#48214)
- Add ``XCom`` validation to prevent empty keys in ``XCom.set()`` and ``XCom.get()`` operations (#46929)
- Add collapsible plugin menu when multiple plugins are present (#55265)
- Add external view plugin categories (admin, browse, docs, user) (#52737)
- Add iframe plugins integration to DAG pages (#52795)
- Add plugin error display in UI with comprehensive error handling (#49643, #49436)
- Add collapsible failed task logs to prevent React error overflow (#54377)
- Add dynamic legend system for calendar view (#55155)
- Add React UI for Edge functionality (#53563)
- Add pending actions display to DAG UI (#55041)
- Add description field for filter parameters (#54903)
- Add Catalan language support to Airflow UI (#55013)
- Add Hungarian language support to Airflow UI (#54716)
- Add map_index validation in categorize_task_instances (#54791)
- Add Grid view UX improvements (#54846)
- Add HITL UX improvements for better user experience (#54990)
- Add async support for Notifiers (AIP-86) (#53831)
- Add filtering capabilities for tasks view (#54484)
- Add asset-based filtering support to DAG API endpoint (#54263)
- Add iframe plugins to navigation (#51706)
- Add RTL (right-to-left) layout support for Arabic and Hebrew (#51376)
- Add test connection button to UI (#51055)
- Add task instance bulk actions endpoint (#50443)
- Add connection bulk deletion functionality (#51201)
- Add pool column to task instances list (#51185)
- Add ``iframe_views`` to backend plugin support (#51003)
- Add keyboard shortcuts to clear and mark state for task instances and DAG runs (#50885)
- Add deadline relationship to DAG runs and deadline model (#50925, #50093)
- Add DAG run deletion UI (#50368)
- Add task instance deletion UI and endpoint (#50235, #50165)
- Switch all airflow logging to structlog (#52651, #55434, #55431, #55638)
- Add Filter Bar to Audit Log (#55487)
- Add Filters UI for Asset View (#54640)
- Update color palette and leverage Chakra semantic tokens (#53981, #55739)
- Improve calendar view UI with enhanced tooltips and visual fixes (#55476)
Bug Fixes
^^^^^^^^^
- Fix DAG list filtering to include ``QUEUED`` runs with null ``start_date`` (#52668)
- Fix XCom deletion failure for mapped task instances through bulk deletion API (#51850)
- Fix XCom deletion failure for mapped task instances (#54954)
- Fix task timeout handling within task SDK (#54089)
- Fix task instance tries API duplicate entries (#50597)
- Fix connection validation and type checking during construction (#54759)
- Fix mapped task instance index display in Task Instances tab (#55363)
- Fix Gantt chart state mismatch with Grid view (#55300)
- Fix Gantt chart status color display issues (#55296)
- Fix XCom mapping for dynamically-mapped task groups (#51556)
- Fix missing ``ti_successes`` and related metrics in Airflow 3.0 Task SDK (#55322)
- Fix bulk operation permissions for connection, pool and variable (#55278)
- Fix ``clearTaskInstances`` API: Restore ``include_past``/``future`` support on UI (#54416)
- Fix migration when XCom has NaN values (#53812)
- Fix HITL related UI schema generated by prek hooks (#55204)
- Fix consistent no-log handling for tasks with try_number=0 in API and UI (#55035)
- Fix timezone conversion in datetime trigger parameters (#54593)
- Fix audit log payload for DAG pause/unpause actions (#55091)
- Fix pushing None as an XCom value (#55080)
- Fix scheduler processing of cleared running tasks stuck in RESTARTING state (#55084)
- Fix XCom deletion failure for mapped task instances (#54954)
- Fix outgoing graph edges should exit opposite of incoming edges (#54789)
- Fix external links in Navigation buttons (#52220)
- Fix Error when viewing DAG details of a no longer configured bundle (#52086)
- Fix compatibility with new numpy and pandas versions (#52071)
- Fix connection recovery from URI when host has protocol (#51953)
- Fix last DAG run not showing on DAG listing (#51115)
- Fix task instance tries API returning duplicate entries (#50597)
- Fix Graph view vanishing and loading issues (#53886, #54756)
- Fix rendered template display formatting for better readability (#53657)
- Fix Grid view expand/collapse button functionality (#54257)
- Fix tooltip visibility and positioning issues (#53913)
- Fix grid keyboard navigation focus management (#54271)
- Fix plugin registration for invalid objects and middleware registration (#55264, #55399)
- Fix external links for plugins with undefined URL routes (#55221)
- Fix language display consistency and flag representation (#51560, #51177)
- Fix RTL layout rendering for Arabic and Hebrew interfaces (#51853)
- Fix graph export cropping when view is partial (#55012)
- Fix log viewer "Toggle Source" to hide only source fields, not all structured log fields (#55474)
- Output on stdout/stderr from within tasks is now filterable in the Sources list in the UI log view (#55508)
- Redact JWT tokens in task logs (#55499)
- Fix grid view to handle long task name (#55332)
- Allow slash characters in Variable keys similar to Airflow 2.x (#55324)
- Fix Grid cache invalidation for multi-run task operations (#55504)
- Fix Gantt chart rendering issues (#55554)
- Fix ``XCom`` access in DAG processor callbacks for notifiers (#55542)
- Fix alignment of arrows in RTL mode for right-to-left languages (#55619)
- Fix connection form extras not inferring correct type in UI (#55492)
- Fix incorrect log timestamps in UI when ``default_timezone`` is not UTC (#54431)
- Fix handling of priority_weight for DAG processor callbacks (#55436)
- Fix pointless requests from Gantt view when there is no Run ID (#55668)
- Ensure filename and ``lineno`` of logger calls are present in Task Logs (#55581)
- Fix DAG disappearing after callback execution in stale detection (#55698)
- Fix DB downgrade to Airflow 2 when fab tables exists (#55738)
- Fix UI stats endpoint causing dashboard loading issues (#55733)
- Fix unintended console output when DAG not found in ``serialized_dag`` table (#54972)
- Fix scheduler handling of orphaned tasks from Airflow 2 during upgrade (#55848)
- Fix logging format to respect existing configuration during upgrade to prevent unexpected log format changes (#55824)
- Fix Grid view crashes when DAG version information is missing (#55771)
- Fix compatibility for custom triggers migrating from Airflow 2.x that use synchronous connection calls (#55799)
- Fix DAG runs triggered from UI incorrectly marked as REST API triggers instead of UI triggers (#54650)
- Fix XCom API responses failing when encountering non-serializable objects by falling back to string representation (#55880)
- Fix asset queue display in UI showing incorrect timestamps for deleted queue events (#54652)
- Fix SQLite database migrations failing due to foreign key constraint handling (#55883)
- Fix DAG deserialization failure when using non-default weight_rule values like 'absolute' (#55906)
- Fix async connection retrieval in triggerer context preventing event loop blocking (#55812)
- Fix Airflow downgrade compatibility by handling serialized DAG format conversion from v3 to v2 (#55975)
- Fix 'All Log Levels' filter not working in task log viewer (#55851)
- Fix Grid view scrollbar overlapping issues on Firefox browser (#55960)
- Fix Gantt chart misalignment with Grid view layout (#55995)
- Fix Grid view task names being extremely collapsed and unreadable when displaying many DAG runs (#55997)
- Fix ``LocalExecutor`` race condition where tasks could start before database state was committed (#56010)
Miscellaneous
^^^^^^^^^^^^^
- Move secrets masker to shared distribution for better modularity (#54449)
- Move email notifications from scheduler to DAG processor for better architecture (#55238)
- Add graph UI load optimization with latest run info endpoint (#53429)
- Optimize UI bundle size by moving translations to dynamic loading (#51735)
- Relocate Task SDK components for improved separation (#55174, #54795)
- Refactor trigger rule utilities and weight rule consolidation (#54797, #53393)
- Remove deprecated Airflow 2.x modules and legacy imports (#50482)
- Clean up unused code and improve module organization (#52176, #52173, #53031)
- Add SQLAlchemy 2.0 CI support for future compatibility (#52233)
- Improve test fixtures and SDK communication testing (#54795, #50603)
- Add translation completeness linting and validation tools (#51166)
- Upgrade to latest versions of important dependencies (#55350)
- Move webserver configuration options to API section (#50693, #50656)
- Improve DAG bundle handling and versioning support (#47592)
- Add database management CLI tools for external database operations (#50657)
- Add comprehensive HITL operator documentation and examples (#54618)
- Add guards for registering middlewares from plugins (#55399)
- Optimize Gantt group expansion with de-bouncing and deferred rendering (#55334)
- Differentiate between triggers and watchers currently running for better visibility (#55376)
- Removed unused config: ``dag_stale_not_seen_duration`` (#55601, #55684)
- Update UI's query client strategy for improved performance (#55528)
- Unify datetime format across the UI for consistency (#55572)
- Mark React Apps as Experimental for Airflow 3.1 release (#55478)
- Improve OOM error messaging for clearer task failure diagnosis (#55602)
- Display responder username for better audit trail in HITL workflows (#55509)
- The constraint file do not contain developer dependencies anymore (#53631)
- Add hyperlinks to ``dag_id`` column in DAG Runs and Task Instances pages for better navigation (#55648)
- Add responsive web design (RWD) support to Grid view (#55745)
Doc Only Changes
^^^^^^^^^^^^^^^^
- Add comprehensive Human-in-the-Loop operator tutorial and examples (#54618)
- Add deadline alerts configuration and usage documentation (#53727)
- Make term Dag consistent in docs task-sdk (#55100)
- Add migration guide for upgrading from legacy SLA functionality to deadline alerts (#55743)
- Add DAG bundles triggerer limitation documentation (#55232)
- Add deadline alerts usage guides and best practices (#53727)
- Remove ``--preview`` flag from ``ruff check`` instructions for Airflow 3 upgrade path (#55516)
- Add documentation for context parameter (#55377)
Airflow 3.0.6 (2025-08-29)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- Fix Connection extra field masking errors when connections use masked values (#54780)
- UI: Fix ``TriggerDagRunLink`` broken page when clicking "Triggered DAG" button (#54760)
- Fix remote logging connection availability in Task SDK supervisor when connections are created via UI (#54720)
- Fix ``task_queued_timeout`` not working after first DAG run by properly resetting ``queued_by_job_id`` (#54604)
- Fix DAG version determination to use bundle path and relative fileloc instead of absolute fileloc (#54483)
- Remove Kerberos replay cache (``KRB5CCNAME`` env) when running tasks with user impersonation (#54672)
- Skip additional span-related database queries when tracing is disabled (#54626)
- Fix ``max_active_tasks`` persisting after removal from DAG code (#54639)
- UI: Automatically switch to the triggered DAG run in Graph/Grid view when manually triggering a DAG run (#54336)
- UI: Fix "Maximum update depth exceeded" errors in Task Log Preview by filtering out empty log entries (#54628)
- Fix custom logging configuration failures preventing triggerer and scheduler startup with simple module paths (#54686)
- Fix MySQL UUID generation in task_instance migration (#54814)
- Only redirect on the dag detail page (#54921)
- Fix local executor task execution (#54922)
Miscellaneous
"""""""""""""
- Add logging when triggerer reaches maximum trigger capacity for better observability (#54549)
- Point deprecation warning in Variable methods to specific alternatives (#54871)
- Point deprecation warning in Connection method to specific alternatives (#54872)
- Bump ``axios`` UI dependency from ``1.8.0`` to ``1.11.0`` (#54733)
- Bump ``pluggy`` to ``1.6.0`` (#54728, #54730)
Doc Only Changes
""""""""""""""""
- Fix broken link for Listener spec (#54535)
- Remove experimental status from ``get_parsing_context`` function (#54802)
- Correct Trigger-Form UI documentation for current Airflow 3 features (#54806)
- Add backfill through UI to docs (#54910)
Airflow 3.0.5 (2025-08-20)
--------------------------
This release has been yanked.
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- UI: Fix JSON field display in dark mode when using system OS theme detection (#54260)
- Restore proper DAG callback execution context (#53684)
- Restore ``get_previous_dagrun`` functionality for task context (#53655)
- Fix scheduler crashes with ``DetachedInstanceError`` when processing executor events (#54334)
- Fix ``DetachedInstanceError`` when accessing ``DagRun.created_dag_version`` (#54362)
- Fix Task SDK to respect custom default queue configuration from config settings (#52786)
- Fix: Cannot edit or delete pools with ``"/"`` in the name in the UI (#54268)
- Fix: Validate and handle invalid ``extra`` field in connections UI and API (#53963, #54034, #54235)
- Fix: Apply DAG permission filter to dashboard (#54215)
- Fix API validation error when DAG runs have bundle_version but no created_dag_version (#54010)
- Fix task configuration defaults not being read from configuration settings (#52871)
- Fix duplicate task group prefixes in task IDs when unmapping ``MappedOperators`` within ``TaskGroups`` (#53532)
- Fix custom ``XCom`` backends not being used when ``BaseXCom.get_all()`` is called (#53814)
- Fix ``xcom_pull`` ignoring ``include_prior_dates`` parameter when ``map_indexes`` is not specified (#53809)
- Allow setting and deleting Variables and XComs from triggers (#53514)
- Fix ``AttributeError`` when reading logs for previous task attempts with ``TaskInstanceHistory`` (#54114)
- Skip database queries for spans and metrics when tracing/metrics are disabled (#54404)
- UI: Fix Graph view edge rendering issues for nested task groups with excessive bends and misalignment (#54412)
- Allow database downgrade from Airflow 3.x to 2.11 (#54399, #54508)
- Reduce excessive warning logs when multiple deferred tasks are queued in triggerer (#54441)
- Fix log retrieval failures for in-progress tasks by properly configuring JWT authentication (#54444)
- Fix DAG import errors for invalid access control roles to persist consistently in UI (#54432)
- Fix task failure callbacks missing ``end_date`` and ``duration`` by populating ``TaskInstance`` data before invoking callbacks (#54458)
- Fix task retry overflow errors when calculating next retry datetime by capping delay to maximum configured value (#54460)
- Add missing ordering to ``AssetEvent`` queries in scheduler to maintain consistent event processing order (#52231)
- Fix XCom lookup failures in nested mapped task groups by correctly resolving ``map_index`` for upstream tasks (#54249)
- UI: Fix task name indentation in Graph view for deeply nested task groups beyond 5 levels (#54419)
- Run failure callbacks for task instances that get stuck in queued state and fail after requeue attempts (#54401)
- Make secrets masking work when connections are loaded from secrets backends (#54574, #54612)
Miscellaneous
"""""""""""""
- Set minimum version for ``common.messaging`` to ``1.0.3`` (#54176)
- Add IP validation to example_dag_decorator DAG (#54208)
Doc Only Changes
""""""""""""""""
- Fix doc redirects for operators moved to the standard provider (#54251)
- Add FAQ entry about testing connections and "Canary" Dag (#54151)
- Add note about ruff rules and preview flag (#53331)
- Fix broken link in advanced logging config docs (#53460)
- Update dag bundles docs; add s3, fix git classpath (#53473)
- Fix example to use proper task context and logging instead of ``dag.log`` (#54463)
- Improve documentation navigation by hiding Public Interface subsections from sidebar while preserving page links (#54465)
Airflow 3.0.4 (2025-08-08)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- Fix scheduler heartbeat timeout failures with intermittent ``DetachedInstanceError`` crashes (#53838) (#53858)
- Fix connection editing where sensitive fields like passwords and extras were lost when updating connections (#53943) (#53973)
- Fix BaseOperator ``on_kill`` functionality not working when tasks are killed externally in TaskSDK (#53718) (#53832)
- Fix TaskInstance notes not refreshing automatically without manual page refresh (#53307) (#54025)
- Fix invalid execution API URLs causing failures in task supervisor (#53082) (#53518)
- Fix task failure callbacks not running on DAG Processor when tasks are externally killed (#53058) (#53143)
- Fix ``task_success_overtime`` configuration option not being configurable (#53342) (#53351)
- Fix CSS warning for nth-child selector (#53982) (#54000)
- Fix DAG filtering where "all" option did not show all DAGs as expected (#53656) (#53672)
- Fix accordion child contents not being visible when content overflows (#53595) (#53602)
- Fix navbar positioning for anchor calculations (#52016) (#53581)
- Fix DagBag safe mode configuration resolution in DAG processor (#52694) (#53507)
- Fix large log reading causing out-of-memory issues in API server (#49470) (#53167)
- Fix connection exceptions consistency between Airflow 2.x and 3.x (#52968) (#53093)
- Remove unnecessary ``group_by`` clause in event logs query for performance (#53733) (#53807)
- Allow remote logging providers to load connections from API Server (#53719) (#53761)
- Add certificate support for API server client communication with self-signed certificates (#53574) (#53793)
- Respect ``apps`` flags for API server command configuration (#52929) (#53775)
- Skip empty DAG run configuration rows and set statement timeout (#50788) (#53619)
- Remove incorrect warning for ``BaseOperator.executor`` attribute (#53496) (#53519)
- Add back DAG parsing pre-import optimization for improved performance (#50371) (#52698)
- Flexible form use ReactMarkdown instead of default Markdown component (#54032) (#54040)
- Unconditionally disable ``start_from_trigger`` functionality (#53744) (#53750)
- Serialize NaN and infinity values to string (#53835) (#53844)
- Make log redaction safer in edge case when redaction has an error (#54046) (#54048)
- Flexible form use ReactMarkdown instead of default Markdown component (#54032) (#54040)
- Fix inconsistent casing in UI of decorated tasks (#54056) (#54092)
Miscellaneous
"""""""""""""
- Fix AIRFLOW_API_APPS constant in API server command (#54007) (#54012)
- Add deprecation notice for using Connection from models in favor of SDK approach (#53594) (#53621)
- Remove remnants of ``~=`` used in requires-python configuration (#52985) (#52987)
- Remove upper-binding for "python-requires" specification (#52980) (#52984)
- Update GitPython from 3.1.44 to 3.1.45 (#53725) (#53731)(#53724) (#53732)
Doc Only Changes
""""""""""""""""
- Update DAG author documentation to use "DAG author" terminology (#53857) (#53950)
- Update architecture diagrams labels from "Webserver(s)" to "API Server(s)" (#53917) (#54020)
- Remove bold formatting for Public Interface documentation in Airflow 3.0+ (#53955) (#53964)
- Add user-facing documentation for running separate Task Execution API server (#53789) (#53794)
- Add documentation for self-signed certificate configuration (#53788) (#53792)
- Update systemd unit files and documentation for Airflow 3.0 compatibility (#52294) (#53609)
- Update public interface documentation to reflect airflow.sdk and AIP-72 changes (#52197) (#53117)
- Update BaseOperator documentation string for clarity (#53403) (#53404)
- Remove extra slash from endpoint URL formatting (#53755) (#53764)
- Clarify our security model for sensitive connection information (#54088) (#54100)
Airflow 3.0.3 (2025-07-14)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- Fix task execution failures with large data by improving internal communication protocol (#51924, #53194)
- Fix reschedule sensors failing after multiple re-queue attempts over long periods (#52706)
- Improve ``xcom_pull`` to cover different scenarios for mapped tasks (#51568)
- Fix connection retrieval failures in triggerer when schema field is used (#52691)
- Add back user impersonation (``run_as_user``) support for task execution (#51780)
- Fix DAG version not updating when bundle name changes without DAG structure changes (#51939)
- Add back ``exception`` to context for task callbacks (#52066)
- Fix task log retrieval for retry attempts showing incorrect logs (#51592)
- Fix data interval handling for DAGs created before AIP-39 during serialization (#51913)
- Fix lingering task supervisors when ``EOF`` is missed (#51180) (#51970)
- Persist ``EventsTimetable``'s description during serialization (#51926)
- Delete import error when a dag bundle becomes inactive (#51921)
- Cleanup import errors during DB migration (#51919)
- Fix ``EOF`` detection of subprocesses in Dag Processor (#51895)
- Stop streaming task logs if end of log mark is missing (#51482)
- Allow more empty loops before stopping log streaming (#52624)
- Fix Jinja2 Template deep copy error with ``dag.test`` (#51673)
- Explicitly close log file descriptor in the supervise function (#51654)
- Improve structured logging format and layout (#51567) (#51626)
- Use Connection Hook Names for Dropdown instead of connection IDs (#51613)
- Add back config setting to control exposing stacktrace (#51617)
- Fix task level alias resolution in structure endpoint (#51579)
- Fix backfill creation to include DAG run configuration from form (#51584)
- Fix structure edges in API responses (#51489)
- Make ``dag.test`` consistent with ``airflow dags test`` CLI command (#51476)
- Fix downstream asset attachment at task level in structure endpoint (#51425)
- Fix Task Instance ``No Status`` Filter (#52154)
- UI: Fix backfill creation to respect run backwards setting from form (#52168)
- UI: Set downstream option to default on task instance clear (#52246)
- UI: Enable iframe script execution (#52568)
- UI: Fix DAG tags filter not showing all tags in UI when tags are greater than 50 (#52714)
- UI: Add real-time clock updates to timezone selector (#52414)
- Improve Grid view performance and responsiveness with optimized data loading (#52718,#52822,#52919)
- Fix editing connection with sensitive extra field (#52445)
- Fix archival for cascading deletes by archiving dependent tables first (#51952)
- Fix whitespace handling in DAG owners parsing for multiple owners (#52221)
- Fix SQLite migration from 2.7.0 to 3.0.0 (#51431)
- Fix http exception when ti not found for extra links API (#51465)
- Fix Starting from Trigger when using ``MappedOperator`` (#52681)
- Add ti information to re-queue logs (#49995)
- Task SDK: Fix ``AssetEventOperations.get`` to use ``alias_name`` when specified (#52324)
- Ensure trigger kwargs are properly deserialized during trigger execution (#52721)
- Fixing bad cadwyn migration for upstream map indexes (#52797)
- Run trigger expansion logic only when ``start_from_trigger`` is True (#52873)
- Fix example dag ``example_external_task_parent_deferrable.py`` imports (#52957)
- Fixes pagination in DAG run lists (#52989)
- Fix db downgrade check condition (#53005)
- Fix log viewing for skipped task (#53028,#53101)
- Fixes Grid view refresh after user actions (#53086)
- Fix ``no_status`` and ``duration`` for grid summaries (#53092)
- Fix ``ti.log_url`` not in Task Context (#50376)
- Fix XCom data deserialization when using ``XCom.get_all()`` method (#53102)
Miscellaneous
"""""""""""""
- Update ``connections_test`` CLI to use Connection instead of BaseHook (#51834) (#51917)
- Fix table pagination when DAG filtering changes (#51795)
- UI: Move asset events to its own tab (#51655)
- Exclude ``libcst`` 1.8.1 for Python 3.9 (#51609)
- UI: Implement navigation on bar click (#50416)
- Reduce unnecessary logging when retrieving connections and variables (#51826)
Doc Only Changes
""""""""""""""""
- Add note about payload size considerations in API docs (#51768)
- Enhance ENV vars and conns visibility docs (#52026)
- Add http-only warning when running behind proxy in documentation (#52699)
- Publish separate docs for Task SDK (#52682)
- Streamline Taskflow examples and link to core tutorial (#52709)
- Refresh Public Interface & align how-to guides for Airflow 3.0+ (#53011)
Airflow 3.0.2 (2025-06-10)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- Fix memory leak in dag-processor (#50558)
- Add back invalid inlet and outlet check before running tasks (#50773)
- Implement slice on LazyXComSequence to allow filtering items from a mapped task(#50117)
- Fix execution API server URL handling for relative paths in KE (#51183)
- Add log lookup exception for Empty operator subtypes (#50325)
- Increase the max zoom on the graph view to make it easier to see small dags on big monitor screens (#50772)
- Fix timezone selection and dashboard layout (#50463)
- Creating backfill for a dag is affecting other dags (#50577)
- Fix next asset schedule and dag card UX (#50271)
- Add bundle path to ``sys.path`` in task runner (#51318)
- Add bundle path to ``sys.path`` in dag processor (#50385)
- Prevent CPU spike in task supervisor when heartbeat timeout exceeded (#51023)
- Fix Airflow Connection Form widget error (#51168)
- Add backwards compatibility shim and deprecation warning for EmailOperator (#51004)
- Handle ``SIGSEGV`` signals during DAG file imports (#51171)
- Fix deferred task resumption in ``dag.test()`` (#51182)
- Fix get dags query to not have join explosion (#50984)
- Ensure Logical date is populated correctly in Context vars (#50898)
- Mask variable values in task logs only if the variable key is sensitive (#50775)
- Mask secrets when retrieving variables from secrets backend (#50895)
- Deserialize should work while retrieving variables with secrets backend (#50889)
- Fix XCom deserialization for mapped tasks with custom backend (#50687)
- Support macros defined via plugins in Airflow 3 (#50642)
- Fix Pydantic ``ForwardRef`` error by reordering discriminated union definitions (#50688)
- Adding backwards compatibility shim for ``BaseNotifier`` (#50340)
- Use latest bundle version when clearing / re-running dag (#50040)
- Handle ``upstream_mapped_index`` when xcom access is needed (#50641)
- Remove unnecessary breaking flag in config command (#50781)
- Do not flood worker logs with secrets backend loading logs (#50581)
- Persist table sorting preferences across sessions using local storage (#50720)
- Fixed patch_task_instance API endpoint to support task instance summaries and task groups (#50550)
- Fixed bulk API schemas to improve OpenAPI compatibility and client generation (#50852)
- Fixed variable API endpoints to support keys containing slashes (#50841)
- Restored backward compatibility for the ``/run`` API endpoint for older Task SDK clients
- Fixed dropdown overflow and error text styling in ``FlexibleForm`` component (#50845)
- Corrected DAG tag rendering to display ``+1 more`` when tags exceed the display limit by one (#50669)
- Fix permission check on the ui config endpoint (#50564)
- Fix ``default_args`` handling in operator ``.partial()`` to prevent ``TypeError`` when unused keys are present (#50525)
- DAG Processor: Fix index to sort by last parsing duration (#50388)
- UI: Fix border overlap issue in the Events page (#50453)
- Fix ``airflow tasks clear`` command (#49631)
- Restored support for ``--local`` flag in ``dag list`` and ``dag list-import-errors`` CLI commands (#49380)
- CLI: Exclude example dags when a bundle is passed (#50401)
- Fix CLI export to handle stdout without file descriptors (#50328)
- Fix ``DagProcessor`` stats log to show the correct parse duration (#50316)
- Fix OpenAPI schema for ``get_log`` API (#50547)
- Remove ``logical_date`` check when validating inlets and outlets (#51464)
- Guard ``ti`` update state and set task to fail if exception encountered (#51295)
Miscellaneous
"""""""""""""
- UI: Implement navigation on bar click (#50416)
- UI: Always Show Trends count in Dag Overview (#50183)
- UI: Add basic json check to variable value
- Remove filtering by last dag run state in patch dags endpoint (#51347)
- Ensure that both public and ui dags endpoints map to DagService (#51226)
- Refresh Dag details page on new run (#51173)
- Log fallback to None when no XCom value is found (#51285)
- Move ``example_dags`` in standard provider to ``example_dags`` in sources (#51275)
- Bring back "standard" example dags to the ``airflow-core`` package (#51192)
- Faster note on grid endpoint (#51247)
- Port ``task.test`` to Task SDK (#50827)
- Port ``dag.test`` to Task SDK (#50300,#50419)
- Port ``ti.run`` to Task SDK execution path (#50141)
- Support running ``airflow dags test`` from local files (#50420)
- Move macros to task SDK ``execution_time`` module (#50940)
- Add a link to the Airflow logo in Nav (#50304)
- UI: Bump minor and patch package json dependencies (#50298)
- Added a direct link to the latest DAG run in the DAG header (#51119,#51148)
- Fetch only the most recent ``dagrun`` value for list display (#50834)
- Move ``secret_key`` config to ``api`` section (#50839)
- Move various ``webserver`` configs to ``fab`` provider (#50774,#50269,#50208,#50896)
- Make ``dag_run`` nullable in Details page (#50719)
- Rename Operation IDs for task instance endpoints to include map indexes (#49608)
- Update default sort for connections and dags (#50600)
- Raise exception if downgrade can't proceed due to no ``ab_user`` table (#50343)
- Enable JSON serialization for variables created via the bulk API (#51057)
- Always display the backfill option in the UI; enable it only for DAGs with a defined schedule (#50969)
- Optimized DAG header to fetch only the most recent DAG run for improved performance (#50767)
- Add ``owner_links`` field to ``DAGDetailsResponse`` for enhanced owner metadata in the API (#50557)
- UI: Move map index column to be in line with other columns when viewing a summary mapped tasks (#50302)
- Separate configurations for colorized and json logs in Task SDK / Celery Executor (#51082)
- Enhanced task log viewer with virtualized rendering for improved performance on large logs (#50746)
Doc Only Changes
""""""""""""""""
- Add dates for Limited Maintenance & EOL for Airflow 2.x (#50794)
- Add Apache Airflow setup instructions for Apple Silicon (#50179)
- Update recommendation for upgrade path to airflow 3 (#50318)
- Add "disappearing DAGs" section on FAQ doc (#49987)
- Update Airflow 3 migration guide with step about custom operators (#50871) (#50948)
- Use ``AssetAlias`` for alias in Asset ``Metadata`` example (#50768)
- Do not use outdated ``schedule_interval`` in tutorial dags (#50947)
- Add Airflow Version in Page Title (#50358)
- Fix callbacks docs (#50377)
- Updating operator extra links doc (#50197)
- Prune old Airflow versions from release notes (#50860)
- Fix types in config templates reference (#50792)
- Fix wrong import for ``PythonOperator`` in tutorial dag (#50962)
- Better structure of extras documentation (#50495)
Airflow 3.0.1 (2025-05-12)
--------------------------
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
- Improves the handling of value masking when setting Airflow variables for enhanced security (#43123)
- Make entire task box clickable to select the task (#49299)
- Vertically align task log header components in full screen mode (#49569)
- Remove ``dag_code`` records with no serialized dag (#49478)
- Clear out the ``dag_code`` and ``serialized_dag`` tables on 3.0 upgrade (#49563)
- Remove extra slash so that the runs tab is selected (#49600)
- Null out the ``scheduler_interval`` field on downgrade (#49583)
- Logout functionality should respect ``base_url`` in api server (#49545)
- Fix bug with showing invalid credentials on Login UI (#49556)
- Fix Dag Code text selection when dark mode is enabled (#49649)
- Bugfix: ``max_active_tis_per_dag`` is not respected by dynamically mapped tasks (#49708)
- Fix infinite redirect caused by mistakenly setting token cookie as secure (#49721)
- Better handle safe url redirects in login form for ``SimpleAuthManager`` (#49697)(#49866)
- API: Add missing ``bundle_version`` to DagRun response (#49726)
- Display bundle version in Dag details tab (#49787)
- Fix gcp remote log module import in airflow local settings (#49788)
- Bugfix: Grid view stops loading when there is a pending task to be expanded (#49772)
- Treat single ``task_ids`` in ``xcom_pull`` the same as multiple when provided as part of a list (#49692)
- UI: Auto refresh Home page stats (#49830)
- UI: Error alert overflows out of the alert box (#49880)
- Show backfill banner after creating a new backfill (#49666)
- Mark ``DAGModel`` stale and associate bundle on import errors to aid migration from 2.10.5 (#49769)
- Improve detection and handling of timed out DAG processor processes (#49868)
- Fix editing port for connections (#50002)
- Improve & Fix grid endpoint response time (#49969)
- Update time duration format (#49914)
- Fix Dashboard overflow and handle no status tasks (#49964)
- Fix timezone setting for logical date input on Trigger Run form (#49662)
- Help ``pip`` with avoiding resolution too deep issues in Python 3.12 (#49853)
- Bugfix: backfill dry run does not use same timezone as create backfill (#49911)
- Fix Edit Connection when connection is imported (#49989)
- Bugfix: Filtering items from a mapped task is broken (#50011)
- Fix Dashboard for queued DagRuns (#49961)
- Fix backwards-compat import path for ``BashSensor`` (#49935)
- Apply task group sorting based on webserver config in grid structure response (#49418)
- Render custom ``map_index_template`` on task completion (#49809)
- Fix ``ContinuousTimetable`` false triggering when last run ends in future (#45175)
- Make Trigger Dag form warning more obvious (#49981)
- Restore task hover and selection indicators in the Grid view (#50050)
- Fix datetime validation for backfills (#50116)
- Fix duration charts (#50094)
- Fix DAG node selections (#50095)
- UI: Fix date range field alignment (#50086)
- Add auto-refresh for ``Stats`` (#50088)
- UI: Fixes validation error and adds error indicator for Params form (#50127)
- fix: wrap overflowing texts of asset events (#50173)
- Add audit log extra to table and improve UX (#50100)
- Handle map indexes for Mapped ``TaskGroup`` (#49996)
- Do not use introspection in migration to fix offline SQL generation (#49873)
- Fix operator extra links for mapped tasks (#50238)
- Fix backfill form (#50249)(#50243)
- UI: Fix operator overflow in graph (#50252)
- UI: Pass ``mapIndex`` to clear the relevant task instances. (#50256)
- Fix markdown rendering on dag docs (#50142)
Miscellaneous
"""""""""""""
- Add ``STRAIGHT_JOIN`` prefix for MySQL query optimization in ``get_sorted_triggers`` (#46303)
- Ensure ``sqlalchemy[asyncio]`` extra is in core deps (#49452)
- Remove unused constant ``HANDLER_SUPPORTS_TRIGGERER`` (#49370)
- Remove sort indicators on XCom table to avoid confusion (#49547)
- Remove ``gitpython`` as a core dependency (#49537)
- Bump ``@babel/runtime`` from ``7.26.0`` to ``7.27.0`` (#49479)
- Add backwards compatibility shim for ``get_current_context`` (#49630)
- AIP-38: enhance layout for ``RunBackfillForm`` (#49609)
- AIP-38: merge Backfill and Trigger Dag Run (#49490)
- Add count to Stats Cards in Dashboard (#49519)
- Add auto-refresh to health section for live updates. (#49645)
- Tweak Execution API OpenAPI spec to improve code Generation (#49700)
- Stricter validation for ``backfill_id`` (#49691)(#49716)
- Add ``SimpleAllAdminMiddleware`` to allow api usage without auth header in request (#49599)
- Bump ``react-router`` and ``react-router-dom`` from 7.4.0 to 7.5.2 (#49742)
- Remove reference to ``root_dag_id`` in dagbag and restore logic (#49668)
- Fix a few SqlAlchemy deprecation warnings (#49477)
- Make default execution server URL be relative to API Base URL (#49747)(#49782)
- Common ``airflow.cfg`` files across all containers in default ``docker-compose.yaml`` (#49681)
- Add redirects for old operators location to standard provider (#49776)
- Bump packaging from 24.2 to 25.0 in ``/airflow-core`` (#49512)
- Move some non-core dependencies to the ``apache-airflow`` meta package (#49846)
- Add more lower-bind limits to address resolution too deep (#49860)
- UI: Add counts to pool bar (#49894)
- Add type hints for ``@task.kuberenetes_cmd`` (#46913)
- Bump ``vite`` from ``5.4.17`` to ``5.4.19`` for Airflow UI (#49162)(#50074)
- Add ``map_index`` filter option to ``GetTICount`` and ``GetTaskStates`` (#49818)
- Add ``stats`` ui endpoint (#49985)
- Add link to tag to filter dags associated with the tag (#49680)
- Add keyboard shortcut for full screen and wrap in logs. (#50008)
- Update graph node styling to decrease border width on tasks in UI (#50047) (#50073)
- Allow non-string valid JSON values in Variable import. (#49844)
- Bump min versions of crucial providers (#50076)
- Add ``state`` attribute to ``RuntimeTaskInstance`` for easier ``ti.state`` access in Task Context (#50031)
- Move SQS message queue to Amazon provider (#50057)
- Execution API: Improve task instance logging with structlog context (#50120)
- Add ``dag_run_conf`` to ``RunBackfillForm`` (#49763)
- Refactor Dashboard to enhance layout (#50026)
- Add the download button on the assets page (#50045)
- Add ``dateInterval`` validation and error handling (#50072)
- Add ``Task Instances [{map_index}]`` tab to mapped task details (#50085)
- Add focus view on grid and graph on second click (#50125)
- Add formatted extra to asset events (#50124)
- Move webserver expose config to api section (#50209)
Doc Only Changes
""""""""""""""""
- Remove flask application configuration from docs for AF3 (#49393)
- Docker compose: airflow-cli to depend on airflow common services (#49318)
- Better upgrade docs about flask/fab plugins in Airflow 3 (#49632)(#49614)(#49628)
- Various Airflow 3.0 Release notes & Updating guide docs updates (#49623)(#49401)(#49654)(#49663)(#49988)(#49954)(#49840)(#50195)(#50264)
- Add using the rest api by referring to ``security/api.rst`` (#49675)
- Add correct redirects for rest api and upgrade docs (#49764)
- Update ``max_consecutive_failed_dag_runs`` default value to zero in TaskSDK dag (#49795) (#49803)
- Fix spacing issues in params example dag (``example_params_ui_tutorial``) (#49905)
- Doc: Fix Kubernetes duplicated version in maintenance policy (#50030)
- Fix links to source examples in Airflow docs (#50082)
- Update ruff instructions for migration checks (#50232)
- Fix example of backfill command (#50222)
- Update docs for running behind proxy for Content-Security-Policy (#50236)
Airflow 3.0.0 (2025-04-22)
--------------------------
We are proud to announce the General Availability of Apache Airflow 3.0 — the most significant release in the project's
history. This version introduces a service-oriented architecture, a stable DAG authoring interface, e