CHANGELOG.md

Rama master · réplica de lectura del repositorio original.

### Watch all release presentations and videos at [ClickHouse Theater](https://presentations.clickhouse.com/) and [YouTube Playlist](https://www.youtube.com/playlist?list=PL0Z2YDlm0b3jAlSy1JxyP8zluvXaN3nxU).

### Table of Contents
**[ClickHouse release v26.10, 2026-10-22](#2610)**<br/>
**[ClickHouse release v26.9, 2026-09-21](#269)**<br/>
**[ClickHouse release v26.8 LTS, 2026-08-27](#268)**<br/>
**[ClickHouse release v26.7, 2026-07-22](#267)**<br/>
**[ClickHouse release v26.6, 2026-06-25](#266)**<br/>
**[ClickHouse release v26.5, 2026-05-21](#265)**<br/>
**[ClickHouse release v26.4, 2026-04-30](#264)**<br/>
**[ClickHouse release v26.3 LTS, 2026-03-26](#263)**<br/>
**[ClickHouse release v26.2, 2026-02-26](#262)**<br/>
**[ClickHouse release v26.1, 2026-01-29](#261)**<br/>
**[Changelog for 2025](https://clickhouse.com/docs/whats-new/changelog/2025/)**<br/>
**[Changelog for 2024](https://clickhouse.com/docs/whats-new/changelog/2024/)**<br/>
**[Changelog for 2023](https://clickhouse.com/docs/whats-new/changelog/2023/)**<br/>
**[Changelog for 2022](https://clickhouse.com/docs/whats-new/changelog/2022/)**<br/>
**[Changelog for 2021](https://clickhouse.com/docs/whats-new/changelog/2021/)**<br/>
**[Changelog for 2020](https://clickhouse.com/docs/whats-new/changelog/2020/)**<br/>
**[Changelog for 2019](https://clickhouse.com/docs/whats-new/changelog/2019/)**<br/>
**[Changelog for 2018](https://clickhouse.com/docs/whats-new/changelog/2018/)**<br/>
**[Changelog for 2017](https://clickhouse.com/docs/whats-new/changelog/2017/)**<br/>

# 2026 Changelog

### <a id="2610"></a> ClickHouse release 26.10, 2026-10-22

#### Backward Incompatible Change
* The query analysis that preceded the analyzer is removed. It has not been selectable since 26.9, when `enable_analyzer` became obsolete; now a server no longer uses it for a query another server sends either, so a cluster whose initiator is older than 26.10 gets the analyzer on this server regardless of what the initiator asks for. Such a mixed cluster can see `NOT_FOUND_COLUMN_IN_BLOCK` on the old initiator, because the two analyses name the result columns differently; set `enable_analyzer = 1` on every server before upgrading any of them. The `use_analyzer_for_mutations` server setting is removed, and `parallel_replicas_only_with_analyzer` is obsolete. [#120311](https://github.com/ClickHouse/ClickHouse/pull/120311) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The predicate push-down that the query analysis preceding the analyzer performed on the AST is removed, and `enable_optimize_predicate_expression` is now obsolete and has no effect. Push-down of a predicate into a subquery is performed by the analyzer as a query plan optimization and is not affected; push-down into a distributed subquery still happens on the AST level and is controlled by `allow_push_predicate_ast_for_distributed_subqueries`. The visible difference is that `EXPLAIN SYNTAX` of a query that is not a `SELECT`, for example `EXPLAIN SYNTAX INSERT INTO ... SELECT ...`, no longer prints a predicate copied into a subquery, a non-aggregate `HAVING` conjunct moved to `WHERE`, or the right-hand table of a `JOIN` wrapped into a subquery. [#121465](https://github.com/ClickHouse/ClickHouse/pull/121465) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `system.metric_log` now uses the `bucketed` schema by default: all profile events and current metrics are stored in a single `metrics` column of type `Map(Enum16(...), Int64)` with the bucketed `Map` serialization and a constant number of 128 buckets, so the table has a few columns instead of thousands, zero values are not stored, and reading one metric reads only one of the 128 buckets. Every metric is also exposed as an `ALIAS` column, so queries such as `SELECT ProfileEvent_Query FROM system.metric_log` continue to work unchanged. On upgrade the existing table is renamed to `metric_log_0`, as with any other change of the structure of a system log table, and the built-in dashboards read `merge('system', '^metric_log')`, so they keep showing the data collected before the upgrade. Add `<schema_type>wide</schema_type>` to the `<metric_log>` section of the server configuration to keep the previous schema. [#118318](https://github.com/ClickHouse/ClickHouse/pull/118318) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124016](https://github.com/ClickHouse/ClickHouse/pull/124016) ([Raúl Marín](https://github.com/Algunenano)).
* `clickhouse-server` can now apply a `seccomp` system call filter at startup. The `seccomp` setting controls refused system calls: `log` only records them, while `trap`, `kill`, and `errno` enforce the policy; `disabled` installs no filter. The `config.xml` shipped with the server packages and the Docker image now sets `seccomp` to `trap`, so a server that runs in a sandbox which refuses the `seccomp` system call must switch it to `log` or `disabled` before the upgrade. Extended-attribute probes used by some hostname-resolution libraries are rejected with `ENOSYS` instead of the configured seccomp action, so server startup still succeeds under the default `trap` mode. You might need to disable seccomp when using executable user-defined functions. [#119545](https://github.com/ClickHouse/ClickHouse/pull/119545) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122577](https://github.com/ClickHouse/ClickHouse/pull/122577) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124856](https://github.com/ClickHouse/ClickHouse/pull/124856) ([Azat Khuzhin](https://github.com/azat)).
* Settings constraints and `allow_feature_tier` are now enforced for users, roles and settings profiles defined by SQL. A statement is refused if it changes a restricted setting of some user indirectly (such as `GRANT`, `DROP ROLE`, `DROP SETTINGS PROFILE` or `CREATE USER OR REPLACE`), sets a value outside its author's constraints, or removes a setting its author is constrained on. A user defined by SQL whose stored settings, roles or assigned settings profiles violate the constraints or `allow_feature_tier` can no longer log in. The `compatibility` setting no longer applies values that settings constraints or `allow_feature_tier` refuse; such settings keep their current value. Login checks use the user's resolved constraints, preserving bounds inherited from the default profile. Closes [#124966](https://github.com/ClickHouse/ClickHouse/issues/124966). [#119472](https://github.com/ClickHouse/ClickHouse/pull/119472) ([Raúl Marín](https://github.com/Algunenano)). [#125582](https://github.com/ClickHouse/ClickHouse/pull/125582) ([Raúl Marín](https://github.com/Algunenano)).
* Overriding a key that is stored in a named collection now requires `SHOW NAMED COLLECTIONS SECRETS` on that collection. This also applies to overrides through an alias (including flat `kafka_*` settings over nested `kafka.*` keys), through a `SETTINGS` clause of a table engine or table function, and through keys in a dictionary source (checked when the dictionary is created). Resolving a relative `url` stored in a named collection against `s3_base` or `url_base` also counts as an override of `url`. Adding a key that the collection does not store remains allowed, and keys marked `NOT OVERRIDABLE` cannot be overridden. The `allow_named_collection_override_by_default` setting is obsolete. Existing tables whose `SETTINGS` or alias arguments override a `NOT OVERRIDABLE` key cannot be loaded after the upgrade, and the server does not start: before upgrading, recreate such tables without the override, or make the key overridable. [#123133](https://github.com/ClickHouse/ClickHouse/pull/123133) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* `max_bytes_before_external_join` and `max_bytes_ratio_before_external_join` are now the spill threshold for every hash-based `join_algorithm`, including `grace_hash`, which previously ignored them and spilled on `max_rows_in_join` / `max_bytes_in_join` instead. Those two settings are a hard cap again for every algorithm: they no longer cause spilling, and a join that spills no longer exceeds them. All join algorithms remain available; `join_algorithm = 'grace_hash'` now needs a non-zero spill threshold, and is skipped in favour of the next algorithm in the list when it has none. Set `legacy_join_size_limits_trigger_spilling = 1` (or `compatibility` to 26.8 or older) to restore the previous behavior. [#117356](https://github.com/ClickHouse/ClickHouse/pull/117356) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#119335](https://github.com/ClickHouse/ClickHouse/pull/119335) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `arrayCount` now returns `UInt64` instead of `UInt32`, so it stays exact for arrays with more than 4294967295 matching elements. Set `array_count_legacy_uint32_result = 1` (or `compatibility = '26.9'`) to restore the previous result type. During a rolling upgrade, enable `array_count_legacy_uint32_result` on upgraded servers for the users under which shard-side queries execute, so distributed queries initiated by not-yet-upgraded servers keep the previous behavior on upgraded shards; remove it after the upgrade is complete. In addition, `length(arrayFilter(func, arr))` is now rewritten to `arrayCount(func, arr)`, which is up to 1.29x faster on long arrays, controlled by `optimize_rewrite_array_filter_length_to_array_count`. [#113023](https://github.com/ClickHouse/ClickHouse/pull/113023) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `accurateCast` from `String` or `FixedString` to a `Nullable` target now throws the error its non-`Nullable` target already raised (`CANNOT_PARSE_TEXT`, `TOO_LARGE_STRING_SIZE`, ...) instead of returning `NULL`, and no longer throws for a row the source marked `NULL`. This affects every use of `accurateCast`: `DeltaLake` writes stop committing an unrepresentable value as `NULL`, `dictGetOrDefault` rejects a default its `Nullable` attribute cannot hold and now converts only the defaults it uses, `IN` under `transform_null_in = 1` no longer matches an unparsable key against a `NULL` in the set, and `cast_ipv4_ipv6_default_on_conversion_error` yields the type's default for an unparsable address. `CAST`, `accurateCastOrNull`, `accurateCastOrDefault`, and ordinary `INSERT` are unchanged, as is what a non-text source does with a value it cannot represent; use `accurateCastOrNull` for the previous behavior. Closes [#120524](https://github.com/ClickHouse/ClickHouse/issues/120524). [#120674](https://github.com/ClickHouse/ClickHouse/pull/120674) ([Groene AI](https://github.com/groeneai)).
* `SELECT ... FINAL` from `SummingMergeTree` no longer removes the rows whose summed columns are all zero, unlike background merges. Before, this removal looked only at the columns the query read, so results depended on them: for example, `SELECT count() FROM table FINAL` could return fewer rows than `SELECT * FROM table FINAL`. Background merges and `OPTIMIZE TABLE ... FINAL` still remove such rows, so filter out the all-zero rows explicitly if a query relied on their removal. [#115032](https://github.com/ClickHouse/ClickHouse/pull/115032) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A read of a `Distributed` or `Remote` table with `skip_unavailable_shards = 1` now fails with `ALL_CONNECTION_TRIES_FAILED` when every shard was skipped before returning any data, instead of returning an empty result, matching what already happens when the only shard is local. Previously an aggregate without `GROUP BY` silently returned zero rows instead of one. The error fails the whole query, so a query that combines such a read with other data, for example a `UNION ALL` with local rows, no longer returns those rows. When a shard is itself a `Distributed` table whose shards were all skipped, the outer query now fails even if another shard answered; `skip_unavailable_shards_mode = 'unavailable_or_exception_before_processing'` keeps the partial result in that case. No setting restores the previous empty result. Closes [#115646](https://github.com/ClickHouse/ClickHouse/issues/115646). [#115775](https://github.com/ClickHouse/ClickHouse/pull/115775) ([Groene AI](https://github.com/groeneai)).
* A `MATERIALIZED` column computed from a column with a `TTL` is now recomputed when the `TTL` resets that column, both during a merge and in `ALTER TABLE ... MATERIALIZE TTL`. Previously it kept the pre-expiry value forever. If the dependent expression is not defined on the expired column's default (for example `intDiv(100, x)` when `x` expires to `0`), the `TTL` merge now throws instead of writing the stale value; make the expression total on the default (`intDivOrZero`, `if(x = 0, ...)`) if such a table must keep merging. Closes [#117310](https://github.com/ClickHouse/ClickHouse/issues/117310). [#117815](https://github.com/ClickHouse/ClickHouse/pull/117815) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* With `input_format_null_as_default = 0`, a JSON `null` for a non-`Nullable` numeric column now raises an exception instead of being silently stored as `0` (or `nan`), consistent with other types and formats. Keep `input_format_null_as_default` at its default `1` to insert the column default instead. [#122137](https://github.com/ClickHouse/ClickHouse/pull/122137) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `ALTER TABLE ... ADD COLUMN` and `MODIFY COLUMN` now apply the `data_type_default_nullable` setting the same way `CREATE TABLE` does: with the setting enabled, a column type written without an explicit `NULL` or `NOT NULL` modifier becomes `Nullable`, so the rows that existed before `ADD COLUMN` read `NULL`, `ADD COLUMN x Int32 DEFAULT NULL` is accepted, and `MODIFY COLUMN x Int64` keeps a `Nullable` column `Nullable` instead of failing. As in `CREATE TABLE`, an `Array`, `Map`, `LowCardinality`, or another type that cannot be inside `Nullable` then requires `NOT NULL`, and so does a `MODIFY COLUMN` of a sorting key column. To keep the previous result of such an `ALTER`, write `NOT NULL` explicitly or run it with `data_type_default_nullable = 0`. Closes [#124504](https://github.com/ClickHouse/ClickHouse/issues/124504). [#124597](https://github.com/ClickHouse/ClickHouse/pull/124597) ([Groene AI](https://github.com/groeneai)).
* `arrayJoin` is now lowered to an `ARRAY JOIN` step by default (setting `query_plan_lower_array_join_function`). The expansion streams through the pipeline instead of being materialized inside the projection, filters on the joined elements are applied before the rows are multiplied, and queries with several `arrayJoin` calls no longer build the whole cross product in memory. As a consequence, a non-deterministic function such as `rand` or `generateUUIDv4` next to `arrayJoin` in the same `SELECT` is now evaluated once per output row, as with the `ARRAY JOIN` clause, instead of once per source row. Set `arrayjoin_nondeterministic_functions_before_expansion = 1` (or `compatibility` to 26.9 or older) to keep the previous behavior. [#117882](https://github.com/ClickHouse/ClickHouse/pull/117882) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#120390](https://github.com/ClickHouse/ClickHouse/pull/120390) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#122240](https://github.com/ClickHouse/ClickHouse/pull/122240) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* A function that depends on blocks or row order, such as `runningDifference`, `neighbor` or `rowNumberInBlock`, next to the `arrayJoin` function now sees the blocks of the expansion, like with the `ARRAY JOIN` clause. Set `legacy_array_join_function_nondeterministic_evaluation = 1` to get the previous behavior. [#123143](https://github.com/ClickHouse/ClickHouse/pull/123143) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* A row policy on a `Distributed` table (or another table that processes the query itself) read through a `Merge` table now makes queries to the `Merge` table fail with `ILLEGAL_PREWHERE` for the users the policy applies to, like queries to that table itself. Before, the `Merge` table applied such a policy after the table had already sorted, limited and computed `extremes`, so queries could return wrong rows and show values of hidden rows in `extremes`. Define the row policy on the underlying tables of the remote servers and on the local child tables instead. This also fixes `NOT_FOUND_COLUMN_IN_BLOCK` on every query over a `Merge` table whose child has a row policy when the `Merge` table also reads a `Distributed` table. [#125765](https://github.com/ClickHouse/ClickHouse/pull/125765) ([Groene AI](https://github.com/groeneai)).
* `clickhouse-local --ignore-error` now reports a statement that failed at execution instead of skipping it silently: previously it printed no message and no exception code, so a failing multi-statement script looked exactly like a passing one, while `clickhouse-client --ignore-error` reported every error all along. `clickhouse-client` also ignored the option entirely when it was combined with `--interactive` and `--query`/`--queries-file` on a terminal, and now applies it to those queries too. The exit code of `clickhouse-client --ignore-error` changes accordingly: it used to be the code of whichever error the last statement of the batch happened to hit, which made the result depend on the position of the failing statement rather than on the batch as a whole, and it is now a success code, matching what `clickhouse-local --ignore-error` has always returned. Scripts that relied on the old exit code to detect failures should drop `--ignore-error` so that the run stops and fails at the first error, or check the reported errors on `stderr`. [#119572](https://github.com/ClickHouse/ClickHouse/pull/119572) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* `ATTACH TABLE ... FROM '<path>'` now requires `READ ON FILE` and `WRITE ON FILE` on the source directory in `user_files`, in addition to `CREATE TABLE`, because the query reads the directory and then moves it out of `user_files` on success. Previously a user could attach and read staged data without those `FILE` grants. Grant the corresponding `FILE` privileges to users that rely on this form. [#120582](https://github.com/ClickHouse/ClickHouse/pull/120582) ([Alexander Gololobov](https://github.com/davenger)).
* A `MySQL` dictionary source created with a `CREATE DICTIONARY` query can no longer enable `enable_local_infile`, at the source or at one of its `REPLICA` entries; loading such a dictionary fails with `BAD_ARGUMENTS`, which with the default `dictionaries_lazy_load` is on first use rather than at `CREATE`. A `REPLICA` that does not set the key inherits the source's value, as the connection pool does. To keep the option, define the dictionary in a server configuration file; `enable_local_infile = 0`, and a `SOURCE(MYSQL(NAME ...))` reference to a named collection where the key was never read, are unaffected. [#120972](https://github.com/ClickHouse/ClickHouse/pull/120972) ([Groene AI](https://github.com/groeneai)).
* The definitions of disks in `storage_configuration` and in `disk` are validated: an element that nothing reads is reported as `UNKNOWN_ELEMENT_IN_CONFIG` instead of being silently ignored, which finds typos and options that do not exist for this disk type, such as `path` in an `S3` disk. If a server does not start after the upgrade because of an element it does not know, remove that element from the configuration, or set `<skip_check_for_incorrect_settings>1</skip_check_for_incorrect_settings>`. [#118440](https://github.com/ClickHouse/ClickHouse/pull/118440) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `system.*_log` tables can now be configured uniformly via `system_tables.*`. In `system.opentelemetry_span_log`, `finish_date` is renamed to `event_date` and kept as an alias. Aliases are now supported in configured `PARTITION BY`, `ORDER BY`, `TTL`, `PRIMARY KEY`, and `SAMPLE BY` expressions, so custom definitions can continue to use `finish_date`. Some tables now fall back to the default TTL when none is configured, so set `ttl` explicitly if that matters. [#121563](https://github.com/ClickHouse/ClickHouse/pull/121563) ([Azat Khuzhin](https://github.com/azat)). [#125564](https://github.com/ClickHouse/ClickHouse/pull/125564) ([Azat Khuzhin](https://github.com/azat)).
* The setting `validate_mutation_query` is deprecated and ignored: mutation queries (`ALTER TABLE ... UPDATE`, `ALTER TABLE ... DELETE`, `DELETE FROM`) are now always validated before being accepted. It was enabled by default since 24.11, and disabling it did not avoid the failure but only deferred it to the background, where the mutation got stuck and required manual intervention. Setting it to `0` no longer lets an invalid mutation be accepted: it is rejected when submitted. [#121419](https://github.com/ClickHouse/ClickHouse/pull/121419) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Settings `query_plan_read_in_order`, `query_plan_optimize_prewhere`, and `query_plan_aggregation_in_order` are obsolete and no longer have any effect. Use `optimize_read_in_order`, `optimize_aggregation_in_order`, and `optimize_move_to_prewhere` instead. [#121390](https://github.com/ClickHouse/ClickHouse/pull/121390) ([Anton Popov](https://github.com/CurtizJ)).
* The setting `use_compact_format_in_distributed_parts_names` is obsolete. The non-compact directory format for the async `INSERT` queue of a `Distributed` table has been removed. The setting was `1` by default since version 21.1. Before upgrading, set it to `1` and let the queue drain. [#120186](https://github.com/ClickHouse/ClickHouse/pull/120186) ([Alexander Tokmakov](https://github.com/tavplubix)).
* The `MergeTree` settings `use_minimalistic_part_header_in_zookeeper` and `use_minimalistic_checksums_in_zookeeper` are obsolete and have no effect: new data parts of `ReplicatedMergeTree` tables are always stored in ZooKeeper as a single node with the compact header, the default since 20.1. An explicit `0` in table metadata or in the `merge_tree` configuration is ignored, and such tables and configurations still load. Part nodes written in the old format by earlier versions are still read. Closes [#122744](https://github.com/ClickHouse/ClickHouse/issues/122744). [#122790](https://github.com/ClickHouse/ClickHouse/pull/122790) ([Groene AI](https://github.com/groeneai)).
* Settings `s3_max_connections` and the `S3` disk's `max_connections` key are obsolete. They have had no effect since 21.4, when the per-endpoint connection pool of the `S3` client they bounded was removed, but were still documented as limiting the number of connections per server. Both are still accepted and do nothing. To bound the connections to object storage, use the server settings `disk_connections_soft_limit`, `disk_connections_store_limit`, `disk_connections_warn_limit`, and `disk_connections_hard_limit` and their `storage_connections_*` and `http_connections_*` counterparts; to bound the request rate, use `s3_max_get_rps` and `s3_max_put_rps`. [#119550](https://github.com/ClickHouse/ClickHouse/pull/119550) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Removed the zero-argument `year` function. Queries that used `year()` must switch to `toYear(today())` instead. [#120268](https://github.com/ClickHouse/ClickHouse/pull/120268) ([Robert Schulze](https://github.com/rschu1ze)).
* Removed the `system.codecs.is_experimental` column. Queries that filtered or selected it should use `tier = 'Experimental'` instead. [#118523](https://github.com/ClickHouse/ClickHouse/pull/118523) ([Raufs Dunamalijevs](https://github.com/rienath)).
* Removed the experimental function `detectTonality`. Queries that use it now fail with `UNKNOWN_FUNCTION`. [#123327](https://github.com/ClickHouse/ClickHouse/pull/123327) ([Groene AI](https://github.com/groeneai)).
* Update the `zxc` library to v0.14.0, which changes the compressed block format of the experimental `ZXC` codec. Data written with `CODEC(ZXC)` by earlier versions cannot be read by the new version and fails with `CANNOT_DECOMPRESS`. If you used the experimental `ZXC` codec, re-encode the affected columns to a different codec before upgrading, or re-insert the data after the upgrade. [#119097](https://github.com/ClickHouse/ClickHouse/pull/119097) ([Actuele AI](https://github.com/actueleai)).
* An aggregate function state that serializes to zero bytes cannot be read back, so the two parameterizations that produced one are now rejected: a `-Resample` combinator over an empty range (`begin >= end`, such as `countResample(10, 5, 1)`) throws `ARGUMENT_OUT_OF_BOUND`, and `covarSampMatrix`, `covarPopMatrix`, and `corrMatrix` with no arguments throw `NUMBER_OF_ARGUMENTS_DOESNT_MATCH`. Both returned an empty array before. No serialized format changes, but a table with such a column no longer loads. Before upgrading, drop the column where the engine supports `DROP COLUMN`, and the whole table where it does not (`Log`, `TinyLog`, `StripeLog`, `Set`, `Join`, `File`). [#119728](https://github.com/ClickHouse/ClickHouse/pull/119728) ([Groene AI](https://github.com/groeneai)).
* A data type naming the state of a function that only works as a window function, such as `AggregateFunction(rank)`, is now rejected with `BAD_ARGUMENTS`: such a state has no serialization, so the column could never hold a value. This applies wherever a type is named, including inside `Array`, `Map`, `Tuple`, and another state's argument list. A table that already stores such a column will no longer attach after the upgrade; drop or rewrite the column before upgrading. [#122148](https://github.com/ClickHouse/ClickHouse/pull/122148) ([Groene AI](https://github.com/groeneai)).
* `rankState` and every other `-State` of a function that only works as a window function, such as `denseRankStateIf` or `initializeAggregation('lagInFrameState', x)`, are now rejected with `BAD_ARGUMENTS` when the query is analyzed, also where the state is never computed (`toTypeName`, `EXPLAIN`, a column the outer query drops), and so is such a type in a binary-encoded `Native` header. Computing such a state always failed, but a view or `CREATE TABLE ... AS SELECT` could still store its type. A table whose engine expressions build such a state, for example in a sorting key, partition key, skip index, or `TTL`, no longer attaches. [#122463](https://github.com/ClickHouse/ClickHouse/pull/122463) ([Groene AI](https://github.com/groeneai)).
* `CREATE TABLE` and full-definition `ATTACH TABLE` now refuse a `CONSTRAINT` name declared more than once, and `ALTER TABLE ... ADD CONSTRAINT` / `MODIFY CONSTRAINT` now refuse an expression containing `arrayJoin` even when the command would install nothing. A table already stored with a repeated constraint name still attaches and keeps enforcing each of its declarations; give each constraint a unique name in new definitions. [#121235](https://github.com/ClickHouse/ClickHouse/pull/121235) ([Groene AI](https://github.com/groeneai)).
* The `Remote`, `RemoteSecure`, and `Cluster` database engines now reject an empty remote database name with `BAD_ARGUMENTS` instead of creating a database that lists and reads no tables. A server that holds such a database, created by an earlier version, does not start after the upgrade, so drop these databases before upgrading. Closes [#122616](https://github.com/ClickHouse/ClickHouse/issues/122616). [#122675](https://github.com/ClickHouse/ClickHouse/pull/122675) ([Groene AI](https://github.com/groeneai)).
* Connections to MySQL (the `mysql` table function, the `MySQL` table and database engines, and `MySQL` dictionaries) that set `ssl_ca` or `ssl_ca_pem` now verify the server certificate chain and host name, after updating `mariadb-connector-c` to v3.4.10. Previously the CA was not enforced, so a connection to a server whose certificate is signed by another CA, or whose certificate does not list the host name or IP address used to connect, succeeded and now fails with `self-signed certificate in certificate chain` or `Hostname verification failed`. Before upgrading, make sure the configured CA signs the MySQL server certificate and that the certificate lists the host name or IP address ClickHouse connects to. [#119510](https://github.com/ClickHouse/ClickHouse/pull/119510) ([Actuele AI](https://github.com/actueleai)).

#### New Feature
* Sets built for `IN` with a subquery or a table now spill to disk by default once query memory usage exceeds half the available memory under the applicable server or user memory limit, calculated at the start of execution (`max_bytes_ratio_before_external_set = 0.5`). The `max_bytes_before_external_set` setting can set an absolute threshold (`max_memory_usage` is not taken into account by the ratio, so use it to spill before that limit is reached), and setting both thresholds to `0` keeps sets in memory. This feature is now generally available. [#123860](https://github.com/ClickHouse/ClickHouse/pull/123860) ([Nihal Z. Miaji](https://github.com/nihalzp)). [#124907](https://github.com/ClickHouse/ClickHouse/pull/124907) ([Nihal Z. Miaji](https://github.com/nihalzp)).
* Support parts of the Trino SQL dialect in regular ClickHouse SQL: `TRY_CAST(x AS T)`, the `ARRAY[...]` array constructor, the type names `ARRAY(T)` and `MAP(K, V)`, and aliases for Trino functions such as `starts_with`, `slice`, `regexp_like`, `day_of_week`, `url_extract_host`, `bitwise_and`, `arbitrary`, and `bool_and`. [#124763](https://github.com/ClickHouse/ClickHouse/pull/124763) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Implemented a generic block nested loop join algorithm. `JOIN` now supports an arbitrary `ON` condition with no equality between the two tables, executing it as a block nested loop join, as slow as a `CROSS JOIN` with a filter, instead of failing with `INVALID_JOIN_ON_EXPRESSION`. The new setting `allow_block_nested_loop_join` turns it on/off. [#123642](https://github.com/ClickHouse/ClickHouse/pull/123642) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123792](https://github.com/ClickHouse/ClickHouse/pull/123792) ([Groene AI](https://github.com/groeneai)).
* Added the function-composition operator `f | g` and the argument placeholders `_`, `_1`, `_2`, ...: for example, `arrayMap(plus(_, 1) | multiply(_, 2), [1, 2, 3])` returns `[4, 6, 8]`. A composition can be used wherever a function is expected, including as an argument of a higher-order function. The operands can be lambdas, function names, nested compositions, or expressions with placeholders, which also work on their own, for example `arrayMap(_1 * 2 + 1, [1, 2, 3])`. The composition is resolved at query analysis time into an ordinary lambda. Note that `|` here is not bitwise OR; use `bitOr` for that. [#116029](https://github.com/ClickHouse/ClickHouse/pull/116029) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The query cache can now store its entries on disk, backed by a preconfigured filesystem cache: this gives the entries more space, makes them survive server restarts, and, with a distributed filesystem cache, provides a way to share them between servers. Select the filesystem cache with the new setting `query_cache_on_disk_cache_name`; see also the new settings `query_cache_on_disk_codec`, `enable_writes_to_query_cache_on_disk`, and `enable_reads_from_query_cache_on_disk`. Closes [#115361](https://github.com/ClickHouse/ClickHouse/issues/115361). [#115382](https://github.com/ClickHouse/ClickHouse/pull/115382) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added a notebook mode to the Web UI (`/play`). After a query is run, a `[+]` button under the result adds further cells, each either a query with its own result or a block of Markdown text, turning the tab into a notebook in the style of Jupyter or Observable. Cells can be run, reordered, inserted and deleted individually; the query editor collapses to a single line when it is not focused, results are bounded and scroll inside themselves, and the whole notebook is restored on reload. Notebook mode is per tab, so a tab that has only one cell looks and behaves exactly as before. [#115378](https://github.com/ClickHouse/ClickHouse/pull/115378) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added support for reading and writing data in the `SQLite` format. [#104510](https://github.com/ClickHouse/ClickHouse/pull/104510) ([Yue Ni](https://github.com/niyue)).
* Added the `Cluster` database engine, which provides real-time access to the tables of a configured cluster database and forwards `SELECT` and `INSERT` queries to it. [#114417](https://github.com/ClickHouse/ClickHouse/pull/114417) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the `Overlay` database engine: `CREATE DATABASE db ENGINE = Overlay(db1, db2, ...)` exposes the union of the tables of existing databases, each table working as an `Alias` table to the table of the first source database that has it. [#121878](https://github.com/ClickHouse/ClickHouse/pull/121878) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `CREATE DATABASE` and `ATTACH DATABASE` now accept a `SETTINGS` clause without an explicit `ENGINE` clause, defaulting to `Atomic`, so a database can define its metadata disk inline the same way a single table does. A single `SETTINGS` clause can now also mix database-engine settings with ordinary query settings. [#118423](https://github.com/ClickHouse/ClickHouse/pull/118423) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `MergeTree` tables in `Ordinary` databases can now be converted to `ReplicatedMergeTree` with `ATTACH TABLE ... AS REPLICATED` or the `convert_to_replicated` flag. [#118757](https://github.com/ClickHouse/ClickHouse/pull/118757) ([Kirill](https://github.com/kirillgarbar)).
* Added query-aware scheduling in workloads: the new workload setting `scheduler` selects how queries of a workload are scheduled — `fifo` (default), `fair` (weighted fair queueing with per-query weights that can be lowered as a query runs), `las` (least-attained-service), or `priority` (by the query setting `workload_priority`). The leaf-node `type` in `system.scheduler` is now `request_queue` instead of `fifo`. [#116800](https://github.com/ClickHouse/ClickHouse/pull/116800) ([Sergei Trifonov](https://github.com/serxa)).
* Queries can now spill to disk under memory pressure based on limits set per workload. The `max_memory_before_spill` and `max_memory_to_spill_ratio` options of `WORKLOAD` define a soft memory limit that triggers spilling for `ORDER BY`, `GROUP BY`, `IN`, `DISTINCT`, and `JOIN` with `grace_hash`. [#119443](https://github.com/ClickHouse/ClickHouse/pull/119443) ([Azat Khuzhin](https://github.com/azat)).
* Quotas can now limit the number of rows written by `INSERT` queries with the new `written_rows` resource, for example `CREATE QUOTA q KEYED BY ip_address FOR INTERVAL 1 minute MAX written_rows = 100000, query_inserts = 250 TO user`. Until now only the written bytes could be limited (`written_bytes`), which is awkward when the natural unit of the workload is rows. The current and maximum values are exposed as `written_rows` and `max_written_rows` in `system.quota_usage`, `system.quotas_usage`, and `system.quota_limits`, and the resource can also be set in `users.xml` as `<written_rows>`. [#118272](https://github.com/ClickHouse/ClickHouse/pull/118272) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the settings `max_temporary_tables`, `max_temporary_table_memory_usage`, `max_temporary_table_size_bytes_compressed`, and `max_temporary_table_size_bytes_uncompressed` to limit the number of temporary tables in a session, the memory usage of `Memory` temporary tables, and the size of `MergeTree` temporary tables. [#122165](https://github.com/ClickHouse/ClickHouse/pull/122165) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the SQL-standard `SIMILAR TO` pattern-matching predicate. [#101514](https://github.com/ClickHouse/ClickHouse/pull/101514) ([Zhao Zheguang](https://github.com/zheguang)).
* Allow `MATERIALIZED` common table expressions inside recursive `WITH`. A materialized helper CTE is evaluated once and every recursive step reads the snapshot, instead of the subquery being re-evaluated on each iteration. The recursive CTE itself still cannot be materialized. [#123452](https://github.com/ClickHouse/ClickHouse/pull/123452) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Query parameters can now be used for primary `USER` and `ROLE` names in `CREATE`, `ALTER`, and `DROP`, and for role and grantee names in `GRANT` and `REVOKE`, for example `GRANT {role:Identifier} TO {user:Identifier}`. [#109696](https://github.com/ClickHouse/ClickHouse/pull/109696) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)).
* `SET DEFAULT ROLE` now accepts an `ON CLUSTER` clause, in either position: `SET DEFAULT ROLE role1 TO user ON CLUSTER my_cluster` and `SET DEFAULT ROLE ON CLUSTER my_cluster role1 TO user`. `CURRENT_USER` is resolved on the initiator before the query is distributed. `SET ROLE` only affects the current session and still takes no `ON CLUSTER` clause. [#118429](https://github.com/ClickHouse/ClickHouse/pull/118429) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added `CREATE OR REPLACE NAMED COLLECTION ...` to replace an existing named collection entirely. [#121569](https://github.com/ClickHouse/ClickHouse/pull/121569) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)).
* The `postgresql` table function and the `PostgreSQL` table engine can now connect to another ClickHouse server over the PostgreSQL protocol. The `query(...)` variant still needs an explicit column list. Added the PostgreSQL-compatibility functions `format_type` and `current_setting`. Also fixed cancellation over the PostgreSQL protocol: a cancel request never cancelled anything. [#112940](https://github.com/ClickHouse/ClickHouse/pull/112940) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Disks with the `plain_rewritable` metadata (`s3_plain_rewritable`, and `azure`/`local` object storages with `metadata_type = plain_rewritable`) support hard links if configured with the new `enable_hard_links` setting, which enables `ALTER` operations and mutations on `MergeTree` tables stored on them. Mutations and partition operations share the unchanged blobs between parts instead of copying them. The setting is off by default, because a disk that has hard links cannot be read by older servers. [#110939](https://github.com/ClickHouse/ClickHouse/pull/110939) ([yanglongwei](https://github.com/ylw510)). [#119304](https://github.com/ClickHouse/ClickHouse/pull/119304) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added `EXCEPT DATA FROM TABLE` / `EXCEPT DATA FROM TABLES` clauses to `BACKUP` queries, so a table's definition can be backed up without its rows, for any table engine that supports backups. Formatting now quotes keyword table names correctly in `EXCEPT DATA FROM TABLES`, so `BACKUP ... ON CLUSTER` round-trips through the formatter. Closes [#111827](https://github.com/ClickHouse/ClickHouse/issues/111827). [#111895](https://github.com/ClickHouse/ClickHouse/pull/111895) ([Aditya Kumar](https://github.com/adityaksolves)). [#122468](https://github.com/ClickHouse/ClickHouse/pull/122468) ([David Meng](https://github.com/davidmenggx)).
* Added five columns to `system.query_log` describing what a query really executed: `used_number_of_joins`, `used_join_algorithms`, `used_join_kinds`, `used_join_strictness`, and `spilled_to_disk`. This makes it possible to find problematic `JOIN` patterns and queries that fall back to external memory across a fleet without reproducing each query. A spilling `SELECT DISTINCT` now records `distinct` in `spilled_to_disk` instead of `[]`, and a spilling block nested loop join records `join`. [#113833](https://github.com/ClickHouse/ClickHouse/pull/113833) ([Matheus Nerone](https://github.com/Manerone)). [#121830](https://github.com/ClickHouse/ClickHouse/pull/121830) ([Groene AI](https://github.com/groeneai)). [#124681](https://github.com/ClickHouse/ClickHouse/pull/124681) ([David Meng](https://github.com/davidmenggx)).
* Added table function `traceView` that renders an OpenTelemetry trace from `system.opentelemetry_span_log` as a call tree with a per-span timeline bar, for debugging where the time of a traced, for example distributed, query was spent. It has a `host` column with the `hostname` of the server that recorded each span, so the spans of the initiator, remote shards, replicas, and stateless workers can be distinguished and filtered with `WHERE host = ...`. [#116325](https://github.com/ClickHouse/ClickHouse/pull/116325) ([Diego Gomes Tomé](https://github.com/diegomestre2)). [#122169](https://github.com/ClickHouse/ClickHouse/pull/122169) ([Diego Gomes Tomé](https://github.com/diegomestre2)). [#123528](https://github.com/ClickHouse/ClickHouse/pull/123528) ([Diego Gomes Tomé](https://github.com/diegomestre2)).
* `clickhouse-client --login` can now authenticate against arbitrary OAuth/OIDC identity providers via endpoint discovery, with explicit endpoint and scope overrides and improved RFC 8628 device-code handling. On Linux desktops it now also passes the client's environment to `xdg-open` when opening the identity-provider login page, so the system browser is detected correctly. Closes [#120127](https://github.com/ClickHouse/ClickHouse/issues/120127). [#110718](https://github.com/ClickHouse/ClickHouse/pull/110718) ([yanglongwei](https://github.com/ylw510)). [#121337](https://github.com/ClickHouse/ClickHouse/pull/121337) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Added interactive client commands `/dialect`, `/lang`, and `/language` to `clickhouse-client` and `clickhouse-local`, so you can switch the active SQL dialect before parsing, including when the current dialect cannot express a `SET` query, for example to leave `kusto` or `promql`. The current non-default dialect is now shown in the prompt, and `SET dialect = ...` keeps the prompt in sync. [#114568](https://github.com/ClickHouse/ClickHouse/pull/114568) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `chdig` [v26.10.1](https://github.com/azat/chdig/releases/tag/v26.10.1): ZooKeeper browser, per-host summary, and better filters. [#124100](https://github.com/ClickHouse/ClickHouse/pull/124100) ([Azat Khuzhin](https://github.com/azat)).
* Support reading the `Parquet` `VARIANT` type (unshredded, as written for example by Spark 4.0) into `Dynamic`, with object values read as `JSON`. A variant column can also be read directly as `JSON` if all its values are objects. Shredded variants are not supported yet. [#117106](https://github.com/ClickHouse/ClickHouse/pull/117106) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* Support the `variant` type of `Iceberg`. Based on the `Parquet` variant reader in [#117106](https://github.com/ClickHouse/ClickHouse/pull/117106). [#117198](https://github.com/ClickHouse/ClickHouse/pull/117198) ([Konstantin Vedernikov](https://github.com/scanhex12)). [#125458](https://github.com/ClickHouse/ClickHouse/pull/125458) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Support the `Parquet` `FILE` logical type (a reference to bytes stored inline or in an external file): it is read as a named `Tuple`, and can be written from a `Tuple` with the corresponding element names with the new setting `output_format_parquet_write_file_logical_type`. [#125132](https://github.com/ClickHouse/ClickHouse/pull/125132) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Lazy type hints for the `JSON` type are now generally available under the `enable_json_lazy_type_hints` setting. The setting remains disabled by default. [#122711](https://github.com/ClickHouse/ClickHouse/pull/122711) ([Pavel Kruglov](https://github.com/Avogar)).
* Add a new input format `Freeform`, which does schema inference for arbitrary tabular text files. Closes [#34006](https://github.com/ClickHouse/ClickHouse/issues/34006). [#43449](https://github.com/ClickHouse/ClickHouse/pull/43449) ([Tri Dao](https://github.com/dmtri35)).
* The `generateRandom` table function and the `GenerateRandom` table engine now support `JSON`, `Dynamic`, `Variant`, `BFloat16`, `Time`, and `Time64`. Generated `JSON` documents have a realistic schema derived from the seed. The new settings `null_ratio`, `max_json_depth`, and `max_json_keys_per_object` control the share of nulls and absent keys, the nesting depth, and the number of keys per object. Generation respects memory limits. `CREATE TABLE ... AS generateRandom(...)` also keeps the `SETTINGS` argument in the stored definition. [#107966](https://github.com/ClickHouse/ClickHouse/pull/107966) ([apst41](https://github.com/apst41)). [#124772](https://github.com/ClickHouse/ClickHouse/pull/124772) ([Vadim Lopatin](https://github.com/buggins)). [#123247](https://github.com/ClickHouse/ClickHouse/pull/123247) ([Raúl Marín](https://github.com/Algunenano)).
* Add the `obfuscate` table function for generating anonymized query results. [#115004](https://github.com/ClickHouse/ClickHouse/pull/115004) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Text indexes with the `keyValuePairs` tokenizer now answer `map['key'] IN ('value1', 'value2')` directly from the index, as they already did for `map['key'] = 'value'`. Added the function `mapContainsKeyValue(map, key, value)`, which returns whether a map contains an entry with the given key and value; in contrast to `map[key] = value`, all entries of a repeated key are considered, and such an index evaluates the function directly. The `mergeTreeTextIndex` table function returns the decoded parts of each token of such an index in the columns `token_key` and `token_value`. [#120244](https://github.com/ClickHouse/ClickHouse/pull/120244) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#120010](https://github.com/ClickHouse/ClickHouse/pull/120010) ([Jimmy Aguilar Mena](https://github.com/Ergus)). [#120183](https://github.com/ClickHouse/ClickHouse/pull/120183) ([Anton Popov](https://github.com/CurtizJ)).
* The `mergeTreeTextIndex` table function now returns posting lists and token positions for a text index in the new virtual columns `row_id` and `positions`. If one of them is selected, the function returns a row per posting. The `positions` column is available for indexes created with `support_phrase_search = 1`. [#125536](https://github.com/ClickHouse/ClickHouse/pull/125536) ([Anton Popov](https://github.com/CurtizJ)).
* Added `mapRemove`, which removes all entries with a specified key from a map, and `mapEntries`, which returns a map's entries as an array of key-value tuples. [#123322](https://github.com/ClickHouse/ClickHouse/pull/123322) ([Minh Vu](https://github.com/fallintoplace)). [#123344](https://github.com/ClickHouse/ClickHouse/pull/123344) ([Minh Vu](https://github.com/fallintoplace)).
* The `-Map` combinator can now be used with non-`Map` columns (the key is passed as the last argument, e.g. `sumMap(value, key)`) and supports aggregate functions with multiple arguments, e.g. `argMaxMap(arg, val, key)`. [#123449](https://github.com/ClickHouse/ClickHouse/pull/123449) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The aggregate functions `avg`, `sum`, `sumWithOverflow`, and `sumKahan` now support the `Interval` data types. The result is an interval of the same unit. [#121458](https://github.com/ClickHouse/ClickHouse/pull/121458) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#121600](https://github.com/ClickHouse/ClickHouse/pull/121600) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The linear regression aggregate functions can now also be called under ClickHouse-style names: `regrSlope`, `regrIntercept`, `regrR2`, `regrCount`, `regrAvgX`, `regrAvgY`, `regrSXX`, `regrSYY`, and `regrSXY`. [#119796](https://github.com/ClickHouse/ClickHouse/pull/119796) ([mosya415](https://github.com/mosya415)).
* Added the `arrayMinIndex` and `arrayMaxIndex` array functions. [#120097](https://github.com/ClickHouse/ClickHouse/pull/120097) ([Minh Vu](https://github.com/fallintoplace)). [#123686](https://github.com/ClickHouse/ClickHouse/pull/123686) ([Minh Vu](https://github.com/fallintoplace)).
* Added the `arrayRemoveAt` array function. [#120445](https://github.com/ClickHouse/ClickHouse/pull/120445) ([Minh Vu](https://github.com/fallintoplace)).
* Added higher-order array functions to pack bits: `arrayPackBitsToUInt64`, `arrayPackBitsToString`, `arrayPackBitsToFixedString`, `arrayPackBitGroupsToUInt64`, `arrayPackBitGroupsToString`, and `arrayPackBitGroupsToFixedString`. [#123478](https://github.com/ClickHouse/ClickHouse/pull/123478) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added SQL functions `normalizedQueryHashUnordered` and `normalizedQueryHashUnorderedOrNull`. Unlike `normalizedQueryHash`, they ignore the order of expression lists such as the `SELECT` list, `GROUP BY` keys, and function arguments, so logically equivalent queries can collapse to one pattern in `query_log`; the `OrNull` variant returns `NULL` instead of throwing when the input is not valid ClickHouse SQL in the current session. [#116345](https://github.com/ClickHouse/ClickHouse/pull/116345) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Add functions `encodeQuotedPrintable` and `decodeQuotedPrintable` for Quoted-Printable encoding (RFC 2045), commonly used in email. [#125454](https://github.com/ClickHouse/ClickHouse/pull/125454) ([Jay Pokale](https://github.com/JayPokale)).
* Added the `sql_compatible_division_for_integers` setting, disabled by default, to make `/` and `divide` return integer quotients truncated toward zero when both operands are integers. It supports integer types up to 256 bits and checks division by zero and result overflow; division involving `Decimal` or floating-point operands is unchanged. [#124490](https://github.com/ClickHouse/ClickHouse/pull/124490) ([Ilya Yatsishin](https://github.com/qoega)).
* Added `SYSTEM RESET FILELOG [db.]table [FILE 'file_name' [OFFSET n | TO END]]`: a `FileLog` table reads all its files again from the beginning, or one file from the beginning, from a given offset, or only from data appended after the command, without recreating the table. Requires the new `SYSTEM RESET FILELOG` privilege. [#123545](https://github.com/ClickHouse/ClickHouse/pull/123545) ([Groene AI](https://github.com/groeneai)).
* The `FileLog` table engine now accepts globs (`*`, `?`, `{a,b}`, `{N..M}`) in the file name of its path, for example `FileLog('/var/lib/clickhouse/user_files/my_app/*.log', 'JSONEachRow')`, and reads only the matching files of that directory, so the compressed files `logrotate` leaves next to the log are not read. A file that is already read keeps being read after it is renamed to a non-matching name, such as `app.log` rotated to `app.log.1`. [#123590](https://github.com/ClickHouse/ClickHouse/pull/123590) ([Groene AI](https://github.com/groeneai)).
* Added the `flat_namespaces` setting to the `DataLakeCatalog` database engine. Set it for `Iceberg` REST catalogs that have only single-level namespaces and reject the `parent` query parameter of the list-namespaces endpoint, such as Apache Polaris federated to AWS Glue: only top-level namespaces are then listed and sub-namespaces are not requested. [#117329](https://github.com/ClickHouse/ClickHouse/pull/117329) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* Added the ClickHouse Keeper `ListWithOptions` operation for filtered direct or recursive listing with optional stats, data, limits, and shuffled results. [#117521](https://github.com/ClickHouse/ClickHouse/pull/117521) ([Antonio Andelic](https://github.com/antonio2368)).
* Added the `MergeTree` setting `min_unreserved_disk_space_for_merge`: background merge selection subtracts the specified amount from the unreserved disk space, so large merges stop being selected before they can take the last free space and starve inserts. Unlike `keep_free_space_bytes`, the protected space remains usable by ClickHouse itself. [#115723](https://github.com/ClickHouse/ClickHouse/pull/115723) ([Valery Petrov](https://github.com/valerypetrov)).
* You can now disable reading through the `executable` table function and from `Executable`/`ExecutablePool` tables with the new `allow_executable_tables` setting, for a deployment that must not run arbitrary server-side scripts. It defaults to `true`, so existing behavior is unchanged. [#120713](https://github.com/ClickHouse/ClickHouse/pull/120713) ([Hanzi Jiang](https://github.com/HanziJiang)).
* Added server setting `ai_function_allow_sql_named_collections`. When disabled, AI functions accept only named collections defined in the server configuration file. [#124999](https://github.com/ClickHouse/ClickHouse/pull/124999) ([George Larionov](https://github.com/george-larionov)).
* Added `<openSSL><server><cipherSuites>` and `<openSSL><client><cipherSuites>` settings to restrict which TLS 1.3 cipher suites a TLS port negotiates. Without them, `cipherList` controls only TLS 1.2 and below. An absent or empty value keeps the OpenSSL defaults. Closes [#119929](https://github.com/ClickHouse/ClickHouse/issues/119929). [#120480](https://github.com/ClickHouse/ClickHouse/pull/120480) ([Groene AI](https://github.com/groeneai)).
* You can now set a custom syslog program name for ClickHouse logs using `logger.syslog.programname`, making it easier to distinguish multiple ClickHouse instances in syslog. The default syslog identifier is now command-specific: remote syslog previously sent `-` as the RFC 5424 `APP-NAME` and now sends the command name (for example `clickhouse-server`), and the multi-purpose binary started as `clickhouse server` or `clickhouse keeper` now uses `clickhouse-server` / `clickhouse-keeper` instead of `clickhouse` for local syslog. [#98789](https://github.com/ClickHouse/ClickHouse/pull/98789) ([François Martin](https://github.com/martinfrancois)).
* Added the `Copy` event and the `source_bucket` and `source_remote_path` columns to `system.blob_storage_log` for server-side object copies (S3 `CopyObject` and `UploadPartCopy`, Azure Blob Storage copies). Azure backups, restores, and `AzureQueue` moves now write to `system.blob_storage_log` too. Closes [#123163](https://github.com/ClickHouse/ClickHouse/issues/123163). [#123178](https://github.com/ClickHouse/ClickHouse/pull/123178) ([Groene AI](https://github.com/groeneai)).
* Added an `is_lazy` column to `system.dictionaries`. It shows whether each dictionary loads on first use, taking into account both its `dictionary_lazy_load` setting and the server setting `dictionaries_lazy_load`. This makes it possible to tell a lazy dictionary that has not been used yet from an eager one that has not been loaded yet; both show `NOT_LOADED`. [#124260](https://github.com/ClickHouse/ClickHouse/pull/124260) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)).
* Added the `replica_group` column to `system.clusters`. For clusters that belong to a `Replicated` database, it shows the replica group of each node as published from the server configuration `replica_group_name`, or its alias `role`. [#120537](https://github.com/ClickHouse/ClickHouse/pull/120537) ([RinChanNOW](https://github.com/RinChanNOWWW)).
* Added `SYSTEM DISABLE ALL FAILPOINTS`, which disables every fail point at once and resumes any thread blocked on a pauseable one. [#118877](https://github.com/ClickHouse/ClickHouse/pull/118877) ([Alexey Milovidov](https://github.com/alexey-milovidov)).

#### Experimental Feature
* Added an experimental columns cache that keeps deserialized columns of wide `MergeTree` parts in memory, one entry per stripe of granules of a column. Compared with the uncompressed cache, this saves time on deserialization and copying. It is enabled by the experimental setting `use_columns_cache`, with `enable_reads_from_columns_cache` and `enable_writes_to_columns_cache` as beta sub-settings, and it applies to tables in databases that assign UUIDs (`Atomic` and `Replicated`). The cache gives its memory back to queries when the server is short of it. [#96844](https://github.com/ClickHouse/ClickHouse/pull/96844) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added experimental `LATERAL JOIN` support, allowing correlated subqueries in the `FROM` clause. This enables the standard SQL pattern for "top N per group" and per-row aggregation queries. Controlled by the `allow_experimental_lateral_join` setting. [#97833](https://github.com/ClickHouse/ClickHouse/pull/97833) ([Dimitri König](https://github.com/dimitri-koenig)).
* Added an experimental `logsql` dialect: LogsQL, the log query language of VictoriaLogs, can now be used to query ClickHouse tables with logs. Enable it with `SET enable_logsql_dialect = 1, logsql_table = '<table>', dialect = 'logsql'` (plus `logsql_database`, `logsql_time_column`, and `logsql_message_column` as needed). LogsQL queries are translated at parse time into ordinary `SELECT` queries, so text filters expect `String`-backed log columns, while the `math` pipe applies raw arithmetic and needs numeric operand columns (`String` field values are not coerced to numbers). [#113207](https://github.com/ClickHouse/ClickHouse/pull/113207) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#123891](https://github.com/ClickHouse/ClickHouse/pull/123891) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the beta `PCO` compression codec for fixed-width numeric columns, based on the [pcodec](https://github.com/pcodec/pcodec) library. Enable it with `enable_pco_codec = 1`. [#110613](https://github.com/ClickHouse/ClickHouse/pull/110613) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added a new implementation of the `unity` catalog type in the `DataLakeCatalog` database engine. It serves both `DeltaLake` and `Iceberg` tables from a single Unity Catalog, detecting the format of each table and routing `Iceberg` tables through the Unity Catalog `Iceberg` REST endpoint. Enable it with the database setting `use_unity_catalog_v2`, or set the session setting of the same name before `CREATE DATABASE`. Switch an existing database with `ALTER DATABASE ... MODIFY SETTING use_unity_catalog_v2 = 1`. [#115879](https://github.com/ClickHouse/ClickHouse/pull/115879) ([Asya Shneerson](https://github.com/asya-ch)).
* Added an experimental embedded `Iceberg` REST catalog server with ClickHouse HTTP authentication. It supports `GET /v1/config`, listing, creating, and checking the existence of namespaces, stored either in memory or in `Keeper`, and listing, creating, loading, checking the existence of, and dropping tables, with their metadata files stored in `S3`. The catalog can also serve TLS via `secure_port`, using the certificate and key in `openSSL.server`, and correctly handles `/` in table and namespace names. Closes [#125762](https://github.com/ClickHouse/ClickHouse/issues/125762). [#118924](https://github.com/ClickHouse/ClickHouse/pull/118924) ([Asya Shneerson](https://github.com/asya-ch)). [#123000](https://github.com/ClickHouse/ClickHouse/pull/123000) ([Asya Shneerson](https://github.com/asya-ch)). [#123331](https://github.com/ClickHouse/ClickHouse/pull/123331) ([Asya Shneerson](https://github.com/asya-ch)). [#124978](https://github.com/ClickHouse/ClickHouse/pull/124978) ([Asya Shneerson](https://github.com/asya-ch)). [#125791](https://github.com/ClickHouse/ClickHouse/pull/125791) ([Groene AI](https://github.com/groeneai)).
* Added experimental XGBoost dictionaries: the `XGBOOST` layout trains a gradient-boosted model from the source table at load time, and `predictXGBoost` runs inference. Enable it with `enable_xgboost`. Available only in builds that include the XGBoost contrib library. [#109999](https://github.com/ClickHouse/ClickHouse/pull/109999) ([Matheus Nerone](https://github.com/Manerone)).
* Added `ipcryptEncrypt`, `ipcryptDecrypt`, `ipcryptPrefixEncrypt` and `ipcryptPrefixDecrypt` functions for format-preserving encryption of IP addresses following the [IPCrypt specification](https://ipcrypt-std.github.io/). The deterministic mode produces consistent output for deduplication, while the prefix-preserving mode maintains subnet relationships. Enable with `SET enable_ipcrypt = 1`. Contributed by Frank Denis. [#123680](https://github.com/ClickHouse/ClickHouse/pull/123680) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added read support for `Iceberg` v3 deletion vectors stored in `Puffin` files. [#110781](https://github.com/ClickHouse/ClickHouse/pull/110781) ([linjiayu1025-collab](https://github.com/linjiayu1025-collab)).
* Added experimental support for `ALTER TABLE ... DROP PARTITION` on standalone `Iceberg` tables and `Iceberg` REST catalogs. On standalone tables, the operation currently requires `Iceberg` format-version 2 with a single non-evolved partition spec. Old date-transform spellings in the partition expression are now evaluated with the `Iceberg` transform semantics too. [#111219](https://github.com/ClickHouse/ClickHouse/pull/111219) ([Den Kalantaevskii](https://github.com/Diskein)). [#123273](https://github.com/ClickHouse/ClickHouse/pull/123273) ([Groene AI](https://github.com/groeneai)). [#122269](https://github.com/ClickHouse/ClickHouse/pull/122269) ([Den Kalantaevskii](https://github.com/Diskein)).
* `parallel_replicas_plan_based` can now distribute queries with an `IN (subquery)` predicate instead of falling back to single-node execution. Closes [#111876](https://github.com/ClickHouse/ClickHouse/issues/111876). [#114086](https://github.com/ClickHouse/ClickHouse/pull/114086) ([Igor Nikonov](https://github.com/devcrafter)).
* With `parallel_replicas_plan_based` enabled, distributed `INSERT ... SELECT` again uses distributed execution when `parallel_distributed_insert_select = 2`, instead of running the `SELECT` on the initiator alone. Closes [#118276](https://github.com/ClickHouse/ClickHouse/issues/118276). [#121097](https://github.com/ClickHouse/ClickHouse/pull/121097) ([Igor Nikonov](https://github.com/devcrafter)).
* Distributed query plans built with `make_distributed_plan` now compress exchanged data with `network_compression_method`; for `ZSTD`, they also use `network_zstd_compression_level` instead of the server's default codec settings. [#120333](https://github.com/ClickHouse/ClickHouse/pull/120333) ([Alexander Gololobov](https://github.com/davenger)).
* Queries executed with the experimental `make_distributed_plan` setting now propagate the OpenTelemetry trace context to stateless workers via the `traceparent` and `tracestate` HTTP headers of the task dispatch request. Each dispatched task produces a `StatelessWorkerClient::sendTask` span on the initiator plus `InterserverIOHTTPHandler` and `DistributedPlanTask::execute` spans on the worker, with the task id, initial query id, plan hash, and exchange streams as attributes, so the whole distributed plan is visible as one trace in `system.opentelemetry_span_log`. [#119010](https://github.com/ClickHouse/ClickHouse/pull/119010) ([Diego Gomes Tomé](https://github.com/diegomestre2)).
* Join runtime filters can now travel across distributed query plan stages with the experimental setting `distributed_plan_join_runtime_filters` (default off). Build-side tasks send partial filters to probe-side scans before their shuffle exchange, reducing the number of rows crossing the network. A filter that is lost, late, or never built cannot change results. [#114822](https://github.com/ClickHouse/ClickHouse/pull/114822) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)).
* Distributed index analysis is now beta. [#123577](https://github.com/ClickHouse/ClickHouse/pull/123577) ([Azat Khuzhin](https://github.com/azat)).
* Added the experimental setting `enable_join_seal_gated_reading` (default off). When enabled together with `enable_join_runtime_filters` and the probe-side join key is a prefix of the table's primary key, the probe side of a hash `JOIN` starts reading only after the build side completes, and the completed runtime filter prunes whole mark ranges by the primary key before read tasks are created, instead of only filtering rows after they were read. [#113512](https://github.com/ClickHouse/ClickHouse/pull/113512) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
* Replaced the experimental settings `query_plan_optimize_join_order_use_conflict_detector_a` and `query_plan_optimize_join_order_use_conflict_detector_c` with `query_plan_optimize_join_order_conflict_detector`, which takes `a`, `c`, or `''` (the default). [#120630](https://github.com/ClickHouse/ClickHouse/pull/120630) ([Fisnik Kastrati](https://github.com/fkastrati)).
* Adaptive codec selection (`enable_adaptive_codec_selection`) also tries each specialized codec followed by the table's default codec, and makes merges of `Float32` and `Float64` columns faster. [#121630](https://github.com/ClickHouse/ClickHouse/pull/121630) ([Raufs Dunamalijevs](https://github.com/rienath)). [#122248](https://github.com/ClickHouse/ClickHouse/pull/122248) ([Raufs Dunamalijevs](https://github.com/rienath)).
* Added the setting `json_extract_named_tuples_as_objects` (default off): when enabled, the `JSONExtract` functions fill named tuples from JSON objects only. By default a JSON array fills a named tuple positionally, so which element lands in which field depends on the order the fields were declared in. [#122637](https://github.com/ClickHouse/ClickHouse/pull/122637) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Experimental `UNIQUE KEY` SST probes now batch lookups with `MultiGet`, so absent keys can be filtered by the SST bloom filter. More unsupported `UNIQUE KEY` operations are now rejected early. [#119997](https://github.com/ClickHouse/ClickHouse/pull/119997) ([johnjing](https://github.com/JingYanchao)). [#123002](https://github.com/ClickHouse/ClickHouse/pull/123002) ([Murphy](https://github.com/murphy-4o)).
* The experimental reader executor (`use_reader_executor = 1`) now plans filesystem cache reads ahead over a window set by `reader_executor_plan_look_ahead` instead of re-checking the cache on every read window. It also no longer fails spuriously with `CANNOT_READ_ALL_DATA` when a concurrent download advances the filesystem cache during a cold read. [#115816](https://github.com/ClickHouse/ClickHouse/pull/115816) ([Sema Checherinda](https://github.com/CheSema)). [#122214](https://github.com/ClickHouse/ClickHouse/pull/122214) ([Sema Checherinda](https://github.com/CheSema)).
* Streaming queries now prefer a `commit_order` projection when one is available for the current read round. [#120420](https://github.com/ClickHouse/ClickHouse/pull/120420) ([Mikhail Artemenko](https://github.com/Michicosun)).
* PromQL again supports the range functions `present_over_time`, `absent_over_time`, `quantile_over_time`, and `predict_linear`, and now also supports `stddev_over_time`, `stdvar_over_time`, `first_over_time`, `ts_of_first_over_time`, `ts_of_last_over_time`, and `mad_over_time`. [#112355](https://github.com/ClickHouse/ClickHouse/pull/112355) ([Valery Petrov](https://github.com/valerypetrov)). [#120352](https://github.com/ClickHouse/ClickHouse/pull/120352) ([Vitaly Baranov](https://github.com/vitlibar)). [#121578](https://github.com/ClickHouse/ClickHouse/pull/121578) ([Vitaly Baranov](https://github.com/vitlibar)). [#121800](https://github.com/ClickHouse/ClickHouse/pull/121800) ([Vitaly Baranov](https://github.com/vitlibar)).
* Added PromQL functions `min_of` and `max_of` to return the smaller or larger of two scalar values. [#121250](https://github.com/ClickHouse/ClickHouse/pull/121250) ([Minh Vu](https://github.com/fallintoplace)).
* PromQL now implements `timestamp` for both direct selectors and general expressions, matching Prometheus 3.5.0 semantics. [#112356](https://github.com/ClickHouse/ClickHouse/pull/112356) ([Valery Petrov](https://github.com/valerypetrov)).
* Support standalone PromQL `step` and `range` scalar functions. The PromQL parser now also accepts bare calls to `start`, `end`, `sort_by_label`, `sort_by_label_desc`, `double_exponential_smoothing`, and `limit_ratio`, so these not yet implemented functions are rejected after parsing instead of failing with a parse error. [#121118](https://github.com/ClickHouse/ClickHouse/pull/121118) ([Minh Vu](https://github.com/fallintoplace)). [#123589](https://github.com/ClickHouse/ClickHouse/pull/123589) ([James Cunningham](https://github.com/JTCunning)).
* `prometheusQuery` now always returns `Float64` values and `DateTime64` timestamps with at least millisecond precision. The time zone of the returned timestamps comes from its `DateTime` or `DateTime64` arguments when they agree, and otherwise from the table. [#120285](https://github.com/ClickHouse/ClickHouse/pull/120285) ([Vitaly Baranov](https://github.com/vitlibar)).
* PromQL now treats Prometheus stale markers as absent in instant selectors and omits them from range selectors. [#121098](https://github.com/ClickHouse/ClickHouse/pull/121098) ([Minh Vu](https://github.com/fallintoplace)).
* PromQL now rejects queries that use the same label in both `on(...)` and `group_left(...)` or `group_right(...)`. [#114376](https://github.com/ClickHouse/ClickHouse/pull/114376) ([Minh Vu](https://github.com/fallintoplace)).
* PromQL now rejects vector selectors whose matchers can all match the empty string. [#121099](https://github.com/ClickHouse/ClickHouse/pull/121099) ([Minh Vu](https://github.com/fallintoplace)).
* PromQL's `stddev` and `stdvar` now return `NaN` instead of `0`. [#122743](https://github.com/ClickHouse/ClickHouse/pull/122743) ([James Cunningham](https://github.com/JTCunning)).
* PromQL now returns `NaN` from date functions when the sample is not finite. [#124300](https://github.com/ClickHouse/ClickHouse/pull/124300) ([James Cunningham](https://github.com/JTCunning)).
* PromQL no longer fails on empty joins. [#124362](https://github.com/ClickHouse/ClickHouse/pull/124362) ([James Cunningham](https://github.com/JTCunning)).
* Aligned PromQL string and comment parsing with Prometheus: quoted strings cannot contain a line feed, and a line comment can end at the end of the query without a space after `#`. Closes [#117812](https://github.com/ClickHouse/ClickHouse/issues/117812). [#124584](https://github.com/ClickHouse/ClickHouse/pull/124584) ([Vitaly Baranov](https://github.com/vitlibar)).
* PromQL range functions over scalars now handle windows with no samples without failing on `NULL`, including when used in nested range functions. [#124586](https://github.com/ClickHouse/ClickHouse/pull/124586) ([Vitaly Baranov](https://github.com/vitlibar)).
* Added deduplication caches for `TimeSeries` metric families and tags, and fixed stale cache entries when the pending rows are full. [#121915](https://github.com/ClickHouse/ClickHouse/pull/121915) ([Vitaly Baranov](https://github.com/vitlibar)). [#122771](https://github.com/ClickHouse/ClickHouse/pull/122771) ([Vitaly Baranov](https://github.com/vitlibar)).
* The `tags` inner table of the `TimeSeries` table engine is split into a `tags` table without `min_time` and `max_time` and a separate table of time ranges. [#121924](https://github.com/ClickHouse/ClickHouse/pull/121924) ([Vitaly Baranov](https://github.com/vitlibar)).
* The inner tables of the `TimeSeries` table engine now use the outer column name `metric_family` instead of `metric_family_name`. [#119226](https://github.com/ClickHouse/ClickHouse/pull/119226) ([Vitaly Baranov](https://github.com/vitlibar)).
* Fixed `ALIAS_REQUIRED` for a join with a `VIEW` written without an alias under the experimental setting `analyzer_inline_views = 1`: the inlined subquery now keeps the view's name as a table expression name, so it needs no alias and stays usable as a column qualifier, with or without the database name. [#118401](https://github.com/ClickHouse/ClickHouse/pull/118401) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reading subcolumns of the experimental `Quantized` codec (`vec.quantized`, `vec.product_quantization_codebook`) through a `Merge` table no longer breaks when another child table of the `Merge` lacks the base column. [#123895](https://github.com/ClickHouse/ClickHouse/pull/123895) ([Groene AI](https://github.com/groeneai)).

#### Performance Improvement
* Large hash tables used for `GROUP BY`, `uniqExact`, and `JOIN` now use up to two times less memory, and aggregation with many keys is faster on many threads. [#125001](https://github.com/ClickHouse/ClickHouse/pull/125001) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The query condition cache now serves conditions that involve the current time, e.g. `WHERE time >= now() - INTERVAL 10 DAY`, when reading from `MergeTree` tables, by deriving a deterministic condition from them with the current-time constant rounded onto a time grid. Closes [#115504](https://github.com/ClickHouse/ClickHouse/issues/115504). [#115522](https://github.com/ClickHouse/ClickHouse/pull/115522) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Enabled the setting `apply_string_filters_during_scan` by default. Substring search conditions on `String` columns (`LIKE`, `position`, `startsWith`, `endsWith`) from `PREWHERE` are pushed down into the column scan, and non-matching values are read as empty strings without copying their data. This makes such queries faster and lowers their memory usage when reading from `MergeTree` tables and from `Parquet` pages whose string values use the `PLAIN`, `DELTA_LENGTH_BYTE_ARRAY`, `DELTA_BYTE_ARRAY`, or dictionary encodings, including reads using the uncompressed cache on object storage. [#115848](https://github.com/ClickHouse/ClickHouse/pull/115848) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124503](https://github.com/ClickHouse/ClickHouse/pull/124503) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#125079](https://github.com/ClickHouse/ClickHouse/pull/125079) ([Groene AI](https://github.com/groeneai)). [#124507](https://github.com/ClickHouse/ClickHouse/pull/124507) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries `GROUP BY key LIMIT n` and `GROUP BY key ORDER BY key LIMIT n` over `MergeTree` tables now read much less data: the aggregation publishes the boundary of its top-K heap to the reading step, which drops rows beyond it before reading the other columns and can skip whole granules through the primary key or a `minmax` skip index. Controlled by the new setting `enable_group_by_top_k_dynamic_filtering`. The same primary-key granule skipping now also applies to `ORDER BY column LIMIT n` when the column is in the primary key. [#121145](https://github.com/ClickHouse/ClickHouse/pull/121145) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries with `ORDER BY <column> LIMIT n` over `Parquet` files now use TopN dynamic filtering with `use_top_k_dynamic_filtering`: rows that cannot enter the top-K are filtered inside the reader before the remaining columns are read, and whole row groups are skipped when their min/max statistics prove they cannot contribute. This works through the `file` and `url` table functions, the `File` and `URL` engines, object storage (`s3`, `Azure`, `HDFS`), and data lake tables such as `Iceberg`. Floating-point sort columns are excluded for now. [#115798](https://github.com/ClickHouse/ClickHouse/pull/115798) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122109](https://github.com/ClickHouse/ClickHouse/pull/122109) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124861](https://github.com/ClickHouse/ClickHouse/pull/124861) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `query_plan_join_shard_by_pk_ranges` is enabled by default. A `full_sorting_merge` or `parallel_full_sorting_merge` `JOIN` of two `MergeTree` tables whose join keys contain a prefix of the primary key of both tables now runs as independent streaming merge joins over primary-key ranges on all threads instead of a single-threaded merge join, several times faster and with several times less memory in our tests. Hash joins are affected only when both join runtime filters and spilling to disk are disabled. Set `query_plan_join_shard_by_pk_ranges = 0` to restore the previous behavior. [#117856](https://github.com/ClickHouse/ClickHouse/pull/117856) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119925](https://github.com/ClickHouse/ClickHouse/pull/119925) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Enabled the text-index postings cache by default with `use_text_index_postings_cache`. On a 1B-row table, a four-token phrase query that previously recomputed posting lists and phrase-search results within one query went from 19 s to 0.12 s. The shared cache is bounded by `text_index_postings_cache_size` (2 GiB by default). [#120677](https://github.com/ClickHouse/ClickHouse/pull/120677) ([Nikita Fomichev](https://github.com/fm4v)). [#123228](https://github.com/ClickHouse/ClickHouse/pull/123228) ([Anton Popov](https://github.com/CurtizJ)).
* With `optimize_aggregation_in_order = 1`, or with the default `optimize_distinct_in_order = 1`, a `GROUP BY` or `DISTINCT` whose keys extend past the table's sorting key no longer re-packs the whole block's grouping key once per run of equal sorting-key values. That cost was quadratic in `max_block_size` once the sorting prefix is near-unique, so `GROUP BY a, b` on a table sorted by `a` is up to 130x faster. Closes [#45386](https://github.com/ClickHouse/ClickHouse/issues/45386). [#120259](https://github.com/ClickHouse/ClickHouse/pull/120259) ([Groene AI](https://github.com/groeneai)).
* Blocks sent back by remote replicas are now marshalled in parallel (`enable_parallel_blocks_marshalling`) when `parallel_replicas_plan_based` is enabled. [#121642](https://github.com/ClickHouse/ClickHouse/pull/121642) ([Igor Nikonov](https://github.com/devcrafter)).
* `GROUP BY` with a `Nullable` key now uses the adaptive aggregator, except when that key is the only one and is a number of up to 8 bytes, a `String`, or a `FixedString`, or when any key is `LowCardinality`. Schema inference makes such keys common in CSV, JSON, and `Parquet` files; high-cardinality aggregations over them become up to 2x faster with up to 2.2x less memory. [#124868](https://github.com/ClickHouse/ClickHouse/pull/124868) ([Groene AI](https://github.com/groeneai)).
* Queries over `Nullable` columns now use the same optimizations as over non-`Nullable` ones: `length`, `empty`, `notEmpty`, and comparisons with an empty string on `Nullable(String)` read the `size` subcolumn, `sum(x + c)` is rewritten to `sum(x) + c * count(x)`, and `uniqExact` over `Nullable` is merged in parallel. Some queries become several times faster. [#125011](https://github.com/ClickHouse/ClickHouse/pull/125011) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The trivial `GROUP BY ... LIMIT n` optimization (setting `optimize_trivial_group_by_limit_query`) now also applies to queries with aggregate functions in the projection when the `GROUP BY` top-K optimization does not apply, for example above `query_plan_max_limit_for_top_k_optimization`: once any aggregation thread exceeds `n + offset` distinct keys, all threads are restricted to a single shared set of kept keys, so the hash tables stay small while the aggregate values of the returned keys remain exact, with up to 7-9x speedups. The optimization no longer fires for queries with `DISTINCT`, `QUALIFY`, `LIMIT AFTER`, `LIMIT UNTIL`, window functions, `arrayJoin`, or `exact_rows_before_limit`, where it could return wrong or inexact results. [#114690](https://github.com/ClickHouse/ClickHouse/pull/114690) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* When `max_threads` is not set explicitly, a query is planned with the fair share of CPU slots of the concurrency control instead of all CPU cores. This increases the throughput and lowers the memory usage under many concurrent queries, while a query on an idle server still uses all cores. It can be disabled with the setting `use_concurrency_control_for_max_threads`. [#125085](https://github.com/ClickHouse/ClickHouse/pull/125085) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A single-level `GROUP BY` result smaller than `max_block_size` is now split into several blocks when it was aggregated by several threads, so the steps after it, such as the probe of a following join, run in parallel instead of on one thread. Previously such queries could be several times slower, most visibly under `compatibility` older than 26.8, which disables `enable_parallel_single_level_merge` and `enable_adaptive_aggregator`. The result of a `GROUP BY` on a single 8- or 16-bit integer key is split the same way. [#121711](https://github.com/ClickHouse/ClickHouse/pull/121711) ([Pratham Nayak](https://github.com/protomn)).
* Restored the batched hash-and-prefetch insert path for `uniq`, `uniqCombined`, and `uniqCombined64` in ordinary `GROUP BY` queries, where they had silently fallen back to a per-row loop and were up to 46% slower. Closes [#120531](https://github.com/ClickHouse/ClickHouse/issues/120531). [#120578](https://github.com/ClickHouse/ClickHouse/pull/120578) ([Groene AI](https://github.com/groeneai)).
* Faster `GROUP BY` over keys that have to be serialized, such as several columns that include a `String`. [#122857](https://github.com/ClickHouse/ClickHouse/pull/122857) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)).
* `GROUP BY` over multiple `LowCardinality` keys now serializes the keys from the per-block dictionaries instead of copying them into full columns first, reducing CPU time and memory usage of aggregation queries over `LowCardinality` dimensions. [#121795](https://github.com/ClickHouse/ClickHouse/pull/121795) ([artemkulyk](https://github.com/artemkulyk)).
* Speed up `GROUP BY` over several `LowCardinality` keys when the same key tuples repeat, by caching aggregate states per tuple of dictionary positions in the serialized aggregation method. [#121998](https://github.com/ClickHouse/ClickHouse/pull/121998) ([artemkulyk](https://github.com/artemkulyk)).
* `GROUP BY` over a single `String` column with only `count` is faster when the grouping state does not fit in the CPU cache, especially with long strings. [#123655](https://github.com/ClickHouse/ClickHouse/pull/123655) ([Groene AI](https://github.com/groeneai)).
* `GROUP BY` queries with many groups that end in `ORDER BY count() ... LIMIT n` and use no other aggregate function are slightly faster when they group by a `String` column or by several columns. [#124277](https://github.com/ClickHouse/ClickHouse/pull/124277) ([Groene AI](https://github.com/groeneai)).
* Share the `GROUP BY` top-K skip boundary between aggregation threads. [#116150](https://github.com/ClickHouse/ClickHouse/pull/116150) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Allow using fused `GROUP BY` top-K with a serialized query plan. [#116163](https://github.com/ClickHouse/ClickHouse/pull/116163) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Added a query-plan optimization, controlled by the new setting `query_plan_aggregation_having_prefilter`, for `HAVING count() <comparison> <constant>` over a `GROUP BY`. While a two-level bucket of the aggregation result is converted to chunks, a group whose `count()` cannot satisfy the bound is now skipped before its key columns are materialized, instead of being materialized and then discarded by the filter above. This speeds up "groups above a threshold" queries over a high-cardinality `GROUP BY` with wide keys, where the discarded keys are most of the conversion. A skipped group is neither filtered nor finalized, so a `HAVING` conjunct written before the bound, and a sibling aggregate's finalization, stop being evaluated on the groups the bound rejects: a query that raised an exception from one of those can now return rows instead. Setting `query_plan_aggregation_having_prefilter = 0`, or `compatibility` to a version below `26.10`, keeps the previous behavior. [#120510](https://github.com/ClickHouse/ClickHouse/pull/120510) ([Groene AI](https://github.com/groeneai)).
* `optimize_aggregators_of_group_by_keys` now works for `LowCardinality` `GROUP BY` keys, so a `HAVING` predicate such as `HAVING min(s) = 'x'` is pushed down to storage and can use skip indexes, as for a plain key. It also eliminates `anyHeavy`, `quantileExact`, `quantileExactLow`, `quantileExactHigh`, `groupBitAnd`, and `groupBitOr` of a `GROUP BY` key. [#110059](https://github.com/ClickHouse/ClickHouse/pull/110059) ([Groene AI](https://github.com/groeneai)).
* `-If` aggregate functions with one `Nullable` argument no longer allocate a buffer the size of the whole block for every group with `optimize_aggregation_in_order` or for every window frame. With small groups or frames this makes them 1.7 to 3.1 times faster on x86_64. [#122505](https://github.com/ClickHouse/ClickHouse/pull/122505) ([Groene AI](https://github.com/groeneai)).
* Speed up `-ForEach` aggregate functions such as `sumForEach` and `avgForEach`: rows are processed in batches instead of calling `add` for each row, and an array of two or more elements is passed to the nested function in one call instead of one call per element. Grouped multi-argument `-If` aggregate functions with a `Nullable` argument or condition are also processed in batches. [#121137](https://github.com/ClickHouse/ClickHouse/pull/121137) ([Valery Petrov](https://github.com/valerypetrov)). [#124207](https://github.com/ClickHouse/ClickHouse/pull/124207) ([Groene AI](https://github.com/groeneai)).
* Speed up serialization of `AggregateFunction` state columns with a fixed-size state, such as `sum`, `count`, `avg`, `min`, `max`, and `any`, and their `-If`, `-Array`, `-Merge`, and `Nullable` variants. This affects writes to `AggregatingMergeTree`, the `Native` format, distributed queries, and spilling aggregate states to disk. [#113962](https://github.com/ClickHouse/ClickHouse/pull/113962) ([Manuel](https://github.com/raimannma)).
* Speed up reading and merging of `uniq` aggregate function states, such as `uniqMerge` over an `AggregatingMergeTree`, by 10-35% depending on state sizes and merge shape. [#111493](https://github.com/ClickHouse/ClickHouse/pull/111493) ([Manuel](https://github.com/raimannma)).
* Faster merging of `uniqCombined` and `uniqCombined64` states with many distinct values into empty states, e.g. in `-Merge` combinators and `AggregatingMergeTree`: up to 6x faster. [#124389](https://github.com/ClickHouse/ClickHouse/pull/124389) ([Manuel](https://github.com/raimannma)).
* Reduce memory usage and improve performance of external `DISTINCT` by avoiding spills for duplicate-heavy input, writing smaller spill blocks, budgeting tail spilling before the final merge, preserving constant payloads, combining chunks before sorting, and retaining fewer rows while restoring order with `LIMIT`. [#121772](https://github.com/ClickHouse/ClickHouse/pull/121772) ([Nihal Z. Miaji](https://github.com/nihalzp)).
* Faster sorted-stream processing on a multi-column key whose leading column has long runs, in `LIMIT BY`, negative `LIMIT BY`, `DISTINCT` in order, `full_sorting_merge` and `partial_merge` `JOIN`, and `ReplacingMergeTree` `FINAL` without vertical `FINAL`: the run of a key prefix is now found once instead of again for every group in it. The query from [#123700](https://github.com/ClickHouse/ClickHouse/issues/123700) is about 3.5x faster. [#123808](https://github.com/ClickHouse/ClickHouse/pull/123808) ([Groene AI](https://github.com/groeneai)).
* Short queries with many keyed aggregations or scalar subqueries over large constant-folded literals are faster because hash-table-stats cache keys no longer hash whole variable-size constant values. [#122738](https://github.com/ClickHouse/ClickHouse/pull/122738) ([Nikita Taranov](https://github.com/nickitat)).
* Reduce memory usage of some window functions by precomputing peer-group boundaries instead of keeping earlier blocks in memory. [#123737](https://github.com/ClickHouse/ClickHouse/pull/123737) ([Mikhail Artemenko](https://github.com/Michicosun)).
* A filter on the `PARTITION BY` columns of a window function is no longer pushed below the window (this reverts [#110114](https://github.com/ClickHouse/ClickHouse/pull/110114)). The push-down made queries up to 8 times slower when the window reads a `JOIN` and the filter removes every row, for example `IN` over an empty subquery, because the join built its right side before the filter could stop the query. Closes [#123875](https://github.com/ClickHouse/ClickHouse/issues/123875). [#125463](https://github.com/ClickHouse/ClickHouse/pull/125463) ([Groene AI](https://github.com/groeneai)).
* `hash` and `parallel_hash` joins now copy right-side output columns directly from the stored blocks instead of calling the stored column object once per row. This speeds up every column type, including `String`, `Array`, `Tuple`, `Nullable`, `Variant`, `Map`, `LowCardinality`, `JSON`, `Dynamic`, `AggregateFunction`, and `QBit`. [#117460](https://github.com/ClickHouse/ClickHouse/pull/117460) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)).
* Speed up hash join output construction in the row-major layout: output reconstruction is up to 2x faster for joins with many rows per key and about 1.5x for wide or `Nullable` payloads. The row-major layout is now also skipped for payloads too narrow to benefit from it. [#121981](https://github.com/ClickHouse/ClickHouse/pull/121981) ([Hechem Selmi](https://github.com/m-selmi)).
* Hash joins with many matches per key are faster when they output a `LowCardinality` column whose blocks have different dictionaries (several parts, `FINAL`, or a `LowCardinality` value computed per block): short ranges from another dictionary are now inserted key by key instead of through temporary columns. Such joins run 4-9x faster, at the 26.6 level or better. [#123900](https://github.com/ClickHouse/ClickHouse/pull/123900) ([Groene AI](https://github.com/groeneai)).
* `RIGHT` and `FULL` joins using `hash`, `parallel_hash`, or `grace_hash` skip the search for unmatched right rows when every right key has matched. With 50 million right keys, that search was about 10% of query time for `hash` (356 ms of 3.6 s) and about 15% for `parallel_hash` (106 ms of 0.7 s). [#124775](https://github.com/ClickHouse/ClickHouse/pull/124775) ([Groene AI](https://github.com/groeneai)).
* A `full_sorting_merge` join now advertises that its result is ordered by the join key, so a sort above it, the pre-sort of the next merge join in a chain or an `ORDER BY`, merges the already sorted streams instead of sorting from scratch. Joining several tables on the same key with `full_sorting_merge` no longer performs one full sort per join. Also fixed the interplay with `query_plan_join_shard_by_pk_ranges`, which could otherwise fail with a logical error or return wrong results for such a chain. [#117786](https://github.com/ClickHouse/ClickHouse/pull/117786) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A join step that has a condition of its own now also joins on equalities implied through other tables, so for `a.x = b.x AND b.x = c.x`, joining `a` with `c` also uses `a.x = c.x`. With `query_plan_optimize_join_order_algorithm = 'dpsize'`, this makes TPC-H Q02 join its correlated subquery to `partsupp` on the part key instead of on the supply cost alone. Closes [#122396](https://github.com/ClickHouse/ClickHouse/issues/122396). [#122445](https://github.com/ClickHouse/ClickHouse/pull/122445) ([Groene AI](https://github.com/groeneai)).
* A `WHERE` condition that no row satisfies is pushed to every `JOIN` input again, including the non-preserved side of an `OUTER JOIN` and a side with no column used above the join, so that input is no longer read and hash-built for nothing. For example, a `UNION ALL` branch whose constant column fails the outer `WHERE` is skipped as in 26.6. Closes [#123702](https://github.com/ClickHouse/ClickHouse/issues/123702). Closes [#123878](https://github.com/ClickHouse/ClickHouse/issues/123878). [#123829](https://github.com/ClickHouse/ClickHouse/pull/123829) ([Groene AI](https://github.com/groeneai)).
* Constant comparison, logical, and null (`IS NULL` / `IS NOT NULL`) filters over materialized constants are now folded and pushed down, reducing unnecessary reads in `WHERE`, `PREWHERE`, and `UNION` branches. Always-true `WHERE` filters are replaced with expressions so join reordering still sees all joined tables as one graph. [#105987](https://github.com/ClickHouse/ClickHouse/pull/105987) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#124056](https://github.com/ClickHouse/ClickHouse/pull/124056) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* `OUTER JOIN` queries can now be faster: when a `WHERE` clause or another `JOIN` later in the query would discard the unmatched rows anyway, ClickHouse converts the join to an `INNER JOIN`. The new setting `query_plan_convert_outer_join_to_inner_join_transitively` controls this, and only takes effect when `query_plan_convert_outer_join_to_inner_join` is enabled. [#120253](https://github.com/ClickHouse/ClickHouse/pull/120253) ([Hechem Selmi](https://github.com/m-selmi)).
* Restored filter push-down over `ANY INNER JOIN`, so primary-key pruning works again and such queries no longer read much more data than necessary. Because `ANY` keeps an arbitrary matching row, `SELECT ... ANY INNER JOIN ... WHERE ...` can now return a different matching row than before. [#120765](https://github.com/ClickHouse/ClickHouse/pull/120765) ([Kirill Shokhin](https://github.com/kirillshokhin)).
* `optimize_and_compare_chain` again derives transitive conditions through a `DateTime` or lower-scale `DateTime64` constant compared with a `DateTime64` expression, as it did before 26.8. For example, `t1.ts < toDateTime('2026-09-16 00:00:00') AND t1.ts >= t2.ts` adds `t2.ts < toDateTime('2026-09-16 00:00:00')`, so the other side of a join can be pruned by its partition and primary key instead of being read in full. Closes [#123706](https://github.com/ClickHouse/ClickHouse/issues/123706). [#123796](https://github.com/ClickHouse/ClickHouse/pull/123796) ([Groene AI](https://github.com/groeneai)).
* Improve join order selection when column statistics are missing: an equi join without NDV statistics is now treated more like a foreign-key-to-primary-key join instead of a cross product, and the greedy join-order algorithm prefers equi-connected pairs over pairs linked only by non-equi predicates. [#120176](https://github.com/ClickHouse/ClickHouse/pull/120176) ([Vladimir Cherkasov](https://github.com/vdimir)). [#121075](https://github.com/ClickHouse/ClickHouse/pull/121075) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Improves `JOIN` cardinality estimation by propagating join-key distinct-value estimates according to join semantics. Outer joins retain the preserved-side estimate, anti joins keep their input estimates, and semi joins narrow only the output side, preventing underestimation that could lead to suboptimal query plans. [#122305](https://github.com/ClickHouse/ClickHouse/pull/122305) ([Christoph Viebig](https://github.com/cv4g)).
* `IEJoin` now treats opposite-direction conditions on unrelated columns as independent conditions instead of as a band, which avoids choosing a join-key pair that passes thousands of times more rows to the residual condition. [#120213](https://github.com/ClickHouse/ClickHouse/pull/120213) ([Vladimir Cherkasov](https://github.com/vdimir)).
* Function `firstNonDefault`, which computes the `USING` columns of `FULL JOIN`, now copies whole ranges of rows instead of making two virtual calls per row. Queries that use a `USING` column of a `FULL JOIN` after the join, for example in `WHERE`, are faster. [#123302](https://github.com/ClickHouse/ClickHouse/pull/123302) ([Groene AI](https://github.com/groeneai)).
* Enabled `enable_join_runtime_filters_index_analysis` by default, so join runtime filters can prune granules during the probe phase. [#121859](https://github.com/ClickHouse/ClickHouse/pull/121859) ([Robert Schulze](https://github.com/rschu1ze)).
* Join runtime filters now run after the query's own `PREWHERE` conditions also for tables without column statistics, as they already did with statistics. Before, a runtime filter on a well-compressed join key could run first and read that key for every row, so enabling `enable_join_runtime_filters` could make a query read several times more data than without it. [#122452](https://github.com/ClickHouse/ClickHouse/pull/122452) ([Groene AI](https://github.com/groeneai)).
* The granule pruning of `enable_join_runtime_filters_index_analysis` now also applies when the left side of a `JOIN` is read from a normal projection. The setting description states that the pruning does not happen with parallel replicas, a distributed query plan, or `FINAL`. [#119041](https://github.com/ClickHouse/ClickHouse/pull/119041) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The query condition cache now records the static `WHERE` conditions of a join whose `PREWHERE` also holds a join runtime filter, so a repeated join skips the granules those conditions exclude instead of reading them again. This helps whenever the runtime filter cannot skip those granules through an index itself, for example when no primary key or skip index covers the join key, or when `enable_join_runtime_filters_index_analysis` is disabled. [#122380](https://github.com/ClickHouse/ClickHouse/pull/122380) ([Groene AI](https://github.com/groeneai)).
* Repeated insertion of empty and single-element `Array` values into in-memory columns is now faster. This happens, for example, during joins. [#117321](https://github.com/ClickHouse/ClickHouse/pull/117321) ([Minh Vu](https://github.com/fallintoplace)).
* With `use_top_k_dynamic_filtering = 1` (the default), a `SELECT ... ORDER BY ... LIMIT n` could read far more data than with the setting off: the internal `__topKFilter` took over the read's `PREWHERE` slot and displaced the query's own condition, and an explicit `PREWHERE` disabled the feature outright. The threshold filter now shares `PREWHERE` with the other conditions and takes the first slot. It is also no longer installed on a read that a sorting projection already delivers in `ORDER BY` order, where it forced a full scan. [#117968](https://github.com/ClickHouse/ClickHouse/pull/117968) ([Groene AI](https://github.com/groeneai)).
* Granules fully filtered by the dynamic threshold of an `ORDER BY ... LIMIT n` read are now recorded in the query condition cache, so repeat runs of such queries skip them at the mark-selection stage instead of re-reading and re-filtering most of the table. On the ClickBench Q24/Q26 shapes (`SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime LIMIT 10`), a warm run drops from about 95 million rows / 12000 granules read to about 1.3 million rows / 160 granules, and from about 15-30 ms to about 8 ms on a 96-core machine. This now also works for `ORDER BY ... LIMIT n` queries over `Parquet` files of `File` tables, with or without a `WHERE` clause: a repeated query reads only the row groups which can hold rows of its result. [#114659](https://github.com/ClickHouse/ClickHouse/pull/114659) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122064](https://github.com/ClickHouse/ClickHouse/pull/122064) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Improve performance of `ORDER BY ... LIMIT` queries that use minmax-index top-K pruning (`use_skip_indexes_for_top_k`) on large tables with many granules. [#116512](https://github.com/ClickHouse/ClickHouse/pull/116512) ([Anton Popov](https://github.com/CurtizJ)).
* Queries with `FINAL` that sort in reverse order of the sorting key, such as `SELECT ... FROM t FINAL ORDER BY key DESC LIMIT n`, now read in order through `Merge` tables over `ReplacingMergeTree` tables too, and the `topKThroughJoin` optimization defers to reverse-order reads below a `LEFT JOIN` when the table supports them. This extends [#115345](https://github.com/ClickHouse/ClickHouse/pull/115345). [#118008](https://github.com/ClickHouse/ClickHouse/pull/118008) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a performance regression in `ORDER BY` merge sort: queries with a sort followed by `LIMIT` no longer do redundant per-output-block work in result-block reservation. [#108179](https://github.com/ClickHouse/ClickHouse/pull/108179) ([Groene AI](https://github.com/groeneai)).
* Setting `optimize_move_to_prewhere_if_final` is now enabled by default, so queries with `FINAL` benefit from the automatic `PREWHERE` optimization. Only deterministic conditions over the sorting key are moved, and query results are not affected. [#121350](https://github.com/ClickHouse/ClickHouse/pull/121350) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* With short-circuit evaluation, non-throwing arguments of `and` and `or` are now ordered by measured cost and selectivity, so cheap filters can avoid evaluating expensive expressions on excluded rows. Controlled by `short_circuit_function_evaluation_reorder_arguments`, enabled by default. [#122508](https://github.com/ClickHouse/ClickHouse/pull/122508) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix a performance regression in short-circuit evaluation of `and` and `or` by avoiding expansion of intermediate results that are only used to update the logical mask. [#120520](https://github.com/ClickHouse/ClickHouse/pull/120520) ([LB7666](https://github.com/acking-you)).
* On a table with no applicable data skipping index, a query no longer stores the same predicate under two query condition cache keys. This halves the number of `O(marks per part)` cache entries such a query allocates, and removes one cache lookup per part. Queries with `distributed_index_analysis` enabled keep both keys. [#116259](https://github.com/ClickHouse/ClickHouse/pull/116259) ([Groene AI](https://github.com/groeneai)).
* `SELECT ... FINAL` with a selective skip-index plan is much faster on tables with many granules: the `use_skip_indexes_if_final_exact_mode` recovery pass now iterates ranges instead of individual granules. [#121799](https://github.com/ClickHouse/ClickHouse/pull/121799) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
* Improve primary index analysis in queries that filter both by the primary key and by other columns, for example `WHERE timestamp BETWEEN ... AND level = 'error'` where `level` is not in the primary key. [#123535](https://github.com/ClickHouse/ClickHouse/pull/123535) ([Anton Popov](https://github.com/CurtizJ)).
* `IS NULL` and `IS NOT NULL` predicates can now prune `MergeTree` parts using the `NULL` row count tracked by `basic` statistics on `Nullable` columns. [#116697](https://github.com/ClickHouse/ClickHouse/pull/116697) ([zoomxi](https://github.com/zoomxi)).
* A `set` skip index on a `Nullable` column again skips granules whose stored set contains `NULL` but not the searched value for `IN` and `=` conditions, restoring the pruning lost in 26.8. Closes [#123705](https://github.com/ClickHouse/ClickHouse/issues/123705). [#123778](https://github.com/ClickHouse/ClickHouse/pull/123778) ([Groene AI](https://github.com/groeneai)).
* Skip indexes (`text`, `bloom_filter`, `tokenbf_v1`, `ngrambf_v1`, `minmax`) and the primary key are now used when the indexed column or the searched constant is wrapped in a conversion that does not change its value, such as `toLowCardinality`, `toNullable`, or a `CAST` to `Nullable` or `LowCardinality`. For example, `m['key'] = 'value'` on `Map(String, LowCardinality(String))` and `hasToken(s, toLowCardinality('x'))` no longer scan all granules. `bloom_filter` and text indexes also work when a `Nullable` indexed column is cast to the non-`Nullable` type, such as `CAST(x, 'String')` or `json.path::String`. [#120220](https://github.com/ClickHouse/ClickHouse/pull/120220) ([Anton Popov](https://github.com/CurtizJ)). [#123239](https://github.com/ClickHouse/ClickHouse/pull/123239) ([Anton Popov](https://github.com/CurtizJ)). [#124220](https://github.com/ClickHouse/ClickHouse/pull/124220) ([Anton Popov](https://github.com/CurtizJ)).
* Speed up planning of queries with skip indexes on tables with many parts and pending patch parts of lightweight updates. [#124931](https://github.com/ClickHouse/ClickHouse/pull/124931) ([Anton Popov](https://github.com/CurtizJ)).
* Speed up queries with multiple `PREWHERE` conditions on AMD Zen 1 and Zen 2 CPUs. [#120283](https://github.com/ClickHouse/ClickHouse/pull/120283) ([Manuel](https://github.com/raimannma)).
* Analysis of a query over a `Merge` table no longer traverses every child table for each query-tree node when `optimize_functions_to_subcolumns` is enabled. The answer is now reused across the query tree, so analysis time grows with the query size plus the number of child tables instead of their product. Closes [#113202](https://github.com/ClickHouse/ClickHouse/issues/113202). [#113367](https://github.com/ClickHouse/ClickHouse/pull/113367) ([Groene AI](https://github.com/groeneai)).
* Queries to the `merge` table function and creation of `Merge` tables are faster when the source tables have many columns: combining the source columns no longer takes quadratic time in the number of columns. [#124115](https://github.com/ClickHouse/ClickHouse/pull/124115) ([Groene AI](https://github.com/groeneai)).
* Faster analysis of queries to tables with many `ALIAS` columns, such as `system.metric_log`: the expression of an `ALIAS` column is analyzed only if the query uses that column. [#123926](https://github.com/ClickHouse/ClickHouse/pull/123926) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reduce aggregate projection analysis overhead for tables with many subcolumns. [#110979](https://github.com/ClickHouse/ClickHouse/pull/110979) ([Rory Shanks](https://github.com/rorylshanks)).
* A condition such as `name IN ('a', 'b')` on the system tables that enumerate tables (`system.tables`, `system.columns`, `system.parts`, `system.replicas`, and others) now narrows the enumeration instead of walking every table of every database, the same way an equality condition already did. This is most visible for databases whose table list comes from a remote system, such as `PostgreSQL`, `SQLite`, and the DataLake catalogs. [#117589](https://github.com/ClickHouse/ClickHouse/pull/117589) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the `external_storage_push_down_limit` setting to reduce data read from external databases for safe `LIMIT` queries. Also fixed wrong results when `LIMIT` plus `OFFSET` does not fit in `UInt64`: `numbers`, `system.numbers`, `generate_series`, `primes`, `system.primes`, and the `PostgreSQL` table engine returned no rows for the common "no limit" idiom `LIMIT 18446744073709551615 OFFSET n`, and a `LIMIT` over a `JOIN` with `numbers`-like sources returned too few rows. [#115238](https://github.com/ClickHouse/ClickHouse/pull/115238) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118131](https://github.com/ClickHouse/ClickHouse/pull/118131) ([Groene AI](https://github.com/groeneai)).
* Push filters into the query fragment executed by parallel replicas so replicas can filter rows and prune data before sending results to the initiator. This also allows join runtime filters to reach the relevant reads. Controlled by `parallel_replicas_filter_pushdown`, now enabled by default; `SET parallel_replicas_filter_pushdown = 0` restores the behavior before 26.10. [#117373](https://github.com/ClickHouse/ClickHouse/pull/117373) ([Nikita Taranov](https://github.com/nickitat)).
* With parallel replicas, `optimize_aggregation_in_order` no longer disables projections: aggregate and normal projections, and `_minmax_count_projection` for `count()` with a partition predicate, are used as with `optimize_aggregation_in_order = 0`. Closes [#123061](https://github.com/ClickHouse/ClickHouse/issues/123061). [#123098](https://github.com/ClickHouse/ClickHouse/pull/123098) ([Groene AI](https://github.com/groeneai)).
* Parallel replicas now apply `enable_join_runtime_filters_index_analysis` to prune probe-side granules locally on each replica, reducing `JOIN` read work. [#123116](https://github.com/ClickHouse/ClickHouse/pull/123116) ([Nikita Taranov](https://github.com/nickitat)).
* Automatic parallel replicas no longer distributes a query when most of its reading would be repeated in full by every replica rather than split between them. The limit is set by the new setting `automatic_parallel_replicas_max_duplicated_read_ratio` (default `0.5`; `1` disables it). [#123332](https://github.com/ClickHouse/ClickHouse/pull/123332) ([Nikita Taranov](https://github.com/nickitat)).
* Automatic parallel replicas for multi-read queries now reuse local plan index analysis in the parallel-replicas plan, reducing extra reads when parallel replicas are enabled. [#115153](https://github.com/ClickHouse/ClickHouse/pull/115153) ([Nikita Taranov](https://github.com/nickitat)).
* Improve reuse of collected query statistics by automatic parallel replicas for equivalent local and distributed plans. [#116781](https://github.com/ClickHouse/ClickHouse/pull/116781) ([Nikita Taranov](https://github.com/nickitat)).
* Automatic parallel replicas now account correctly for the size of aggregated results on more aggregation paths, so the cost model no longer treats shipping aggregate states as free and is less likely to choose parallel replicas when a query is cheaper locally. [#119499](https://github.com/ClickHouse/ClickHouse/pull/119499) ([Nikita Taranov](https://github.com/nickitat)).
* Automatic parallel replicas now collect planning statistics for the table read that is actually split between replicas, including after join reordering and with lazy materialization, improving the decision to distribute a query. [#122906](https://github.com/ClickHouse/ClickHouse/pull/122906) ([Nikita Taranov](https://github.com/nickitat)).
* Avoid building the parallel replicas candidate plan when cached statistics already show the query should stay on a single replica. [#124983](https://github.com/ClickHouse/ClickHouse/pull/124983) ([Nikita Taranov](https://github.com/nickitat)).
* Added quick checks to skip automatic parallel replicas optimization for queries unsupported by parallel replicas, including keyless aggregations whose aggregate state owns memory and therefore never produces the probe statistics the optimization needs. [#120861](https://github.com/ClickHouse/ClickHouse/pull/120861) ([Nikita Taranov](https://github.com/nickitat)). [#125470](https://github.com/ClickHouse/ClickHouse/pull/125470) ([Nikita Taranov](https://github.com/nickitat)).
* Automatic parallel replicas (`automatic_parallel_replicas_mode`) can now be applied to queries with `ORDER BY ... WITH FILL`. [#123929](https://github.com/ClickHouse/ClickHouse/pull/123929) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries that search for several tokens at once with a text index, such as `hasAllTokens`, are now faster. The automatic choice of algorithm for intersecting posting lists is improved too, and is controlled by the new setting `text_index_postings_intersection_algorithm`. [#121038](https://github.com/ClickHouse/ClickHouse/pull/121038) ([Anton Popov](https://github.com/CurtizJ)).
* Queries that use a text index read large posting lists faster and with fewer disk operations. The improvement is most noticeable for searches for frequent tokens in big tables. [#121040](https://github.com/ClickHouse/ClickHouse/pull/121040) ([Anton Popov](https://github.com/CurtizJ)).
* A query that searches a text index and also selects a column no index covers no longer reads that column for every row of every granule the index selected. On a 1M-row table with a 1055-byte payload and 64 matching rows, `SELECT ts, payload FROM t WHERE hasToken(message, 'needle')` now reads 578 KiB instead of 532 MiB. [#120517](https://github.com/ClickHouse/ClickHouse/pull/120517) ([Groene AI](https://github.com/groeneai)).
* A query that reads a `text` index no longer asks the filesystem for each index substream's on-disk name and size once per part; both are now taken from the part's checksums, which already hold them. On an 8M-row table split into 400 parts, `SELECT count() FROM logs WHERE hasToken(message, 'needle')` drops from a median 12 ms to 9 ms, with 42% less system time. [#120875](https://github.com/ClickHouse/ClickHouse/pull/120875) ([Groene AI](https://github.com/groeneai)).
* Speed up queries that filter by a text index together with other `PREWHERE` conditions, such as the dynamic filter of `ORDER BY ... LIMIT` queries. [#124880](https://github.com/ClickHouse/ClickHouse/pull/124880) ([Anton Popov](https://github.com/CurtizJ)).
* Speed up `LIKE`, `ILIKE`, `startsWith`, and `endsWith` evaluated by a text-index dictionary scan (`use_text_index_like_evaluation_by_dictionary_scan`): the scan now seeks the sorted dictionary to the block range an anchored pattern can match, and tests a block's tokens for the pattern's mandatory literal in one pass instead of running the matcher on every token. [#120495](https://github.com/ClickHouse/ClickHouse/pull/120495) ([Groene AI](https://github.com/groeneai)).
* Repeated broad `LIKE` and `ILIKE` queries over a text index no longer rescan the index dictionary of every part only to abandon the scan again: the decision is cached (setting `use_text_index_pattern_bypass_cache`). [#113572](https://github.com/ClickHouse/ClickHouse/pull/113572) ([Rory Shanks](https://github.com/rorylshanks)).
* A text index with the `splitByNonAlpha` or `array` tokenizer now answers single-needle `multiSearchAny`, `multiSearchAnyUTF8`, `multiSearchAnyCaseInsensitive`, and `multiSearchAnyCaseInsensitiveUTF8` with the same dictionary scan that `LIKE '%needle%'` uses, so a needle inside a word, such as `PointerExc` in `NullPointerException`, no longer reads the whole column. [#124520](https://github.com/ClickHouse/ClickHouse/pull/124520) ([Groene AI](https://github.com/groeneai)).
* `multiSearchAny` over a map element can now use a `mapValues(...)` ngram or token text index instead of being rejected by index analysis. [#121787](https://github.com/ClickHouse/ClickHouse/pull/121787) ([Groene AI](https://github.com/groeneai)).
* Optimize `SELECT count()` queries answered from a text index with a compressed posting list codec (`bitpacking`, `pfor`). [#123294](https://github.com/ClickHouse/ClickHouse/pull/123294) ([Anton Popov](https://github.com/CurtizJ)).
* Reduced the CPU overhead of reading and intersecting text index posting lists in lazy apply mode. [#123786](https://github.com/ClickHouse/ClickHouse/pull/123786) ([Anton Popov](https://github.com/CurtizJ)). [#123873](https://github.com/ClickHouse/ClickHouse/pull/123873) ([Anton Popov](https://github.com/CurtizJ)).
* Merges of tables with a `text` index are now faster. Pages of a source part's row ids that hold consecutive values are stored as a first value plus a count, so they are neither bit-packed nor decoded per posting row id and need no storage for packed values. [#120982](https://github.com/ClickHouse/ClickHouse/pull/120982) ([Groene AI](https://github.com/groeneai)).
* Merging positional text indexes is faster: position lists are now merged segment by segment instead of being materialized and sorted in a temporary `roaringish` structure. [#121976](https://github.com/ClickHouse/ClickHouse/pull/121976) ([Elmi Ahmadov](https://github.com/ahmadov)).
* Added the `pfor` codec for text-index posting lists. It improves compression ratio and decoding speed. Enable it per index with `posting_list_codec = 'pfor'`, or for all text indexes of a table with the `text_index_posting_list_codec` setting. [#118123](https://github.com/ClickHouse/ClickHouse/pull/118123) ([Elmi Ahmadov](https://github.com/ahmadov)).
* Building text indexes during `INSERT` is now faster, especially for indexes with the `bitpacking` codec. [#113856](https://github.com/ClickHouse/ClickHouse/pull/113856) ([Anton Popov](https://github.com/CurtizJ)).
* Building a text index over a `LowCardinality(String)` or `Array(LowCardinality(String))` column is faster: each distinct value is tokenized once per batch of rows instead of at every occurrence. `MATERIALIZE INDEX` over a column with 50 distinct values takes about 33% less time. [#122409](https://github.com/ClickHouse/ClickHouse/pull/122409) ([Groene AI](https://github.com/groeneai)).
* Improve performance of the `ngrams` tokenizer. This speeds up `tokens(s, 'ngrams', N)`, `ngrams(s, N)`, `hasAnyTokens`, `hasAllTokens`, and `hasPhrase` with the `ngrams` tokenizer, as well as inserts into tables with an `ngrambf_v1` or `text(tokenizer = ngrams(N))` index. `tokens(URL, 'ngrams', 3)` over ClickBench URLs takes 26% (aarch64) to 36% (x86_64) less time. [#122328](https://github.com/ClickHouse/ClickHouse/pull/122328) ([Groene AI](https://github.com/groeneai)).
* Reading columns and subcolumns of `MergeTree` tables no longer rebuilds the serialization of a column for every reader: the serialization the data part already holds is reused. The gain is largest for `JSON` columns with many typed paths, where the whole set of path serializations used to be rebuilt per read task and per filter step. [#120632](https://github.com/ClickHouse/ClickHouse/pull/120632) ([Pavel Kruglov](https://github.com/Avogar)).
* Reduce per-block overhead when reading `MergeTree` parts with small blocks: column-stream names and substream-cache keys are no longer rebuilt for every block, and reads skip patch-part handling when there are none, virtual-column name lookups, and some small-column reallocations. This makes queries with a low `max_block_size` faster. [#124816](https://github.com/ClickHouse/ClickHouse/pull/124816) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124817](https://github.com/ClickHouse/ClickHouse/pull/124817) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reduce CPU overhead of reads from `MergeTree` tables by not formatting a tracing span name for every block. [#124820](https://github.com/ClickHouse/ClickHouse/pull/124820) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reading many columns from Compact parts is up to 2% faster by avoiding per-call copies of column metadata. [#122840](https://github.com/ClickHouse/ClickHouse/pull/122840) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)).
* Restore parallel prefix deserialization when reading an `Object` or `JSON` column from a Wide part: marks loading no longer queues behind one mutex. Closes [#114136](https://github.com/ClickHouse/ClickHouse/issues/114136). [#114252](https://github.com/ClickHouse/ClickHouse/pull/114252) ([Groene AI](https://github.com/groeneai)).
* Queries that read subcolumns of `JSON` columns with many typed paths are faster: resolving a subcolumn by name visits only the typed paths that can match it. For a `JSON` column with 500 typed paths, reading subcolumns from 96 parts takes about half the time. [#122029](https://github.com/ClickHouse/ClickHouse/pull/122029) ([Rory Shanks](https://github.com/rorylshanks)).
* Reduce serialization construction when reading `JSON` subcolumns from `MergeTree` parts by reusing safe cached subcolumn serializations. [#110957](https://github.com/ClickHouse/ClickHouse/pull/110957) ([Rory Shanks](https://github.com/rorylshanks)).
* Improve nested `JSON` path read performance with `map_with_buckets` by reusing thread-local serializations for `Dynamic` values. [#113344](https://github.com/ClickHouse/ClickHouse/pull/113344) ([Rory Shanks](https://github.com/rorylshanks)).
* Writing `JSON` columns with many paths in shared data is faster: the writer no longer rebuilds the statistics of the column for every block. For merges of parts with small granules, this saves up to 13% of the merge time. [#121987](https://github.com/ClickHouse/ClickHouse/pull/121987) ([Rory Shanks](https://github.com/rorylshanks)).
* Make constructing a `JSON` data type that uses `SKIP REGEXP` cheaper. The patterns are now compiled once per type instead of once for every declared path, so a column declaring 183 typed paths and 6 `SKIP REGEXP` entries performs 6 regular expression compilations where it previously performed 1098. This matters most for wide `JSON` schemas, where the patterns are often long alternations that are expensive to compile. [#122941](https://github.com/ClickHouse/ClickHouse/pull/122941) ([arc1999](https://github.com/arc1999)).
* Read only the matching `Map` subcolumn for safe `LIKE` predicates over `Map` values. [#119809](https://github.com/ClickHouse/ClickHouse/pull/119809) ([Minh Vu](https://github.com/fallintoplace)).
* Improve `SELECT` performance by avoiding lock contention when getting mutation snapshots with many concurrent background fetches. [#102382](https://github.com/ClickHouse/ClickHouse/pull/102382) ([c-end](https://github.com/c-end)).
* Support `PREWHERE` (including the automatic move of `WHERE` conditions by `optimize_move_to_prewhere`) for `Memory` tables: only the columns of the conditions are read at first, and the remaining columns are read only for the blocks where some rows pass, and only for the passing rows. This is especially beneficial for tables with `SETTINGS compress = true`, because for a selective condition most columns are never decompressed. Additionally, `SELECT count() FROM table` on a `Memory` table is now served from metadata, and `system.columns` shows real per-column sizes for `Memory` tables. [#116248](https://github.com/ClickHouse/ClickHouse/pull/116248) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `PREWHERE` over a `Memory` table is evaluated in multiple steps, like for `MergeTree` (`enable_multiple_prewhere_read_steps`): the columns of each next condition are read only for the rows that passed the previous ones. For a table with `compress = true`, a condition on a wide column next to a selective one no longer decompresses that column for every block, which makes such queries up to several times faster. [#122604](https://github.com/ClickHouse/ClickHouse/pull/122604) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries with `ORDER BY ... LIMIT n` over `Memory` tables use TopN dynamic filtering: rows that cannot get into the result are dropped while reading, before the other columns are read or decompressed with `compress = true`. [#123787](https://github.com/ClickHouse/ClickHouse/pull/123787) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries whose only aggregate functions are `min` and `max` of columns of `Parquet` files read with the `file` table function or the `File` engine, without `GROUP BY` or filters (for example ClickBench Q7), now take the values from the column chunk statistics instead of reading the data. Controlled by the new setting `optimize_min_max_from_files`. [#124201](https://github.com/ClickHouse/ClickHouse/pull/124201) ([Groene AI](https://github.com/groeneai)).
* `ORDER BY ... LIMIT` over a single `Parquet` file reads first the row groups whose min/max statistics are best for the result, so the top-K threshold skips the other row groups: `SELECT URL FROM file('hits.parquet') ORDER BY UserID DESC LIMIT 10` skips 207 of its 226 row groups at 4 threads. New profile event `ParquetTopKSkippedRowGroups`. [#123639](https://github.com/ClickHouse/ClickHouse/pull/123639) ([Groene AI](https://github.com/groeneai)).
* Row group pruning by dictionaries in the `Parquet` reader is faster for an equality or a short `IN` list on a column with large dictionaries, such as `WHERE UserID = ...` over ClickBench `hits.parquet`: the hashes of the dictionary values are no longer sorted unless a column chunk is checked against many values. [#123579](https://github.com/ClickHouse/ClickHouse/pull/123579) ([Groene AI](https://github.com/groeneai)).
* `Parquet` readers now check a column chunk's bloom filter before reading its dictionary page. A row group that the bloom filter rules out no longer reads the much larger dictionary page to reach the same conclusion, so `input_format_parquet_dictionary_filter_push_down` costs nothing on files that carry both, while pruning as much as before. [#120907](https://github.com/ClickHouse/ClickHouse/pull/120907) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `Parquet` footer-size reads are now tuned by storage type: remote and object-storage reads use more optimistic footer-size estimates to fetch the footer in one read instead of two, reducing footer-read latency, while local-file reads again use a 64 KiB tail so short queries over many local `Parquet` files read less data. [#115686](https://github.com/ClickHouse/ClickHouse/pull/115686) ([UnamedRus](https://github.com/UnamedRus)). [#124883](https://github.com/ClickHouse/ClickHouse/pull/124883) ([Groene AI](https://github.com/groeneai)).
* Avoid a redundant object-storage GET request when reading random-access formats from object storage. The generic from-start read-ahead is no longer issued for files larger than a single read buffer, where it could not reach the footer. Also, `input_format_allow_seeks = 0` is now honored by the `Parquet` reader, which previously sought regardless of the setting: with seeks disabled the whole object is read sequentially instead, which can raise memory usage for queries that set it. [#115681](https://github.com/ClickHouse/ClickHouse/pull/115681) ([UnamedRus](https://github.com/UnamedRus)).
* Speed up writing `Parquet` bloom filters: instead of deduplicating the value hashes in a hash set to determine the filter size, the writer now builds a filter sized for the number of values and folds it down to the size implied by `output_format_parquet_bloom_filter_bits_per_value`. This removes the worst-case slowdown on data that is adversarial for the hash set, and lowers the worst-case memory usage. [#119230](https://github.com/ClickHouse/ClickHouse/pull/119230) ([Armin Braun](https://github.com/original-brownbear)).
* Parse some GeoParquet metadata lazily. [#122693](https://github.com/ClickHouse/ClickHouse/pull/122693) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* Snappy decompression on AArch64 now uses NEON for short-offset copies, as upstream snappy does by default, which makes reading Snappy-compressed `Parquet` with repetitive string columns up to ~9% faster on ARM servers. The ARMv8.0 compatibility build (`NO_ARMV81_OR_HIGHER`) is unchanged. [#124608](https://github.com/ClickHouse/ClickHouse/pull/124608) ([Groene AI](https://github.com/groeneai)).
* Reading `CSV` with `input_format_csv_trim_whitespaces = 0` no longer prevents the parser from scaling across CPU cores. Each field used to copy a `shared_ptr` shared by all parsing threads, turning its reference counter into a contention point. On a 96-core machine, reading 200 million rows became 4.6 times faster and used 23 times less CPU. [#117251](https://github.com/ClickHouse/ClickHouse/pull/117251) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `Iceberg` partition pruning now uses the partition value of a `day`, `month`, `year`, or `hour` transform for a filter that wraps the partition source column in a monotonic function, such as `WHERE toDate(ts) = '2024-01-20'`, including for partition values from before 1970. Closes [#103433](https://github.com/ClickHouse/ClickHouse/issues/103433). [#118606](https://github.com/ClickHouse/ClickHouse/pull/118606) ([Konstantin Vedernikov](https://github.com/scanhex12)). [#119302](https://github.com/ClickHouse/ClickHouse/pull/119302) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Faster planning of queries to `Iceberg` tables with many manifest files: when a manifest file header has the `schema-id` key and that schema is already known from `metadata.json`, the schema copy stored in the manifest header is no longer parsed and compared with it. ClickHouse now also writes the `schema-id` key into the headers of manifest files it creates. [#122574](https://github.com/ClickHouse/ClickHouse/pull/122574) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* An `INSERT` into an `Iceberg` table that loses the commit race now reuses its manifests on retry and rewrites only the manifest list and the metadata file, instead of regenerating all manifests. [#122096](https://github.com/ClickHouse/ClickHouse/pull/122096) ([Asya Shneerson](https://github.com/asya-ch)).
* Restore the summary-based trivial `count()` optimization for `Iceberg` tables when the snapshot summary contains a safe row count. The optimization is skipped when equality deletes are present, because their recorded count is not the number of deleted rows. [#123267](https://github.com/ClickHouse/ClickHouse/pull/123267) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* `Iceberg` min/max hyperrectangles are now built only for columns the filter uses. [#122681](https://github.com/ClickHouse/ClickHouse/pull/122681) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* Restore min/max pruning of `Iceberg` data files for a `Decimal(38, 38)` column. A manifest bound of magnitude above roughly 0.7 could not be widened inside `Int128` and was dropped, leaving the column without a min/max condition; it is now saturated at the largest magnitude the precision allows, which is still outside every value the file can hold. [#119712](https://github.com/ClickHouse/ClickHouse/pull/119712) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Queries to `system.iceberg_history` with filters on `table` now avoid listing unrelated data lake catalog namespaces. [#113867](https://github.com/ClickHouse/ClickHouse/pull/113867) ([Den Kalantaevskii](https://github.com/Diskein)).
* Improve HTTP upload and download efficiency for `S3`, `Azure`, the `url` table function and engine, dictionaries, and data lake catalogs. Uploads use 1 MiB chunks instead of 8 KiB chunks, reducing the number of system calls for a 15 MB body from 1817 to 15 and avoiding redundant copies. Reads from `Azure` also avoid an intermediate 8 KiB buffer. [#116222](https://github.com/ClickHouse/ClickHouse/pull/116222) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reading from object storage with `remote_filesystem_read_method = 'read'` now keeps full-size network reads. Before, a short read at the end of one range capped every later read of the same file, which could make a scan hundreds of times slower, for example a compact part on an encrypted `S3` disk without the filesystem cache. [#123777](https://github.com/ClickHouse/ClickHouse/pull/123777) ([Groene AI](https://github.com/groeneai)).
* Reads that are served from the OS page cache no longer consume the budget of the local read bandwidth throttler (`max_local_read_bandwidth_for_server` and `max_local_read_bandwidth`), so a query over cached data is not slowed down to the configured disk bandwidth. This works with the default `local_filesystem_read_method = 'pread_threadpool'`, with `pread`, and for the files of the filesystem cache; the `read` method cannot detect cached reads and is accounted as before. [#118495](https://github.com/ClickHouse/ClickHouse/pull/118495) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Populating the filesystem cache on a cache miss is faster: the writer now reserves space ahead instead of taking the cache lock on every write. [#123255](https://github.com/ClickHouse/ClickHouse/pull/123255) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Loading the metadata of a large `s3_plain_rewritable` disk is several times faster. Previously it made one `ListObjects` request per directory and enumerated the `__meta` directory by a single paginated listing, so a disk with tens of thousands of directories needed tens of thousands of sequential-ish requests. The listings are now split by key prefix and run in parallel, and the files of all directories are collected by a few listings of the whole disk. For a disk with 36 thousand directories holding half a million objects, the load time dropped from 116 to 37 seconds. [#121412](https://github.com/ClickHouse/ClickHouse/pull/121412) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Metadata transactions of unrelated tables on a `plain_rewritable` disk no longer wait for each other: the disk-wide lock used to be held across all object storage requests of a commit, so one long `MOVE PARTITION`, `DROP TABLE`, or part removal blocked every merge, insert, and `CREATE TABLE` on the same disk. The lock is now taken per affected path, and the in-memory metadata is updated by replaying the recorded changes. Also fixes a permanent leak of objects when the server was killed during `DROP TABLE` (or another removal) on a `plain_rewritable` disk: the leftovers of such a removal are now marked and deleted when the disk metadata is loaded on the next start. [#118490](https://github.com/ClickHouse/ClickHouse/pull/118490) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Removing data parts from a disk with `metadata_type = plain_rewritable` (cleanup of outdated parts, `TRUNCATE`, `DROP PARTITION`, `DROP TABLE`) no longer copies every file of the part to a temporary object before deleting it, and no longer holds the disk metadata lock for those copies. The part directory is removed with one batched delete request. Closes [#123275](https://github.com/ClickHouse/ClickHouse/issues/123275). [#123303](https://github.com/ClickHouse/ClickHouse/pull/123303) ([Groene AI](https://github.com/groeneai)).
* Reduce metadata update cost and lock contention for `s3_plain_rewritable` disks with many parts by sharing unchanged directory entries between snapshots. [#119625](https://github.com/ClickHouse/ClickHouse/pull/119625) ([Rory Shanks](https://github.com/rorylshanks)).
* Avoid object reads for unchanged directories when a plain-rewritable disk refreshes its metadata. [#119760](https://github.com/ClickHouse/ClickHouse/pull/119760) ([Rory Shanks](https://github.com/rorylshanks)).
* `OPTIMIZE TABLE ... FINAL` on a non-replicated `MergeTree` table now assigns and runs the merges for all partitions at once instead of processing them one partition at a time. [#109004](https://github.com/ClickHouse/ClickHouse/pull/109004) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `TTL DELETE` merges can now use the vertical merge algorithm on `ReplacingMergeTree`, `CollapsingMergeTree`, and `VersionedCollapsingMergeTree`, where `vertical_merge_optimize_ttl_delete` previously applied only to plain `MergeTree`. [#119886](https://github.com/ClickHouse/ClickHouse/pull/119886) ([Murphy](https://github.com/murphy-4o)).
* Allow vertical merges to produce `Wide` parts with `Packed` storage instead of forcing the horizontal merge algorithm. [#125760](https://github.com/ClickHouse/ClickHouse/pull/125760) ([alesapin](https://github.com/alesapin)).
* Improved merge performance for `SummingMergeTree` tables by skipping unnecessary aggregate-column materialization. [#120852](https://github.com/ClickHouse/ClickHouse/pull/120852) ([Eduard Karacharov](https://github.com/korowa)).
* Many queued lightweight deletes with an `IN` condition that reach the same part in one mutation are now applied together, instead of evaluating each `IN` predicate over the whole part separately. Controlled by the new `MergeTree` setting `enable_row_mask_update_coalescing`. [#122718](https://github.com/ClickHouse/ClickHouse/pull/122718) ([Sergey Kuznetsov](https://github.com/skuznetsov-clickhouse)).
* Optimize the insert backpressure path in `MergeTree` by short-circuiting the parts-count check. [#114754](https://github.com/ClickHouse/ClickHouse/pull/114754) ([johnjing](https://github.com/JingYanchao)).
* Reduced merge-selection work when all storage volumes allow merges. [#119782](https://github.com/ClickHouse/ClickHouse/pull/119782) ([Rory Shanks](https://github.com/rorylshanks)).
* Reduce merge-selection CPU overhead for tables with column TTLs by avoiding copies of TTL descriptions. [#119644](https://github.com/ClickHouse/ClickHouse/pull/119644) ([Rory Shanks](https://github.com/rorylshanks)).
* Speed up decompression of codec chains. The first codec now reads the compressed block in place and the last codec writes the result in place, unless that codec requires a padded buffer (`LZ4`, `ZXC`). [#120043](https://github.com/ClickHouse/ClickHouse/pull/120043) ([Raufs Dunamalijevs](https://github.com/rienath)).
* Faster decompression of `T64`-encoded columns. [#120414](https://github.com/ClickHouse/ClickHouse/pull/120414) ([Raufs Dunamalijevs](https://github.com/rienath)). [#122442](https://github.com/ClickHouse/ClickHouse/pull/122442) ([Raufs Dunamalijevs](https://github.com/rienath)).
* A query with `arrayJoin` and a small `LIMIT` again shrinks the read block size instead of reading whole blocks on every thread for a few output rows, fixing a regression from 26.6. Closes [#119193](https://github.com/ClickHouse/ClickHouse/issues/119193). [#120214](https://github.com/ClickHouse/ClickHouse/pull/120214) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Chained `ARRAY JOIN`s no longer materialize the whole block of a lazily replicated array column before expanding it, which cuts their peak memory by several times. [#121397](https://github.com/ClickHouse/ClickHouse/pull/121397) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Avoid copying whole columns when slicing them, including in `ARRAY JOIN`. [#118788](https://github.com/ClickHouse/ClickHouse/pull/118788) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Improve `ARRAY JOIN` performance for queries with wide columns that are not expanded, by avoiding unnecessary copies and preserving lazy replication through filtering and sorting. [#124727](https://github.com/ClickHouse/ClickHouse/pull/124727) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Reduced the peak memory usage of `RESTORE` and of incremental `BACKUP`: the metadata file of a backup is now parsed while it is read, instead of being read into memory whole first. For backups of millions of files this saves gigabytes. [#125712](https://github.com/ClickHouse/ClickHouse/pull/125712) ([Julia Kartseva](https://github.com/jkartseva)).
* Faster startup of `clickhouse-local` and other tools. Time zone lookup tables are built 2.4 to 5.6 times faster and use half the memory (2.3 MiB instead of 4.6 MiB per loaded time zone). The `clickhouse` binary also starts faster when it is not in the page cache, with a 1.40x speedup measured for `clickhouse local`. [#125665](https://github.com/ClickHouse/ClickHouse/pull/125665) ([Raúl Marín](https://github.com/Algunenano)). [#125672](https://github.com/ClickHouse/ClickHouse/pull/125672) ([Raúl Marín](https://github.com/Algunenano)).
* A full scan of a complex-key `hashed`, `sparse_hashed`, `hashed_array`, or `range_hashed` dictionary (`SELECT ... FROM <dictionary>` or the `dictionary` table function) is faster: each reading stream now deserializes only the keys of its own block instead of one thread deserializing all keys before reading starts. A scan with `LIMIT` deserializes only the blocks it reads. A full scan also no longer keeps all deserialized keys in memory at once. [#123883](https://github.com/ClickHouse/ClickHouse/pull/123883) ([Groene AI](https://github.com/groeneai)).
* Added the `SIEVE` cache eviction policy for in-memory caches such as the mark cache and the uncompressed cache. Enable it with the corresponding `*_cache_policy` server settings, for example `mark_cache_policy` or `uncompressed_cache_policy`. It is not available for the filesystem cache. [#123435](https://github.com/ClickHouse/ClickHouse/pull/123435) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The symbolized stack trace of a failed query is now built only when the error message will be logged or the trace will be stored in `system.query_log`. Previously each failing `clickhouse-local` invocation built the symbol index of the whole binary to produce a string that was then discarded. [#120660](https://github.com/ClickHouse/ClickHouse/pull/120660) ([Groene AI](https://github.com/groeneai)).
* Reduces the CPU overhead of the workload scheduler thread by updating the `system.scheduler` introspection metrics more cheaply. A saturated scheduler can now serve about 15% more resource requests per second. [#121958](https://github.com/ClickHouse/ClickHouse/pull/121958) ([Sergei Trifonov](https://github.com/serxa)).
* PromQL now copies label matchers across binary operators for the labels the operator matches series by, so `b / on(job) a{job="x"}` reads only the matching series of `b`. As in Prometheus, duplicate left series of a one-to-one match in a group the right side does not have no longer fail the query. The new setting `promql_push_down_label_matchers` turns this off. [#122972](https://github.com/ClickHouse/ClickHouse/pull/122972) ([Valery Petrov](https://github.com/valerypetrov)). [#125267](https://github.com/ClickHouse/ClickHouse/pull/125267) ([Groene AI](https://github.com/groeneai)).
* Accelerate exact PromQL label matching on `TimeSeries` tables with a text index on the `tags` map. [#120083](https://github.com/ClickHouse/ClickHouse/pull/120083) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Use more compact codecs for generated `timestamp` and `value` columns in the samples and recent samples tables of the `TimeSeries` engine: `Delta`, `T64`, and `ZSTD(3)` for `timestamp`, and `ALP` plus `ZSTD(3)` for `value`. [#120454](https://github.com/ClickHouse/ClickHouse/pull/120454) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Reduce memory usage of the `timeSeries*ToGrid` aggregate functions. [#124773](https://github.com/ClickHouse/ClickHouse/pull/124773) ([Valery Petrov](https://github.com/valerypetrov)).
* Filtering and sorting are now faster: the helpers that build filter masks, count filter bytes, and check for an identity permutation use AVX2 or AVX-512 on x86 instead of SSE2, and are vectorized on ARM where they were not. [#121403](https://github.com/ClickHouse/ClickHouse/pull/121403) ([Raúl Marín](https://github.com/Algunenano)).
* Faster filtering of numeric columns on ARM (up to 1.28x) and on AMD Zen 4 (up to 2.8x), speeding up queries with `WHERE` and multi-step `PREWHERE`. [#122893](https://github.com/ClickHouse/ClickHouse/pull/122893) ([Raúl Marín](https://github.com/Algunenano)).
* Filters on numeric columns that keep few rows are faster. [#124024](https://github.com/ClickHouse/ClickHouse/pull/124024) ([Raúl Marín](https://github.com/Algunenano)).
* Filtering numeric columns no longer over-allocates the result buffer (by up to 2x) when the size of the result is known in advance. Closes [#123546](https://github.com/ClickHouse/ClickHouse/issues/123546). [#123576](https://github.com/ClickHouse/ClickHouse/pull/123576) ([Raúl Marín](https://github.com/Algunenano)).
* Vectorized comparisons of `UUID`, `IPv4`, and `IPv6` columns with columns or constants of the same type, which makes such comparisons several times faster. Comparisons of `FixedString(16)` values are now branchless too, making them faster when it is unpredictable whether the values are equal. [#120193](https://github.com/ClickHouse/ClickHouse/pull/120193) ([Manuel](https://github.com/raimannma)).
* Comparisons `<`, `>`, `<=`, and `>=` of `IPv6` and `FixedString(16)` values are 1.4 to 1.8 times faster on CPUs with AVX-512. [#125151](https://github.com/ClickHouse/ClickHouse/pull/125151) ([Groene AI](https://github.com/groeneai)).
* `LIKE`, `ILIKE`, `match`, and their variants are faster when the pattern is not a constant, up to 5 times with 8 threads. [#122764](https://github.com/ClickHouse/ClickHouse/pull/122764) ([Groene AI](https://github.com/groeneai)). [#122843](https://github.com/ClickHouse/ClickHouse/pull/122843) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)).
* Faster `multiMatchAny` and related functions with non-constant needles when `reject_expensive_hyperscan_regexps` is disabled. [#122842](https://github.com/ClickHouse/ClickHouse/pull/122842) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)).
* `CAST` of strings to `Nullable(Bool)` (also `accurateCastOrNull(s, 'Bool')` and `toBool` of a `Nullable(String)`) is up to 7 times faster with 8 threads, and `arrayIntersect`, `arrayUnion`, `arraySymmetricDifference`, and `arrayJaccardIndex` over non-numeric elements such as `DateTime64`, `UUID`, or `Tuple` are up to 18 times faster with 8 threads: they no longer allocate memory and update profile events for every row. `multiMatchAny` and the other multi-regexp functions with non-constant needles now add their cache hit and miss counts once per block. [#122920](https://github.com/ClickHouse/ClickHouse/pull/122920) ([Groene AI](https://github.com/groeneai)).
* Speed up extracting a value from a `Map` with `LowCardinality` keys. The key is now resolved to its dictionary position once instead of being compared against every entry of a row, so the cost of `m[key]` no longer grows with the position of the key in the row. Also fix extracting a value from a `Map` whose key is `NaN`, or holds a `NULL` inside a composite key: such a key used to compare equal to every other key. [#120453](https://github.com/ClickHouse/ClickHouse/pull/120453) ([Pavel Kruglov](https://github.com/Avogar)).
* `L2DistanceTransposed`, `cosineDistanceTransposed`, `dotProductTransposed`, and their `...Quantized` variants with precision `1` are now computed directly from the packed sign bit plane with per-byte lookup tables, 2 to 3 times faster than before. The new setting `qbit_one_bit_symmetric_distance` additionally reduces the reference vector to its signs and derives the result from the Hamming distance between the two sign vectors, computed with XOR and popcount, which is up to 10 times faster. [#120594](https://github.com/ClickHouse/ClickHouse/pull/120594) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Speed up `randomHadamardTransform` and vector quantization for vector dimensions of the form 2^k × 12 or 2^k × 20, such as 768 or 1536, on x86 with AVX2: 1.9x to 2.5x higher throughput and 50% to 60% lower median latency. [#120874](https://github.com/ClickHouse/ClickHouse/pull/120874) ([Jingxin (Philip) Li](https://github.com/philip-jingxin)).
* `hasAll` and `hasAny` on integer arrays use SIMD on both x86 and ARM, including `Nullable` arrays and `hasAny`, which previously used a scalar loop. `hasAny` is up to 15x faster, and `hasAll` becomes several times faster on ARM. [#123420](https://github.com/ClickHouse/ClickHouse/pull/123420) ([Groene AI](https://github.com/groeneai)).
* `hasAny` over `Array(String)` looks each element up in a hash set, instead of comparing it with every string, when its second argument is a constant array of four or more strings of up to 256 bytes and a block has enough rows to pay for building the set. With 32 strings this uses 87% less CPU. Closes [#122333](https://github.com/ClickHouse/ClickHouse/issues/122333). [#122507](https://github.com/ClickHouse/ClickHouse/pull/122507) ([Groene AI](https://github.com/groeneai)).
* Vectorize `least` and `greatest` for `Nullable` numeric arguments and for three or more numeric arguments, and `clamp` for numeric arguments. These cases were previously evaluated row-by-row and are now 3-7 times faster. [#109783](https://github.com/ClickHouse/ClickHouse/pull/109783) ([Manuel](https://github.com/raimannma)).
* Speed up `round`, `roundBankers`, `floor`, `ceil`, and `trunc` with a scale argument on `Float32` and `Float64` columns by using AVX2 on x86 instead of SSE4.1, and speed up rounding of integers narrower than 64 bits on ARM by vectorizing it. [#122032](https://github.com/ClickHouse/ClickHouse/pull/122032) ([Raúl Marín](https://github.com/Algunenano)).
* `modulo` and `moduloOrNull` of two unsigned integers of different widths are faster again, as before 26.9, when the wider one is `UInt128` (about 3 to 4 times, for example `UInt128 % UInt16`) or the dividend is `UInt256` (1.6 to 2 times). [#123170](https://github.com/ClickHouse/ClickHouse/pull/123170) ([Groene AI](https://github.com/groeneai)).
* `exp10` is up to 2x faster by splitting its argument with `trunc` instead of the `modf` library call, with bitwise-identical results. [#122263](https://github.com/ClickHouse/ClickHouse/pull/122263) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Speed up `entropy` and multi-argument `uniqExact` on short strings by optimizing `SipHash` for short inputs. Hash values are unchanged. [#123658](https://github.com/ClickHouse/ClickHouse/pull/123658) ([Groene AI](https://github.com/groeneai)).
* `sumMap`, `minMap`, `maxMap`, `sumMapWithOverflow`, and `sumMapFiltered` over arrays are up to 2.3 times faster for integer, `Enum`, `Date`, `Date32`, and `DateTime` keys. [#124340](https://github.com/ClickHouse/ClickHouse/pull/124340) ([Groene AI](https://github.com/groeneai)).
* `h3PolygonToCells` and `h3PolygonToCellsWithContainment` are several times faster on polygons much smaller than a cell (9x on the query from the issue). This brings `h3PolygonToCells` back to the speed it had on such polygons before 26.8.12 and 26.9. Closes [#123877](https://github.com/ClickHouse/ClickHouse/issues/123877). [#123911](https://github.com/ClickHouse/ClickHouse/pull/123911) ([Groene AI](https://github.com/groeneai)).
* `isValidUTF8` validates a `String` or `FixedString` column with one `simdutf` call over the whole column instead of one call per row, which makes it up to 2.8 times faster on short strings. [#123835](https://github.com/ClickHouse/ClickHouse/pull/123835) ([Groene AI](https://github.com/groeneai)).
* Speed up `lowerUTF8` and `upperUTF8` on text whose non-ASCII characters are mostly two-byte UTF-8 (`U+0080` to `U+07FF`, e.g. Latin-1, Greek, Cyrillic): the column is processed in one pass, these characters are mapped from a table built from ICU, and ICU is called only for the rest of a row that contains other characters. [#123867](https://github.com/ClickHouse/ClickHouse/pull/123867) ([Groene AI](https://github.com/groeneai)).
* `lower`, `upper`, and the ASCII path of `lowerUTF8` and `upperUTF8` are now faster by using AVX2 or AVX-512 depending on the CPU. [#121374](https://github.com/ClickHouse/ClickHouse/pull/121374) ([Manuel](https://github.com/raimannma)).
* Speed up `lengthUTF8` and other `UTF-8` functions on long strings by counting code points 64 bytes at a time, on both x86 and ARM. [#122049](https://github.com/ClickHouse/ClickHouse/pull/122049) ([Raúl Marín](https://github.com/Algunenano)).
* Fixed ARM performance regressions in `indexOf` with a constant needle and in text output formats over non-ASCII strings. Closes [#123338](https://github.com/ClickHouse/ClickHouse/issues/123338). Closes [#123346](https://github.com/ClickHouse/ClickHouse/issues/123346). [#123377](https://github.com/ClickHouse/ClickHouse/pull/123377) ([Raúl Marín](https://github.com/Algunenano)).
* UTF-8 string functions such as `substringUTF8` are faster on long ASCII strings on CPUs with AVX-512. [#120088](https://github.com/ClickHouse/ClickHouse/pull/120088) ([Minh Vu](https://github.com/fallintoplace)).
* Improved performance of functions `extractKeyValuePairs` and `extractKeyValuePairsWithEscaping`. [#119361](https://github.com/ClickHouse/ClickHouse/pull/119361) ([Anton Popov](https://github.com/CurtizJ)).
* Function `mortonEncode` now resolves the type of each argument and the ratios of a constant range mask once per block instead of once per row, which makes it up to 2x faster. [#123354](https://github.com/ClickHouse/ClickHouse/pull/123354) ([Groene AI](https://github.com/groeneai)).
* Speed up `mortonDecode` on AArch64 by decoding with compile-time shift-and-mask sequences instead of a lookup table. The output is unchanged on every architecture. [#116205](https://github.com/ClickHouse/ClickHouse/pull/116205) ([Groene AI](https://github.com/groeneai)).
* `encrypt` and `aes_encrypt_mysql` are faster on values shorter than 16 bytes in the `aes-*-ecb` and `aes-*-cbc` modes (up to 2x for `encrypt` with `aes-128-ecb`). [#124103](https://github.com/ClickHouse/ClickHouse/pull/124103) ([Groene AI](https://github.com/groeneai)).
* Speed up `hex` and `unhex` with optimized `AVX` and `AVX2` implementations when they are available. [#123511](https://github.com/ClickHouse/ClickHouse/pull/123511) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Faster `toMillisecond`, `toMicrosecond`, and `toNanosecond` on `DateTime64`. [#122852](https://github.com/ClickHouse/ClickHouse/pull/122852) ([Samuel Krempaský](https://github.com/NIKTONIKTO717)).
* Speed up `reverse` for `String` and `FixedString` values. [#119462](https://github.com/ClickHouse/ClickHouse/pull/119462) ([Minh Vu](https://github.com/fallintoplace)).
* `arrayDistinct` is faster: it now does a single hash-table lookup per element instead of two, and `Decimal` arrays use a specialized fast path. [#113966](https://github.com/ClickHouse/ClickHouse/pull/113966) ([Manuel](https://github.com/raimannma)). [#121646](https://github.com/ClickHouse/ClickHouse/pull/121646) ([Minh Vu](https://github.com/fallintoplace)).
* Specialize `has` and `indexOf` searches in non-nullable numeric arrays when the needle is constant. [#119090](https://github.com/ClickHouse/ClickHouse/pull/119090) ([Minh Vu](https://github.com/fallintoplace)).
* `arrayFilter` is faster when the lambda keeps all or none of the elements of a block. [#119816](https://github.com/ClickHouse/ClickHouse/pull/119816) ([Minh Vu](https://github.com/fallintoplace)).
* Improve performance of `bitmapAndCardinality`, `bitmapOrCardinality`, `bitmapXorCardinality`, `bitmapAndnotCardinality`, and `bitmapHasAny` for `UInt64` and `Int64` bitmaps, which previously had to materialize the intersection to compute the result. On AArch64, `bitmapAndCardinality`, `bitmapOrCardinality`, `bitmapXorCardinality`, and `bitmapAndnotCardinality` no longer hit a rare code-layout-dependent slow path. [#120615](https://github.com/ClickHouse/ClickHouse/pull/120615) ([Ashish kumar choubey](https://github.com/aschoube)). [#123675](https://github.com/ClickHouse/ClickHouse/pull/123675) ([Groene AI](https://github.com/groeneai)).
* Restored the existing `Decimal` fast paths in `arrayCompact`. [#121245](https://github.com/ClickHouse/ClickHouse/pull/121245) ([Minh Vu](https://github.com/fallintoplace)).
* `uniqUpTo` now stops hashing after it exceeds its threshold. [#121247](https://github.com/ClickHouse/ClickHouse/pull/121247) ([Minh Vu](https://github.com/fallintoplace)).
* Restored the `FixedString` fast path for `arrayEnumerateUniq` and `arrayEnumerateDense`. [#121338](https://github.com/ClickHouse/ClickHouse/pull/121338) ([Minh Vu](https://github.com/fallintoplace)).
* `arrayReverse` is now faster for `Decimal` and wide integer arrays. [#121342](https://github.com/ClickHouse/ClickHouse/pull/121342) ([Minh Vu](https://github.com/fallintoplace)). [#121644](https://github.com/ClickHouse/ClickHouse/pull/121644) ([Minh Vu](https://github.com/fallintoplace)).
* Avoid copying `String` columns for `substring(s, 1)`. [#119820](https://github.com/ClickHouse/ClickHouse/pull/119820) ([Minh Vu](https://github.com/fallintoplace)).
* Skip KMP preprocessing for impossible `hasSubstr` searches. [#119824](https://github.com/ClickHouse/ClickHouse/pull/119824) ([Minh Vu](https://github.com/fallintoplace)).
* Speed up `arrayMin` and `arrayMax` for `Decimal` and `DateTime64` arrays. [#123507](https://github.com/ClickHouse/ClickHouse/pull/123507) ([Minh Vu](https://github.com/fallintoplace)).
* Speed up `arrayUniq` for `Decimal32`, `Decimal64`, and `Decimal128` arrays. [#122121](https://github.com/ClickHouse/ClickHouse/pull/122121) ([Minh Vu](https://github.com/fallintoplace)).
* Allow a single condition to constrain multiple key columns during index analysis. For example, if the primary key is `(toYYYYMMDD(dt), toDate(dt), dt)` and the query has `WHERE dt >= toDateTime('2026-01-10 00:00:00')`, all three key columns are now used for pruning, instead of only one of them as before. This behavior is controlled by the setting `analyze_index_with_multiple_key_columns_per_condition` (enabled by default). [#96225](https://github.com/ClickHouse/ClickHouse/pull/96225) ([Nihal Z. Miaji](https://github.com/nihalzp)).
* Left-side columns absent from a hash join's output (for example the join key in `SELECT count() ... LEFT JOIN ...`) are now pruned when only a spill threshold is set, as with the default `max_bytes_ratio_before_external_join`, instead of being carried through the pipeline. [#108191](https://github.com/ClickHouse/ClickHouse/pull/108191) ([Groene AI](https://github.com/groeneai)).
* Joins on null-safe comparison (`<=>`) of nullable keys use fixed-width hash keys instead of a serialized tuple. [#120596](https://github.com/ClickHouse/ClickHouse/pull/120596) ([Manuel](https://github.com/raimannma)).
* Fixed a quadratic slowdown of `JOIN` with `join_algorithm = 'hash'` when `ON` has a non-equality condition or several `OR` conditions. [#121209](https://github.com/ClickHouse/ClickHouse/pull/121209) ([nt](https://github.com/namtran1812)).
* Faster parsing of floating-point numbers from text formats. Update `fast_float` to v8.3.2. [#121383](https://github.com/ClickHouse/ClickHouse/pull/121383) ([Actuele AI](https://github.com/actueleai)).
* Query plan optimizations can now use the sort order reported by any table engine, not only `MergeTree`, avoiding unnecessary sorting. [#121858](https://github.com/ClickHouse/ClickHouse/pull/121858) ([Ivan Zhukov](https://github.com/zhkvia)).
* Slightly reduce the per-block overhead of aggregation by not copying the columns of the block when preparing the aggregate function instructions. [#124819](https://github.com/ClickHouse/ClickHouse/pull/124819) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `GROUP BY` by several keys, which requires the serialization of keys, serializes the keys of a block in chunks that fit in the CPU cache instead of all at once. This makes such queries faster when many threads are used, for example, ClickBench Q18 is 37% faster on a 96-core machine. [#124955](https://github.com/ClickHouse/ClickHouse/pull/124955) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the setting `cluster_function_wait_for_replica_ms` (default `0`, disabled), which lets cluster table functions wait for the replica assigned a file to connect before giving that file to another replica. This improves file distribution when replicas start at different times. Work stealing now takes files from the active replica with the most remaining bytes without scanning all unprocessed files. [#125611](https://github.com/ClickHouse/ClickHouse/pull/125611) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* The `SLRU` filesystem cache policy now moves a partially downloaded file segment to the protected queue on a cache hit, as it does for a fully downloaded one. Before, a scan could evict the data of narrow reads from the probationary queue. [#125657](https://github.com/ClickHouse/ClickHouse/pull/125657) ([Sema Checherinda](https://github.com/CheSema)).

#### Improvement
* Lightweight updates (the `UPDATE` statement and patch parts) are now generally available instead of Beta. The `enable_lightweight_update` setting is now in the production feature tier. [#125719](https://github.com/ClickHouse/ClickHouse/pull/125719) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Cluster discovery is no longer experimental: clusters with a `discovery` section in `remote_servers` work without enabling `allow_experimental_cluster_discovery`. The setting is now obsolete. [#123917](https://github.com/ClickHouse/ClickHouse/pull/123917) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The Kusto (KQL) and Trino query dialects were promoted from experimental to beta. They are still enabled with the `allow_experimental_kusto_dialect` and `enable_trino_dialect` settings. [#124750](https://github.com/ClickHouse/ClickHouse/pull/124750) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Tables and databases can now be addressed through URL paths in the HTTP interface by default: `curl http://localhost:8123/my_db/my_table.csv` now works without extra configuration. The settings `http_allow_path_requests`, `http_allow_database_as_path`, `http_allow_table_as_file`, and `http_allow_filters_as_path` are enabled by default; `http_allow_filters_as_unrecognized_url_parameters` stays disabled. [#121143](https://github.com/ClickHouse/ClickHouse/pull/121143) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The `all_...` union tables over the system log tables (`system.all_query_log`, `system.all_text_log`, and so on) are now created by default: the default server configuration enables the `create_union_system_log_tables` section with `merge_rotated_tables`, so queries over the rotated versions of a log no longer have to remember constructs like `merge(system, '^query_log')`. The `cluster` setting is shown as a commented-out example, because the table definitions would reference the cluster and require it to exist. [#117943](https://github.com/ClickHouse/ClickHouse/pull/117943) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* In a `DataLakeCatalog` database with `allow_experimental_iceberg_compaction = 1`, only `Iceberg` tables are now cached. `DeltaLake` and `Paimon` tables no longer keep the first schema they saw, so `SELECT *` and `DESCRIBE` pick up added columns without recreating the database or restarting the server. Closes [#122548](https://github.com/ClickHouse/ClickHouse/issues/122548). [#122568](https://github.com/ClickHouse/ClickHouse/pull/122568) ([Groene AI](https://github.com/groeneai)).
* The background memory worker now corrects the global memory tracker from real memory usage on every tick (server setting `memory_worker_correct_memory_tracker`, enabled by default). Previously the tracker was a pure counter that was re-baselined only at startup and when it went negative, so upward accounting drift was permanent: servers were observed refusing every query with `MEMORY_LIMIT_EXCEEDED`, reporting more than 100 GiB in use while actually using 7 GiB, until restart. The drift itself remains visible as the difference between the `MemoryTrackingUncorrected` and `MemoryTracking` metrics. [#118094](https://github.com/ClickHouse/ClickHouse/pull/118094) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The AI function per-query quotas are now disabled by default: `ai_function_max_input_tokens_per_query`, `ai_function_max_output_tokens_per_query`, and `ai_function_max_api_calls_per_query` default to `0`, meaning no limit. To keep the previous caps, set the three settings explicitly or use `compatibility = '26.9'`. [#121424](https://github.com/ClickHouse/ClickHouse/pull/121424) ([George Larionov](https://github.com/george-larionov)).
* Named `Tuple` columns in `Pretty` formats are now displayed as subcolumns, with the element names shown in additional header and footer lines. Controlled by the new setting `output_format_pretty_named_tuples_as_subcolumns` (enabled by default). A named tuple rendered inside one cell, for example nested in an `Array` or when the setting is disabled, is still displayed according to `output_format_pretty_named_tuples_as_json`. [#78927](https://github.com/ClickHouse/ClickHouse/pull/78927) ([Engel Danila](https://github.com/aaaengel)).
* Show the query result in `Pretty` formats as soon as it arrives. Previously a query that returns a few rows early and then keeps reading, such as `SELECT DISTINCT` over a large table, displayed nothing until it finished or was cancelled. A later fix prevents a background write error after the query has finished from surfacing as a late `LOGICAL_ERROR`. [#117005](https://github.com/ClickHouse/ClickHouse/pull/117005) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#125028](https://github.com/ClickHouse/ClickHouse/pull/125028) ([Groene AI](https://github.com/groeneai)).
* Display non-printable control characters as Unicode "Control Pictures" in the `Vertical` and `Pretty*` output formats by default, so they are no longer silently swallowed and no longer deform the table. `TAB`, the line feed, and `ESC` are excluded, because a terminal interprets them rather than swallowing them. The new setting `output_format_pretty_display_control_characters` can be disabled to restore raw-byte output. [#101774](https://github.com/ClickHouse/ClickHouse/pull/101774) ([Yash ](https://github.com/Onyx2406)).
* Expired authentication methods are now removed from a user automatically. Every `ALTER USER` drops the authentication methods whose `VALID UNTIL` deadline has already passed and keeps all the others, so short-lived credentials can be rotated indefinitely with `ALTER USER ... ADD IDENTIFIED WITH ... VALID UNTIL ...` without the expired ones accumulating against `max_authentication_methods_per_user`. A method added by the same statement is not affected, and a user whose every method is expired keeps them rather than being left with no way to authenticate. [#118880](https://github.com/ClickHouse/ClickHouse/pull/118880) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* On `ReplicatedMergeTree`, the sync value `3` (`mutations_sync`, `lightweight_deletes_sync`, `alter_sync`) now waits for the active replicas, as documented, instead of all of them, so an inactive replica no longer causes `UNFINISHED`. This also affects `OPTIMIZE`, `TRUNCATE`, `DROP`/`DETACH PARTITION`, `REPLACE PARTITION`, and `MOVE PARTITION TO TABLE`. [#120592](https://github.com/ClickHouse/ClickHouse/pull/120592) ([Nikolay Degterinsky](https://github.com/evillique)).
* A number written directly as the first argument of `CAST` is now read with the target type when that type reads it more precisely than the number itself carries it, such as a `Decimal` or an integer wider than 64 bits. `Nullable`, `Array`, and `LowCardinality` of those types take the same exact path. [#116043](https://github.com/ClickHouse/ClickHouse/pull/116043) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `TTL ... GROUP BY (a, b, c)` is now accepted and means the same as `TTL ... GROUP BY a, b, c`, matching how `ORDER BY (a, b, c)` works. Previously the parenthesized form was rejected with `TTL Expression GROUP BY key should be a prefix of primary key`. [#118426](https://github.com/ClickHouse/ClickHouse/pull/118426) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `sequenceMatch`, `sequenceCount`, and `sequenceMatchEvents` now accept a `DateTime64` timestamp instead of failing. Events are ordered at the column's full precision, so two events within the same second are no longer ambiguous. [#118427](https://github.com/ClickHouse/ClickHouse/pull/118427) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `argMin` and `argMax` now consistently return the value from the first row among rows with the same minimum or maximum within a block, matching `argMinIf` and `argMaxIf`. [#122892](https://github.com/ClickHouse/ClickHouse/pull/122892) ([Raúl Marín](https://github.com/Algunenano)).
* Added the setting `analyzer_compatibility_allow_cte_redefinition`. When enabled, a CTE name may be defined more than once in a single `WITH` clause; a later definition shadows the earlier one as the query analysis before 24.3 did. `MATERIALIZED` CTEs and CTEs in `WITH RECURSIVE` cannot be redefined. [#121623](https://github.com/ClickHouse/ClickHouse/pull/121623) ([Dmitry Novik](https://github.com/novikd)).
* Added settings `semi_join_include_columns_from_both_sides` and `anti_join_include_columns_from_both_sides` (enabled by default) for `SEMI` and `ANTI` joins in the analyzer. When enabled, behavior is unchanged: columns from both sides are accessible after the join. Disable either setting to enforce SQL-standard semantics, where only the preserved side's columns are visible after the join. [#96662](https://github.com/ClickHouse/ClickHouse/pull/96662) ([Zhang Yifan](https://github.com/zhangyifan27)).
* The setting `aggregate_function_input_format` with the values `value` and `array` now works in every input format, including `RowBinary`, `Native`, and `Parquet`, and for `AggregateFunction` nested in `Array`, `Tuple`, and `Map`. In the JSON formats the value is now a JSON value, for example `{"x": [1, 2, 3]}`, not a string containing it. [#121120](https://github.com/ClickHouse/ClickHouse/pull/121120) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Compression codec names and their keyword arguments are now matched case-insensitively, so `CODEC(zstd)` works like `CODEC(ZSTD)` and `ALP(std)` like `ALP(STD)`. Previously a codec whose name is mixed-case, such as `Delta`, `DoubleDelta`, `Gorilla` or `Quantized`, could not be used in the `default_compression_codec`, `marks_compression_codec` and `primary_key_compression_codec` settings, nor in the `<compression>` section of the server configuration, because those paths upper-cased the codec name. [#119313](https://github.com/ClickHouse/ClickHouse/pull/119313) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added aliases `allow_suspicious_indexes`, `ignore_data_skipping_indexes`, `force_data_skipping_indexes`, and `secondary_indexes_enable_bulk_filtering` for the settings spelled with `indices`, so both plural forms are accepted. [#122565](https://github.com/ClickHouse/ClickHouse/pull/122565) ([Mohammed Sohail](https://github.com/msohail22)).
* `APPLY (f, 'prefix')` now keeps the table qualifier in generated column names. [#122702](https://github.com/ClickHouse/ClickHouse/pull/122702) ([Groene AI](https://github.com/groeneai)).
* Async insert now supports eligible single-block local `INSERT ... SELECT` queries, with synchronous fallback for larger, multi-block, empty, transactional, or non-parallel-quorum inserts. [#111663](https://github.com/ClickHouse/ClickHouse/pull/111663) ([Mikhail f. Shiryaev](https://github.com/Felixoid)).
* Added the opt-in `optimize_row_order_if_no_order_by` setting for `MergeTree` tables with an empty sorting key (`ORDER BY ()` / `ORDER BY tuple()`). When enabled, it applies row-order optimization automatically to inserts into such tables. [#104591](https://github.com/ClickHouse/ClickHouse/pull/104591) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `WHATIF` can now estimate hypothetical projections without reading the whole table: past `max_rows_to_scan` rows it estimates from a sample of granules and reports the resulting range in `marks_span`, `max_rows_to_read` lowers that budget, and it follows `(prefer|force)_optimize_projection`. It now also uses the real projection-optimizer weighing logic, so its verdicts, reasons, and pruned ranges track the optimizer more closely. [#118233](https://github.com/ClickHouse/ClickHouse/pull/118233) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#121766](https://github.com/ClickHouse/ClickHouse/pull/121766) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#122060](https://github.com/ClickHouse/ClickHouse/pull/122060) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#123540](https://github.com/ClickHouse/ClickHouse/pull/123540) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). [#123587](https://github.com/ClickHouse/ClickHouse/pull/123587) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Added the setting `parallel_replicas_for_queries_with_multiple_tables` (default `true`) to control whether parallel replicas are used for queries joining multiple tables. When disabled, parallel replicas are not used for queries with `JOIN`, where the non-leftmost side is read in full on every replica. The setting does not affect a `UNION` query without a `JOIN`, where each branch is an independent single-table read, nor `ARRAY JOIN`. [#76595](https://github.com/ClickHouse/ClickHouse/pull/76595) ([Igor Nikonov](https://github.com/devcrafter)).
* `max_joined_block_size_rows` and `max_joined_block_size_bytes` now also bound the result blocks of `parallel_hash` and `grace_hash` joins: squashing of the join output no longer merges blocks past these limits. Closes [#124198](https://github.com/ClickHouse/ClickHouse/issues/124198). [#124304](https://github.com/ClickHouse/ClickHouse/pull/124304) ([Groene AI](https://github.com/groeneai)).
* Added the setting `query_plan_filter_push_down_below_limit_by` (default `1`), controlling whether filters on `LIMIT BY` key columns are pushed below the `LIMIT BY` step. Setting it to `0` restores the plan shape used before 26.7, for queries where the pushed predicate makes index analysis costlier than the scan it saves. [#121675](https://github.com/ClickHouse/ClickHouse/pull/121675) ([Groene AI](https://github.com/groeneai)).
* Added the `use_aggregation_memory_tracker` setting to choose how `GROUP BY` measures memory when deciding on two-level aggregation. `compatibility` below `26.9` restores the measurement used before 26.7. [#124439](https://github.com/ClickHouse/ClickHouse/pull/124439) ([Hechem Selmi](https://github.com/m-selmi)).
* `system.query_log` and `system.processes` now report the metrics of each task of a distributed query plan (`make_distributed_plan = 1`) in a row of its own: a task is a secondary query with its own `query_id`, correlated to the statement that started it by `initial_query_id`. The initiator's row reports its own work instead of the volume its tasks read. [#120949](https://github.com/ClickHouse/ClickHouse/pull/120949) ([Groene AI](https://github.com/groeneai)).
* `system.query_log` now records the databases, tables, and columns read by a replica that executes a query plan shipped to it, which happens with `parallel_replicas_plan_based`. [#122657](https://github.com/ClickHouse/ClickHouse/pull/122657) ([Igor Nikonov](https://github.com/devcrafter)).
* Queries with `send_logs_level` now also forward text logs from stateless `make_distributed_plan` workers through the coordinator, so worker-side exception context reaches the client. [#118603](https://github.com/ClickHouse/ClickHouse/pull/118603) ([Antonio Filipovic](https://github.com/antoniofilipovic)).
* The setting `parallel_replicas_insert_select_local_pipeline` is obsolete. It was a workaround for a performance problem fixed in 25.5, and it duplicated `parallel_replicas_local_plan`, which already decides whether the initiator reads its own share of a distributed `INSERT SELECT`. Use `parallel_replicas_local_plan = 0` to leave all the reading to the remote replicas. [#121653](https://github.com/ClickHouse/ClickHouse/pull/121653) ([Igor Nikonov](https://github.com/devcrafter)).
* Distributed `SELECT` queries now emit one OpenTelemetry span per remote shard read, with attributes such as `clickhouse.cluster`, `clickhouse.shard_num`, `clickhouse.processed_stage`, `clickhouse.query_id`, `clickhouse.initial_query_id`, and `clickhouse.target_host`, so slow or failed shards can be identified directly from `system.opentelemetry_span_log`. [#115784](https://github.com/ClickHouse/ClickHouse/pull/115784) ([Diego Gomes Tomé](https://github.com/diegomestre2)).
* Reads with parallel replicas are now covered by OpenTelemetry spans: a summary span with the totals of the coordination, one span per announcement and per read request handled by the coordinator, and `clickhouse.replica_num` on the per-replica spans, so the work distribution between replicas can be inspected in `system.opentelemetry_span_log`. [#118570](https://github.com/ClickHouse/ClickHouse/pull/118570) ([Diego Gomes Tomé](https://github.com/diegomestre2)).
* Added profile events for network activity: `DNSRequests`, `DNSRequestMicroseconds`, and `DNSRequestError`; `DNSReverseRequests`, `DNSReverseRequestMicroseconds`, and `DNSReverseError`; `TLSHandshakes`, `TLSHandshakeMicroseconds`, and `TLSHandshakeErrors`; `TLSServerHandshakes`, `TLSServerHandshakeMicroseconds`, and `TLSServerHandshakeErrors`; and `HTTPServerConnectionsErrors`. DNS now exposes timing as well as error counts, and TLS handshakes are now observable. [#116136](https://github.com/ClickHouse/ClickHouse/pull/116136) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the `AsyncInsertFlush` profile event. Compare it with `AsyncInsertQuery` to see how many queries are batched into one asynchronous `INSERT` flush. [#120187](https://github.com/ClickHouse/ClickHouse/pull/120187) ([Sema Checherinda](https://github.com/CheSema)).
* Add the `RuntimeFilterLookupsBeforeBuildFinished` profile event, counting blocks that a JOIN runtime filter passed through unfiltered because its build side had not finished yet (for example with `join_overflow_mode = 'break'` and a finite `max_rows_in_join` / `max_bytes_in_join`). [#107108](https://github.com/ClickHouse/ClickHouse/pull/107108) ([Groene AI](https://github.com/groeneai)).
* Added `last_error_symbols` and `last_error_lines` to `system.errors` and `system.error_log`. Closes [#74561](https://github.com/ClickHouse/ClickHouse/issues/74561). [#100374](https://github.com/ClickHouse/ClickHouse/pull/100374) ([Narasimha Pakeer](https://github.com/npakeer)).
* `system.processors_profile_log` now has entries for queries that fail with an exception, and new columns `exception_code` and `exception`. [#121503](https://github.com/ClickHouse/ClickHouse/pull/121503) ([Aleksandr Efimov](https://github.com/alexandrefimov)).
* `DROP TABLE` and `DROP DATABASE` now write a `RemovePart` event to `system.part_log` for every part of the dropped table, as merges, `DROP PARTITION`, and `TRUNCATE` already do. [#122093](https://github.com/ClickHouse/ClickHouse/pull/122093) ([Garrett Thomas](https://github.com/garrettthomaskth)).
* Added the server setting `min_allocation_size_to_log_stack_trace`. When a single charge to the global memory tracker reaches it, the stack trace is written to the server log and to `system.trace_log` with trace type `MemoryLargeAllocation`. This helps attribute a global tracked total that has grown far beyond real memory usage, a state in which the server refuses every allocation while far below its limit. Default `0` keeps it off. [#119527](https://github.com/ClickHouse/ClickHouse/pull/119527) ([Groene AI](https://github.com/groeneai)).
* Show throttler and workload IO scheduler wait time in the client progress bar (`ThrottlerSleepMicroseconds`, `SchedulerIOReadWaitMicroseconds` and `SchedulerIOWriteWaitMicroseconds` are summed into the `waited` figure, and the stalled sections of the bar are colored yellow). [#118536](https://github.com/ClickHouse/ClickHouse/pull/118536) ([Azat Khuzhin](https://github.com/azat)).
* `EXPLAIN ANALYZE time=1` shows more information about time and parallelism of every step. [#114786](https://github.com/ClickHouse/ClickHouse/pull/114786) ([Kirill Kopnev](https://github.com/Fgrtue)).
* Peak-memory reporting under `EXPLAIN ANALYZE` now accounts for aggregation correctly. [#123324](https://github.com/ClickHouse/ClickHouse/pull/123324) ([Azat Khuzhin](https://github.com/azat)).
* Removed nine unneeded `ProfileEvents`: the eight never-incremented `DistributedIndexAnalysisParts`, `ParallelReplicasDeniedRequests`, `WasmMemoryAllocated`, `S3QueueSetFileProcessingMicroseconds`, `S3QueueSetFileProcessedMicroseconds`, `S3QueueSetFileFailedMicroseconds`, `FileSegmentRemoveMicroseconds`, and `ExternalDataSourceLocalCacheReadBytes`, plus the duplicate `CachedReadBufferPredownloadedBytes`, which always matched `CachedReadBufferPredownloadedFromSourceBytes`. This reduces per-query memory and bookkeeping overhead; queries and dashboards should use `CachedReadBufferPredownloadedFromSourceBytes`. [#123040](https://github.com/ClickHouse/ClickHouse/pull/123040) ([Raúl Marín](https://github.com/Algunenano)). [#123042](https://github.com/ClickHouse/ClickHouse/pull/123042) ([Raúl Marín](https://github.com/Algunenano)).
* Killing a query that is loading skip-index marks from object storage now stops the read promptly: the remote read path checks query cancellation before each buffer fill, instead of reading on for minutes after the query was already cancelled. [#113436](https://github.com/ClickHouse/ClickHouse/pull/113436) ([Groene AI](https://github.com/groeneai)).
* Reduced cancellation latency for `GROUP BY ... WITH TOTALS` queries: `KILL QUERY` now interrupts totals accumulation within a single chunk instead of waiting for all aggregate states to be merged. [#119470](https://github.com/ClickHouse/ClickHouse/pull/119470) ([Roman Vasin](https://github.com/rvasin)).
* `KILL QUERY` and Ctrl+C in `clickhouse-client` now cancel long-running functions inside `WHERE` and `HAVING` filters instead of waiting for the function to finish. [#112705](https://github.com/ClickHouse/ClickHouse/pull/112705) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `KILL MUTATION`, `SYSTEM STOP MERGES`, `DETACH`/`DROP` of a table, `REPLACE`/`ATTACH PARTITION ... FROM` and server shutdown now stop a running mutation within a fraction of a second when the mutation's predicate has an `IN (subquery)` whose subquery works without reading new data (for example a cross join). Previously they waited until the whole set was built, and `REPLACE`/`ATTACH PARTITION ... FROM` gave up with `TIMEOUT_EXCEEDED` after 120 seconds. `KILL MUTATION` now also stops a mutation that is already writing its new part. [#124122](https://github.com/ClickHouse/ClickHouse/pull/124122) ([Groene AI](https://github.com/groeneai)).
* `KILL QUERY` and `max_execution_time` (with the default `timeout_overflow_mode = 'throw'`) now interrupt writing the `Parquet` format in the middle of a block that is split into many row groups (for example with a small `output_format_parquet_row_group_size`), instead of only after the whole block is written. [#125489](https://github.com/ClickHouse/ClickHouse/pull/125489) ([Groene AI](https://github.com/groeneai)).
* `DROP TABLE`, `DETACH TABLE`, server shutdown, and `ALTER TABLE ... MODIFY SETTING refresh_statistics_interval` no longer wait for a background refresh of the statistics cache of a `MergeTree` table to read every part. On object storage with thousands of parts this wait could take minutes per table and delay a restart. Closes [#125136](https://github.com/ClickHouse/ClickHouse/issues/125136). [#125165](https://github.com/ClickHouse/ClickHouse/pull/125165) ([Groene AI](https://github.com/groeneai)).
* `DROP TABLE` and other operations that cancel merges and mutations no longer wait for an in-progress `vector_similarity` index build to finish. [#123062](https://github.com/ClickHouse/ClickHouse/pull/123062) ([Raúl Marín](https://github.com/Algunenano)). [#123739](https://github.com/ClickHouse/ClickHouse/pull/123739) ([Raúl Marín](https://github.com/Algunenano)).
* `KILL QUERY WHERE query_id = '<id>'` now cancels the caller's own query without requiring `SELECT` on `system.processes`. For a caller holding neither that grant nor `KILL QUERY`, an id that is not running as them cancels nothing and returns no rows. Any other `WHERE` condition keeps requiring the privileges it requires today. [#117999](https://github.com/ClickHouse/ClickHouse/pull/117999) ([Groene AI](https://github.com/groeneai)).
* Access to `system.jemalloc_profile_text`, `system.jemalloc_sampled_allocations`, `system.jemalloc_bins`, `system.jemalloc_arena_bins`, and `system.jemalloc_stats` now requires `SYSTEM_JEMALLOC`; the heap profiles flushed by `system.jemalloc_profile_text` and `system.jemalloc_sampled_allocations` are also removed after being returned. [#121622](https://github.com/ClickHouse/ClickHouse/pull/121622) ([Azat Khuzhin](https://github.com/azat)). [#121634](https://github.com/ClickHouse/ClickHouse/pull/121634) ([Azat Khuzhin](https://github.com/azat)).
* Added the `access_control_improvements.readonly_restricts_set_role` server setting, `false` by default. When enabled, `SET ROLE` is refused with `READONLY` in a session or query whose `readonly` setting is not zero. Previously `SET ROLE` was the only identity-changing statement that `readonly` did not refuse. [#118021](https://github.com/ClickHouse/ClickHouse/pull/118021) ([Groene AI](https://github.com/groeneai)).
* Secret arguments of table functions, table engines, database engines, backup engines, and dictionary sources are now declared next to each feature and are masked consistently in `SHOW CREATE`, system tables, query logs, and `EXPLAIN AST`, instead of relying on a central hardcoded parser list. [#123364](https://github.com/ClickHouse/ClickHouse/pull/123364) ([Raúl Marín](https://github.com/Algunenano)).
* The `PostgreSQL`, `MaterializedPostgreSQL`, `DataLakeCatalog`, `Cluster`, `Remote`, and `HDFS` database engines key the exemption for the server's replay of its own stored metadata on the metadata loader instead of on the query being internal. A `HDFS` database restored with `RESTORE DATABASE` is now checked against `remote_url_allow_hosts`. [#120280](https://github.com/ClickHouse/ClickHouse/pull/120280) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Allow `remote_url_allow_hosts` to match the bucket name of an `S3` URL. [#124453](https://github.com/ClickHouse/ClickHouse/pull/124453) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* Support proxy credentials in the `http_proxy` and `https_proxy` environment variables. Previously the userinfo part of the proxy URI was discarded, so ClickHouse could not use an authenticating proxy. Proxy credentials are sent only to the proxy, rather than to the target server inside a `CONNECT` tunnel. [#123069](https://github.com/ClickHouse/ClickHouse/pull/123069) ([Akanshaak](https://github.com/Akanshaak)). [#124396](https://github.com/ClickHouse/ClickHouse/pull/124396) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Interserver authentication failures on a listener that uses the PROXY protocol now record the client address from the PROXY header in `system.session_log` when `auth_use_forwarded_address` is enabled, instead of the proxy address. [#125326](https://github.com/ClickHouse/ClickHouse/pull/125326) ([Groene AI](https://github.com/groeneai)).
* Added the `tcp_with_proxy_allow_interserver_mode` server setting, which allows disabling the native protocol's interserver mode on `tcp_with_proxy_port`. [#123188](https://github.com/ClickHouse/ClickHouse/pull/123188) ([Pradeep Chhetri](https://github.com/chhetripradeep)).
* Reject unexpected packets in the native protocol before deserializing their payload, avoiding unnecessary processing of data types and aggregate function states before returning `UNEXPECTED_PACKET_FROM_CLIENT`. [#119368](https://github.com/ClickHouse/ClickHouse/pull/119368) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the setting `kill_throw_if_noop`, enabled by default, which makes local `KILL QUERY` and `KILL MUTATION` statements throw `NO_QUERY_TO_KILL` when their selection produces no eligible target. `ON CLUSTER` execution remains tolerant of empty matches. Set `kill_throw_if_noop = false` to preserve the previous behavior. Related [#83348](https://github.com/ClickHouse/ClickHouse/issues/83348). [#85023](https://github.com/ClickHouse/ClickHouse/pull/85023) ([zwy991114](https://github.com/zwy991114)).
* Merges and queries on `S3` disks no longer hold one HTTP connection per open stream for their whole duration once connections are scarce. Previously this let a few merges of tables with many substreams, for example a `JSON` column, saturate `disk_connections_hard_limit`, stall each other, and fail every other request to the disks with `HTTP_CONNECTION_LIMIT_REACHED`. Once the `Disk` connection group is at or above `disk_connections_soft_limit`, reads from `S3` disks fetch each buffer fill with a separate request and return the connection to the pool between fills; the new profile event `ReadBufferFromS3RequestsCut` counts such requests. [#119730](https://github.com/ClickHouse/ClickHouse/pull/119730) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#124701](https://github.com/ClickHouse/ClickHouse/pull/124701) ([Groene AI](https://github.com/groeneai)).
* The `filesystem_cache_boundary_alignment` setting can now be changed per query. For disk reads, `0` means to use the cache configuration's `boundary_alignment`; for non-disk reads, `0` still disables alignment. [#118791](https://github.com/ClickHouse/ClickHouse/pull/118791) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Added the filesystem cache setting `use_real_disk_size` (disabled by default), which accounts for cache reservations in filesystem allocation units instead of raw bytes. It makes `FilesystemCacheSize`, eviction-size metrics, and `current_size` in `system.filesystem_cache_settings` approximate the disk footprint. The accounting is block-aligned rather than exact and can include reserve-ahead bytes of in-flight reads. [#79490](https://github.com/ClickHouse/ClickHouse/pull/79490) ([codeworse](https://github.com/codeworse)).
* Custom HTTP headers configured for S3 are now handled case-insensitively by name, so a header spelled `X-Amz-Meta-Foo` reaches Google Cloud Storage translated to `x-goog-meta-foo`, as `x-amz-meta-foo` already did. [#120014](https://github.com/ClickHouse/ClickHouse/pull/120014) ([Sema Checherinda](https://github.com/CheSema)).
* Better error messages for S3 request retries: the HTTP response code is no longer printed as `18446744073709551615` when the request received no response at all, and the misleading message about a possible time skew is no longer printed on every retry. [#118439](https://github.com/ClickHouse/ClickHouse/pull/118439) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Allow `DESCRIBE FILESYSTEM CACHE` on the introspection port. [#115894](https://github.com/ClickHouse/ClickHouse/pull/115894) ([Groene AI](https://github.com/groeneai)).
* Files on encrypted disks now support positional reads, so a table with a `UNIQUE KEY` on an encrypted disk can be loaded again after a server restart. [#119991](https://github.com/ClickHouse/ClickHouse/pull/119991) ([johnjing](https://github.com/JingYanchao)).
* The `url` table function and the `URL` table engine now expand glob patterns lazily instead of materializing every address before the query starts. `glob_expansion_max_elements` now limits how many addresses a query actually generates rather than how large the pattern is, so a query that stops early - for example under a `LIMIT` - can use a pattern that describes far more addresses than the limit. [#115680](https://github.com/ClickHouse/ClickHouse/pull/115680) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `Iceberg` manifest writing now uses sequence-number inheritance: `ADDED` entries have `NULL` `sequence_number` and `file_sequence_number` unless compaction assigns an explicit sequence number. [#120031](https://github.com/ClickHouse/ClickHouse/pull/120031) ([Asya Shneerson](https://github.com/asya-ch)).
* `Iceberg` manifest statistics are now written for `boolean` and `fixed` columns. [#121548](https://github.com/ClickHouse/ClickHouse/pull/121548) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* When a table is created through a catalog, `CREATE TABLE catalog.table ENGINE = IcebergS3` can now omit explicit credentials and reuse the catalog credentials. [#114477](https://github.com/ClickHouse/ClickHouse/pull/114477) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* `ALTER TABLE ... EXECUTE expire_snapshots` now works for `Iceberg` tables in transactional catalogs such as Iceberg REST and Unity. [#123248](https://github.com/ClickHouse/ClickHouse/pull/123248) ([alesapin](https://github.com/alesapin)).
* A `DataLakeCatalog` database whose catalog cannot be built no longer reports its tolerated table-listing failure as an error of the query that enumerated it: the skipped database is now logged at `debug` instead of `error`, so a query such as `SELECT ... FROM system.tables` or `system.iceberg_history` no longer sends a server-side exception to a client whose `send_logs_level` includes `error`. Set `show_data_lake_catalogs_in_system_tables = 1` to have the real exception raised instead. [#121475](https://github.com/ClickHouse/ClickHouse/pull/121475) ([Groene AI](https://github.com/groeneai)).
* When a `DeltaLake` write commit fails, ClickHouse now always reports the commit error rather than an error raised while removing the data files the failed `INSERT` had already uploaded, and every removal that fails during that cleanup is logged with the path it was left on. Previously the first failing removal replaced the commit error in the response, so the reason nothing was written was lost and no path was logged. Closes [#122023](https://github.com/ClickHouse/ClickHouse/issues/122023). [#122123](https://github.com/ClickHouse/ClickHouse/pull/122123) ([Groene AI](https://github.com/groeneai)).
* Reading an `Iceberg` table whose position delete file has no `pos` or `file_path` column now fails with `ICEBERG_SPECIFICATION_VIOLATION` naming the file and the missing column, instead of a `LOGICAL_ERROR` (missing `pos`) or a misleading `UNKNOWN_IDENTIFIER` (missing `file_path`). [#123583](https://github.com/ClickHouse/ClickHouse/pull/123583) ([Groene AI](https://github.com/groeneai)).
* `Paimon` `BOOLEAN` columns are now read as `Bool` instead of `Int8`, matching `Iceberg` and `DeltaLake`. `DESCRIBE` and `toTypeName` now report `Bool`, and text formats render the values as `false` and `true` instead of `0` and `1`. [#119404](https://github.com/ClickHouse/ClickHouse/pull/119404) ([JIaQi Tang](https://github.com/JiaQiTang98)).
* Avro files now include per-column size statistics in their metadata. [#121392](https://github.com/ClickHouse/ClickHouse/pull/121392) ([Konstantin Vedernikov](https://github.com/scanhex12)).
* The `Parquet` reader now rejects a column chunk whose metadata has a negative `num_values`, `total_uncompressed_size`, or `unencoded_byte_array_data_bytes` with `INCORRECT_DATA` instead of failing later with a misleading memory-allocation error. It also caps memory estimates derived from these fields at the reader's watermarks and ignores pre-allocations too large to ever succeed, so absurdly large values no longer cause an internal error. Closes [#124224](https://github.com/ClickHouse/ClickHouse/issues/124224). [#124330](https://github.com/ClickHouse/ClickHouse/pull/124330) ([Groene AI](https://github.com/groeneai)).
* Bumped the bundled `ORC` library so that an `ORC` file declaring a compression codec the reader does not implement is rejected with an error naming the codec, instead of an internal error that debug and sanitizer builds treat as a failed assertion. [#115889](https://github.com/ClickHouse/ClickHouse/pull/115889) ([Groene AI](https://github.com/groeneai)).
* `INSERT ... VALUES` over a stream that fails mid-row now reports that read failure directly, instead of retrying the failed read while recovering from a parse error. This affects an HTTP request body that is shorter than its declared `Content-Length` and is streamed to the input format. [#121916](https://github.com/ClickHouse/ClickHouse/pull/121916) ([Groene AI](https://github.com/groeneai)).
* ClickHouse Keeper no longer drops the client session when the outgoing request queue stays full for longer than `operation_timeout_ms`. The enqueue wait is now three times `session_timeout_ms`, so a short burst of backpressure no longer invalidates the session's ephemeral nodes and watches. [#121559](https://github.com/ClickHouse/ClickHouse/pull/121559) ([Kseniia Sumarokova](https://github.com/kssenii)).
* ClickHouse Keeper now reports orphaned nodes in a snapshot as `CORRUPTED_DATA` instead of an opaque internal error. With `keeper_server.remove_orphaned_nodes_on_startup = true` and `keeper_server.digest_enabled = false`, it can also remove those nodes during startup so the snapshot still loads. [#95761](https://github.com/ClickHouse/ClickHouse/pull/95761) ([Seva Potapov](https://github.com/seva-potapov)).
* The Keeper `rcfg` command now honors its `retry` count per action, stops once `max_total_wait_time_ms` is exhausted, and rejects a negative `retry` before running any action. [#122758](https://github.com/ClickHouse/ClickHouse/pull/122758) ([Groene AI](https://github.com/groeneai)).
* Updated `NuRaft` to fix handling of abandoned peers when a removed server rejoins. [#120736](https://github.com/ClickHouse/ClickHouse/pull/120736) ([Antonio Andelic](https://github.com/antonio2368)).
* Added the setting `get_zookeeper_lock_acquire_timeout_ms` so Keeper-backed queries such as reads from `system.zookeeper`, `zookeeperSessionUptime`, and `SYSTEM RECONNECT ZOOKEEPER` can fail fast while another thread is reconnecting Keeper. [#94836](https://github.com/ClickHouse/ClickHouse/pull/94836) ([c-end](https://github.com/c-end)).
* Activating a replica whose `active` or `is_active` node in `Keeper` is still held by another session is now reported as `REPLICA_ALREADY_EXISTS` instead of `LOGICAL_ERROR`, so it no longer aborts the server in debug and sanitizer builds. The same applies when that node is rewritten while it is being removed. [#114408](https://github.com/ClickHouse/ClickHouse/pull/114408) ([Groene AI](https://github.com/groeneai)).
* The ZooKeeper client, the Graphite metrics writer, the Redis source, `system.zookeeper_info`, the distributed query exchange, and host names in `listen_host` and `bind_host` now resolve through the ClickHouse DNS cache, so these lookups are cached, visible in `system.dns_cache` and subject to `dns_allow_resolve_names_to_ipv4` and `dns_allow_resolve_names_to_ipv6`. Previously they resolved names on their own, bypassing the cache. [#120831](https://github.com/ClickHouse/ClickHouse/pull/120831) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the `http_x_clickhouse_format_overrides_output_format` setting (default `true`). When disabled, the `X-ClickHouse-Format` HTTP header only sets `default_format`, as it did before 26.8, instead of overriding a query's `FORMAT` clause via `output_format`. [#121132](https://github.com/ClickHouse/ClickHouse/pull/121132) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The Prometheus HTTP API accepts `RFC3339` times with `Z` or a `+hh:mm` offset and fractional seconds in `time`, `start`, and `end`, as Prometheus does, so clients like vmalert work. String time arguments of `prometheusQuery`, `timeSeriesSelector`, and the `timeSeries*ToGrid` functions accept `RFC3339` too. [#122808](https://github.com/ClickHouse/ClickHouse/pull/122808) ([Valery Petrov](https://github.com/valerypetrov)).
* PromQL regex matchers that list plain literals, such as `{job=~"api|web"}`, now use the primary key and return exactly the listed values, as in Prometheus. Before, only the first alternative was anchored at the start and only the last at the end, so such a matcher also returned values like `api-server` or `myweb`. [#122796](https://github.com/ClickHouse/ClickHouse/pull/122796) ([Valery Petrov](https://github.com/valerypetrov)).
* PromQL instant queries now return series in a deterministic order: `topk` and `bottomk` at the root are sorted by value, as Prometheus returns them, and other instant queries are sorted by their labels, like range queries, so repeated requests list the series in the same order. [#122347](https://github.com/ClickHouse/ClickHouse/pull/122347) ([James Cunningham](https://github.com/JTCunning)). [#122976](https://github.com/ClickHouse/ClickHouse/pull/122976) ([Valery Petrov](https://github.com/valerypetrov)).
* `/api/v1/format_query` and PromQL serialization now print durations in Prometheus units, so `sum(rate(x[5m]))` formats as itself rather than as `sum(rate(x[300]))`. [#120939](https://github.com/ClickHouse/ClickHouse/pull/120939) ([Valery Petrov](https://github.com/valerypetrov)).
* PromQL now validates label names in `label_join` and `label_replace`. [#118190](https://github.com/ClickHouse/ClickHouse/pull/118190) ([Minh Vu](https://github.com/fallintoplace)).
* A Prometheus remote-write or remote-read request whose body cannot be decompressed is now answered with `400` instead of `500`, and the error names the size of the block and the uncompressed length it declares. [#120938](https://github.com/ClickHouse/ClickHouse/pull/120938) ([Valery Petrov](https://github.com/valerypetrov)).
* Added the `merge_selector_min_age_to_disable_right_tail_heuristic` `MergeTree` setting. It lets the `Simple` and `StochasticSimple` merge selectors include small parts at the right end of a range in merge candidates once every part in the range is old enough. [#122638](https://github.com/ClickHouse/ClickHouse/pull/122638) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Added the `MergeTree` settings `merge_selector_small_parts_min_count`, `merge_selector_small_parts_threshold`, and `merge_selector_small_parts_max_age` to reduce the number of merge operations under rapid small-part insertion by requiring a minimum batch size before merging small fresh parts. [#100436](https://github.com/ClickHouse/ClickHouse/pull/100436) ([Amos Bird](https://github.com/amosbird)).
* Calculate projections using the query analyzer and reject missing projection columns rather than writing incomplete projection parts. Row policies and `additional_table_filters` for `system.one` no longer affect projection calculation. [#118375](https://github.com/ClickHouse/ClickHouse/pull/118375) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `ALTER TABLE ... ADD COLUMN ..., MODIFY ORDER BY ...` now rejects a new sorting key that uses a subcolumn of the added `DEFAULT` or `MATERIALIZED` column, and its error message names whether the expression kind is `DEFAULT` or `MATERIALIZED` instead of always saying `default expression`. [#121873](https://github.com/ClickHouse/ClickHouse/pull/121873) ([GON](https://github.com/gonappuccino)).
* An unresolved name in a column `DEFAULT`, `MATERIALIZED`, or `ALIAS` expression is now reported by the name the user wrote, instead of an internal `<column>_tmp_alter<random>` alias that appears nowhere in the query and also broke the "Maybe you meant" hint. A sorting, primary, or partition key that contains a nullable column or a constant now names that column or constant. [#116871](https://github.com/ClickHouse/ClickHouse/pull/116871) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Web UI: save and load the list of tabs, their queries, and query parameters as a Markdown file. The tab bar is now always shown, and the connection settings always live behind the key button in its corner. [#121234](https://github.com/ClickHouse/ClickHouse/pull/121234) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Web UI: added a `Format` button to the query editor, which formats the query with `formatQuery`. [#121652](https://github.com/ClickHouse/ClickHouse/pull/121652) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#122477](https://github.com/ClickHouse/ClickHouse/pull/122477) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Web UI: clicking a cell of a `JSON` column pretty-prints the value with two-space indentation and syntax highlighting. [#118640](https://github.com/ClickHouse/ClickHouse/pull/118640) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The graph view of the Web UI for `EXPLAIN ... graph = 1` now follows the light/dark theme instead of always drawing a white canvas with black lines, and its labels use a sans-serif font instead of the Graphviz default. Rendering a graph also works when `play.html` is opened as a local file. [#116208](https://github.com/ClickHouse/ClickHouse/pull/116208) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* In the Web UI, the digits of a long number in the result table are now grouped by a thin gap, as in `1 234 567`, instead of an underline. The gap is an empty element, so selecting or copying the number still yields the digits alone. [#119839](https://github.com/ClickHouse/ClickHouse/pull/119839) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Show a checkmark on the Play UI result-table copy button for 1.2 s after a cell value is copied to the clipboard. [#120617](https://github.com/ClickHouse/ClickHouse/pull/120617) ([yanglongwei](https://github.com/ylw510)).
* In the Web UI, the browser window title of a renamed query tab is now exactly the tab's name. [#122067](https://github.com/ClickHouse/ClickHouse/pull/122067) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Web UI: when a login last succeeded with an empty password, a stale password autofilled by the browser's password manager for that server and user is no longer used. [#118638](https://github.com/ClickHouse/ClickHouse/pull/118638) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The schema visualizer now supports hiding and showing table categories by clicking their legend labels, with a Hide all / Show all control. [#124963](https://github.com/ClickHouse/ClickHouse/pull/124963) ([Mark Needham](https://github.com/mneedham)).
* The command-line client no longer shows as-you-type hints on a query recalled from the history or otherwise displayed at once, so pressing Up/Down keeps navigating the history until the line is edited. [#115065](https://github.com/ClickHouse/ClickHouse/pull/115065) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The progress bar in `clickhouse-client` and `clickhouse-local` no longer flickers when the query result is redirected to a pipe or a file. [#123549](https://github.com/ClickHouse/ClickHouse/pull/123549) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A bare table name can now use the archive path syntax in `clickhouse-local`, so `SELECT * FROM 'archive.tar.zst::data.native'` reads a file inside an archive, like `SELECT * FROM file('archive.tar.zst::data.native')`. The same applies to tables of a `URL` database with a `file://` target and of a `Filesystem` database. [#118170](https://github.com/ClickHouse/ClickHouse/pull/118170) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Clean up client output when a web terminal or SSH session is closed during a running query, avoiding spurious write-buffer errors. [#125558](https://github.com/ClickHouse/ClickHouse/pull/125558) ([Groene AI](https://github.com/groeneai)).
* A musl-built ClickHouse binary is now self-contained enough to run in a `FROM scratch` container image with no extra files: CA certificates are embedded, `localhost` resolves to loopback even without `/etc/hosts`, and DNS no longer falls back to the unreachable localhost when `/etc/resolv.conf` is missing. [#115389](https://github.com/ClickHouse/ClickHouse/pull/115389) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Hedged requests are now supported on macOS. Previously `use_hedged_requests` was silently ignored on macOS and distributed queries always used non-hedged connections. [#119915](https://github.com/ClickHouse/ClickHouse/pull/119915) ([Raúl Marín](https://github.com/Algunenano)).
* Determine the number of CPUs from the kernel's sysfs lists instead of the libc, so per-CPU structures and `auto` thread counts do not depend on the libc's interpretation of the affinity mask. [#118912](https://github.com/ClickHouse/ClickHouse/pull/118912) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Workloads created without a parent workload are now allowed; they become children of an implicit root workload, so several workload trees can coexist. [#118578](https://github.com/ClickHouse/ClickHouse/pull/118578) ([Sergei Trifonov](https://github.com/serxa)).
* `RANDOMIZE FOR` is now applied on each replica of a refreshable materialized view independently, so a random replica performs each refresh instead of the same one every time. Replicas no longer agree on `next_refresh_time`, so the value reported by `system.view_refreshes` on one replica is the time that replica would refresh at, not necessarily when the refresh happens. Closes [#78410](https://github.com/ClickHouse/ClickHouse/issues/78410). [#112979](https://github.com/ClickHouse/ClickHouse/pull/112979) ([Groene AI](https://github.com/groeneai)).
* `FileLog` now reads a file again from the beginning if, at the next read, it is shorter than the offset already read from it (for example after `logrotate` with `copytruncate`), instead of failing every read of the table with `CANNOT_READ_ALL_DATA` until the file grows back to its old size. [#123390](https://github.com/ClickHouse/ClickHouse/pull/123390) ([Groene AI](https://github.com/groeneai)).
* A relative path in the `FileLog` table engine is now resolved against `user_files_path`, like in the `file` table function, so `FileLog('nginx/', 'JSONEachRow')` reads `<user_files_path>/nginx/`. A relative path that already points inside `user_files_path` from the server's working directory keeps its meaning. [#123397](https://github.com/ClickHouse/ClickHouse/pull/123397) ([Groene AI](https://github.com/groeneai)).
* Connections from the `MongoDB` table engine, table function, and dictionary source now identify themselves to the MongoDB server with the ClickHouse name and version in the driver handshake metadata. MongoDB operators can tell ClickHouse connections apart from other clients in server logs, `currentOp` output, and monitoring tools. [#122205](https://github.com/ClickHouse/ClickHouse/pull/122205) ([bisht2050](https://github.com/bisht2050)).
* Lookups of `YTsaurus` dictionaries can now be throttled. [#100044](https://github.com/ClickHouse/ClickHouse/pull/100044) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* AI functions using the `openai`, `huggingface`, or `tei` provider send the query ID and the initial query ID in the `X-ClickHouse-Query-Id` and `X-ClickHouse-Initial-Query-Id` HTTP headers. [#123622](https://github.com/ClickHouse/ClickHouse/pull/123622) ([George Larionov](https://github.com/george-larionov)).
* `aiFilter` now requests structured JSON Schema output. [#121526](https://github.com/ClickHouse/ClickHouse/pull/121526) ([Andrew Wilkins](https://github.com/axw)).
* WebAssembly UDF calls with `ABI BUFFERED_V1` now measure the whole input batch instead of one serialization per row, so usage is priced by the stream the guest actually receives for formats whose cost is not additive across rows. [#118989](https://github.com/ClickHouse/ClickHouse/pull/118989) ([Vasily Chekalkin](https://github.com/bacek)).
* Allow the `ON CLUSTER` clause for `RENAME TABLE` and `EXCHANGE TABLES` on a `Replicated` or Cloud `Shared` database when the cluster name matches the database's automatic cluster name. The clause is treated as a no-op and the database engine propagates the DDL as usual, which lets scripts written for non-`Replicated` databases be ported without rewriting `ON CLUSTER` clauses. [#106192](https://github.com/ClickHouse/ClickHouse/pull/106192) ([Groene AI](https://github.com/groeneai)).
* Stored HTTP handler queries no longer break when a request supplies `max_parser_depth`, `max_parser_backtracks`, or `max_query_size`, and the `CREATE HANDLER`, `ALTER HANDLER`, and `DROP HANDLER` documentation is now published. [#114883](https://github.com/ClickHouse/ClickHouse/pull/114883) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Improve memory tracking for query sources, including recursive CTEs and external data sources, so their container allocations count toward memory limits. [#124952](https://github.com/ClickHouse/ClickHouse/pull/124952) ([Nikita Taranov](https://github.com/nickitat)).
* A subquery with its own `SETTINGS` clause now plans `GROUP BY` using the settings after constraints and `max_threads` clamping are applied, instead of reapplying the clause as written. This prevents nested `SETTINGS max_threads = ...` from bypassing settings constraints or expanding the aggregation pipeline to an excessive number of streams. [#123156](https://github.com/ClickHouse/ClickHouse/pull/123156) ([Groene AI](https://github.com/groeneai)).
* Improved the error message of `parseDateTime`. [#120489](https://github.com/ClickHouse/ClickHouse/pull/120489) ([Robert Schulze](https://github.com/rschu1ze)).
* Restoring an invalid data part from backup now reports `BACKUP_DAMAGED` instead of `CORRUPTED_DATA`. [#113085](https://github.com/ClickHouse/ClickHouse/pull/113085) ([Alexander Tokmakov](https://github.com/tavplubix)).
* Deserializing an aggregate function state that declares an oversized element count now reports `TOO_LARGE_ARRAY_SIZE` and names both the offending count and the limit, instead of a bare `POCO_EXCEPTION` carrying no numbers. [#121381](https://github.com/ClickHouse/ClickHouse/pull/121381) ([Groene AI](https://github.com/groeneai)).
* A setting given a value of the wrong type or out of range now names the setting and the value in the error message. [#116882](https://github.com/ClickHouse/ClickHouse/pull/116882) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* When a dictionary with a `PostgreSQL` source and a custom `QUERY` returns its columns in an order that does not match the dictionary structure, the error now names the destination dictionary column and the column order the query must return, instead of only reporting that a value could not be parsed. [#114212](https://github.com/ClickHouse/ClickHouse/pull/114212) ([Groene AI](https://github.com/groeneai)).
* A `PostgreSQL` dictionary whose `QUERY` returns fewer columns than the dictionary structure now fails to load with a `NUMBER_OF_COLUMNS_DOESNT_MATCH` error that names the column order the `QUERY` must use, instead of a `LOGICAL_ERROR`. A `QUERY` that returns too many columns gets the same column-order hint. [#125397](https://github.com/ClickHouse/ClickHouse/pull/125397) ([Groene AI](https://github.com/groeneai)).
* A column matcher (`*`, `t.*`, `COLUMNS('...')`) inside `additional_table_filters`, `additional_result_filter` or a row policy is now rejected with a clear `BAD_ARGUMENTS` diagnostic instead of the obscure `Unqualified matcher * cannot be resolved. There are no table sources`. `CREATE` / `ALTER ROW POLICY` with a matcher now fails immediately. A matcher inside a subquery of such a filter, for example `x IN (SELECT * FROM allowed)`, keeps working. [#119250](https://github.com/ClickHouse/ClickHouse/pull/119250) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A matcher (`*`, `t.*`, `COLUMNS(...)`) used as a whole `LIMIT AFTER` or `LIMIT UNTIL` boundary is now rejected with `UNEXPECTED_EXPRESSION`, as in `WHERE`, instead of failing with `std::out_of_range` or silently using only its first column. [#123309](https://github.com/ClickHouse/ClickHouse/pull/123309) ([Groene AI](https://github.com/groeneai)).
* `CREATE FUNCTION` now rejects a SQL user-defined function whose lambda arguments cannot be evaluated, instead of storing a function that later fails with `BAD_ARGUMENTS` on every call. Closes [#121197](https://github.com/ClickHouse/ClickHouse/issues/121197). [#121370](https://github.com/ClickHouse/ClickHouse/pull/121370) ([Groene AI](https://github.com/groeneai)).
* A correlated subquery in `HAVING` of a `WITH TOTALS` query that also uses `max_rows_to_group_by` with `group_by_overflow_mode = 'any'` (and `totals_mode` other than `after_having_exclusive`) now fails with `NOT_IMPLEMENTED` instead of a logical error. [#122601](https://github.com/ClickHouse/ClickHouse/pull/122601) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The `mergeTreeAnalyzeIndexes` and `mergeTreeAnalyzeIndexesUUID` table functions now reject a `vector_search_index_analysis` limit above `max_limit_for_vector_search_queries`, the bound under which the plan optimization rewrites a vector search query. A larger limit previously made the vector similarity index reserve memory proportional to it and exhaust the server's memory. [#122154](https://github.com/ClickHouse/ClickHouse/pull/122154) ([Groene AI](https://github.com/groeneai)).
* An unconditioned `CROSS JOIN` is no longer reported as `INNER` in `EXPLAIN` and in the join columns of `system.query_log` when the `dpsub` join order algorithm runs with a conflict detector enabled. Query results were not affected. [#122236](https://github.com/ClickHouse/ClickHouse/pull/122236) ([Fisnik Kastrati](https://github.com/fkastrati)).
* Formatting a query where an `IN` is the left operand of a tuple element (`(x IN (1)).1`) inside a multi-argument function call is now stable across a format-parse-format round trip. [#124325](https://github.com/ClickHouse/ClickHouse/pull/124325) ([Groene AI](https://github.com/groeneai)).
* Exchanging `-Merge` aggregate states with older servers now stays compatible: the state version of a `-Merge` function follows the state version of the function it merges only for peers that announce the new protocol revision `54494`. [#123070](https://github.com/ClickHouse/ClickHouse/pull/123070) ([Groene AI](https://github.com/groeneai)).
* An aggregate function parameter whose value has no unambiguous bare literal form, such as a `Decimal`, a wide integer, a `UUID`, an `IPv4` or `IPv6`, or a non-negative `Int64`, is now always printed with an explicit `::Type` suffix, and inside an array or tuple parameter the suffix goes on the elements. Such a parameter therefore reads back as the same value for every family, instead of only for the ones that had opted in. The typed form also travels over the `Native` protocol, so an insert into a column whose parameter was spelled without the type is now rejected there too, as it already was locally. [#121725](https://github.com/ClickHouse/ClickHouse/pull/121725) ([Groene AI](https://github.com/groeneai)).
* Diagnostics about slowly acquired locks are now logged at `Information` level instead of `Warning`, so they no longer reach clients that use `send_logs_level = 'warning'` while still being written to the server log with a stack trace. [#123797](https://github.com/ClickHouse/ClickHouse/pull/123797) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Adaptive aggregation now logs less noisily by reducing the severity of its messages. [#123312](https://github.com/ClickHouse/ClickHouse/pull/123312) ([Azat Khuzhin](https://github.com/azat)).
* `kqlRangeCount` now rejects the `Int256` minimum span with step `-1` instead of incorrectly reporting an empty range. [#117212](https://github.com/ClickHouse/ClickHouse/pull/117212) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A distributed query whose shards plan aggregation with the experimental `enable_cascades_optimizer` no longer hits the logical error `GroupingAggregatedTransform pushed bucket N twice` on the initiator. [#119814](https://github.com/ClickHouse/ClickHouse/pull/119814) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Local execution of distributed query plans (`distributed_plan_execute_locally = 1`) now materializes constant columns in its in-memory exchange like serialized exchanges do, fixing a query that could fail with a `LOGICAL_ERROR` when it sorted by a constant expression. [#124097](https://github.com/ClickHouse/ClickHouse/pull/124097) ([Groene AI](https://github.com/groeneai)).
* The internal function `__bitSwapLastTwo` now raises the same `BAD_ARGUMENTS` error for unsupported argument types when the expression is JIT-compiled as when it is interpreted. [#117613](https://github.com/ClickHouse/ClickHouse/pull/117613) ([Groene AI](https://github.com/groeneai)).
* Fix a UBSan failure during parallel parsing of `TabSeparated` input with an empty buffer. [#122636](https://github.com/ClickHouse/ClickHouse/pull/122636) ([Ray Winkelman](https://github.com/rwinkelman)).
* The `ErrorCode` registry is now sparse, so ClickHouse can reserve gaps between error codes without preallocating counters for them. [#111796](https://github.com/ClickHouse/ClickHouse/pull/111796) ([Maxim Orlovsky](https://github.com/UberDever)).
* Fixed a signed integer overflow when a negative or huge `lock_acquire_timeout` value is used as the timeout for scheduling a background task. [#117898](https://github.com/ClickHouse/ClickHouse/pull/117898) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Cancelled marks-loading background tasks are no longer counted as errors in `system.errors` and `system.error_log`. Previously `ASYNC_LOAD_CANCELED` was recorded during normal operation whenever a reader was dropped before its asynchronously loaded marks were needed. [#118928](https://github.com/ClickHouse/ClickHouse/pull/118928) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* `ALTER TABLE ... EXECUTE expire_snapshots` for `Iceberg` tables now counts a file as deleted only when its removal succeeds, reports the removals that return an error in a new `failed_deletions_count` metric like `remove_orphan_files`, and counts a file that several expired manifests list only once. Closes [#125332](https://github.com/ClickHouse/ClickHouse/issues/125332). [#125379](https://github.com/ClickHouse/ClickHouse/pull/125379) ([Groene AI](https://github.com/groeneai)).
* Reading an archive that libarchive fails to open without giving a reason (for example a `.tar.gz` with a truncated gzip trailer) now throws `CANNOT_UNPACK_ARCHIVE` instead of `STD_EXCEPTION` (`fmt::format_error: string pointer is null`). Closes [#125620](https://github.com/ClickHouse/ClickHouse/issues/125620). [#125650](https://github.com/ClickHouse/ClickHouse/pull/125650) ([Groene AI](https://github.com/groeneai)).
* `RESTORE` from a tar backup with a non-empty `password` setting now fails with `NOT_IMPLEMENTED`, like `BACKUP` to a tar archive, instead of `LOGICAL_ERROR`. Closes [#125648](https://github.com/ClickHouse/ClickHouse/issues/125648). [#125666](https://github.com/ClickHouse/ClickHouse/pull/125666) ([Groene AI](https://github.com/groeneai)).

#### Bug Fix (user-visible misbehavior in an official stable release)
* Fixed silent data loss when writing to an S3 key that already holds an object, for example `INSERT INTO FUNCTION s3(...)` with `s3_truncate_on_insert = 1`. Closes [#114348](https://github.com/ClickHouse/ClickHouse/issues/114348). [#118829](https://github.com/ClickHouse/ClickHouse/pull/118829) ([Sema Checherinda](https://github.com/CheSema)).
* Fix possible data loss when a part's `checksums.txt` is repaired (recalculated and written on load, or by `CHECK TABLE`) and the server loses power afterwards. The metadata rewrite is now durable and atomic (write to a temporary file, fsync, atomically rename), and an empty `checksums.txt` is now recalculated on load instead of detaching the whole part as broken. [#111414](https://github.com/ClickHouse/ClickHouse/pull/111414) ([Groene AI](https://github.com/groeneai)).
* Fix a part being detached as broken, losing its rows, when `columns.txt` or `metadata_version.txt` survives a power loss at length zero. Such a file is now treated like an absent one, which the load path already handles safely. [#119582](https://github.com/ClickHouse/ClickHouse/pull/119582) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix data loss when loading a part with a broken projection: a marks file damaged by an interrupted write could be accepted as an empty part instead of being detached as broken. [#119586](https://github.com/ClickHouse/ClickHouse/pull/119586) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed replica recovery of a `Replicated` database deleting the data of a `KeeperMap` table, for example after the server was killed during an `ALTER` of that table. A `KeeperMap` table that still exists in Keeper is now kept or renamed with its data, instead of being dropped and re-created empty. Closes [#111957](https://github.com/ClickHouse/ClickHouse/issues/111957). [#122917](https://github.com/ClickHouse/ClickHouse/pull/122917) ([Groene AI](https://github.com/groeneai)).
* `DROP DATABASE` of a `Replicated` database now drops a materialized view with `ENGINE = TimeSeries` instead of failing with `It's not initial query` and leaving the database half-dropped. [#123663](https://github.com/ClickHouse/ClickHouse/pull/123663) ([Groene AI](https://github.com/groeneai)).
* Fixed data loss when a single `ALTER TABLE` renamed a `Nested` member, or another column with a dot in its name, and modified the renamed column. Such an `ALTER` is now rejected with `NOT_IMPLEMENTED`, as it already was for other columns. [#122794](https://github.com/ClickHouse/ClickHouse/pull/122794) ([Groene AI](https://github.com/groeneai)).
* Fixed merges deleting all rows of a part whose rows TTL was never calculated, although none of its rows had expired: a part attached with `ATTACH PARTITION ... FROM` from a table without that rows TTL, or written before `MODIFY TTL` with `materialize_ttl_after_modify = 0` added it, for example after `MODIFY COLUMN ... REMOVE TTL`. The rows were lost in background TTL merges and in merges with fully expired parts. Closes [#121124](https://github.com/ClickHouse/ClickHouse/issues/121124). [#123068](https://github.com/ClickHouse/ClickHouse/pull/123068) ([Groene AI](https://github.com/groeneai)).
* Fixed a race in `MergeTree` where an `ALTER TABLE ... UPDATE` or `DELETE` mutation submitted concurrently with another mutation could be silently skipped on some parts while being reported as done. [#124167](https://github.com/ClickHouse/ClickHouse/pull/124167) ([Murphy](https://github.com/murphy-4o)).
* A mutation on a `Replicated*` table whose predicate contains a lambda can no longer prune to too few partitions when a lambda parameter name matches an `ALIAS` or `EPHEMERAL` column definition. Previously some matching rows were left unmutated while the mutation still finished successfully. [#121193](https://github.com/ClickHouse/ClickHouse/pull/121193) ([Groene AI](https://github.com/groeneai)).
* Fix background merges with TTL silently stopping for the whole server. A merge with TTL that was selected and then dropped before it ran — for example because its table was dropped right after the expired rows were inserted — leaked one of the `max_number_of_merges_with_ttl_in_pool` slots, which are shared by all tables. After two such merges no table would delete expired rows any more until the server was restarted. [#121236](https://github.com/ClickHouse/ClickHouse/pull/121236) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Make the asynchronous startup of a `ReplicatedMergeTree` table robust against a concurrent `DETACH`, which could previously hang the startup forever or leave background tasks running on a table that was already shut down. [#121672](https://github.com/ClickHouse/ClickHouse/pull/121672) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix corrupted shared data of nested `JSON` objects after a failed attempt to parse a value during insert. It could lead to errors like `Unexpected version of JSON type binary encoding` or `Unknown type code` when reading the data after merges or when serializing it in binary formats. [#123898](https://github.com/ClickHouse/ClickHouse/pull/123898) ([Rory Shanks](https://github.com/rorylshanks)).
* The server no longer terminates when a transient error such as `MEMORY_LIMIT_EXCEEDED`, a network error, or concurrent `SYSTEM RESTART REPLICA` happens while outdated data parts are loading in the background. Such errors are now retried later instead of aborting the server. [#117981](https://github.com/ClickHouse/ClickHouse/pull/117981) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed silent row loss when a `MergeTree` table with a projection is read with a `WHERE` in which an `AND` is used with negative polarity, under a `NOT`, as an operand of a comparison, or as the branch condition of `if`/`multiIf`/`CASE`, and one of that `AND`'s conjuncts references a column the projection does not store. [#120286](https://github.com/ClickHouse/ClickHouse/pull/120286) ([Groene AI](https://github.com/groeneai)).
* Fixed some `SELECT` queries silently returning fewer rows when the table has a projection that exists only in some of its parts, for example one added with `ALTER TABLE ... ADD PROJECTION` after data was inserted, and `query_plan_remove_unused_columns` is `0`, which `compatibility` up to `25.11` sets. [#124022](https://github.com/ClickHouse/ClickHouse/pull/124022) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong query results with skip indexes on `SummingMergeTree`, `AggregatingMergeTree`, `CoalescingMergeTree`, and `GraphiteMergeTree` tables: with `optimize_on_insert`, the default, an `INSERT` built every skip index whose expression is not a plain column from the rows before they were merged, so queries using the index could skip granules with matching rows. `SummingMergeTree` also no longer keeps rows whose summed columns total zero because of such an index. [#123291](https://github.com/ClickHouse/ClickHouse/pull/123291) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results when a `WHERE` clause mixing `AND` and `OR` was evaluated with skip indexes (`use_skip_indexes_for_disjunctions`, enabled by default). If a `minmax` or `set` index could not evaluate one atom of the predicate for a granule, the per-atom verdicts recorded for that granule were shifted, so one atom's verdict was applied to another and granules holding matching rows were skipped. [#124303](https://github.com/ClickHouse/ClickHouse/pull/124303) ([Groene AI](https://github.com/groeneai)).
* Fixed a query silently returning fewer rows after an unrelated `JOIN` had run: a join runtime filter moved into a read's `PREWHERE` made the read record a false "no matching rows" verdict in the query condition cache. [#119823](https://github.com/ClickHouse/ClickHouse/pull/119823) ([Groene AI](https://github.com/groeneai)).
* Fixed the query condition cache serving one session's granule verdicts to another when the two differ only in a setting a conversion function captured, such as `precise_float_parsing`, which silently dropped matching rows from the second session's result. [#119168](https://github.com/ClickHouse/ClickHouse/pull/119168) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `LEFT`, `RIGHT`, `FULL`, and `ANY` outer `JOIN` silently dropping non-matched rows when the `WHERE` filter reads a `Date32` or `Enum` column of the non-preserved side. Closes [#121115](https://github.com/ClickHouse/ClickHouse/issues/121115). [#121181](https://github.com/ClickHouse/ClickHouse/pull/121181) ([Groene AI](https://github.com/groeneai)).
* Fixed a join runtime filter silently dropping rows whose join key is a float. When the build side had a single distinct value the filter replaced set membership with `equals`, but a hash join compares float keys bitwise, so `NaN` did not match itself and `0.0` matched `-0.0`. [#119720](https://github.com/ClickHouse/ClickHouse/pull/119720) ([Groene AI](https://github.com/groeneai)).
* Fix wrong values after lazy materialization expands a sparse column during a `JOIN`: rows could receive values from other rows. [#124442](https://github.com/ClickHouse/ClickHouse/pull/124442) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
* `ALTER TABLE ... EXECUTE expire_snapshots` on an `Iceberg` table now always keeps the head snapshot of every branch, even when the table property `history.expire.min-snapshots-to-keep` is `0`. Previously such a table could lose all its data files. Closes [#124333](https://github.com/ClickHouse/ClickHouse/issues/124333). [#125365](https://github.com/ClickHouse/ClickHouse/pull/125365) ([Groene AI](https://github.com/groeneai)).
* Fixed an `ssl_certificate` authentication bypass where an embedded NUL byte in a certificate's Common Name (for example `john\0.evil.com`) was truncated server-side to `john`, letting a holder of such a certificate authenticate as a different user. [#117403](https://github.com/ClickHouse/ClickHouse/pull/117403) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Fixed one failed write to a non-replicated `MergeTree` table's deduplication log (with `non_replicated_deduplication_window` greater than zero) making every following `INSERT` into that table fail with `Cannot write to canceled buffer` until the server was restarted. The log now replaces a writer that a failed flush left unusable, no longer finalizes such a writer while rotating or shutting down, and no longer stops removing its outdated files, which could also make `ALTER TABLE ... MODIFY SETTING non_replicated_deduplication_window` fail. [#118350](https://github.com/ClickHouse/ClickHouse/pull/118350) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results after `ALTER TABLE ... MODIFY COLUMN <column> <type> AFTER <other>` on a column used in `PARTITION BY`. Closes [#92834](https://github.com/ClickHouse/ClickHouse/issues/92834). [#120862](https://github.com/ClickHouse/ClickHouse/pull/120862) ([Groene AI](https://github.com/groeneai)).
* Fixed a merge or a mutation making a table unreadable (`Stream ... is not found`) when a column that a part does not hold is dropped and its name is also the name of a subcolumn of another column, for example `a.size0` next to an `Array` column `a`. [#118246](https://github.com/ClickHouse/ClickHouse/pull/118246) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix wrong rows returned by `ORDER BY ... LIMIT` over a `MergeTree` table read in the order of a function of its sorting key, for example `ORDER BY toStartOfMonth(date) DESC, i`, when the sorting-key column uses sparse serialization. [#109424](https://github.com/ClickHouse/ClickHouse/pull/109424) ([Groene AI](https://github.com/groeneai)).
* `ALTER TABLE ... DROP PARTITION` and other partition commands with an invalid or out-of-range `Date` or `DateTime` literal, such as `'2024-02-30'` or `'9999-01-01'`, no longer silently act on a different partition. They now raise an error. [#122419](https://github.com/ClickHouse/ClickHouse/pull/122419) ([DeviousCardi](https://github.com/DeviousCardi)).
* Fixed wrong query results and unsorted parts in `MergeTree` tables whose sorting, primary or partition key reads a subcolumn of a column (for example ``PRIMARY KEY cityHash64(`c1.1`)`` or `ORDER BY t.a`) when a column `TTL` or an `UPDATE` changes that column. A column `TTL` on such a column is now refused as on any other key column, `UPDATE` of a column that a `MATERIALIZED` key column of this kind is computed from is refused, and TTL merges of a table already stored with such a `TTL` fail instead of writing unsorted parts. [#123628](https://github.com/ClickHouse/ClickHouse/pull/123628) ([Groene AI](https://github.com/groeneai)).
* After a canary OOM, only the merges and mutations that are already running are cancelled. Newly scheduled ones now run as usual instead of all later merges and mutations staying cancelled until server restart. Closes [#121748](https://github.com/ClickHouse/ClickHouse/issues/121748). [#123870](https://github.com/ClickHouse/ClickHouse/pull/123870) ([Javier G. Sogo](https://github.com/jgsogo)).
* Fixed a spurious `Cannot schedule a task: no free thread ... (CANNOT_SCHEDULE_TASK)` error that rejected a valid `SELECT` over a `MergeTree` table when it had more parts to analyse than the query had threads and their analysis took longer than `lock_acquire_timeout`. Index analysis now runs one pool job per thread instead of one per part, which also makes it faster for queries over many parts. [#116822](https://github.com/ClickHouse/ClickHouse/pull/116822) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results and `CANNOT_READ_ALL_DATA` or `INCORRECT_DATA` errors when reading `MergeTree` parts in packed storage (`min_bytes_for_full_part_storage`), for example with parallel replicas, with a patch part of a lightweight update or delete, or when reading a `JSON` subcolumn. [#116920](https://github.com/ClickHouse/ClickHouse/pull/116920) ([Groene AI](https://github.com/groeneai)).
* Fixed `ILLEGAL_COLUMN` (`Illegal column ... of first argument of function to...`) raised by every query touching a column that was dropped and re-added with a different type, for as long as a part still carried the old column. The re-added column now reads as its default, as it does once the drop's mutation rewrites the part. Queries served from a projection no longer return the old values of such a column. [#118585](https://github.com/ClickHouse/ClickHouse/pull/118585) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed reading stale data, or a `Sizes of nested column and null map of Nullable column are not equal` / `Got empty stream for SerializationLowCardinality keys` error, from a column that was turned into `ALIAS` or `EPHEMERAL` (or dropped while its partition was detached) and later made physical again, when a mutation that rewrites only some columns (for example `MATERIALIZE INDEX`) ran on a Wide part in between. [#124516](https://github.com/ClickHouse/ClickHouse/pull/124516) ([Groene AI](https://github.com/groeneai)).
* Fixed a use-after-free (on some versions a crash with `Pure virtual function called!`) on `INSERT`, merge or `SELECT ... FINAL` of a `SummingMergeTree` table with a `Nested` column whose name ends in `Map` and whose element type has `LowCardinality` inside an `Array` or `Map`, for example `Array(LowCardinality(String))`. [#125533](https://github.com/ClickHouse/ClickHouse/pull/125533) ([Groene AI](https://github.com/groeneai)).
* Fix a `MergeTree` table with a rows `TTL` and a `text` index never dropping its expired parts: the `TTLDrop` merge failed with `Text index transform for index ... not found` and retried forever. [#113385](https://github.com/ClickHouse/ClickHouse/pull/113385) ([Groene AI](https://github.com/groeneai)).
* Fixed dropping whole parts with TTL (`ttl_only_drop_parts = 1`) still reading all data from large, fully expired parts. [#108626](https://github.com/ClickHouse/ClickHouse/pull/108626) ([Michael Jarrett](https://github.com/EmeraldShift)).
* Fixed a server crash on `CREATE TABLE` or `ALTER TABLE ... MODIFY TTL` when a `TTL` expression uses the result of `arrayMap` with a constant lambda, such as `toString(arrayMap(x -> CAST(1, 'Dynamic'), arr))`. [#122868](https://github.com/ClickHouse/ClickHouse/pull/122868) ([Groene AI](https://github.com/groeneai)).
* Fixed `NOT_FOUND_COLUMN_IN_BLOCK` (`Not found column _row_exists in block`) that permanently blocked all further mutations of a `MergeTree` table after a lightweight `DELETE` run with `lightweight_delete_mode = 'lightweight_update'` was followed by a heavyweight `ALTER TABLE ... UPDATE` and then by another mutation rewriting all columns. Closes [#115570](https://github.com/ClickHouse/ClickHouse/issues/115570). [#115665](https://github.com/ClickHouse/ClickHouse/pull/115665) ([Groene AI](https://github.com/groeneai)).
* Fixed `ALTER DELETE` and `ALTER UPDATE` silently skipping rows when the predicate references an `ALIAS` column whose definition contains a non-deterministic function such as `now`. [#117273](https://github.com/ClickHouse/ClickHouse/pull/117273) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a patch part whose `source_parts.dat` lost its content to a zeroed block loading as an empty index, which silently unapplied an acknowledged lightweight `UPDATE` and then let background cleanup delete the only copy of it. Such a part is now detached as broken. [#119585](https://github.com/ClickHouse/ClickHouse/pull/119585) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `ALTER TABLE ... DELETE` and `ALTER TABLE ... UPDATE` skipping a whole partition when the parts the initiating replica holds in it are all empty: rows another replica had acknowledged in that partition survived the mutation on every replica. [#119149](https://github.com/ClickHouse/ClickHouse/pull/119149) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a pending lightweight `DELETE` read with `apply_mutations_on_fly = 1` evaluating its predicate over patch parts created after it, which made rows disappear from the result and reappear after the mutation materialized. Closes [#118049](https://github.com/ClickHouse/ClickHouse/issues/118049). [#118919](https://github.com/ClickHouse/ClickHouse/pull/118919) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#119306](https://github.com/ClickHouse/ClickHouse/pull/119306) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed lightweight `UPDATE` for parts adopted from another table via `REPLACE PARTITION` or `ATTACH PARTITION FROM`: after such a part was cloned again (by a mutation that does not modify it, by a clone instead of a fetch of a replicated mutation, or by a move between disks), applying patch parts could silently drop updated rows in release builds and raise the `current_version != existing_version` exception in debug and sanitizer builds. [#115979](https://github.com/ClickHouse/ClickHouse/pull/115979) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed lightweight `UPDATE ... ON CLUSTER` on `ReplicatedMergeTree` tables running on every replica of a shard instead of once per shard, which applied assignments such as `SET v = v + 1` once per replica. A plain `UPDATE` in a `Replicated` database with more than one shard had the same problem. [#122393](https://github.com/ClickHouse/ClickHouse/pull/122393) ([Pratham Nayak](https://github.com/protomn)).
* Fixed a concurrent heavyweight `ALTER TABLE ... UPDATE`/`DELETE` silently dropping an acknowledged lightweight `UPDATE` on a plain `MergeTree` table. [#119155](https://github.com/ClickHouse/ClickHouse/pull/119155) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed mutations such as `ALTER TABLE ... UPDATE` failing with `CANNOT_READ_ALL_DATA` on every attempt, and never completing, when they read a column together with its `LowCardinality` subcolumn in a Wide part: a `Dynamic` column and its `LowCardinality(String)` subcolumn, or a `Map` with `map_serialization_version = 'with_buckets'` and its `keys` or `values` subcolumn. The problem appeared in 26.6. [#123160](https://github.com/ClickHouse/ClickHouse/pull/123160) ([Groene AI](https://github.com/groeneai)).
* Fixed an acknowledged lightweight `UPDATE` being invisible on `ReplicatedMergeTree` when it targets a column whose `RENAME COLUMN` mutation has not materialized yet. [#119144](https://github.com/ClickHouse/ClickHouse/pull/119144) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a lightweight `UPDATE` made before `ALTER TABLE ... RENAME COLUMN` on a `ReplicatedMergeTree` table being lost when a merge ran before the rename's mutation (for example with `apply_patches_on_merge = 0`): queries returned the old value, and the mutation then dropped the update from disk. [#124653](https://github.com/ClickHouse/ClickHouse/pull/124653) ([Groene AI](https://github.com/groeneai)).
* Fixed a wrong result of a mutation of a `MergeTree` table when several pending mutations are executed as a single task and a patch part, created by a lightweight `UPDATE`, has a version in between: the patch was applied before the commands that precede it, so they were evaluated over data they must not see. [#119127](https://github.com/ClickHouse/ClickHouse/pull/119127) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `ReplicatedMergeTree` TTL processing stalling indefinitely when a TTL merge that drops fully expired parts was larger than half the free disk space. Such a merge deletes every row and therefore needs no room for its source parts, but it was postponed on their size, and the postponed entry kept occupying the only `max_replicated_merges_with_ttl_in_queue` slot, so no TTL merge ran and no space was ever freed. It is now admitted when the table's only TTL is an unconditional rows TTL. Closes [#119775](https://github.com/ClickHouse/ClickHouse/issues/119775). [#119983](https://github.com/ClickHouse/ClickHouse/pull/119983) ([Groene AI](https://github.com/groeneai)).
* Fixed `ATTACH PARTITION` on `ReplicatedMergeTree` failing to rename a broken part in `detached/` aside, so every later `ATTACH PARTITION` for that partition kept failing on it. [#116734](https://github.com/ClickHouse/ClickHouse/pull/116734) ([Groene AI](https://github.com/groeneai)).
* Fixed the replication queue never draining when a part is present locally but its node in `ZooKeeper` is missing: the `GET_PART` entry for such a part re-downloaded it and failed with `DUPLICATE_DATA_PART` on every retry. [#119189](https://github.com/ClickHouse/ClickHouse/pull/119189) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Writing the empty covering part of a replicated table now happens before the covered parts are removed from the working set, so an exception during the operation no longer leaves the table with unexpected data parts and readonly after reload. [#122610](https://github.com/ClickHouse/ClickHouse/pull/122610) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Background Keeper users now retry a `get_zookeeper_lock_acquire_timeout_ms` timeout like other Keeper errors, so a `ReplicatedMergeTree` table attached during a transient Keeper outage no longer stays read-only. [#124327](https://github.com/ClickHouse/ClickHouse/pull/124327) ([Groene AI](https://github.com/groeneai)).
* A table read by a scalar subquery inside a `CONSTRAINT` is now a loading dependency of the table that declares the constraint. Previously such a table could be dropped, after which the server did not start, because attaching the dependent table executes the subquery. [#114923](https://github.com/ClickHouse/ClickHouse/pull/114923) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a server startup failure after creating a table with a `quantile*` state column whose level is a `Decimal`, a wide integer, a `UUID`, an `IPv4`, or an `IPv6`. Such a level was persisted into the metadata as a quoted string, which reparsed as a `String` and was rejected, so metadata loading failed with `CANNOT_CONVERT_TYPE` and the server did not start. The same `CREATE` also failed outright on a `Replicated` database. [#119810](https://github.com/ClickHouse/ClickHouse/pull/119810) ([Groene AI](https://github.com/groeneai)).
* A table whose stored definition carries a column `TTL` inherited from its source table now loads again on engines that do not support `TTL` (`Merge`, `Buffer`, `Distributed`, `Remote`). Previously such a table could not be loaded, which failed the load of the whole database, and with `async_load_databases = 0` the server refused to start. The `TTL` stays in the definition and has no effect; `ALTER TABLE ... MODIFY COLUMN ... REMOVE TTL` removes it. [#122273](https://github.com/ClickHouse/ClickHouse/pull/122273) ([Mohammed Sohail](https://github.com/msohail22)).
* Fix a `Remote` or `Cluster` database without a local shard making `SELECT count()` through the `merge` table function return `0`, and making the server fail to start when a dictionary reads one of its tables. Closes [#122612](https://github.com/ClickHouse/ClickHouse/issues/122612). [#122661](https://github.com/ClickHouse/ClickHouse/pull/122661) ([Groene AI](https://github.com/groeneai)).
* Fix `Part ... intersects previous part ...` when attaching a `MergeTree` table after `REPLACE PARTITION`, if parts outdated by an earlier `DROP PART` were still on disk. The server could fail to start because of it. [#120980](https://github.com/ClickHouse/ClickHouse/pull/120980) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix `Part ... intersects next part ...` when loading a `MergeTree` table after `REPLACE PARTITION` into it ran concurrently with `MOVE PARTITION` from it. The server could fail to start because of it. [#123780](https://github.com/ClickHouse/ClickHouse/pull/123780) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `S3`, `HDFS`, `AzureBlobStorage`, and `URL` tables over a named collection with `format = 'auto' NOT OVERRIDABLE` failing to load with `Override not allowed for 'format'`, which could stop the server from starting. The format that ClickHouse inferred and stored at `CREATE` time is now accepted when the table is loaded again; a format written by the user over the locked value is still refused. [#124215](https://github.com/ClickHouse/ClickHouse/pull/124215) ([Groene AI](https://github.com/groeneai)).
* Restored compatibility with Azure Blob Storage endpoints that omit the `ETag` header. [#120419](https://github.com/ClickHouse/ClickHouse/pull/120419) ([alesapin](https://github.com/alesapin)).
* Fix `Part ... intersects previous part ...` when loading a `MergeTree` table after a `TRUNCATE`, `DROP PARTITION` or `MOVE PARTITION TO TABLE` failed to commit (e.g. because of a memory limit or a disk error) and a merge ran before the restart. The server could fail to start because of it. [#122375](https://github.com/ClickHouse/ClickHouse/pull/122375) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A `PROJECTION` that could not be analyzed when the table was loaded at server startup is no longer deleted from the table's persisted metadata by the next `ALTER`, which also orphaned its already-materialized data with no error at any step. While such a declaration is present, an `ALTER` that rewrites the table metadata is now refused (`ALTER TABLE ... DROP PROJECTION` and `CLEAR PROJECTION` still work), so it survives until the cause is removed or the user drops it. [#118378](https://github.com/ClickHouse/ClickHouse/pull/118378) ([Groene AI](https://github.com/groeneai)).
* Fixed non-deterministic conditions being moved to `PREWHERE` in `SELECT ... FINAL` when `optimize_move_to_prewhere` is enabled. [#109538](https://github.com/ClickHouse/ClickHouse/pull/109538) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Fixes wrong results with `query_plan_optimize_lazy_final` when the primary key is nullable and holds a `NULL`: the `NULL`-keyed row was silently missing from the result of a `FINAL` query. [#112857](https://github.com/ClickHouse/ClickHouse/pull/112857) ([Groene AI](https://github.com/groeneai)).
* Fixed rows silently dropped by index analysis when a `DateTime64` column is compared with a constant of another scale and the sorting or partition key contains a `CAST` of that column, such as `ORDER BY d::String`. [#118688](https://github.com/ClickHouse/ClickHouse/pull/118688) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix rows silently dropped by partition, primary-key, and skip-index pruning, and by `Iceberg` partition pruning, when a string constant is compared with a `DateTime` or `DateTime64` column whose type name carries no time zone and the effective `session_timezone` differs from the one that column's type was built with. [#120114](https://github.com/ClickHouse/ClickHouse/pull/120114) ([Groene AI](https://github.com/groeneai)).
* Fixed a `minmax` skip index, a `set` skip index, part-level minmax pruning, and statistics-based part pruning dropping rows that hold a `NaN` when the query negates a float comparison, for example `WHERE NOT (val BETWEEN 0 AND 3)`. Closes [#106533](https://github.com/ClickHouse/ClickHouse/issues/106533). Closes [#106948](https://github.com/ClickHouse/ClickHouse/issues/106948). [#107074](https://github.com/ClickHouse/ClickHouse/pull/107074) ([Groene AI](https://github.com/groeneai)).
* Fixed silently missing rows for a range predicate on `toDayOfWeek(key, mode)` with the Sunday-first modes `2` and `3` over a date primary key. [#117683](https://github.com/ClickHouse/ClickHouse/pull/117683) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `ngrambf_v1`, `tokenbf_v1`, and `sparse_grams` skip indexes silently dropping matching rows when an `IPv6` column is compared with a string literal, as in `WHERE ip = '2001:db8::'`. [#113157](https://github.com/ClickHouse/ClickHouse/pull/113157) ([Groene AI](https://github.com/groeneai)).
* Fixed a `set` skip index whose expression has type `Dynamic` or `Variant`, which includes an undeclared `JSON` path such as `j.a`, silently returning no rows for `!=` and `IS NULL`. [#120780](https://github.com/ClickHouse/ClickHouse/pull/120780) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results when a query converts a `Date` or `Date32` column to `DateTime` and the column's values leave the range of day numbers that fits into 32-bit seconds. Key analysis, statistics part pruning, and reading in sorting-key order assumed the conversion preserves order, so a filter could skip parts holding matching rows or answer `count` from metadata without filtering, and `ORDER BY` could return rows out of order. Affects `toDateTime`, `toDateTime32`, `CAST(... AS DateTime)`, and `accurateCast`. [#117031](https://github.com/ClickHouse/ClickHouse/pull/117031) ([Groene AI](https://github.com/groeneai)).
* Fix rows silently dropped by primary-key pruning when `toDayOfWeek` is applied to a `DateTime` column whose time zone differs from the session one, or over a `Date32`/`DateTime64` key that leaves the `Date` range. `toDayOfWeek` with a Sunday-first `week_mode` can use the primary key again. [#118655](https://github.com/ClickHouse/ClickHouse/pull/118655) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed rows silently missing from a query over a table whose primary key is a `Tuple` with a `Nullable` element holding `NULL`: primary-key analysis dropped whole granules of matching rows. [#118945](https://github.com/ClickHouse/ClickHouse/pull/118945) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix a `JSONAllPaths` skip index silently pruning every granule for a filter on a subcolumn inside a declared typed path of a `JSON` column (for example `json.a.b` for `JSON(a JSON)` or `JSON(a Tuple(b Int64))`). Such a filter no longer uses the index. [#119321](https://github.com/ClickHouse/ClickHouse/pull/119321) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix rows silently dropped by primary key and partition pruning when a `MergeTree` key expression applies a timezone-observing function such as `toYYYYMM`, `toString` or a `CAST` to `String` to a `DateTime`/`DateTime64` column and a compared constant declares a different timezone. Both a scalar comparison constant and an `IN`-set element are affected, including when the differing timezone sits under an `Array`, `Map`, `Tuple`, `Nullable`, `LowCardinality` or `Variant` wrapper that only one side carries, when a custom type name such as `SimpleAggregateFunction` wraps that composite on one side only, and when a sibling leaf differs only by a custom type name (`Bool` against `UInt8`). Also fix the same loss when a compared `Tuple` and the key column's disagree on explicit element names, which `toJSONString` renders as an object against an array; no timezone difference is needed. [#113781](https://github.com/ClickHouse/ClickHouse/pull/113781) ([Groene AI](https://github.com/groeneai)).
* Fixed rows silently dropped, or counted without being filtered, when a `DateTime64` constant of a finer scale than the column is compared with `!=`, `NOT IN`, or `IN`: index analysis truncated the constant and treated the truncated value as the exact point the query asked for, pruning the granule or partition that held a matching row. [#118957](https://github.com/ClickHouse/ClickHouse/pull/118957) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed index analysis for a condition that wraps the primary key in a widening integer cast such as `toInt64` and then applies another monotonic function such as `negate` or `abs`. Such a condition could silently drop matching rows, or fail with `LOGICAL_ERROR: abs's argument does not match the expected data type`. Present since 26.5. [#116699](https://github.com/ClickHouse/ClickHouse/pull/116699) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results from primary-key analysis for predicates such as `toDate(dt64_key) >= ...` and `toDateTime(date32_key) >= ...`: at the default `date_time_overflow_behavior = 'ignore'` the conversion wraps for values the result type cannot hold, so matching rows silently disappeared and `count()` could overcount. [#117394](https://github.com/ClickHouse/ClickHouse/pull/117394) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed partition pruning skipping a partition whose rows match a filter like `NOT (c <=> f(partition_key))` when `f` returns `NULL` for that partition's key value (for example `moduloOrNull` or `nullIf`). Such queries returned fewer rows than with `use_partition_pruning = 0`. [#124041](https://github.com/ClickHouse/ClickHouse/pull/124041) ([Groene AI](https://github.com/groeneai)).
* Fixed `use_skip_indexes = 0` also disabling part pruning by the partition-key min-max index since 26.5, which made such queries read every part of the table. [#123403](https://github.com/ClickHouse/ClickHouse/pull/123403) ([Ashish Kumar Choubey](https://github.com/aschoube)).
* Fixed missing rows in queries mixing `AND` and `OR` when a `minmax` or `set` skip index over an expression with `mortonEncode` or `hilbertEncode` is combined with other skip indexes (`use_skip_indexes_for_disjunctions`, enabled by default). Closes [#124452](https://github.com/ClickHouse/ClickHouse/issues/124452). [#124784](https://github.com/ClickHouse/ClickHouse/pull/124784) ([Groene AI](https://github.com/groeneai)).
* Fix a query that compared a `Dynamic` or `JSON` column with a constant, such as `d = '2024-01-05'` or `d = toDate('2024-01-05')`, and returned fewer rows when the table was ordered by a transform of that column, such as `ORDER BY d::String`. Such a comparison no longer uses the primary key; compare the key expression itself, as in `d::String = '2024-01-05'`, to keep using it. [#124476](https://github.com/ClickHouse/ClickHouse/pull/124476) ([Ashish Kumar Choubey](https://github.com/aschoube)).
* Fixed a `text` index dropping matching rows when a `FixedString` is involved in the comparison. [#118972](https://github.com/ClickHouse/ClickHouse/pull/118972) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed wrong results of `hasPhrase`, `hasToken`, `hasAnyTokens`, and `hasAllTokens` over a text index with a `postprocessor` or `preprocessor` when direct read from the text index could not answer the function, for example `NOT hasPhrase` with a phrase whose tokens are all removed by a stop-word postprocessor. [#125118](https://github.com/ClickHouse/ClickHouse/pull/125118) ([Groene AI](https://github.com/groeneai)).
* Fixed text-index search functions returning wrong results, including silently dropped rows, on a column whose text index declares a preprocessor. [#117774](https://github.com/ClickHouse/ClickHouse/pull/117774) ([Elmi Ahmadov](https://github.com/ahmadov)).
* Fixed wrong results after a mutation of a wide data part that does not store a column added by a metadata-only `ALTER TABLE ... ADD COLUMN`: the mutation wrote a column statistic for that column, so statistics-based part pruning could drop a part holding matching rows and `min` and `max` could be answered with the pre-`ALTER` default. [#121936](https://github.com/ClickHouse/ClickHouse/pull/121936) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results for `min`, `max`, and filtered `count` on a `MergeTree` column that carries a column `TTL`. When the `TTL` emptied the column, the part kept a statistic for it and queries were answered from that stale statistic instead of from the rows the part stores. [#122058](https://github.com/ClickHouse/ClickHouse/pull/122058) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results from `use_query_condition_cache_for_top_k`: `SELECT ... SAMPLE ... ORDER BY ... LIMIT n` no longer reuses cache entries written by the same query without `SAMPLE`, and Top-K query condition cache entries are no longer written for reads with on-the-fly mutations or patch parts. Closes [#122564](https://github.com/ClickHouse/ClickHouse/issues/122564). [#122538](https://github.com/ClickHouse/ClickHouse/pull/122538) ([Nikita Fomichev](https://github.com/fm4v)). [#122617](https://github.com/ClickHouse/ClickHouse/pull/122617) ([Nikita Fomichev](https://github.com/fm4v)).
* Fix an exception `NOT_FOUND_COLUMN_IN_BLOCK` and, in other queries, wrong results when a lambda argument has the same name as an identifier of an expression that is bound to an alias outside of that lambda, for example `SELECT number + 1 AS n, arrayMap(number -> number + n, [1, 2]) FROM numbers(2)`. [#115544](https://github.com/ClickHouse/ClickHouse/pull/115544) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed wrong results when a `GROUP BY`, `DISTINCT`, `LIMIT BY`, or window `PARTITION BY` key was `plus` or `minus` of a partition column with a non-integer constant. [#120182](https://github.com/ClickHouse/ClickHouse/pull/120182) ([Groene AI](https://github.com/groeneai)).
* Fixed a regression in 26.8 where columns of a parameterized view could not be qualified by the view name (`SELECT pv.col FROM pv(p = 1)` failed with `UNKNOWN_IDENTIFIER`) and a parameterized view used as a `JOIN` operand required an explicit alias. Closes [#119837](https://github.com/ClickHouse/ClickHouse/issues/119837). [#112481](https://github.com/ClickHouse/ClickHouse/pull/112481) ([Groene AI](https://github.com/groeneai)). [#120698](https://github.com/ClickHouse/ClickHouse/pull/120698) ([Dmitry Novik](https://github.com/novikd)).
* Fixed `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN` and `CANNOT_PARSE_TEXT` when a `PREWHERE` guard predicate and a throwing expression, such as a `CAST` or `toUInt64(substring(...))`, end up in the same read step: either because they read subcolumns of the same column, for example a `JSON` or `Map` column, or because a JOIN runtime filter's key computation was merged into the guarded subquery. Such a query used to fail because the throwing expression was evaluated on rows the guard had rejected. [#111954](https://github.com/ClickHouse/ClickHouse/pull/111954) ([Groene AI](https://github.com/groeneai)).
* Fix wrong window function results when a `QUALIFY` condition, or a comparison derived from it by `optimize_and_compare_chain`, turns into an `indexHint` that prunes rows below the window. Closes [#121177](https://github.com/ClickHouse/ClickHouse/issues/121177). [#122975](https://github.com/ClickHouse/ClickHouse/pull/122975) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
* Fixed wrong aggregate values with `optimize_aggregation_in_order_limit` when the `ORDER BY` is a strict prefix of the `GROUP BY` key and a group's rows span multiple parts. Closes [#116849](https://github.com/ClickHouse/ClickHouse/issues/116849). [#117573](https://github.com/ClickHouse/ClickHouse/pull/117573) ([Konstantin Bogdanov](https://github.com/thevar1able)).
* Fixed wrong results of `sum`, `min`, `max`, and `avg` over arithmetic with a `Decimal` operand and a constant, for example `min` greater than `max`, a changed `sum` of a division, or a missing `DECIMAL_OVERFLOW`: `optimize_arithmetic_operations_in_aggregate_functions` no longer moves such arithmetic out of the aggregate. [#117779](https://github.com/ClickHouse/ClickHouse/pull/117779) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix `<agg>IfOrNullIf` and `<agg>IfOrDefaultIf` ignoring the outer condition when used without `GROUP BY` keys or as a window function with non-`Nullable` arguments. [#122908](https://github.com/ClickHouse/ClickHouse/pull/122908) ([Raúl Marín](https://github.com/Algunenano)).
* Fixed a possible hang of a distributed `GROUP BY` query with `uniqExact` and `distributed_aggregation_memory_efficient = 0` when merging large states of two-level aggregation. [#123352](https://github.com/ClickHouse/ClickHouse/pull/123352) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed missing rows of an empty grouping set `()` in `GROUP BY GROUPING SETS` queries without aggregate functions. For example, `SELECT number FROM numbers(3) GROUP BY GROUPING SETS ((number), ())` returned 3 rows instead of 4, and `GROUPING SETS (())` returned one row per input row. Closes [#61461](https://github.com/ClickHouse/ClickHouse/issues/61461). [#123539](https://github.com/ClickHouse/ClickHouse/pull/123539) ([Groene AI](https://github.com/groeneai)).
* The `uniq` aggregate function returned garbage (for example `18446743978444128518`) for cardinalities above roughly thirty billion, because its state kept only 32 bits of each hash and the correction formula overflowed. The state now additionally keeps a small sample of full 64-bit hashes (state version `1`) which the estimate switches to at about two billion elements: `SELECT uniq(number) FROM numbers_mt(100000000000)` returns `99360630405` (−0.64%), and the estimate stays accurate (about 1%) up to trillions. Below two billion nothing changes: the estimates, the memory usage, and the state format are exactly as before, and states written by older versions remain readable and merge correctly in mixed-version clusters. [#115393](https://github.com/ClickHouse/ClickHouse/pull/115393) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix wrong results for `uniq*`, `GROUP BY`, `LIMIT n BY`, `ORDER BY` after `GROUP BY`, and window `PARTITION BY` over `toString` of a `DateTime`/`DateTime64` in a time zone with a daylight-saving transition, and over `toFixedString` of a `String`. Both functions claimed to be injective while mapping distinct arguments onto the same result, and the default-on optimizations that take that claim as a proof then eliminated the wrapper. [#117238](https://github.com/ClickHouse/ClickHouse/pull/117238) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `optimize_rewrite_aggregate_function_with_if` (enabled by default) turning a short-circuit-guarded aggregate such as `sum(if(number = 0, 0, intDiv(42, number)))` into a query failure: the rewritten `sumIf(intDiv(42, number), number != 0)` evaluates its argument for every row, so the guarded expression ran on the rows the condition excluded and threw. The rewrite is now refused when the lifted branch relies on short-circuit evaluation. Closes [#85784](https://github.com/ClickHouse/ClickHouse/issues/85784). [#118697](https://github.com/ClickHouse/ClickHouse/pull/118697) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results and errors of `min`, `max`, `sum`, and `avg` over an arithmetic operation with a constant with `optimize_arithmetic_operations_in_aggregate_functions` enabled (the default): on an `Array` or `Tuple` operand (`min(arr * -1)`), on an `IPv4` or `IPv6` operand (`sum(ip * 1)` threw `ILLEGAL_TYPE_OF_ARGUMENT`), for `avg` of a date, time, or interval (`avg(t + INTERVAL 500 MILLISECOND)`), for multiplication or division by zero, infinity, or `NULL` (`min(x / -0.0)`), and for a day or longer added to a `DateTime` or a `Time64` across a DST change or a month end. Closes [#122647](https://github.com/ClickHouse/ClickHouse/issues/122647). [#123908](https://github.com/ClickHouse/ClickHouse/pull/123908) ([Groene AI](https://github.com/groeneai)).
* Fixed `hasAll`, `hasAny`, `hasSubstr`, `startsWith`, and `endsWith` returning wrong results on `Array(Nullable(T))`. A `NULL` element was compared as whatever value was physically stored beneath it, so these functions disagreed with `has` on identical input: `hasAll([1, 2, NULL, 4], [0])` returned `1` while `has([1, 2, NULL, 4], 0)` returned `0`. Closes [#68310](https://github.com/ClickHouse/ClickHouse/issues/68310). [#116239](https://github.com/ClickHouse/ClickHouse/pull/116239) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results from JIT-compiled conversions between integers and `Decimal`. With the default `compile_expressions = 1`, a conversion the destination could not hold silently wrapped instead of raising `DECIMAL_OVERFLOW` once the expression crossed `min_count_to_compile_expression`. Closes [#118958](https://github.com/ClickHouse/ClickHouse/issues/118958). [#118994](https://github.com/ClickHouse/ClickHouse/pull/118994) ([Groene AI](https://github.com/groeneai)).
* Fixed `if`, `multiIf`, and `CASE` returning a wrapped value instead of raising `DECIMAL_OVERFLOW` once JIT-compiled, when a `Decimal`, `DateTime64`, or `Time64` branch is lifted to a larger scale (with default settings, after the query had run a few times). [#123186](https://github.com/ClickHouse/ClickHouse/pull/123186) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results of `extract` and `replaceRegexpAll` (and its alias `regexp_replace`) under the default-enabled `optimize_rewrite_regexp_functions`: `extract` could return a different occurrence when a leading greedy `^.*` was stripped, or a truncated result for a pattern ending with `.*$` whose only parentheses are an inline flag group, such as `(?i)b.*$`; `replaceRegexpAll` could silently lose replacements when a pattern that can match more than once (with an inline `(?m)` flag, or a tail that can match the empty string, such as `o*$`) was rewritten to `replaceRegexpOne`; and both mishandled a trailing `.*$` or `$` that is literal text inside an unterminated `\Q`. [#117228](https://github.com/ClickHouse/ClickHouse/pull/117228) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#117227](https://github.com/ClickHouse/ClickHouse/pull/117227) ([Alexey Milovidov](https://github.com/alexey-milovidov)). [#118361](https://github.com/ClickHouse/ClickHouse/pull/118361) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed rows silently missing from the result of `match` with a pattern that has a zero-allowing quantifier behind an inline flag group or an empty `\Q\E` quote. [#118351](https://github.com/ClickHouse/ClickHouse/pull/118351) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed the best-effort date/time parsers reading one byte past the end of the value. The parsed result of a row of a `String` column could depend on the text of the following row: a valid value was rejected as `NULL` or zero, or `addMonths` and the rest of the add-interval family returned date and time fields taken from the neighbouring row. [#122246](https://github.com/ClickHouse/ClickHouse/pull/122246) ([Groene AI](https://github.com/groeneai)).
* Fixed `toStartOfMinute`, `toStartOfHour`, `toStartOfDay`, `toStartOfFiveMinutes`, `toStartOfTenMinutes`, `toStartOfFifteenMinutes`, `timeSlot`, `toTimeWithFixedDate`, `toStartOfInterval` and `dateTrunc` returning a value one second later, possibly in the next interval, for a `DateTime64` before 1970 with a fractional part. For example, `toStartOfMinute(toDateTime64('1969-12-31 23:58:59.500', 3))` returned `23:59:00`. Also fixed `toStartOfInterval` with an origin and a `MONTH`, `QUARTER` or `YEAR` interval returning a value after its argument when the argument was earlier in the day than the origin, and rounding a bucket start clipped to the end of a shorter month to the previous bucket. [#122138](https://github.com/ClickHouse/ClickHouse/pull/122138) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed accepting a column whose two serialization streams map to the same file name (for example ``Nullable(JSON(`null` Int64))`` or ``Tuple(`a` Tuple(`b` UInt64), `a.b` UInt64)``). Such a table was created successfully, but its parts were written with both streams in one file and could not be read, merged, or checked. Creating such a column now fails with `BAD_ARGUMENTS`, as a collision between two different columns already did. Existing tables still attach, with the collision logged. [#115632](https://github.com/ClickHouse/ClickHouse/pull/115632) ([Pavel Kruglov](https://github.com/Avogar)).
* Fix a valid `JOIN ON` query rejected with `TYPE_MISMATCH` by `optimize_redundant_comparisons`, which is enabled by default since 26.7. It affects a join expression whose `AND` has a `Nothing`-typed operand, for example a join key compared against an `ARRAY JOIN []` alias. [#117476](https://github.com/ClickHouse/ClickHouse/pull/117476) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results, or an `ILLEGAL_COLUMN` error, of a `JOIN` whose condition equates both a column and an expression of that column (for example `ON t1.k = t2.k AND intDiv(t1.k, 2) = intDiv(t2.k, 2)`) when a filter on that expression is pushed down to the other side of the join: the pushed-down copy applied the expression twice. Closes [#122613](https://github.com/ClickHouse/ClickHouse/issues/122613). [#122686](https://github.com/ClickHouse/ClickHouse/pull/122686) ([Groene AI](https://github.com/groeneai)).
* Fixed `SAMPLE` on the first table of a `JOIN` being applied to the other joined tables too, which returned too few rows, or failed with `SAMPLING_NOT_SUPPORTED` when a joined table had no sampling key. [#124767](https://github.com/ClickHouse/ClickHouse/pull/124767) ([awss](https://github.com/awss1i)).
* Fixed missing rows from a `Distributed` or `remote` query whose `WHERE` has a `NOT` over an `AND` containing a conjunct that cannot be converted to an AST, such as a lambda or `dictGet`: the pushed-down predicate became stronger than the original one. [#117620](https://github.com/ClickHouse/ClickHouse/pull/117620) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix wrong results when a query selects an `ALIAS` column together with a window function with `enable_analyzer = 1`, over a `Distributed` table (or `remote`) with two or more shards, or over a table read with parallel replicas. Values could be returned in the wrong columns with no error, or the query could fail with `NUMBER_OF_COLUMNS_DOESNT_MATCH` or `CANNOT_PARSE_DATETIME`. [#116498](https://github.com/ClickHouse/ClickHouse/pull/116498) ([Groene AI](https://github.com/groeneai)).
* Fixed a distributed `SELECT` silently returning a partial result when `connection_pool_max_wait_ms` is set to a finite value and `skip_unavailable_shards = 1`. A shard whose replica the initiator had no free connection slot for was reported as an unavailable shard and dropped, so a two-shard query could return one shard's rows with exit code `0`. Such a shard now fails the query with `ALL_CONNECTION_TRIES_FAILED`, the same error the default `skip_unavailable_shards = 0` already produces. Closes [#120832](https://github.com/ClickHouse/ClickHouse/issues/120832). [#120969](https://github.com/ClickHouse/ClickHouse/pull/120969) ([Groene AI](https://github.com/groeneai)).
* Fixed asynchronous TLS connections occasionally failing with an unrelated OpenSSL error left by another connection on the same OS thread. This could terminate distributed queries with a misleading network exception. [#116692](https://github.com/ClickHouse/ClickHouse/pull/116692) ([Dmitriy Borisenko](https://github.com/otselnik)).
* Fixed a query over a `Distributed` table silently returning too few rows when its `WHERE` calls `joinGet`: the condition was sent to the shards, and each one answered the call against its own local `Join` table rather than the initiator's. `joinGet` is now treated as non-deterministic and stays on the initiator. [#121822](https://github.com/ClickHouse/ClickHouse/pull/121822) ([Nikita Taranov](https://github.com/nickitat)).
* Fix `THERE_IS_NO_COLUMN` for a `Distributed` query that has an `IN` over the sharding key outside its `WHERE`, in the `SELECT` projection, in `GROUP BY`, in `ORDER BY`, or in `LIMIT BY`. The default-enabled `optimize_skip_unused_shards_rewrite_in` rewrote that `IN` per shard, which changed the result column's name, so the initiator could not bind the column the shard returned. [#117252](https://github.com/ClickHouse/ClickHouse/pull/117252) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed cancelling a distributed, `s3Cluster` or parallel-replicas `SELECT` taking up to `receive_timeout` (300 s by default) to take effect, and never reaching some of the remote sources, when one of them was draining the packets left over after a `LIMIT` was satisfied. [#118696](https://github.com/ClickHouse/ClickHouse/pull/118696) ([Groene AI](https://github.com/groeneai)).
* Fixed a server abort (`vector[] index out of bounds`) when reading a `Distributed` table with `prefer_global_in_and_join = 1` and an `IN` expression that the analyzer shares between two parent expressions. The local to `GLOBAL IN` rewrite now rewrites the `IN` function in place, so every reference to a shared node is promoted. [#120681](https://github.com/ClickHouse/ClickHouse/pull/120681) ([Groene AI](https://github.com/groeneai)).
* Fixed `INSERT INTO <replicated table> SELECT ... FROM url(...)`, and the same shape with `s3` or `azureBlobStorage`, with parallel replicas enabled inserting the data once per replica. Closes [#120485](https://github.com/ClickHouse/ClickHouse/issues/120485). [#120488](https://github.com/ClickHouse/ClickHouse/pull/120488) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix `INSERT INTO <Distributed table> SELECT ... FROM url(...)` (and `s3`, `azureBlobStorage`, ...) with parallel replicas enabled inserting the data once per shard of the destination cluster: the query forwarded to the shards now names the `*Cluster` table function with the cluster of the `Distributed` table, so every shard reads only its share of the files. Such an `INSERT` also no longer writes into the shards' local tables when the destination has a deterministic sharding key, because the rows a cluster table function hands to a shard cannot satisfy that key and the resulting placement made `optimize_skip_unused_shards` return partial results; it falls back to the ordinary `INSERT ... SELECT` instead, which places the rows through the `Distributed` sink. [#121070](https://github.com/ClickHouse/ClickHouse/pull/121070) ([Igor Nikonov](https://github.com/devcrafter)).
* Fixed duplicated rows returned by `ANY RIGHT JOIN` with `any_join_distinct_right_table_keys = 1` when parallel replicas are enabled. [#123683](https://github.com/ClickHouse/ClickHouse/pull/123683) ([Groene AI](https://github.com/groeneai)).
* Fixed `CREATE MATERIALIZED VIEW ... POPULATE` and `CREATE TABLE ... AS SELECT` in a `Replicated` database with parallel replicas enabled: each replica sent its population to other replicas, so its rows landed in other replicas' tables (duplicated there, missing on the replica itself), or the query failed with `UNKNOWN_TABLE`. The population now ignores the query's parallel replicas and `parallel_distributed_insert_select` settings, so each replica fills only its own table. Closes [#124446](https://github.com/ClickHouse/ClickHouse/issues/124446). [#124477](https://github.com/ClickHouse/ClickHouse/pull/124477) ([Groene AI](https://github.com/groeneai)).
* Fix `S3` client refresh overwriting static authentication settings and breaking subsequent queries. [#120433](https://github.com/ClickHouse/ClickHouse/pull/120433) ([alesapin](https://github.com/alesapin)).
* Fix a server abort when an Azure Blob Storage endpoint reports an asynchronous blob copy as completed without the optional `x-ms-copy-source` or `x-ms-copy-status-description` header. [#120935](https://github.com/ClickHouse/ClickHouse/pull/120935) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reads from Azure Blob Storage now verify that the returned range starts at the requested offset, so an endpoint or proxy that ignores the request `Range` fails the read with `HTTP_RANGE_NOT_SATISFIABLE` instead of delivering the wrong bytes. [#120955](https://github.com/ClickHouse/ClickHouse/pull/120955) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* A bounded read from Azure Blob Storage whose response ends before the requested right bound is now reopened and, if the endpoint keeps answering short, fails with `UNEXPECTED_END_OF_FILE` instead of silently returning a truncated file. [#120963](https://github.com/ClickHouse/ClickHouse/pull/120963) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix a buffer over-read on the sequential read path from Azure Blob Storage: an endpoint that answers a ranged request with more data than was requested could deliver bytes from outside the requested range to the caller. The end of a read is now decided by the locally set right bound, not by the `Content-Length` of the response, and a read is pinned to the listed generation of the blob with `If-Match`, so an object replaced during a read fails with `AZURE_OBJECT_CHANGED_DURING_READ` instead of being truncated (new setting `azure_validate_etag_on_read`, enabled by default). [#120910](https://github.com/ClickHouse/ClickHouse/pull/120910) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a disk with `object_storage_type = 'local'` and a relative `path` removing its own root directory once the last blob in it was deleted. Anything else stored in that directory was lost with it; in particular, a disk whose `path` pointed at the directory of its own `local` metadata became unusable. [#119119](https://github.com/ClickHouse/ClickHouse/pull/119119) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `S3` requests that keep failing with a retryable error being attempted `(s3_retry_attempts + 1)^2` times instead of `s3_retry_attempts + 1`. Fixed an error that a plain-HTTP `S3` endpoint sends before reading the whole request body (for example `NoSuchUpload` for an expired multipart upload) being retried as a `Broken pipe` network error instead of being reported. Closes [#124326](https://github.com/ClickHouse/ClickHouse/issues/124326). [#124557](https://github.com/ClickHouse/ClickHouse/pull/124557) ([Groene AI](https://github.com/groeneai)).
* Fixed a heap use-after-free in the `s3`, `azureBlobStorage`, `hdfs`, and similar object-storage table functions and engines when an exception such as `MEMORY_LIMIT_EXCEEDED` was thrown while creating the reader of a file in a row input format such as `CSV`. [#125090](https://github.com/ClickHouse/ClickHouse/pull/125090) ([Groene AI](https://github.com/groeneai)).
* Fixed exceptions such as `UNKNOWN_CODEC` when reading `MergeTree` tables through the userspace page cache (`use_page_cache_for_disks_without_file_cache = 1` with `remote_filesystem_read_method = 'read'`, or `use_page_cache_for_local_disks = 1`): files kept inside packed archives (statistics, skip indices, parts in packed storage) were read from the wrong offset. Closes [#122157](https://github.com/ClickHouse/ClickHouse/issues/122157). [#122949](https://github.com/ClickHouse/ClickHouse/pull/122949) ([Groene AI](https://github.com/groeneai)).
* Fix Parquet `time32`/`time64` columns being read as `DateTime64` and then silently shifted by `session_timezone` when inserted into a `Time64` target column. Parquet `TIME` logical types now map directly to `Time64`, and the missing `Time64 -> Time` cast is added. [#106019](https://github.com/ClickHouse/ClickHouse/pull/106019) ([Shaohua Wang](https://github.com/tiandiwonder)).
* Fixed a `Parquet` reader bug where a filter on an integer column read with a type hint that reorders the file's statistics (for example `UInt32` over a plain `INT32` column, `Enum8` over a `UINT_8` one, or `IPv4` over a `UINT_64` one) could silently return fewer rows than the file holds. Such statistics are no longer used for row group or page pruning, and a `DATE` value outside the requested type's range now raises the same error with pruning as without it. Closes [#118376](https://github.com/ClickHouse/ClickHouse/issues/118376). [#121033](https://github.com/ClickHouse/ClickHouse/pull/121033) ([Groene AI](https://github.com/groeneai)).
* Fixed an infinite loop in the Parquet reader on a ZSTD or Brotli compressed page that decompresses to fewer bytes than its header declares. Such a query could not be stopped by `max_execution_time` or `KILL QUERY`; it now fails with `INCORRECT_DATA`. Closes [#123274](https://github.com/ClickHouse/ClickHouse/issues/123274). [#123298](https://github.com/ClickHouse/ClickHouse/pull/123298) ([Groene AI](https://github.com/groeneai)).
* Fix `Unexpected number of rows in column subchunk` (`LOGICAL_ERROR`) when filtering a Parquet file whose `Bool` column is RLE-encoded (pyarrow's default for data page v2). The filter is now applied while decoding RLE-encoded and dictionary-encoded pages, which also makes `PREWHERE` over dictionary-encoded columns of Parquet files without a page index faster (ClickBench q31 1.92 s to 1.21 s). Closes [#123483](https://github.com/ClickHouse/ClickHouse/issues/123483). [#123618](https://github.com/ClickHouse/ClickHouse/pull/123618) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results when reading a `Parquet` file with a structure hint whose type differs from the stored one. Row-group and page pruning, bloom filters, and dictionary filters now hash and interpret the query constant in the stored physical type rather than the hinted one, which fixes cases such as narrowing, signedness flips, `Enum`, `Date`/`Date32`/`Time`, length-mismatched `String`/`FixedString`, and `Bool` over an integer column. Reading a `UINT_8` column with a `Bool` hint now also treats every nonzero value as `true`, so `x = true`, `x IN (true)`, and `WHERE x` agree. [#123864](https://github.com/ClickHouse/ClickHouse/pull/123864) ([Groene AI](https://github.com/groeneai)). [#121077](https://github.com/ClickHouse/ClickHouse/pull/121077) ([Groene AI](https://github.com/groeneai)). [#124285](https://github.com/ClickHouse/ClickHouse/pull/124285) ([Groene AI](https://github.com/groeneai)).
* Fixed `Parquet` output failing with `Capacity error: array cannot contain more than 2147483646 bytes` or `Uncompressed page is too big` when a batch of rows held more than 2 GiB of `String` or `FixedString` data. A batch is now bounded by bytes as well as by rows, which also lowers the peak memory of the write. [#122482](https://github.com/ClickHouse/ClickHouse/pull/122482) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed `SELECT ... FORMAT Arrow` failing with `Method getDataAt is not supported` for `JSON`, `Dynamic`, and `QBit` columns, and silently exporting heap addresses instead of the aggregate state for `AggregateFunction` columns. Added the `output_format_arrow_unsupported_types` setting (`throw` / `text` / `binary`, default `binary`), which supersedes `output_format_arrow_unsupported_types_as_binary`. Such columns can now also be read back into their original type, which previously failed or silently corrupted the value. `INSERT INTO ... ENGINE = ArrowFlight` now follows this setting too; it previously rejected such columns unconditionally, so an insert that used to fail may now write an opaque column. Set `output_format_arrow_unsupported_types = 'throw'` to keep rejecting it. [#115250](https://github.com/ClickHouse/ClickHouse/pull/115250) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix `O(n^2)` decompression of gzip/zlib/deflate streams whose single DEFLATE block spans the whole stream, the shape zlib-ng produces at compression level 1 (the default of the official .NET SDK): a 24 MiB gzip HTTP body that took 11 s to ingest in 26.7 now takes 0.12 s, identical to a multi-block body, with memory bounded by the buffer size instead of the block size. [#114059](https://github.com/ClickHouse/ClickHouse/pull/114059) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed a segmentation fault or `std::length_error` when reading `TSV` formats with `input_format_tsv_crlf_end_of_line = 1` if an escape sequence ended exactly at the end of a read buffer. [#123861](https://github.com/ClickHouse/ClickHouse/pull/123861) ([Groene AI](https://github.com/groeneai)).
* Fixed lost rows when a query over a `MySQL`, `PostgreSQL`, `SQLite`, or ODBC/JDBC table compares a column, string, number, or `Decimal`, with an `Enum` constant, also inside a tuple, an `IN` list, `Nullable`, `Variant`, or `Dynamic`, at any depth. The filter sent to the external database used the `Enum`'s number, while ClickHouse compares a string column with the `Enum`'s name, and `PostgreSQL` rejected such a query. An `IN` list or row of `Variant` or `Decimal` constants is no longer sent as one quoted string. [#123843](https://github.com/ClickHouse/ClickHouse/pull/123843) ([Groene AI](https://github.com/groeneai)).
* Fixed wrong results of a negated `IN` with a `NULL` in the list, e.g. `x NOT IN (1, NULL)` or `NOT (x IN (1, NULL))`, over `MySQL`, `PostgreSQL`, `SQLite`, and `ODBC` tables: the filter pushed to the external database now leaves the `NULL` members out of the list, the way ClickHouse evaluates the set. [#124339](https://github.com/ClickHouse/ClickHouse/pull/124339) ([Groene AI](https://github.com/groeneai)).
* Fixed silent data loss when an `EmbeddedRocksDB` table is renamed. `RENAME TABLE` now moves the table's `RocksDB` directory to the table's new location, so the rows are still there after a server restart. Previously the directory was left behind, and after a restart the table opened a fresh empty `RocksDB` while the real rows stayed orphaned under the old path. [#112224](https://github.com/ClickHouse/ClickHouse/pull/112224) ([Groene AI](https://github.com/groeneai)).
* `optimize_inverse_dictionary_lookup` (enabled by default) no longer rewrites `dictGet(dict, attr, key) = const` when a `Variant`, `Dynamic`, or `JSON` appears in the type of `key` or in the dictionary's declared key type. The rewritten predicate is not equivalent there, so the query could fail with `CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN`, `ILLEGAL_TYPE_OF_ARGUMENT`, or `NO_COMMON_TYPE`, or return wrong results. [#120888](https://github.com/ClickHouse/ClickHouse/pull/120888) ([Groene AI](https://github.com/groeneai)).
* Fixed SQL injection through identifiers in the `PostgreSQL` integration: relation, schema, column, publication, and replication-slot names used by the `PostgreSQL` table engine, the `postgresql` table function, `PostgreSQL` dictionary sources, and `MaterializedPostgreSQL` were escaped with backslashes, which PostgreSQL does not honour inside a quoted identifier, so a remote name containing a double quote could end the identifier and let the rest execute on the PostgreSQL server. Relations and columns whose names contain a double quote or a backslash can now be read and replicated, including those listed in `materialized_postgresql_tables_list`; an empty element in that setting is now rejected. [#119216](https://github.com/ClickHouse/ClickHouse/pull/119216) ([Groene AI](https://github.com/groeneai)). [#119103](https://github.com/ClickHouse/ClickHouse/pull/119103) ([Groene AI](https://github.com/groeneai)).
* Fixed `PARALLEL WITH`, `EXECUTE AS`, and the `QueryRunner` table engine skipping the access checks of the `CREATE` statements they execute, which allowed creating tables and databases without the corresponding privileges and defeated the `readonly` and `allow_ddl` settings. [#119848](https://github.com/ClickHouse/ClickHouse/pull/119848) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed an `HDFS` database serving a table from its internal cache without the `READ ON HDFS` grant. The grant and the `remote_url_allow_hosts` filter were checked only while a table name was resolved for the first time, so once any user holding the grant had read a name, every user with `SELECT` on the database was served the contents of the file, and that table kept being served after its host was removed from the allow-list. `EXISTS TABLE`, which requires only `SHOW TABLES`, answered from the same cache and so reported which names other users had resolved. Both checks now run before the cache is consulted, and the existence probe no longer consults it. [#119523](https://github.com/ClickHouse/ClickHouse/pull/119523) ([Groene AI](https://github.com/groeneai)).
* Fixed filesystem containment checks accepting a path with an embedded NUL byte. A `DeltaLakeLocal` or `IcebergLocal` table could be pointed outside of `user_files` this way. [#119846](https://github.com/ClickHouse/ClickHouse/pull/119846) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed several engines disclosing an `Alias` table's target schema to a user who holds no privilege on that target. Reading the metadata of an `Alias` through any of them now requires table-level `SHOW COLUMNS` on its target, which is what a `DESCRIBE` of the target already requires. [#117572](https://github.com/ClickHouse/ClickHouse/pull/117572) ([Groene AI](https://github.com/groeneai)).
* `remote(<cluster name>, 'db', 'table', 'user', 'password')`, and the same form of `remoteSecure` and of the `Remote` and `RemoteSecure` table engines, no longer write the password into `system.query_log`, `system.text_log`, and the server log in cleartext. [#121474](https://github.com/ClickHouse/ClickHouse/pull/121474) ([Groene AI](https://github.com/groeneai)).
* Fixed SQL injection through identifiers when inserting into a table with the `SQLite` engine. It is now also possible to insert into a SQLite table whose name, or column name, legitimately contains a double quote or a backslash. [#117421](https://github.com/ClickHouse/ClickHouse/pull/117421) ([Groene AI](https://github.com/groeneai)).
* Fixed credential masking for `nats_url`, `rabbitmq_address`, the `jdbc` and `odbc` connection string, and the `format_avro_schema_registry_url`, `url_base`, and `s3_base` settings in `SHOW CREATE`, `system.tables`, and the query text in `system.query_log`. Closes [#121257](https://github.com/ClickHouse/ClickHouse/issues/121257). [#121425](https://github.com/ClickHouse/ClickHouse/pull/121425) ([Groene AI](https://github.com/groeneai)).
* Fixed a `Backup` database engine printing the credentials of its destination in plaintext. `SHOW CREATE DATABASE`, `system.databases.engine_full`, `system.query_log`, and the messages that reject a malformed locator no longer expose an `AzureBlobStorage` `account_key`, a connection string, or a presigned URL regardless of `format_display_secrets_in_show_and_select`. [#117429](https://github.com/ClickHouse/ClickHouse/pull/117429) ([Groene AI](https://github.com/groeneai)).
* `KILL TRANSACTION ... ON CLUSTER`, `KILL PART_MOVE_TO_SHARD ... ON CLUSTER`, `SYSTEM STOP THREAD FUZZER ON CLUSTER`, `SYSTEM START THREAD FUZZER ON CLUSTER`, and `SYSTEM RESET COVERAGE ON CLUSTER` no longer run without checking any privilege. Each now requires what its non-`ON CLUSTER` spelling has always required. [#118360](https://github.com/ClickHouse/ClickHouse/pull/118360) ([Groene AI](https://github.com/groeneai)).
* Fixed metadata disclosure through a chain of `Alias` tables. `DESCRIBE`, `hasColumnInTable`, `system.columns`, `system.tables`, `system.projections`, `system.constraints`, and `system.data_skipping_indices` now require the privilege on every table an `Alias` resolves through, as reading its rows already did. [#118967](https://github.com/ClickHouse/ClickHouse/pull/118967) ([Groene AI](https://github.com/groeneai)).
* Fixed a bug where `REVOKE GRANT OPTION FOR` was accepted but then omitted from `SHOW GRANTS`, from `system.grants`, and from the persisted access metadata whenever another revoked database, table, column, or named-collection name shared a prefix with it. The grant option therefore came back after a server restart, and the user could pass it on with `GRANT CURRENT GRANTS`. [#122078](https://github.com/ClickHouse/ClickHouse/pull/122078) ([Groene AI](https://github.com/groeneai)).
* Fixed settings constraints from a settings profile not being enforced against the other setting assignments in the same statement. `SET profile = 'p', <setting> = <value>` could override a `CONST`, `MIN`/`MAX`, or `readonly` constraint that profile `p` installs. The same escape existed through an in-query `SETTINGS` clause, HTTP URL parameters, and Arrow Flight session options. [#120376](https://github.com/ClickHouse/ClickHouse/pull/120376) ([Raúl Marín](https://github.com/Algunenano)).
* Fixed row policies on `Memory` tables being ignored when a read is executed from a serialized query plan, such as a distributed plan or a parallel-replicas plan. Since `Memory` gained `PREWHERE` support, such reads could return rows the policy excludes. [#122503](https://github.com/ClickHouse/ClickHouse/pull/122503) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed an integer underflow in the `MySQL` protocol handshake handler that could cause the server to buffer an unbounded amount of data from an unauthenticated client. [#115776](https://github.com/ClickHouse/ClickHouse/pull/115776) ([Dmitrii Tishkin](https://github.com/dmi03)).
* Statistics-based part pruning is now disabled for tables with row policies. [#122035](https://github.com/ClickHouse/ClickHouse/pull/122035) ([alesapin](https://github.com/alesapin)).
* A row policy on a `Set`-engine table was accepted but never applied: the right side of `IN` read the prebuilt set directly and returned the rows the policy hides. Such a query now fails with `ACCESS_DENIED`, because the `Set` engine has no read path that could filter those rows, and `IN set_table` now requires `SELECT` on all columns of the table. On `Replicated` databases, `ALTER ... DELETE`, `ALTER ... UPDATE`, `DELETE`, and `UPDATE` now run the initiator's checks as the submitting user, so the same row-policy and access checks apply there too; with `ON CLUSTER`, run the query on a host that has the right-hand `Set` table, or enable `distributed_ddl_use_initial_user_and_roles` on every host and set `distributed_ddl_entry_format_version` to at least `8`. Queries that used a `Set` table covered by a database-wide row policy ran unfiltered before and now throw. [#120312](https://github.com/ClickHouse/ClickHouse/pull/120312) ([Harikrishnan Prabakaran](https://github.com/harikrishnan94)). [#122141](https://github.com/ClickHouse/ClickHouse/pull/122141) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fixed the `Kafka` and `NATS` table engines ignoring `remote_url_allow_hosts`: broker addresses from `kafka_broker_list`, `nats_url`, and `nats_server_list` are now checked against the remote host filter. A forbidden address is rejected at `CREATE TABLE` with `UNACCEPTABLE_URL`, as the `RabbitMQ` engine already does. [#121825](https://github.com/ClickHouse/ClickHouse/pull/121825) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
* HTTP headers in `HTTP` dictionary sources are now hidden in `SHOW CREATE DICTIONARY`, `system.tables`, and query logs, like passwords. They can still be shown with `display_secrets_in_show_and_select`. [#122673](https://github.com/ClickHouse/ClickHouse/pull/122673) ([Matheus Nerone](https://github.com/Manerone)).
* Fixed secret settings of `S3Queue` and `AzureQueue` tables written with the legacy `s3queue_` prefix, such as `s3queue_after_processing_move_secret_access_key`, being shown in cleartext in `SHOW CREATE TABLE`, `system.tables`, `system.query_log`, and the server log. They are now hidden as they are without the prefix. [#123060](https://github.com/ClickHouse/ClickHouse/pull/123060) ([Groene AI](https://github.com/groeneai)).
* Column-level `SELECT` privileges are now enforced for columns used only insid