integration-test/AGENTS.md
Rama master · réplica de lectura del repositorio original.
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Integration Test Guidelines
These instructions apply to `integration-test/` and supplement the repository-level instructions.
## Test discovery and categories
- Name concrete IT classes to match Failsafe's default includes: `IT*`, `*IT`, or `*ITCase`.
Prefer the existing `IoTDB...IT` convention; put variant numbers before the suffix, such as
`IoTDBActiveRegionScan2IT`, not `IoTDBActiveRegionScanIT2`. An `IoTDB` prefix or a `Test` suffix
alone does not make a class discoverable.
- Every test method must have an effective JUnit `@Category`, declared on the method, its class,
or a superclass. All test profiles filter by category; `@Test` and `@RunWith` alone are not
sufficient. In a class using method-level categories without a class/superclass category,
annotate every new `@Test` method, even when neighboring methods already have categories.
- Prefer a class-level category when all cases share the same requirements. Preserve method-level
categories when cases need different profiles; adding a broad class category can enable cases
in unintended environments. Inherited categories are valid and need not be copied to subclasses.
- Select categories by the actual data model and topology, using the profile mapping below and
`pom.xml`. Shared packages such as `session/it` contain both tree and table tests; the package
name alone does not determine the category. Check the corresponding CI workflow for specialized
categories such as `DailyIT` and multi-cluster tests.
- Follow the neighboring tests' runner and environment lifecycle conventions. Ordinary ITs use
`@RunWith(IoTDBTestRunner.class)`; retain inherited or specialized runners for parameterized
fixtures such as `AbstractSchemaIT` instead of replacing them.
## Credentials and deployment defaults
- Use `EnvFactory.getEnv()` connection helpers or client builders with their default credentials
when a test does not need a specific user.
- When constructing clients or helpers that require explicit default credentials, use
`SessionConfig.DEFAULT_USER` and `SessionConfig.DEFAULT_PASSWORD`. This includes
`NodesSupplier`, pools, and direct RPC helpers; do not hardcode `"root"` as the default password.
Default credentials can differ between IoTDB and downstream editions.
- Keep a client's business connections and discovery/recovery helpers on the same credentials.
Check helper methods as well as the main test body: a builder may use the correct defaults
while a manually constructed discovery client silently uses different ones.
- Explicit credentials are appropriate for users created by the test and for authentication or
invalid-password scenarios. Make that intent clear, and clean up users created by the test.
- When a test creates users or roles with explicit credentials, make each password 12-32
characters long with no spaces, drawn from the supported credential charset (letters,
digits, `_ - ! @ # $ % ^ & * ( ) + =`), containing at least one lowercase letter, one
uppercase letter, one digit, and one special character, and different from the username;
usernames use the same charset and a 4-32 character length (for example `newuser` with
`IoTDB!Test2024`). Trivial placeholders such as `123456` or `password` pass on a default
local cluster yet are exactly what stricter password policies reject, and they encourage
weak secrets to spread into real configurations through copied test code. Reserve weak or
illegal credentials for scenarios that deliberately test them.
- Obtain node addresses and allocated ports from the test environment or node wrappers. Configure
required consensus protocols, replication factors, and other non-default settings before
initializing the cluster.
## Asynchronous behavior and failure recovery
- Use bounded condition waits, such as Awaitility or environment status helpers, for startup,
discovery, leader election, and recovery. Prefer observable conditions over fixed sleeps.
- When no read-only condition exposes the state being awaited, use a finite wait derived from the
running server's effective configuration and the required propagation rounds, with a documented
safety margin. Reuse a feature-specific wait helper; do not guess a fixed duration or assume
the test JVM's configuration singleton reflects a separately launched node's configuration.
- Wait for each relevant layer separately. A node status reported by ConfigNode does not prove
that an existing client has refreshed its endpoint snapshot or recovered its connection.
- For locally stopped nodes, use `EnvUtils.getNodeStatusAfterLocalStop()` where applicable;
graceful process termination behaves differently on Unix and Windows. Use the appropriate
status expectation for forced termination.
- Keep connection/socket timeouts distinct from query timeouts and the overall assertion deadline.
Allow for startup and election costs on CI; use deliberately short timeouts only when needed by
the scenario. Do not retry non-idempotent setup statements just to hide a short timeout.
- For tests that require a stale discovery snapshot or a particular failover order, control
background refresh and assert the preconditions before triggering recovery.
- Investigate authentication, configuration, and transport failures before increasing a wait
timeout. A retained client snapshot can mean that discovery never succeeded. Include useful
endpoint/status context and preserve underlying failures when adding diagnostics.
- Do not label a failure as environmental or intermittent based only on one failed run.
## Isolation and assertions
- Reuse existing test classes and fixtures, and share cluster startup and teardown across
compatible cases within a class or suite whenever possible. Avoid starting and stopping a
cluster for each test method or creating an extra cluster when an existing test cluster can
cover the scenario. Use a separate cluster or restart only when required by the tested lifecycle,
topology, configuration, or isolation.
- Keep tests that share a cluster independent of execution order: isolate their data and restore
changed state or configuration using the repository's test helpers. Wait for asynchronous
cleanup effects, such as quota accounting after schema deletion, before the next method's setup;
successful cleanup SQL alone does not prove that dependent cluster state has converged.
- Do not use global database-count statements such as `COUNT DATABASE` or `COUNT DATABASE root.**`
in ITs, including helper methods and wait conditions. Limit counts to an explicit test-owned
database path or test-specific path pattern, or assert the existence or absence of the specific
test databases. Do not assume that the cluster's total database count equals the number created
by the test; pre-existing databases can vary across test environments.
- Close sessions, pools, and result sets; shut down test-owned executors and clean the cluster
even when assertions fail. Use try-with-resources and `finally` as appropriate.
- Assert the behavior under test and its recovery result. Only tolerate expected cleanup errors
in narrowly scoped cleanup code; do not swallow failures from the operation being verified.
- When enabling an old test or changing its expected result, verify the current intended
client/server contract and preserve the original regression input and execution path. For an
expected rejection, assert the specific exception and status/message so that an unrelated
transport or setup failure cannot satisfy the test.
- Do not weaken assertions, remove cluster categories, or add `@Ignore` merely to make a failing
test pass CI. If a known product defect must be quarantined, retain the reproducer and document
the defect, affected topology, and tracking reference; report skipped cases separately.
- Account for platform semantics in process-exit assertions: POSIX exposes only the low 8 bits
of an exit status. When checking a portable process status derived from a `TSStatusCode`,
compare the low 8 bits of both values (`& 0xFF`); keep full status-code checks for RPC responses.
### Result ordering across DataNodes
- When row order is not part of the query's contract and may differ across DataNodes, use
`TestUtils.treeResultSetEqualTestUnordered(sql, expectedHeader, expectedRetArray)` for tree-model
SQL cases, or
`TestUtils.tableResultSetEqualTestUnordered(sql, expectedHeader, expectedRetArray, database)`
for table-model SQL cases (`database` may be `null` when no `USE` is needed).
When the test already owns a result set, use
`TestUtils.assertResultSetEqualUnordered(resultSet, expectedHeader, expectedRows)` and close
the original result set after the assertion. Reuse these helpers instead of writing per-node
comparison loops in individual tests.
- These helpers check every node independently, including headers, values, and duplicate counts.
They use `ClusterTestResultSet.getNodeResultSets` to read each DataNode's underlying result set.
The default `ClusterTestResultSet` compares nodes at the same cursor position; collecting or
sorting rows after reading through that wrapper cannot avoid row-order mismatches. Do not
replace the helpers with set equality or single-node checks that weaken these assertions.
- Keep ordered assertions when SQL guarantees the final result order. For `LIMIT`/`OFFSET`,
ensure the selected rows are deterministic before applying unordered equality; sorting cannot
repair a nondeterministic subset. Add sufficient `ORDER BY` tie-breakers when needed.
## Verification
This module is excluded from the default build. Run the commands below from the repository root,
always enabling `-P with-integration-tests` when building or testing this module.
Format Java changes with `mvn spotless:apply -pl integration-test -P with-integration-tests`.
### Selection check
After adding or renaming tests, changing categories or inheritance, or editing CI test selection,
run this source-level check from the repository root before the Maven build:
```bash
python3 .github/scripts/check-it-selection.py
```
- The check detects non-matching class names, missing effective categories, and missing or
duplicated shard candidates. Also inspect the intended profile's `groups`/`excludedGroups`,
POM excludes, and `@Ignore`; the check does not prove that a particular CI job selects a test.
Running a test in an IDE or explicitly with `-Dit.test` does not prove default CI discovery.
- When editing CI sharding, reuse `.github/scripts/build-it-shard-list.sh`: shard all candidate
classes by fully qualified name and let Failsafe apply category filters. Do not grep source
files for category names; that misses inherited categories. Preserve POM excludes by using
`failsafe.includesFile`, and reject empty lists because Failsafe falls back to default includes.
Verify that shards cover each candidate exactly once and that their selected-test union matches
the unsharded run for the same profile.
### Targeted test runs
Run only the affected IT classes or methods for a focused change. Select the matching test profile:
| Data model | Default topology | JUnit category | Test profile |
| --- | --- | --- | --- |
| Tree | 1 ConfigNode + 1 DataNode | `LocalStandaloneIT` | `SimpleIT` (default) |
| Tree | 1 ConfigNode + 3 DataNodes | `ClusterIT` | `ClusterIT` |
| Table | 1 ConfigNode + 1 DataNode | `TableLocalStandaloneIT` | `TableSimpleIT` |
| Table | 1 ConfigNode + 3 DataNodes | `TableClusterIT` | `TableClusterIT` |
For example, run one tree-model cluster test method:
```bash
mvn clean verify -DskipUTs \
'-Dit.test=ClassName#methodName' \
-DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false \
-pl integration-test -am -PClusterIT -P with-integration-tests
```
Use `-Dit.test=ClassName` to run a whole class. Replace `-PClusterIT` with the profile needed by the
test; it may be omitted for the default `SimpleIT` profile.
- Check `integration-test/target/failsafe-reports` to confirm that the intended classes and
methods actually ran, including expected parameterized variants, and inspect skipped counts.
A successful Maven exit is insufficient: the no-tests flags allow upstream reactor modules
to have no matching tests and can also hide an incorrectly selected IT.
- For changes involving distributed state in tests enabled for both single- and multi-DataNode
profiles, verify the affected cases in both profiles. A 1C1D pass does not establish 1C3D behavior.
- Report the tested revision, edition, profile, and result accurately. Passing an upstream build
does not establish that the corresponding downstream edition was tested.
### Node templates and IDE runs
Build the node templates before an IDE run or a direct Failsafe invocation, and rebuild them after
code changes:
```bash
mvn clean package -DskipTests -pl integration-test -am -P with-integration-tests
```
The `clean verify` command above already includes this packaging step. In IntelliJ, enable the
`with-integration-tests` profile in the Maven sidebar before running test cases directly.
### Edge tests
Build the Edge distribution before running its dedicated Tree/Table read-write IT:
```bash
mvn clean package -DskipTests -pl distribution -am
mvn verify -DskipUTs -Dit.test=IoTDBEdgeBasicIT \
-DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false \
-pl integration-test -am -P EdgeIT -P with-integration-tests
```