docs/content-as-code.md
Rama main · réplica de lectura del repositorio original.
# Content as code architecture
Content as code turns Lightdash resources into portable, reviewable files that
can be downloaded from one instance and uploaded to another. This document
describes the responsibility boundary between the CLI and the backend, how the
portable representation is built, and where validation belongs.
The core rule is:
> The backend owns Lightdash domain behavior. The CLI owns local file
> processing and delegates resource validation and persistence to `/code`
> endpoints.
This keeps every API client consistent, prevents domain rules from being
reimplemented in the CLI, and lets the backend apply authorization and
validation against current server capabilities.
## Responsibility boundary
| Layer | Responsibilities |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Common | Defines the portable resource and API response types used by the CLI and backend. |
| Backend controller | Exposes authenticated `/code` endpoints and generated OpenAPI contracts. |
| Backend service | Loads resources, builds the portable representation, validates uploads, resolves identity, calculates changes, and persists them. |
| CLI | Selects the project or organization, reads and writes files, chooses deterministic filenames, calls the `/code` API, and presents progress and per-file failures. |
The CLI should not contain lists of valid domain values, compare uploaded
resources with database resources, or coordinate lower-level create and update
endpoints. For example, it should not know which scopes are valid for a custom
role or calculate which scopes need to be added and removed.
## Implementation structure
Content as code is a first-class subsystem rather than a set of unrelated CLI
handlers:
- Portable contracts are split by resource under
`packages/common/src/types/contentAsCode/`. The existing `coder.ts` exports
remain as a compatibility facade.
- `ProjectCoderController` owns project routes and
`OrganizationCoderController` owns organization routes. The two controllers
are separate because TSOA controllers have one route prefix. Canonical routes
use `/code/{resource}`; the previous resource-first routes remain deprecated
compatibility aliases until after 17 August 2026.
- Backend resource handlers own externalization, reference conversion,
comparison, and persistence. `CoderService` remains as a compatibility facade
while callers migrate to those handlers.
- CLI resources use `CodeResourceDefinition` adapters for scope, discovery,
parsing, identity, filenames, dependencies, and ordering. Charts, dashboards,
nested spaces, scheduled content, AI-agent bulk operations, and data-app
bundles retain specialized orchestration where their layouts require it.
- Filename allocation is centralized and handles Unicode normalization,
bounded names, stable collision suffixes, case-insensitive collisions, safe
paths, and reuse of an existing file owned by the same portable identity.
New resources should use these extension points instead of adding another
standalone download/upload implementation.
## Homepages
Published homepage layouts can be exported with `lightdash download
--include-homepages`, `--homepages "Team homepage"`, or `--include-all`.
They are stored in `homepages/*.yml` under the content root. The exact homepage
name is its project-scoped identity; duplicate names are rejected and changing
the name creates a separate homepage.
Uploads resolve content slugs and organization group names in the destination.
The existing upload flow imports homepage files after their referenced content.
`lightdash upload --homepages "Team homepage"` imports only that homepage, so its
referenced content must already exist. Imports save drafts by default; add
`--publish` to publish the layout and apply its exported default/group/role
assignments. `--skip-homepages` excludes homepage files from an upload.
Internal content links use `lightdash://dashboards/<slug>/view` (or `saved`,
`spaces`, and `apps`) in files and become destination project links on upload.
External links and live collection rules remain unchanged. Announcement entries,
personalized results, signed app thumbnails, and organization homepage settings
are not exported. Imports require the existing homepage builder feature and
homepage management permission; they do not enable it or publish announcements.
## Download flow
1. The CLI resolves the project or organization and the local content root.
2. The CLI calls the resource's backend `GET .../code/{resource}` endpoint.
3. The backend checks access and loads the database resources.
4. The backend converts each database resource into its portable as-code type.
5. The CLI serializes the returned resources into deterministic YAML files.
Building the portable representation is a backend responsibility because it
requires knowledge of the database model and the public resource contract. The
representation should:
- include a format `version`;
- contain semantic, portable fields rather than database implementation
details;
- omit UUIDs, timestamps, ownership metadata, and other instance-specific
values unless they are part of the portable identity;
- use stable ordering for unordered collections so repeated downloads produce
clean diffs;
- use names or slugs that have a defined identity rule on upload.
The CLI may sort YAML keys and normalize filenames, but it should not reshape
the resource returned by the backend.
## Upload flow
1. The CLI discovers and parses the relevant YAML files.
2. The CLI sends each portable document to the resource's backend `POST
.../code/{resource}` endpoint.
3. The backend authenticates and authorizes the request.
4. The backend validates the portable document and its domain invariants.
5. The backend resolves the existing resource using the documented portable
identity.
6. The backend creates the resource, applies the required update, or determines
that there are no changes.
7. The backend returns a `CREATE`, `UPDATE`, or `NO_CHANGES` action.
8. The CLI aggregates those actions and reports any per-file failures.
Sending resources individually is useful when files are independent. A bad
file can fail without preventing valid sibling files from being processed. The
backend still owns the transaction and atomicity of each individual resource.
Omitting a file does not delete the remote resource unless a resource explicitly
defines deletion semantics.
## Validation model
Validation is split by ownership rather than duplicated across layers.
### CLI validation
The CLI validates only local file and bundle concerns:
- the expected directory exists and contains supported files;
- YAML can be parsed;
- filenames can be generated safely and deterministically;
- files that would represent the same portable identity are reported as a
bundle conflict;
- processing continues after an independent file fails.
Parsed YAML is treated as unknown input. Shared parsers may validate structural
contract concerns needed for safe local processing, including the document
shape, version, content type, and portable identity. The backend repeats the
required contract checks and remains responsible for domain rules.
The CLI must not ship a snapshot of server domain rules such as supported
permission scopes, allowed transitions, or mutable fields. Structural parsing
must preserve documented legacy behavior, including resources where
`contentType` was historically optional and unknown keys were accepted.
### Backend validation
The backend validates everything needed to understand and persist the
resource:
- authentication and authorization for the target project or organization;
- supported format version and expected fields;
- field types and required values;
- references to other Lightdash resources;
- current server capabilities and allowed enum-like values;
- resource-specific invariants and immutable fields;
- uniqueness and identity constraints;
- whether the requested operation is a create, update, or no-op.
TypeScript types and the generated OpenAPI schema document the API contract,
but they do not replace runtime validation in the service. YAML, older clients,
and direct API callers can all provide values that were not produced by the
current TypeScript code.
Validation errors should identify the invalid field or value. The CLI adds the
source filepath to the backend error when reporting a failed document.
## Project and organization content
Project content uses project-scoped `/code` endpoints and is selected through
the normal project download and upload workflow. Organization content uses
organization-scoped `/code` endpoints and an explicit `--organization` mode.
Both scopes use the standard `lightdash/` content root. Organization resources
live in resource-specific subdirectories rather than an additional
`organization/` directory. For example:
```text
lightdash/
custom-roles/
developer-view-only.yml
```
The CLI success message refers to the shared `lightdash/` parent because future
organization resources will use the same root.
## Resource catalog
| Resource | Scope | Portable identity | Primary location |
| -------------------- | ------------ | -------------------------------- | ---------------------------------------- |
| Charts | Project | Slug | `lightdash/charts/` or nested spaces |
| SQL charts | Project | Slug | Chart layouts using `.sql.yml` |
| Dashboards | Project | Slug | `lightdash/dashboards/` or nested spaces |
| Spaces | Project | Full hierarchy path | `.space.yml` files |
| Virtual views | Project | Slug | `lightdash/virtual-views/` |
| AI agents | Project | Slug | `lightdash/ai-agents/` (+ `<slug>/knowledge-documents/`) |
| Scheduled deliveries | Project | Project-scoped slug | `lightdash/scheduled-deliveries/` |
| Alerts | Project | Project-scoped slug | `lightdash/alerts/` |
| Google Sheets syncs | Project | Project-scoped slug | `lightdash/google-sheets/` |
| External connections | Project | Project-scoped slug | `lightdash/external-connections/` |
| Custom roles | Organization | Exact role name | `lightdash/custom-roles/` |
| Users | Organization | Lowercase primary email | `lightdash/users/` |
| Groups | Organization | Exact, case-sensitive group name | `lightdash/groups/` |
| User attributes | Organization | Exact attribute name | `lightdash/user_attributes/` |
| Data App themes | Organization | Immutable organization slug | `lightdash/themes/<slug>/` |
Data apps and organization Data App themes are deliberately outside the
single-document YAML resource registry because they are multi-file bundles.
They reuse shared path-safety, dependency-ordering, and reporting utilities
without pretending to be single-document resources.
Project APIs use `/api/v1/projects/{projectUuid}/code/{resource}`. Organization
APIs use `/api/v2/orgs/{orgUuid}/code/{resource}`. The resource segments are
`charts`, `sqlCharts`, `dashboards`, `spaces`, `virtualViews`, `aiAgents`,
`scheduledDeliveries`, `alerts`, `googleSheets`, `externalConnections`,
`roles`, `users`, `groups`, and `userAttributes`.
The legacy resource-first routes are deprecated in OpenAPI and should not be
used by new clients.
Organization themes use the organization-design package endpoints instead of a
`/code/{resource}` endpoint because their portable representation is an
uncompressed tar archive rather than one JSON document.
## AI agent knowledge documents
An agent's knowledge documents travel with the agent. The YAML lists them
under `knowledgeDocuments`, and each document's text lives in its own file
beside the YAML so a change in Git is a readable diff:
```
lightdash/ai-agents/
revenue-agent.yml
revenue-agent/
knowledge-documents/
revenue-definitions.md
refund-policy.md
```
```yaml
knowledgeDocuments:
- name: Revenue definitions
file: revenue-agent/knowledge-documents/revenue-definitions.md
alwaysIncludeInContext: true
- name: Refund policy
file: revenue-agent/knowledge-documents/refund-policy.md
alwaysIncludeInContext: false
```
`file` is relative to the YAML's directory and must stay inside it; `.md`
files are markdown documents and `.txt` files plain text, and download writes
each document back with the matching extension. The CLI inlines the content
for the API; the server never sees file paths. Download
writes only the documents the agent owns alone: documents shared with other
agents or with the whole organization are not exported and an upload never
touches them.
The list follows the same rule as `skills`: omitted means "leave the agent's
documents as they are", present means authoritative. Upload matches by
document name, creates the missing ones, rewrites changed content, toggles
`alwaysIncludeInContext`, and deletes the agent-owned documents the list no
longer names. Unchanged content is skipped, so a repeat upload does not
regenerate the document's AI summary. The per-document size limit and the
organization quota apply as in the UI; a document that fails them fails only
its agent.
With `content_as_code.sync: true`, every uploaded document is recorded as
Git-backed. The agent settings page then shows it as managed by Content as
Code and refuses edits, usage changes and deletion from the UI for everyone;
the repo is the only writer until the document is removed from the repo and
uploaded again. Documents nobody uploaded stay editable, so one agent can mix
repo-managed and UI-managed documents. Without sync, upload is a plain copy
and the UI stays editable, like charts.
## Optional project resources
Virtual views, AI agents, alerts, scheduled deliveries, Google Sheets syncs,
and external connections are opt-in for download through their resource
selectors, `--include-*` flags, or `--include-all`. Upload processes matching
files already present on disk unless the corresponding `--skip-*` option is
supplied. This distinction is intentional: a normal download must not delete
or rewrite optional local resources that were not fetched.
Google Sheets documents contain portable destination metadata. Content-as-code
uploads validate the payload and permissions but do not call Google Drive to
validate the spreadsheet URL. This allows CI and service-account deployments
without the uploader's personal Google OAuth token. Executing an enabled sync
still requires usable Google credentials for the scheduler owner.
## Organization Data App themes
Themes live under `lightdash/themes/<slug>/` with a strict
`lightdash-theme.yml` manifest and optional files directly inside `css/`,
`fonts/`, `images/`, and `instructions/`. The immutable, organization-scoped
manifest slug is the portable identity. Existing imports preserve the design
UUID and default status; a new slug creates a non-default theme.
The endpoints are:
- `GET /api/v1/org/designs/` to list themes in the authenticated organization;
- `GET /api/v1/org/designs/{uuidOrSlug}/package` to export one canonical tar;
- `PUT /api/v1/org/designs/package` to atomically import one canonical tar.
The shared validation contract in `packages/common/src/ee/designs/` is used by
both the backend package parser and the CLI's local-directory preflight. This
is an intentional exception to the normal single-document responsibility
split: the CLI must reject an unsafe or incomplete local bundle before any
organization resource is mutated, while the backend must repeat every security
and domain check before persistence.
Download stages and validates every remote package before atomically replacing
the local `themes/` directory, so a failed batch preserves the previous local
snapshot. Upload preflights every local theme first, processes organization
documents in dependency order, then imports themes sequentially. Each theme
replacement is atomic, but the batch is not: successful siblings remain active
when another import fails, and the summary names completed slugs and failures.
Missing or empty local theme directories are no-ops. Omitting one theme never
deletes it remotely, and theme sync never changes the organization default.
There are no standalone theme commands or theme-specific include, skip, or only
flags.
## External connections
Data-app external connections (see `docs/data-apps.md` → _External
connections_) are enterprise-only and both download and upload require the
admin-only `manage:ExternalConnection` scope. When `--include-all` reaches
them implicitly on a non-enterprise server or without that permission, the
CLI warns and skips; the explicit `--external-connections <slugs...>` and
`--include-external-connections` selectors fail loudly instead.
Documents live in `lightdash/external-connections/<slug>.yml`:
```yaml
contentType: external_connection
version: 1
slug: stripe-api
name: Stripe API
authType: api_key # none | api_key | bearer_token | google_service_account
origin: https://api.stripe.com
allowBrowserImages: false
allowDataAppBuilderLinking: false
instructions: null
allowedPathPrefixes: []
allowedMethods:
- GET
allowedContentTypes:
- application/json
responseMaxBytes: 1048576
requestMaxBytes: 262144
timeoutMs: 10000
rateLimitPerMinute: null
apiKeyName: Authorization
apiKeyLocation: header
oauthScopes: null
customHeaders: null
```
The endpoints are:
- `GET /api/v1/projects/{projectUuid}/code/externalConnections`
- `POST /api/v1/projects/{projectUuid}/code/externalConnections/{slug}`
The project-scoped `slug` column is the portable identity. It is generated
from the name at creation, stays stable across renames, and is unique among
live connections per project (a partial unique index; generation also reserves
soft-deleted slugs). Changing the slug in a file creates a new connection;
omitted files never delete remote connections.
**Secrets never appear in documents or API reads.** At upload time the CLI
reads `LIGHTDASH_EXTERNAL_CONNECTION_SECRET_<SLUG>` (slug uppercased with
hyphens as underscores) and sends the value alongside the document. With the
variable set, the secret is stored (an unchanged secret still reports
`NO_CHANGES`); unset, an existing connection keeps its stored secret, while
creating a connection whose `authType` requires a secret fails with an error
naming the expected variable. Clearing a secret is not supported as code —
use the settings UI. A stray `secret` key in a YAML file is stripped and
warned about, never uploaded.
App↔connection links are not part of these documents — each data app's
bundle manifest (`lightdash-app.yml`) carries them as
`externalConnections: [{alias, connectionSlug}]`, resolved by slug in the
target project on app upload. See `docs/data-apps.md` → _Data apps as code_.
## Spaces and access
Project space definitions use `.space.yml` files. New flat downloads place them
under `lightdash/spaces/`:
```text
lightdash/
spaces/
finance.space.yml
```
Use `--root-spaces` to write new flat space files at the `lightdash/` root for
the legacy layout. With `--nested`, each definition stays in its per-space
folder alongside that space's `charts/` and `dashboards/` directories.
Existing files are always updated in their current location, so downloading
does not duplicate or automatically move root, `spaces/`, or nested files.
Upload discovers all of these layouts recursively.
Each space file contains:
```yaml
contentType: space
version: 1
spaceName: Finance
slug: company/finance
access:
inheritParentPermissions: false
projectMemberAccessRole: viewer
users:
- email: owner@example.com
role: admin
groups:
- name: Finance team
role: editor
```
The endpoints are:
- `GET /api/v1/projects/{projectUuid}/code/spaces`
- `POST /api/v1/projects/{projectUuid}/code/spaces`
The previous `GET` and `POST /api/v1/projects/{projectUuid}/spaces/code` routes
are deprecated compatibility aliases shared with the other migrated resources.
The full hierarchy path in `slug` is the portable identity. Uploads process
parents before descendants and create missing spaces unless
`--skip-space-create` is used. Changing the path creates another space; missing
files do not move, delete, or revoke a remote space.
For compatibility with older downloads that only contain a leaf space file,
metadata-only uploads can recreate missing ancestor folders using names derived
from each path segment. Version 1 files with an explicit `access` policy remain
strict: every parent must already exist or have its own successfully applied
space file. With `--skip-space-create`, missing spaces and their dependent
content are reported as skipped while unrelated existing content continues.
When `access` is present it replaces the complete direct policy: inheritance,
the all-project-members role, direct human users, and direct groups. Empty
arrays explicitly remove grants. Effective inherited access and expanded group
members are never serialized. Legacy space files without `version` and
`access` remain valid and update metadata without changing access.
If direct access contains a principal without a portable identity, download
still writes the space as a legacy metadata-only file and warns that access was
omitted. Re-uploading that file leaves access unchanged; it never applies a
partial policy that could revoke the unsupported grant. Descendant spaces are
still downloaded independently when their own access is portable.
Full project downloads and uploads include spaces by default. Use
`--skip-spaces` to leave them alone or `--spaces-only` to operate only on space
definitions. `--root-spaces` is a download-only legacy layout option and cannot
be combined with `--nested`. Filtered chart and dashboard operations never
reconcile unrelated space access. They still maintain metadata-only files for
the referenced spaces so names remain portable; an existing versioned `access`
block is preserved rather than replaced by embedded metadata.
If the access-aware space endpoint is unavailable, a normal full download
continues with the legacy metadata-only spaces embedded in chart and dashboard
responses. `--spaces-only` remains strict because it has no content response to
use as a fallback.
Changing a space name or access policy requires actual permission to manage the
resolved space. Broad content-as-code permission does not bypass restricted
space authorization, and an unchanged legacy metadata-only file is treated as
a no-op after confirming the caller can view the space.
Keep organization deployments in this order:
1. `lightdash upload --organization`
2. Project spaces
3. Project content
## Custom roles reference implementation
Custom roles are the reference implementation for organization-scoped content
as code.
The portable document is defined by `CustomRoleAsCode` in
`packages/common/src/types/roles.ts`:
```yaml
version: 1
name: Developer view only
description: Can view production content
level: project
scopes:
- view:Dashboard
- view:SavedChart
```
The endpoints are:
- `GET /api/v2/orgs/{orgUuid}/code/roles`
- `POST /api/v2/orgs/{orgUuid}/code/roles`
The backend implementation lives in
`packages/backend/src/services/RolesService/RolesService.ts`. It:
- exports user-defined roles and sorts their scopes;
- rejects unknown fields, unsupported versions, invalid names, duplicate or
unknown scopes, and invalid scope/level combinations;
- identifies a role by its exact name within the organization, matching the
database uniqueness constraint;
- creates a missing role;
- compares description and scopes for an existing role;
- treats the role level as immutable;
- applies scope additions and removals and supports clearing a description;
- returns `CREATE`, `UPDATE`, or `NO_CHANGES`;
- does not delete roles that are absent from the local directory.
For compatibility, an existing role may retain a legacy scope that is no
longer assignable at its level. Scope-level validation is applied to newly added
scopes, so downloading and uploading an unchanged legacy role remains a no-op.
The CLI implementation lives under
`packages/cli/src/handlers/organizationContent/`. It only:
- reads and writes `lightdash/custom-roles/*.yml`;
- treats a missing or empty custom-roles directory as a no-op;
- creates safe filenames and disambiguates normalized filename collisions;
- rejects duplicate role names within the local bundle;
- calls the two `/code/roles` endpoints;
- continues after a backend rejection and prints the filepath with the error;
- aggregates backend actions into created, updated, unchanged, and failed
counts.
It intentionally does not list roles through the general roles API, validate
scope names, calculate scope diffs, or call the lower-level create and patch
role endpoints.
## Users
Organization users are stored under `lightdash/users/*.yml`:
```yaml
version: 1
email: analyst@example.com
firstName: Alex
lastName: Example
disabled: false
role:
type: system
name: editor
```
The endpoints are:
- `GET /api/v2/orgs/{orgUuid}/code/users`
- `POST /api/v2/orgs/{orgUuid}/code/users`
Email is the portable identity and is normalized to lowercase. An upload
creates a missing organization member or reconciles the existing member's
organization role, disabled state, and supplied names. `firstName` and
`lastName` are independently optional: omitting either preserves that name on
an existing user, so older version 1 files remain valid. Supplied names are
trimmed; an explicit empty string clears that name. Downloads include both
names, including empty strings when a name is unset. A custom role is
referenced by its exact organization-level role name, so organization uploads
process custom roles before users.
Credentials are not portable, so authentication status is not written to user
files. A missing user is staged without an authentication method and reported
as awaiting authentication. Uploads never add or remove credentials. Omitting a
user file does not remove the remote user.
Invitations are a separate side effect and are not sent by default. Passing
`lightdash upload --organization --send-invites` sends invitations only to
eligible staged users. Authenticated users, disabled users, and users with a
valid invitation are skipped. Without that flag, users authenticate through
the instance's existing domain, SSO, or manually triggered invitation flows.
Uploads also preserve the organization admin invariant. Enabled admin
promotions are processed before admin demotions or disables, and the backend
rejects any individual operation that would leave the organization without an
enabled authenticated admin.
## Groups
Organization groups are stored under `lightdash/groups/*.yml`:
```yaml
version: 1
name: Finance
members:
- alice@example.com
- bob@example.com
```
The endpoints are:
- `GET /api/v2/orgs/{orgUuid}/code/groups`
- `POST /api/v2/orgs/{orgUuid}/code/groups`
The exact, case-sensitive group name is the portable identity. Upload creates a
missing group or replaces the complete membership of an existing group. Every
member email must resolve to the primary email of a non-internal user in the
destination organization before any mutation begins. Empty membership is
represented explicitly as `members: []`.
Group files intentionally exclude UUIDs, project roles, space access, AI-agent
access, user attributes, and ownership metadata. Changing the name creates a
new group and leaves the original group intact; missing files do not delete
groups.
Organization uploads preflight theme packages, then run remote phases
sequentially: custom roles, users, groups, user attributes, and themes. A failed document phase
prevents later phases from starting, so group emails are resolved only after
the complete users phase has succeeded and theme imports begin only after the
user attributes phase succeeds.
SCIM and content as code should not manage the same group. Groups do not yet
record management provenance, so this limitation cannot be enforced by the
first version.
## User attributes as code
`lightdash download --organization` writes one YAML file per attribute under
`lightdash/user_attributes/`. Use `--path` to select a separate content root for
each organization, as with users and groups.
```yaml
version: 1
name: team_id
description: Team access
attributeDefaults: null
users:
- email: jane@example.com
values: [a, b]
groups:
- name: team-a-viewers
values: [a]
```
The endpoints are `GET` and `POST /api/v2/orgs/{orgUuid}/code/userAttributes`.
Both require permission to manage the target organization. References use
organization member emails (case-insensitive) and exact group names, not UUIDs.
Unknown or ambiguous references and duplicate assignments are rejected before
that attribute is changed. Values support multiple strings.
`lightdash upload --organization` applies attributes after users and groups.
Each file replaces the attribute's description, defaults, and complete user and
group assignment lists. Use `users: []`, `groups: []`, and
`attributeDefaults: null` to clear them. Reapplying the same document reports
`unchanged`. Missing files do not delete remote attributes, and changing a name
creates a new attribute. Uploads without this folder leave attributes unchanged.
This does not lock UI editing or configure project-level group roles.
## Adding another content-as-code resource
Use this sequence when adding a new resource:
1. Define a versioned portable type, parser, resource identity, and typed API
responses in the appropriate `packages/common/src/types/contentAsCode/`
module.
2. Add methods to the appropriate scope controller without changing an existing
route family or operation ID.
3. Implement a backend resource handler for externalization, runtime validation,
reference conversion, identity resolution, comparison, and persistence.
4. Return explicit create, update, and no-change outcomes from the backend.
5. Register a `CodeResourceDefinition` with its folder, accepted extensions,
identity, parser, filename profile, and dependency metadata. Connect the
definition to the project or organization orchestration for its API calls and
permission checks. Use a specialized hook only when the resource is not a
normal YAML document.
6. Keep organization resources under `lightdash/<resource>/` and project
resources within the existing project content layout.
7. Add a schema coverage contract under
`packages/backend/src/contentAsCode/fieldCoverage/`. Name the test after the
resource key so the registry test can verify that every registered resource
is covered.
8. Add parser, filename, adapter, backend-domain, and round-trip tests, including
partial failures and dependency skipping where applicable.
## Evolving an existing resource
Portable contracts are intentionally curated and do not automatically mirror
database models. Adding a field to a chart, dashboard, or another domain model
does not automatically expose it in YAML. Each new field must be classified as
portable, read-only metadata, or internal state.
For a portable field, update the document type, externalization, create/update
mapping, comparison logic, generated schema, and round-trip tests together. If
the new meaning is incompatible with existing files, increment the resource
version and add an explicit normalizer or migration path. Never spread an
entire persistence model into a portable document: UUIDs, ownership, secrets,
and instance-specific state must remain excluded.
### Schema coverage guardrail
Every registered single-document resource has a field coverage contract in
`packages/backend/src/contentAsCode/fieldCoverage/`. These unit tests compare
the top-level fields in the generated OpenAPI schema for the domain model with
the fields in its portable content-as-code document. A model field cannot be
silently added without making one of these decisions:
- Add the field to the portable document and its adapters.
- Add the field to `skippedModelFields` when excluding it is intentional.
- Add a document-only field to `documentOnlyFields` when it has no direct model
equivalent, such as `version` or `contentType`.
The lists are exact contracts rather than permanent allowlists. A test also
fails when an entry becomes stale because the model and document now share the
field. `registry.test.ts` ensures that adding a resource to
`CONTENT_AS_CODE_VERSIONS` also requires adding a matching coverage test.
When a model changes, regenerate the OpenAPI schema before running a focused
contract test. CI does this through `build`, but focused local test runs do not:
```bash
pnpm generate-api:backend
pnpm -F backend exec vitest run \
src/contentAsCode/fieldCoverage/chart.test.ts \
--config vitest.config.ts
```
An unclassified chart field fails with an actionable error:
```text
[content-as-code:chart] SavedChartDAO has new fields not covered by ChartAsCode:
- newField
Add them to ChartAsCode and its adapters, or intentionally add them to
skippedModelFields in chart.test.ts.
```
Run every coverage contract with:
```bash
pnpm -F backend exec vitest run \
src/contentAsCode/fieldCoverage \
--config vitest.config.ts
```
This is a schema coverage contract, not a replacement for behavior tests. It
catches unclassified top-level fields; resource adapters and round-trip tests
must still prove that portable fields download, compare, and upload correctly.
### Dashboard control sections
When `dashboard-controls` is enabled, `config.controlSections` downloads each
section with a readable `slug`. Filters, parameter controls and date zoom
controls refer to it with `sectionSlug`. A section written without a `slug`
takes the slug of its name. Files never hold section or control ids.
Section slugs come from the name on every download and are never saved, like
tab slugs; repeated names are all suffixed `-1`, `-2` in list order. A slug in
an uploaded file only resolves references inside that file. An upload keeps a
saved section's internal UUID only when no other section shares its slug base
(the name) on the saved dashboard or in the file; a renamed or repeated section
gets a new UUID v4. Invalid section slugs are rejected with the allowed
characters, duplicate section slugs with the positions and names of the
sections that share them, and unknown `sectionSlug` references with the valid
slugs listed (or a hint to add `config.controlSections` when the file has
none).
With the flag off, downloads keep saved sections without their IDs, write
`sectionSlug` references and omit parameter controls. Uploads resolve section
slugs with either flag state, without enabling parameter controls or replacing
their saved values while the flag is off. With the flag off, uploads match a
saved section by its saved slug, else the slug of its name, never giving two
sections the same saved section, and save each section's slug.
### Dashboard parameter controls
When dashboard controls are enabled, dashboard files can save controls under
`config.parameterControls`. Each control has a unique lowercase `slug`, a
`label`, `value`, `valueType` (`type` and `multiple`), and a `parameters` map.
Downloads derive the slug from the label, else the first parameter key, and
never save it; repeats are all suffixed `-1`, `-2`. An upload keeps a saved
control's internal ID only when no other control shares its slug base on the
saved dashboard or in the file; a relabelled or repeated control gets a new ID. Duplicate slugs are rejected.
Each parameter rule is `all`, `{ only: [tile-slug] }`, or
`{ except: [tile-slug] }`. A key has at most one default (`all` or `except`),
and explicit assignments cannot overlap another assignment. `only: []` assigns
no tiles. Hidden controls still apply their assignments and values. A present
empty list means no saved controls and an empty parameter mirror; an absent list
uses legacy top-level parameters. With a saved list, upload derives that mirror
from control values.
### Dashboard control order
`config.controlOrder` is the order of filters, parameter controls and sections
on the bar, as `{ kind, slug }` refs (`kind` is `filter`,
`parameter` or `section`) to controls in the same file. With
`dashboard-controls` enabled, a dashboard with a saved order downloads its full
reconciled order and a `slug` on every dimension and metric filter; a
never-reordered dashboard downloads without either. Filter slugs come from the
field id on every download, suffixed `-1`, `-2` when several filters share a
field, and are never saved.
The bar follows `controlOrder` entry by entry, whatever the order of
`filters.dimensions`, `filters.metrics`, `parameterControls` or
`controlSections`; a section's members follow their entries inside the
section. An entry with `newLine: true` starts a new bar line, and
every other entry shares the line of the entry before it. A line start on a
section member passes to the next entry in the list outside a section; it is
dropped if there is none. Without a saved
order, every section starts its own line. Controls left out of
`controlOrder` go to the end of their container, inline; to place a new
control, add its entry. With dashboard controls enabled, AI agent reads match
downloads but always include filter slugs, so an agent can write a list. An
upload, from the CLI or an AI agent, stores a non-empty list as given.
Slugs in an uploaded file only resolve `controlOrder` refs inside that file.
An upload keeps a dimension filter's ID only when it is the only filter on its
table and field on the saved dashboard and in the file; filters sharing a field
get new IDs, never each other's. Unknown, duplicate and `dateZoom`
refs are rejected with the valid slugs listed, and refs with the wrong shape
fail validation; parameter refs need
`config.parameterControls` in the file. When dashboard controls are on, a file
without `controlOrder`, or with an empty one, resets the bar to the default
order. With the flag off,
downloads omit filter slugs and `controlOrder`, and uploads ignore them,
keeping the saved order. Scheduled delivery filters carry no slugs.
Charts, dashboards, SQL Runner charts, spaces, and data apps reserve slugs
within their project, including slugs owned by soft-deleted rows. The database
uniqueness constraints are authoritative. Normal creation derives a slug from
the name and appends `-1`, `-2`, and so on after exact indexed candidate
probes. Content-as-code and promotion may request an exact slug: an available
slug is preserved, an active matching resource is updated by the upsert flow,
and a deleted-resource conflict is rejected rather than silently reused.
The same slug may exist in different projects. Slugs remain portable,
human-readable project-scoped selectors, while UUIDs are the canonical internal
identity for relationships and unscoped references. Lookup code must always
include the project scope or reject ambiguity; it must never select an arbitrary
match.
If implementing the CLI requires importing backend domain maps, reproducing
validation rules, fetching current resources to calculate a diff, or calling
several persistence endpoints, the responsibility is probably in the wrong
layer and should move behind the resource's `/code` endpoint.