docs-for-dev/MODULES.md

Rama main · réplica de lectura del repositorio original.

# Modules and installation

[Handbook](README.md) · [Related guide](TESTING.md)

DeepTutor has one source repository and disjoint Python distributions. `deeptutor`
owns the shared turn runtime, CLI and Python SDK. Feature distributions add code
into the same package namespace without overwriting core files. `deeptutor-cli`
is a compatibility metapackage: it depends on core and owns no Python files or
console scripts. `deeptutor[full]` is an installation profile, not a permanent
metapackage that would prevent removing individual applications.

The modular distribution family starts with 1.7.0 and must be published together
at each release. The 1.6.15 and earlier packages use the previous layout and do
not have these boundaries. Local wheel builds are also supported.

## Source checkout

```sh
python -m pip install -r requirements/full.txt
python -m pip install -e . --no-deps
(cd web && npm ci --legacy-peer-deps)
deeptutor module list
deeptutor module disable solve
deeptutor module enable solve
```

In a checkout all source modules are present. `enable` and `disable` exercise the
same assembly rules as wheel installs. `install` and `uninstall` deliberately
require a wheel environment, so they never delete tracked source files. Source
requirements are generated by `python scripts/sync_module_requirements.py`.
Use `--check` in CI to detect drift. `requirements/core.txt` and `server.txt`
are minimal; `full.txt` supplies the ordinary complete source/Docker runtime.
Source archives and Docker images are recognized by the project file and module
build script, without requiring `.git`. Optional source dependencies can be
added with `python scripts/install_extras.py <extra>`; this resolves module
requirements without installing duplicate feature wheels over the source tree.

## Optional document parsers

For Docling, LiteParse, MarkItDown, or PyMuPDF4LLM, replace `<engine>` below
with `docling`, `liteparse`, `markitdown`, or `pymupdf4llm`. The Document Parsing
settings page does not install packages into a running server.

- **Docker:** add `parse-<engine>` to `DEEPTUTOR_EXTRAS` in the existing
  deployment's environment configuration, preserving any other declared extras.
  Recreate the same service with its existing configuration and mounts (for
  Compose, use the same project and Compose files with `docker compose up -d`).
  The entrypoint reapplies these extras on each startup. Installation failures
  are nonfatal: inspect the service's startup logs before rechecking availability.
  Data mounts preserve user data, not Python packages in the container.
- **Source:** stop DeepTutor, then run
  `python scripts/install_extras.py parse-<engine>` from the checkout using the
  server's Python environment. Restart the same instance.
- **Wheels:** stop DeepTutor, then run
  `deeptutor module install parser-<engine>` in the server's Python environment.
  Restart with the same runtime home. Do not use this command over a source tree.

Return to Document Parsing settings and choose **Recheck availability**. This
only refreshes engine availability; it does not install packages. Docling's
remote-server mode remains usable without the local package. Model downloads
are a separate action after the local package becomes available.

## Independent wheel installation

```sh
python scripts/build_modules.py --output dist/modules
python -m venv /tmp/deeptutor-modules
/tmp/deeptutor-modules/bin/python -m pip install --find-links dist/modules deeptutor
/tmp/deeptutor-modules/bin/deeptutor module install solve --wheelhouse dist/modules
/tmp/deeptutor-modules/bin/deeptutor module install renderer-manim --wheelhouse dist/modules
/tmp/deeptutor-modules/bin/deeptutor module disable solve
/tmp/deeptutor-modules/bin/deeptutor module enable solve
/tmp/deeptutor-modules/bin/deeptutor module uninstall solve --dry-run
/tmp/deeptutor-modules/bin/deeptutor module uninstall solve
```

Use the equivalent `Scripts/python.exe` and `Scripts/deeptutor.exe` on Windows.
Run commands with the same `DEEPTUTOR_HOME` as the application. Python packages
are installed per environment; enabled state lives in
`<home>/data/system/modules.json` and applies to that deployment, not one learner.
`--wheelhouse` supplements the package index with locally built wheels.

Install `server` for the API, then `web` for packaged Web assets. The Web wheel
requires assets produced by `scripts/prepare_web_package.py` for `deeptutor start`;
a source development server uses the checkout's `web/` directory. Build and
release automation prepares Web assets before building the wheel family.

## Upgrade from 1.6.15

Stop the old process and back up its runtime home. Build the candidate wheels
with packaged Web assets first. In a disposable copy of the old environment,
upgrade with `python -m pip install --upgrade --find-links dist/modules
"deeptutor[full]==<candidate-version>"`. If the legacy `deeptutor-cli` is
installed, first uninstall it with `python -m pip uninstall deeptutor-cli`,
then run the upgrade command, optionally adding the new compatibility alias.
The legacy CLI owns the same Python files as the new core: pip installs its
new core dependency before removing the old CLI, which deletes the new files.
Removing the old CLI first avoids this collision and retains runtime-home data.
For CLI-only migration, install `deeptutor-cli==<candidate-version>` after that
uninstall. Use the exact candidate version to avoid resolving an
older published release. Keep `DEEPTUTOR_HOME` unchanged, run `pip check`,
inspect `deeptutor module list`, and exercise chat, persisted sessions and Web
startup before upgrading the real environment. Never test against the only copy
of user data. The bare core upgrade does not retain the old full-app profile.

`deeptutor/__version__.py` is the version source for all builds, including the
source CLI compatibility package. Extras are templates in
`tool.deeptutor.optional-dependencies`; both setuptools and the module builder
resolve their internal pins to that version. A release bump must not leave
dependencies pointing at the previous module family.

## Ownership and assembly

`deeptutor/modules/manifests/*.json` is the authoritative catalog. Each manifest
owns paths, dependency requirements, turn capabilities, loop extensions, tool
names, API routers and authorization policies, CLI groups, navigation paths and
background service callbacks. Longest matching path wins. Shared protocols and
lightweight context metadata stay in core; feature implementations stay in their
own distribution. The builder rejects overlapping wheel files.

| Module family | Contents |
| --- | --- |
| core | Durable turns, chat, ask-questions, shared tools, CLI/SDK, settings and sessions |
| server / web | API and authentication adapters / packaged Next.js assets |
| solve / questions / research | Planning loop / question generation / research pipeline |
| visualize / renderer-manim | HTML/SVG/chart visualizers / optional Manim pipeline |
| video-lesson / audio-overview | Browser lessons / audio overview generation |
| knowledge / parsing | Retrieval and ingestion / document parsing implementations |
| reading / mastery / books | Reading and watching / guided learning / book application |
| memory / notebook / co-writer / calendar | Optional application services and their routes |
| partners / providers | Companion runtime and channel SDKs / optional model-provider SDKs |
| parser-* | Additional parser dependency distributions, requiring parsing |

Dependencies are explicit and acyclic. For example, reading currently requires
knowledge, parsing, mastery and notebook because its application uses those
stores. Books also requires questions, visualize and mastery for its standard
chapter blocks and selected-source grounding. Optional integrations are listed
separately: Calendar works without Mastery, Research and Questions work without
Knowledge, and Parsing never imports knowledge storage. Selecting an unavailable
integration returns an installation hint. Knowledge Frontier requires both
Research and Knowledge; a tool contribution can declare additional requirements
without forcing those modules onto every use of its owning application.
Disabling a dependency is blocked until its enabled dependents are
disabled. Uninstalling a dependency is blocked while any installed dependent
remains, including disabled dependents. Enabling/installing resolves dependencies
in order. Core cannot be disabled or removed.

The loader does not import absent or disabled capability/loop implementations.
Request schemas are lazy. Loop extensions are instantiated once per turn and
activation is re-evaluated as context changes; instances are never shared across
turns. Managed plugin revocation is still checked. Background start/stop callbacks
remain paired for partial-startup rollback. Existing API authentication policies
remain on the contributed routers; module state is not a permission grant.

`GET /api/modules` requires authentication and exposes installed, enabled and
ready separately. The Runtime settings page displays these states and offline
commands. Sidebar and page access use navigation ownership; API route assembly
also excludes absent modules, including feature-specific paths in shared routers.
Readiness checks package requirements, matching core/module versions and external
executables. Configured provider credentials and service connectivity remain in
the existing settings readiness system.

## Lifecycle guarantees

Official modules share one interpreter. Running application containers hold a
shared environment lease; module changes require an exclusive lease, across
processes. A second lease protects deployment selection when different Python
environments share the same home. Stop CLI turns and API workers before changing packages or selection.
Direct external `pip` commands cannot be prevented by this advisory lock.

Install first resolves dependencies, rejects replacement of an existing package
version, and builds the exact resolved artifacts before modifying the environment.
If installation fails after adding packages, those newly added distributions are
removed, including when saving selection fails. Malformed selection is rejected
before any package mutation. Selection is atomically replaced only after package operations succeed.
Use a new environment or an explicit environment upgrade for dependency conflicts.
This is not an atomic filesystem transaction across a machine crash; standard pip
repair/reinstallation remains the recovery path for an interrupted machine.

Uninstall removes only the selected distribution. Shared dependencies are retained
because other applications may use them. User settings, sessions, notebooks,
knowledge bases, generated artifacts and credentials are never purged. Reinstall
uses the retained data. There is no implicit data-deletion command.

Parser package installation from a running settings/setup session now returns an
offline module command. It no longer upgrades packages beneath active turns.
Model-weight downloads remain background jobs, because they do not alter Python
code. Third-party managed plugins retain their existing isolated environments.

## Third-party worker extensions

Managed plugin workers use their own environments and lifecycle, separately
from official module wheels. See [plugin streaming and cancellation](PLUGINS.md#worker-protocol-and-cancellation)
and the [runnable example](examples/learning_echo/README.md).

## Validation

```sh
python scripts/sync_module_requirements.py --check
python -m pytest tests/modules tests/test_packaging_metadata.py
python scripts/build_modules.py --output dist/modules
python scripts/check_module_install.py --wheelhouse dist/modules --extended
```

The install smoke creates a disposable virtual environment, uses a local mock
model (no provider credentials), checks core chat and durable replay, installs
Solve, runs it, uninstalls it, verifies persisted files, checks core again, then
reinstalls Solve. Extended mode installs Parsing, Research and Calendar without
Knowledge or Mastery, exercises parsing and its cache plus independent research
and calendar paths, then removes each module while verifying Solve and retained
data. The source isolation tests also prohibit undeclared imports in a fresh
subprocess for each of the 25 modules and assemble their contributions.
Manim additionally needs system rendering libraries, FFmpeg and
LaTeX. A render and cancellation test should accompany changes to its renderer.
The Web application remains one frontend distribution; independently replaceable
third-party frontend bundles are outside this Python module contract.

## Implementation and verification

- [deeptutor/modules/manifests/](../deeptutor/modules/manifests/)
- [scripts/build_modules.py](../scripts/build_modules.py)
- [scripts/check_module_install.py](../scripts/check_module_install.py)