README.md
Rama main · réplica de lectura del repositorio original.
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="./ui/public/brand/deltallm-delta-lockup-on-dark.svg">
<source media="(prefers-color-scheme: light)" srcset="./ui/public/brand/deltallm-delta-lockup-on-light.svg">
<img alt="DeltaLLM" src="./ui/public/brand/deltallm-delta-lockup-on-light.svg" width="236">
</picture>
</p>
[](https://github.com/deltawi/deltallm/actions/workflows/ci.yml)
[](https://docs.deltallm.io/)
[](https://github.com/deltawi/deltallm/releases)
[](https://www.python.org/downloads/)
[](LICENSE)
[](https://github.com/deltawi/deltallm/commits/main)
[](https://github.com/deltawi/deltallm/stargazers)
[](https://railway.com/deploy/deltallm?utm_medium=integration&utm_source=template&utm_campaign=deltallm)
**Run an LLM service for your teams or customers.**
Connect model providers, create plans with model access, pricing, and rate limits,
and assign them to organizations. Apps use an OpenAI-compatible API. You manage
access and usage through the Admin UI.
Self-hosted. MIT licensed.
[SSO](docs/features/authentication.md#single-sign-on) and
[audit logs](docs/features/audit-log.md) included.
⭐ If DeltaLLM is useful to you, [star the repository](https://github.com/deltawi/deltallm)
so other teams can find it.
## Customer Plans
Define a Growth plan with two models, set their prices and limits, and assign it
to five customer organizations. When the plan changes, review and activate a new
version without configuring each customer separately.
[Service tiers](docs/admin-ui/tiers.md) manage these plans. Assign organizations to
follow the active version so they receive plan updates together.
## Admin UI
Manage organizations, access, and usage in the browser.
## One Endpoint For Your Apps
```python
from openai import OpenAI
# Before: call OpenAI directly.
client = OpenAI(api_key="sk-...")
# After: send the same OpenAI-compatible requests through DeltaLLM.
client = OpenAI(
base_url="http://localhost:4002/v1",
api_key="sk-deltallm-key",
)
```
Your application keeps its OpenAI request format. DeltaLLM handles provider credentials, routing, policy checks, caching, failover, and spend tracking.
## What It Handles
- **Unified API** - Use one OpenAI-compatible endpoint across OpenAI, Anthropic, Azure OpenAI, Bedrock, Gemini, Groq, and other providers.
- **Scoped API keys** - Issue virtual keys with model allowlists, rate limits, budgets, owners, and expiry.
- **Customer plans** - Assign versioned model packages to organizations through service tiers, with model access, rate limits, and pricing managed together.
- **Routing and failover** - Route by strategy, retry failed deployments, and separate provider credentials from application code.
- **Batch API** - Run embeddings and non-streaming chat completions asynchronously through OpenAI-compatible files and batches, even when upstream providers are synchronous.
- **MCP gateway** - Register external MCP servers and expose approved tools through controlled gateway flows.
- **Guardrails** - Detect PII and prompt injection before provider calls.
- **Spend and usage** - Attribute cost by key, team, organization, model, and provider.
- **Admin UI** - Manage models, credentials, route groups, teams, users, usage, audit logs, and settings in the browser.
- **Operations** - Export Prometheus metrics, request logs, audit events, cache behavior, and health checks.
## Performance
Latest complete run: 11 October 2026. All four rates passed.
| Rate | Duration | Successful / scheduled | p95 / p99 | Result |
| --- | ---: | ---: | ---: | --- |
| 50 RPS | 600 s | 30,000 / 30,000 | 22.86 / 26.15 ms | Passed |
| 100 RPS | 600 s | 60,000 / 60,000 | 23.53 / 28.09 ms | Passed |
| 200 RPS | 600 s | 120,000 / 120,000 | 28.16 / 41.67 ms | Passed |
| 500 RPS | 600 s | 300,000 / 300,000 | 95.15 / 179.39 ms | Passed |
Synthetic one-token provider on a twelve-CPU, 12 GiB VM. See the
[RPS results](https://docs.deltallm.io/en/latest/project/issue-320-rps-report/)
for setup and limits.
## Quick Start With Docker
Use Docker Compose for the shortest local evaluation path. It starts DeltaLLM, PostgreSQL, and Redis.
```bash
git clone https://github.com/deltawi/deltallm.git
cd deltallm
cp config.example.yaml config.yaml
```
For a first local request, enable one-time model bootstrap in `config.yaml`:
```yaml
general_settings:
model_deployment_source: db_only
model_deployment_bootstrap_from_config: true
```
Generate the required secrets:
```bash
python3 -c 'import secrets; print("DELTALLM_MASTER_KEY=sk-" + secrets.token_hex(20) + "A1")'
python3 -c 'import secrets; print("DELTALLM_SALT_KEY=" + secrets.token_hex(32))'
```
Create `.env` with the generated values and a provider key:
```env
DELTALLM_MASTER_KEY=sk-your-generated-master-key
DELTALLM_SALT_KEY=your-generated-salt-key
OPENAI_API_KEY=sk-your-openai-key
PLATFORM_BOOTSTRAP_ADMIN_EMAIL=admin@example.com
PLATFORM_BOOTSTRAP_ADMIN_PASSWORD=ChangeMe123!
```
Start the stack:
```bash
docker compose --profile single up -d --build
```
Wait for the health endpoint to return `{"status":"ok"}`, then send a request.
In this terminal, export the same master key you put in `.env`: Compose reads that
file for its containers, but it does not export variables into your shell.
```bash
curl http://localhost:4002/health/liveliness
export DELTALLM_MASTER_KEY='paste-your-generated-master-key'
curl http://localhost:4002/v1/chat/completions \
-H "Authorization: Bearer $DELTALLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Hello from DeltaLLM"}
]
}'
```
Open the Admin UI at `http://localhost:4002`. Got your first answer back? A
[GitHub star](https://github.com/deltawi/deltallm) helps other teams find DeltaLLM.
Full Docker setup, environment variables, the multi-instance evaluation profile, and Dockerfile
usage are covered in the [Docker guide](docs/getting-started/docker.md). Compose is not the
production high-availability reference.
## Deployment Paths
| Path | Use it for | Start here |
| --- | --- | --- |
| Railway | Managed evaluation with hosted PostgreSQL and Redis | [Railway deployment](docs/deployment/railway.md) |
| Docker Compose | Local evaluation, demos, small teams, simple self-hosting | [Docker guide](docs/getting-started/docker.md) |
| Kubernetes | Multi-instance production, ingress, autoscaling, and managed infrastructure | [Kubernetes guide](docs/deployment/kubernetes.md) |
| Local development | Backend, UI, and contributor workflow from the repository | [Installation guide](docs/getting-started/installation.md) |
Moving from LiteLLM? The import tool converts a LiteLLM `config.yaml` and lists each setting
that it could not import. See [Move from LiteLLM](docs/guides/migrate-from-litellm.md).
## Documentation
- [Getting started](docs/getting-started/index.md)
- [Gateway usage examples](docs/getting-started/quickstart.md)
- [Configuration reference](docs/configuration/index.md)
- [Model deployments](docs/configuration/models.md)
- [Features](docs/features/index.md)
- [Admin UI guide](docs/admin-ui/index.md)
- [API reference](docs/api/index.md)
- [Deployment guide](docs/deployment/index.md)
- [Security model](docs/security/index.md)
## Contributing
- [Report issues](https://github.com/deltawi/deltallm/issues)
- [Request features](https://github.com/deltawi/deltallm/discussions)
- [Report a vulnerability privately](SECURITY.md)
- PRs are welcome. Start with the [local installation guide](docs/getting-started/installation.md).
- Documentation changes follow the [documentation contribution guide](CONTRIBUTING_DOCS.md).
If DeltaLLM solves a problem for you, consider starring the repository. Share your
use case in [Discussions](https://github.com/deltawi/deltallm/discussions) so we can
improve the workflows that matter to you.
## License
See [LICENSE](LICENSE).