Runtime management source notes

Overview

This system transitions cc-lb from static TOML configuration files to a dynamic, database-backed runtime management model. You no longer define upstreams, principals, or plugin chains in cc-lb.toml. Instead, all runtime configurations are stored in the database and managed dynamically. This change allows you to add, update, or delete resources on the fly without restarting the proxy. You can perform these operations through the web dashboard or directly via the admin v1 REST API.

Architecture

The runtime architecture relies on a lock-free DynamicView snapshot managed via an ArcSwap primitive. When configuration changes occur in the database, the server builds a new immutable DynamicView snapshot and swaps it atomically. This design ensures that active request-handling threads never block on configuration updates.

In multi-replica deployments, replicas synchronize their local views using Postgres LISTEN/NOTIFY channels for sub-second updates. If a notification is missed, a background reconciler polls the database every 60 seconds as a fallback. The system uses a single rebind primitive to safely commit staged plugin slots and rebuild the routing table.

Admin v1 REST API

Administrative operations use the providers configured in admin.auth.providers. A static-token provider accepts its environment-backed Bearer token in the Authorization header.

The default admin listener is [::1]:9090; use http://[::1]:9090 for the examples below. Proxy traffic uses a separate listener on port 8080 (http://localhost:8080/v1/messages).

Local proxy key files

If an operator keeps a local proxy client key file such as a local file, treat it as a client-side cache of the plaintext key returned once by POST /admin/v1/principals/{id}/keys. Rotating a managed key is a two-step lifecycle: issue the replacement key, update the local key file atomically with mode 0600, verify the proxy request path with the new key, then revoke the old key through the admin API. The server never rewrites operator key files automatically.

Upstreams API

Method	Path	Auth	Request Body	Response Body	Error Codes
POST	/admin/v1/upstreams	Bearer	UpstreamCreateBody	UpstreamResponse	conflict, invalid_location
GET	/admin/v1/upstreams	Bearer	None	List of UpstreamResponse	None
GET	/admin/v1/upstreams/{id}	Bearer	None	UpstreamResponse	upstream_not_found
PUT	/admin/v1/upstreams/{id}	Bearer	UpstreamUpdateBody	UpstreamResponse	stale_revision, upstream_not_found
DELETE	/admin/v1/upstreams/{id}	Bearer	None	None	referenced_by, stale_revision
POST	/admin/v1/upstreams/{id}/enable	Bearer	None	UpstreamResponse	stale_revision
POST	/admin/v1/upstreams/{id}/disable	Bearer	None	UpstreamResponse	stale_revision

UpstreamCreateBody

{
  "name": "upstream-name",
  "kind": "anthropic_api_key",
  "base_url": "https://api.anthropic.com",
  "api_key_env": "ANTHROPIC_API_KEY"
}

api_key_env names a variable in the server process’s environment. The example uses ANTHROPIC_API_KEY, which must be set before creating the upstream. The API reads its value and encrypts the credential for database storage; the variable name is not stored as a live secret reference. You can alternatively supply api_key_value, but supplying both fields returns conflicting_api_key. Keep literal credentials out of saved commands and source control.

UpstreamResponse

{
  "id": "<UPSTREAM_ID>",
  "name": "upstream-name",
  "kind": "anthropic_api_key",
  "enabled": true,
  "warmup_enabled": true,
  "warmup_dialect_plugin": null,
  "spec_revision": 1,
  "status": {
    "last_apply_error": null,
    "last_apply_at_unix_secs": null,
    "last_warmup_at_unix_secs": null
  }
}

spec_revision is the user-managed optimistic-lock counter used by If-Match and ETag. Background controller writes (status/lease/secret/token) never bump it. The nested status object holds system-managed operational state: only the apply daemon and warmup observer write here. warmup_enabled defaults to true for anthropic_oauth upstreams and false for other kinds; see docs/upstream-warmup.md.

Principals API

Method	Path	Auth	Request Body	Response Body	Error Codes
POST	/admin/v1/principals	Bearer	CreatePrincipalBody	PrincipalResponse	conflict, unsupported_cache_keepalive_llm_judge
GET	/admin/v1/principals	Bearer	None	List of PrincipalResponse	None
GET	/admin/v1/principals/{id}	Bearer	None	PrincipalResponse	unknown_principal
PUT	/admin/v1/principals/{id}	Bearer	UpdatePrincipalBody	PrincipalResponse	stale_revision, unsupported_cache_keepalive_llm_judge
DELETE	/admin/v1/principals/{id}	Bearer	None	None	stale_revision
POST	/admin/v1/principals/{id}/enable	Bearer	None	PrincipalResponse	stale_revision
POST	/admin/v1/principals/{id}/disable	Bearer	None	PrincipalResponse	stale_revision
PUT	/admin/v1/principals/{id}/allowed_models	Bearer	AllowedModelsBody	PrincipalResponse	stale_revision

CreatePrincipalBody

{
  "name": "principal-name",
  "kind": "machine",
  "allowed_models": ["claude-sonnet-4-5-20250929"],
  "default_limits": [],
  "cache_keepalive": {
    "enabled": true,
    "refresh_lead_time_5m_secs": 30,
    "refresh_lead_time_1h_secs": 300,
    "max_refreshes_per_session": 12,
    "max_total_duration_secs": 14400,
    "snapshot_max_bytes": 524288,
    "classifier": {
      "extra_wait_for_user_tools": [],
      "treat_end_turn_as_ambiguous": false
    }
  }
}

cache_keepalive is an optional object. Omit it (or pass null) to disable the prompt-cache keep-alive feature for this principal. When present, enabled is required. Full field semantics, defaults, and provider-gating rules are documented in docs/cache-keepalive-operator.md. cache_keepalive.classifier.llm_judge is reserved for a future release; create/update requests that set it return unsupported_cache_keepalive_llm_judge.

PrincipalResponse

{
  "id": "<PRINCIPAL_ID>",
  "name": "principal-name",
  "kind": "machine",
  "enabled": true,
  "revision": 1,
  "allowed_models": ["claude-sonnet-4-5-20250929"],
  "default_limits": [],
  "cache_keepalive": null
}

The cache_keepalive field mirrors whatever was persisted on the principal, or null when unset. PATCH /admin/v1/principals/{id} with "cache_keepalive": null clears the config.

Each principal receives a built-in subscription-preference Router chain entry at creation, with order 0. Use the plugin-chain endpoints below to remove the entry or change its position among other router plugins.

Plugins API

Method	Path	Auth	Request Body	Response Body	Error Codes
GET	/admin/v1/plugins/registry	Bearer	None	RegistryListResponse	None
GET	/admin/v1/plugins/registry/{id}	Bearer	None	RegistryEntryResponse	unknown_registry_entry
GET	/admin/v1/plugins/registry/{id}/references	Bearer	None	RegistryReferencesResponse	unknown_registry_entry
PATCH	/admin/v1/plugins/registry/{id}	Bearer	PatchRegistryBody	RegistryEntryResponse	stale_revision
DELETE	/admin/v1/plugins/registry/{id}	Bearer + If-Match	None	None	referenced_by, stale_revision
DELETE	/admin/v1/plugins/registry/{id}?cascade=references	Bearer + If-Match + X-Reference-Fingerprint	None	RegistryCascadeDeleteResponse	reference_fingerprint_required, references_changed, stale_revision
GET	/admin/v1/principals/{principal_id}/plugin-chain	Bearer	None	ChainListResponse	None
POST	/admin/v1/principals/{principal_id}/plugin-chain	Bearer	InsertChainBody	PluginChainEntry	conflict
POST	/admin/v1/principals/{principal_id}/plugin-chain/reorder	Bearer	ReorderBody	None	stale_revision
POST	/admin/v1/principals/{principal_id}/plugin-chain/rebalance	Bearer	None	None	None
PUT	/admin/v1/plugin-chain-entries/{id}	Bearer	PluginChainEntryUpdate	PluginChainEntry	stale_revision
DELETE	/admin/v1/plugin-chain-entries/{id}	Bearer	None	None	stale_revision

InsertChainBody

{
  "slot": "router",
  "wasm_registry_id": "<PLUGIN_ID>",
  "config": {},
  "position": null
}

RegistryEntryResponse includes the inspected plugin metadata name, nullable version, SHA-256, supported slots, revision, and a live refcount. The refcount is computed from both principal plugin-chain bindings and upstream warmup dialect plugin bindings.

GET /admin/v1/plugins/registry/{id}/references returns a reference_fingerprint plus the concrete chain and warmup resources that currently reference the plugin. Use that fingerprint when intentionally deleting a referenced plugin with ?cascade=references; stale fingerprints are rejected with 409 references_changed so operators see a fresh impact list before removing bindings.

Status and Export API

Method	Path	Auth	Request Body	Response Body	Error Codes
GET	/admin/v1/status	Bearer	None	StatusResponse	storage_unavailable
GET	/admin/v1/export	Bearer	None	ExportResponse	storage_unavailable