Skip to content

Deployment smoke test

This smoke test validates a running service deployment through the public HTTP API. It uses the self-contained examples/smoke_human_workflow.yaml spec and covers health, readiness, auth behavior, spec validation, run creation, human resume, audit listing, credential inventory, and environment policy updates.

Compose stack

Prepare a production env file:

cp deploy/production.env.example deploy/production.env

deploy/production.env is intentionally ignored by source control. Replace all placeholder tokens and passwords before binding a deployment to a real network. For local CI/smoke runs, the documented placeholder values are used only to exercise the auth path.

Start the API and PostgreSQL:

docker compose --env-file deploy/production.env up --build -d

Run the smoke test against the unversioned routes:

python scripts/deployment_smoke.py \
  --base-url http://127.0.0.1:8000 \
  --api-key replace-admin-token \
  --operator-api-key replace-operator-token \
  --reviewer-api-key replace-reviewer-token \
  --viewer-api-key replace-viewer-token \
  --expect-auth

Run the same smoke test against the versioned routes:

python scripts/deployment_smoke.py \
  --base-url http://127.0.0.1:8000 \
  --api-prefix /v1 \
  --api-key replace-admin-token \
  --operator-api-key replace-operator-token \
  --reviewer-api-key replace-reviewer-token \
  --viewer-api-key replace-viewer-token \
  --expect-auth

Shut the stack down when finished:

docker compose --env-file deploy/production.env down

PostgreSQL coverage

The compose stack uses PostgreSQL by default through KNEO_SERV_DATABASE_URL=postgresql://...@db:5432/..., so the smoke path above also validates PostgreSQL-backed run state, checkpoints, continuations, queue records, locks, audit events, and project metadata.

For a separately managed PostgreSQL database, start the API with KNEO_SERV_DATABASE_URL set and run the same scripts/deployment_smoke.py commands against that service URL.

Staging and remote smoke

Prepare a staging env file from the example:

cp deploy/staging.env.example deploy/staging.env

Replace every placeholder token, database password, provider key, and telemetry endpoint before use. For compose-based staging rehearsals, set KNEO_SERV_ENV_FILE so the API container reads the staging file:

python scripts/validate_staging_env.py deploy/staging.env

The validator fails if required staging settings are missing, scoped API roles are incomplete, payload telemetry capture is enabled, provider secret checks are disabled, or placeholder values remain.

For repeatable staging gates, render the local file from secret-backed environment variables instead of editing it by hand:

export KNEO_STAGING_API_KEYS="operator:<operator-token>:operator;reviewer:<reviewer-token>:reviewer;viewer:<viewer-token>:viewer"
export KNEO_STAGING_ADMIN_API_KEY=<admin-token>
export KNEO_STAGING_SPEC_SIGNING_KEY=<signing-key>
export KNEO_STAGING_DATABASE_URL=<postgresql-dsn>
export KNEO_STAGING_POSTGRES_PASSWORD=<compose-db-password>
export KNEO_STAGING_OTEL_EXPORTER_OTLP_ENDPOINT=<otel-endpoint>
export KNEO_STAGING_OPENAI_API_KEY=<openai-key>
export KNEO_STAGING_MCP_API_KEY=<mcp-key>
python scripts/render_staging_env.py --output deploy/staging.env

KNEO_SERV_ENV_FILE=./deploy/staging.env \
  docker compose --env-file deploy/staging.env up --build -d

For a remote staging deployment, run the smoke script against the public staging URL with scoped keys:

export KNEO_STAGING_BASE_URL=https://staging.example.com
export KNEO_STAGING_OPERATOR_TOKEN=<operator-token>
export KNEO_STAGING_REVIEWER_TOKEN=<reviewer-token>
export KNEO_STAGING_VIEWER_TOKEN=<viewer-token>
python scripts/deployment_smoke.py --api-prefix /v1 --expect-auth

The operator key validates specs, creates runs, reads audit and credential inventory, and updates policy state. The reviewer key resumes the human task. The viewer key is optional; when supplied, the smoke verifies that policy writes are rejected with 403.

The full staging release gate validates deploy/staging.env, derives the scoped smoke tokens from KNEO_SERV_API_KEYS, and runs the /v1 remote smoke:

python scripts/staging_release_gate.py \
  --env-file deploy/staging.env \
  --base-url https://staging.example.com

The GitHub Actions Staging Gate workflow runs the same renderer and release gate from the staging environment secrets. Dispatch it with the deployed staging URL after the service is reachable.

See also