Skip to content

Deployment smoke test

This smoke test validates a running service deployment through the public HTTP API. It uses the self-contained examples/smoke_human_workflow.yaml spec and covers health, readiness, auth behavior, spec validation, run creation, human resume, audit listing, credential inventory, and environment policy updates.

Compose stack

Prepare a production env file:

cp deploy/production.env.example deploy/production.env

deploy/production.env is intentionally ignored by source control. Replace all placeholder tokens and passwords before binding a deployment to a real network. For local CI/smoke runs, the documented placeholder values are used only to exercise the auth path.

Start the API and PostgreSQL:

docker compose --env-file deploy/production.env up --build -d

Run the smoke test against the unversioned routes:

python scripts/deployment_smoke.py \
  --base-url http://127.0.0.1:8000 \
  --api-key replace-admin-token \
  --operator-api-key replace-operator-token \
  --reviewer-api-key replace-reviewer-token \
  --viewer-api-key replace-viewer-token \
  --expect-auth

Run the same smoke test against the versioned routes:

python scripts/deployment_smoke.py \
  --base-url http://127.0.0.1:8000 \
  --api-prefix /v1 \
  --api-key replace-admin-token \
  --operator-api-key replace-operator-token \
  --reviewer-api-key replace-reviewer-token \
  --viewer-api-key replace-viewer-token \
  --expect-auth

Shut the stack down when finished:

docker compose --env-file deploy/production.env down

PostgreSQL coverage

The compose stack uses PostgreSQL by default through KNEO_SERV_DATABASE_URL=postgresql://...@db:5432/..., so the smoke path above also validates PostgreSQL-backed run state, checkpoints, continuations, queue records, locks, audit events, and project metadata.

For a separately managed PostgreSQL database, start the API with KNEO_SERV_DATABASE_URL set and run the same scripts/deployment_smoke.py commands against that service URL.

Staging and remote smoke

Prepare a staging env file from the example:

cp deploy/staging.env.example deploy/staging.env

Replace every placeholder token, database password, provider key, and telemetry endpoint before use. For compose-based staging rehearsals, set KNEO_SERV_ENV_FILE so the API container reads the staging file:

python scripts/validate_staging_env.py deploy/staging.env

The validator fails if required staging settings are missing, scoped API roles are incomplete, payload telemetry capture is enabled, provider secret checks are disabled, or placeholder values remain.

For repeatable staging gates, render the local file from secret-backed environment variables instead of editing it by hand:

export KNEO_STAGING_API_KEYS="operator:<operator-token>:operator;reviewer:<reviewer-token>:reviewer;viewer:<viewer-token>:viewer"
export KNEO_STAGING_ADMIN_API_KEY=<admin-token>
export KNEO_STAGING_SPEC_SIGNING_KEY=<signing-key>
export KNEO_STAGING_DATABASE_URL=<postgresql-dsn>
export KNEO_STAGING_POSTGRES_PASSWORD=<compose-db-password>
export KNEO_STAGING_OTEL_EXPORTER_OTLP_ENDPOINT=<otel-endpoint>
export KNEO_STAGING_OPENAI_API_KEY=<openai-key>
export KNEO_STAGING_MCP_API_KEY=<mcp-key>
python scripts/render_staging_env.py --output deploy/staging.env

KNEO_SERV_ENV_FILE=./deploy/staging.env \
  docker compose --env-file deploy/staging.env up --build -d

For a remote staging deployment, run the smoke script against the public staging URL with scoped keys:

export KNEO_STAGING_BASE_URL=https://staging.example.com
export KNEO_STAGING_OPERATOR_TOKEN=<operator-token>
export KNEO_STAGING_REVIEWER_TOKEN=<reviewer-token>
export KNEO_STAGING_VIEWER_TOKEN=<viewer-token>
python scripts/deployment_smoke.py --api-prefix /v1 --expect-auth

The operator key validates specs, creates runs, reads audit and credential inventory, and updates policy state. The reviewer key resumes the human task. The viewer key is optional; when supplied, the smoke verifies that policy writes are rejected with 403.

The full staging release gate validates deploy/staging.env, derives the scoped smoke tokens from KNEO_SERV_API_KEYS, and runs the /v1 remote smoke:

python scripts/staging_release_gate.py \
  --env-file deploy/staging.env \
  --base-url https://staging.example.com

The GitHub Actions Staging Gate workflow runs the same renderer and release gate from the staging environment secrets. Dispatch it with the deployed staging URL after the service is reachable.

Self-hosted staging rehearsal (no standing staging)

A release cycle without a standing staging deployment can still run both staging gates against a real deployed instance: deploy the published release-candidate image on any Docker host and point the gates at it. This is the procedure that first closed the gate at v0.9.0rc1 (it had been deferred for four releases for want of a staging environment).

  1. Generate real secret values (no placeholders — the validator rejects them) and export the eight KNEO_STAGING_* variables from the renderer section above. Generate strong random tokens; for a compose-hosted rehearsal, point KNEO_STAGING_DATABASE_URL at the compose service host (@db:5432).
  2. Store the same values as GitHub environment secrets so the hosted workflow renders an env that matches the deployment:
gh api -X PUT repos/<owner>/<repo>/environments/staging --silent
for var in KNEO_STAGING_API_KEYS KNEO_STAGING_ADMIN_API_KEY \
    KNEO_STAGING_SPEC_SIGNING_KEY KNEO_STAGING_DATABASE_URL \
    KNEO_STAGING_POSTGRES_PASSWORD \
    KNEO_STAGING_OTEL_EXPORTER_OTLP_ENDPOINT \
    KNEO_STAGING_OPENAI_API_KEY KNEO_STAGING_MCP_API_KEY; do
  printenv "$var" | gh secret set "$var" --env staging
done
  1. Deploy the rc image itself, pinned by digest (verify the cosign signature first — see security_hardening.md), under the rendered staging env:
python scripts/render_staging_env.py --output deploy/staging.env
docker pull ghcr.io/<owner>/kneo-serv@sha256:<rc-digest>
docker tag  ghcr.io/<owner>/kneo-serv@sha256:<rc-digest> \
            ghcr.io/<owner>/kneo-serv:latest   # local-only alias for compose
KNEO_SERV_ENV_FILE=./deploy/staging.env \
  docker compose --env-file deploy/staging.env up -d --no-build
  1. Run the script gate locally against the instance (scripts/staging_release_gate.py --env-file deploy/staging.env --base-url http://127.0.0.1:8000), then dispatch the hosted Staging Gate workflow with a URL the GitHub runner can reach.

Reachability caveat: GitHub-hosted runners have no IPv6 egress — a direct IPv6 URL fails with [Errno 101] Network is unreachable even when the host is genuinely on the IPv6 internet. If the host has no public IPv4, front the instance with a short-lived tunnel and pass the tunnel URL instead:

cloudflared tunnel --url http://localhost:8000 --no-autoupdate &
# grep the https://<random>.trycloudflare.com URL from its output
gh workflow run staging-gate.yml -f staging_url=https://<random>.trycloudflare.com

Auth stays enforced for the whole exposure window — the gate's --expect-auth checks depend on it.

  1. Tear down afterwards: kill the tunnel, docker compose --env-file deploy/staging.env down -v, delete the rendered deploy/staging.env, and remove the local latest alias. The GitHub environment secrets from step 2 are rehearsal-scoped — re-render them against real keys and a managed DSN if you later stand up permanent staging.

See also