Skip to content

Troubleshooting

A symptom-indexed runbook for the Dashboard. Start with the health triage table, then find your symptom. Every API error also carries a request id — see Using the request id to trace a specific failure.

Health triage (/api/healthz)

GET /api/healthz is always 200 and reports the real state in its body — start here:

{ "live": true, "ready": false,
  "checks": { "store": true, "recovery_mode": true, "oidc": "configured" } }
Body shows Meaning Do
ready: true store reachable and not in recovery healthy — /api/readyz is 200
checks.store: false the state store is unreachable check KNEO_DASH_DB_URL / the DB / the volume mount; readyz is 503 until it answers
checks.recovery_mode: true the instance is in post-restore recovery reconcile it — kneo-dash recover --status then --reconcile (Backup & recovery)
checks.oidc not configured OIDC settings missing/incomplete in oidc mode check the KNEO_DASH_OIDC_* vars (environment)

/api/readyz returns 503 whenever ready is false (store down or recovery) — that's the signal your proxy de-routes on. /api/livez stays 200 regardless (it's liveness only — never wire a probe to restart on a store hiccup).

Symptoms

Can't log in, or logged in with no access

  • Denied right after OIDC login — your identity mapped to no role (default-deny). Fix the KNEO_DASH_OIDC_ROLE_MAP / role claim, or set a bootstrap admin (security hardening · environment).
  • App refuses to start in static mode — static needs KNEO_DASH_DEV_MODE=1 (dev only); production must be KNEO_DASH_AUTH_MODE=oidc.

A 403 on an action you expected to work

The BFF enforces a capability, not a role — your effective Access map doesn't grant it. An admin can adjust it in Settings › Access. Remember saving Access re-authenticates every session — so a just-changed grant applies on your next request.

Suddenly redirected to login (401)

Your session hit its absolute or idle timeout, or an admin/rotation invalidated it (KNEO_DASH_SESSION_TTL_SECONDS / SESSION_IDLE_SECONDS; rotating SESSION_SECRET logs everyone out). Just log back in.

The environment switcher is empty / "using the default profile"

No Connections are configured — add environment → profile mappings in Settings › Connections. With none, the BFF falls back to default_profile.

The live trace won't stream (SSE)

  • A 503 + Retry-After on the stream = the per-operator concurrent-stream cap (KNEO_DASH_SSE_MAX_STREAMS_PER_OPERATOR); close other open traces.
  • Nothing streams behind a proxy = the proxy is buffering SSE; disable response buffering for /api/* (the example nginx.conf does).

"Platform is busy" banner (503/429)

Backpressure from kneo-serv — the SPA backs off and retries with a countdown. Transient; if it persists, the platform (not the Dashboard) is saturated.

"State changed / refresh" (409)

An optimistic-concurrency conflict — someone else changed the run state, or the annotation / saved-filter / setting you edited, since you loaded it. Refresh and re-apply. (Cancelling an already-terminal run is not an error — it's a 200 no-op.)

The live kneo-serv is below the version floor for that facet — it degrades (the filter is dropped, not the page). See the platform compatibility floors.

Using the request id

Every /api/* error envelope includes a request_id, and the BFF sets it on the X-Request-Id response header and its access-log line. To trace one failure:

  1. Copy the request_id from the error (the SPA surfaces it on error cards).
  2. Grep the BFF access log for that id to find the exact request + status.
  3. For a platform error, the envelope preserves the upstream request_id — quote it when correlating with kneo-serv logs / support.

This is the fastest way to turn "it failed" into a specific, correlatable call.

  • Deployment — probes + the post-deploy checklist.
  • Upgrade — if a version bump won't come ready.
  • Backup & recovery — the recovery/reconcile flow.
  • API contract §12 — the full error-code table behind the envelopes.