MET-WP11-THOTH-API-01-C - Thoth API runtime and migration operations evidence

Status: DRAFT / PROPOSED. Evidence collection: NOT STARTED. No action authorization.
Programme: Thoth Metrics. Parent: #981 (MET-WP11-THOTH-API-01, CRITICAL). Programme ledger: #766. Owning issue: this issue. Owner/approver: CTO.
Proposed risk: CRITICAL (it selects the production migration/deployment model that E0 and E then execute).
Stage: C of the approved #981 decomposition (A -> A2 -> B, C in parallel, D -> E0 -> E).

Why this issue exists now

The approved #981 specification (original body, Amendment 1 6083041997 §C, Amendment 2 6097640294 §14, CTO approval 6126801903) requires a separate bounded C gate. It also says C may run in parallel with B/D but must be accepted before E0. Until now no owning issue existed for C. Without it, the CG-13 gaps recorded in #981 stay unmapped: production compute/deployment path, database migration execution path, deployment approval mechanism, and rollback/restore procedure. B is still CHANGES REQUIRED with H2 and H3 pending (#988, #989), so collecting C evidence now keeps it off the critical path.

This issue is a control ledger only. Publishing it was authorized by the CTO on 2026-10-29 in the project control thread, for exactly this one issue. That grant authorizes no other mutation, provider access or operation.

Required outcome

One frozen, independently reviewed and CTO-approved operations model for the inactive v1.9.0 Thoth API. It must state:

real deployment mechanism (test/non-production and production)
authorized non-agent deployment actor
safe migration execution model: Option A or Option B (below)
fleet/start serialization required by the selected option
backup/snapshot prerequisite and restore capability
forward-repair procedure
candidate image identity verification (exact GHCR digest)
service health/smoke route and its startup window
observation evidence route
rollback / stop conditions

Migration execution options (from Amendment 1, CR-1)

• Option A - automatic migration on thoth init. Permitted only with a proven single migration runner / serialized startup contract. No concurrent-replica safety assumption may be invented.
• Option B - explicit operator-run thoth migrate under separate migration authorization while candidate rollout is held. Normal rollout starts only after migration verification.

Operational timing evidence (Amendment 2 §14)

selected model and real rolling-deployment behaviour
concurrent-start prevention/proof if Option A
duration of the twenty v1.9.0 migration train
interaction with service health/startup window
lock-timeout / failure posture
backup/restore evidence where required

Carried B findings to close here or in E0

• B-OPS-01 (MET-WP11-THOTH-API-01 - Thoth Metrics API integration and inactive-release readiness #981 6239896043): PostgreSQL >= 15, single runner, production-shaped upgrade and revert-via-image, forward repair/restore.
• MIG-B-03: production-shaped v1.8 upgrade, old-binary read/write/delete smoke, new-binary startup, previous-binary rollback without database down, bounded failure/retry. The rehearsal belongs to E0, but C must define how it will be run.
• Migration rollback doctrine is forward-only (MIG-B-06: 12 unguarded IF EXISTS down files). C must not assume down-migration rollback in production.

Evidence route and safety rules

• No AI agent reads production secrets, uses deployment credentials, reads secret-bearing provider state or starts a deployment.
• Facts come from a human operator (CTO or delegated) as sanitized statements or redacted artefacts. Control records them here verbatim and attributes them.
• Read-only repository evidence (workflows, Dockerfile, thoth init/migrate code paths, release workflow) may be gathered by agents under separate read authorization.
• Any provider read, even read-only, needs its own explicit grant naming the provider and scope.
• If the facts cannot be established through an approved sanitized route: HOLD.

Information needed from the operator

1. Where does the Thoth API run today in test/staging and in production (platform, service shape, replica count)?
2. How does a new image reach each environment today (manual, CI-triggered, platform auto-deploy from GHCR tag)? Who performs it?
3. How do migrations run today: on container start via thoth init, by an operator command, or otherwise? Can more than one replica start at once?
4. Database: PostgreSQL major version in production and test; managed backup/snapshot capability and the restore procedure; who can run it.
5. Health/smoke route used after deploys, and the platform's startup/health timeout.
6. How the running image digest can be verified after deploy without secrets.
7. Where post-deploy observation evidence (logs, error rates) can be read, and by whom.

Non-goals

No deployment, migration execution, release, provider write, credential use, Sphinx/dashboard work, collection, rebuild or activation. This does not change B, D, E0 or E gates, and OPS-03/OPS-04 (#799) remain not direct prerequisites (Amendment 1 §16).

Current action authorization

Next gate

Operator supplies the sanitized facts above -> control drafts the C operations model here -> fresh independent CRITICAL specification review (cross-family) -> separate CTO approval -> C accepted -> E0 may be specified once D's release artefact exists.

Current gate: C DRAFT / EVIDENCE NOT STARTED. #984 DRAFT / B CHANGES REQUIRED. E0 and E NOT AUTHORIZED.