Skip to content

test(acquire): validate mirror history end to end - #248

Merged
toddysm merged 3 commits into
mainfrom
feature/mirror-history/e2e-163-25e59c6f
Oct 27, 2026
Merged

toddysm merged 3 commits into
mainfrom
feature/mirror-history/e2e-163-25e59c6f

Conversation

@toddysm

@toddysm toddysm commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #163. Completes the remaining acceptance work for #157, whose close-out stays separate.

  • Add a repeatable, same-repository-PR/manual validation workflow calling the existing mirror and scan/promote/delete workflows, without bypassing the HIGH vulnerability gate.
  • Build a tiny scratch-image upstream fixture entirely inside this run’s two disposable ghcr.io/toddysm/{quarantine,golden}/mirror-history-e2e-<run-id>-<attempt>/image packages. Normal image packages are not modified.
  • Check the initial copy, promotion and deletion, an actual history skip, forced copying with exactly one appended entry, and the rendered Acquisition card through real package/issue/dashboard HTTP servers.
  • Require package-read/deletion credentials before publishing; reject unsafe fixture coordinates; always attempt guarded cleanup, including partial setup failures, and require verified package absence.
  • Retain credential-free JSON snapshots and rendered fixture cards as run artifacts for 14 days. Document normal execution and interrupted-run cleanup.
  • Expose mirror workflow copied/skipped/digest outputs and an optional unique graph-event artifact name; existing callers retain their default artifact name and behavior.

Validation and review

Testing mode: hosted, normal GitHub Actions and Copilot review authorized.

Tested head: f6f2c5f5f0436a7413f7093809c9e298a4cc639f.

Local validation on Python 3.12/macOS arm64:

  • python -m pytest -q tests/mirror_history: 20 passed, including fixture guards, lifecycle failure cases, cleanup authorization/verification, real HTTP service startup/termination, credential-log redaction, and assertion-disable rejection.
  • Existing per-project tests: cssc_common 10 passed, packages-service 8 passed, dashboard-web 25 passed, issues-service 12 passed (75 local tests total).
  • MYPYPATH=apps/python-app/libs/cssc_common mypy --python-executable .venv/bin/python --follow-imports=silent tests/mirror_history/mirror_history_e2e.py: clean.
  • python -m compileall -q tests/mirror_history: clean. Editor-specific diagnostics are not exposed in this session; CLI syntax/type diagnostics were used instead.
  • actionlint 1.7.12: both changed workflows clean.
  • shellcheck -S warning 0.11.0: all 18 new workflow shell blocks clean.
  • 20 local documentation links checked; git diff --check clean.
  • Staged-data scan and manual review found no provider tokens, private keys, literal credentials, or credential-bearing URLs. Authentication remains environment-based.

Local graph-library/graph-service dependency restoration was blocked by the configured package mirror’s missing LadybugDB wheel; no dependency constraints were changed. The hosted checks job runs all libraries and services, including those graph suites, before any live registry mutation.

Self-review: safety/diagnostic findings addressed before publishing; no remaining findings. Acceptance requires every lifecycle job, cleanup, CodeQL, and settled review to pass; local or partial workflow success is not sufficient.

Final current-head evidence

Validated head: 364e225446e831788bec30d34dff749469f28a6d.

Complete hosted acceptance: https://github.com/toddysm/cssc-framework/actions/runs/40562725339 — success, including verified deletion of both disposable packages.

  • 182 hosted tests passed: 36 lifecycle regressions, 14 shared-library tests, 65 graph-library tests, 25 dashboard tests, 22 graph-service tests, 12 issue-service tests, and 8 package-service tests.
  • Actual initial mirror, vulnerability-gated promotion/deletion, history skip, forced copy, live service HTTP/rendered dashboard checks, cleanup and final acceptance all passed. The override-notification job was intentionally skipped because approval overrides are disabled for the clean fixture.
  • Persisted JSON/HTML artifacts were downloaded and independently verified: initial = 1 entry / 4 quarantine versions; promoted and skipped = 1 unchanged entry / only the original history version; forced = 2 entries with the original entry preserved and exactly one forced append.
  • Both scratch-fixture packages were deleted and independently verified absent by cleanup.
  • CodeQL passed. Copilot completed current-head review with no open findings; all four initial review threads have fix-SHA replies and are resolved. No administrative merge or protection changes.
  • 95 affected local tests and type diagnostics passed at the exact head; all three changed workflow schemas and 18 shell blocks are clean at warning severity.

The live test exposed and fixed two real producer defects: the registry reader now follows GHCR blob CDN redirects without forwarding registry authorization or exposing signed queries, and normal promotion excludes fallback-referrer index tags (matching the SBOM path) so they cannot race image-owned referrer cleanup. Initial historical results above are superseded by this current-head evidence.

Exercise the existing mirror and promotion workflows with guarded disposable registry fixtures, actual service HTTP calls, rendered dashboard assertions, and verified cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 18:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T18:25:55.849307Z f6f2c5f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f6f2c5f5f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/mirror_history/mirror_history_e2e.py
Comment thread tests/mirror_history/mirror_history_e2e.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cleanup can leave fixtures behind after a package failure or falsely report success against the wrong fixture attempt on a partial rerun.

2 open issues
What changed in this PR

Adds end-to-end validation of mirror history across the existing acquisition and promotion workflows, covering the lifecycle acceptance work in #163.

Changes:

  • Adds a hosted workflow using disposable registry fixtures to verify copying, promotion, deletion, history skips, forced copying, and dashboard rendering.
  • Exposes mirror workflow outputs and configurable graph-event artifact names while preserving existing defaults.
  • Adds validation tests and documentation for execution, evidence, and cleanup.
File Description
tests/​mirror_history/​test_mirror_history_e2e.py Tests lifecycle assertions, fixture safety, cleanup, and HTTP service handling.
tests/​mirror_history/​mirror_history_e2e.py Implements live validation, evidence capture, and guarded cleanup.
docs/​reference/​mirror-history.md Links to the end-to-end validation guide.
docs/​guides/​acquire/​validating-mirror-history.md Documents execution, acceptance criteria, artifacts, and recovery.
docs/​guides/​acquire/​README.md Adds the validation guide to the acquisition index.
docs/​contributing/​workflow-naming.md Defines the validation workflow naming category.
.github/​workflows/​validate-mirror-history.yml Orchestrates checks, fixture lifecycle validation, and cleanup.
.github/​workflows/​_mirror-image.yml Exposes copy, skip, and digest outputs plus configurable artifact names.

Review effort: Balanced

Comment thread .github/workflows/validate-mirror-history.yml Outdated
Comment thread tests/mirror_history/mirror_history_e2e.py Outdated
Follow GHCR blob redirects without forwarding registry authorization, redact signed download diagnostics, check untagged referrer removal, retry bounded visibility failures, and preserve fixture identity and independent cleanup on reruns.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The credentialed GHCR publication, promotion, and deletion lifecycle needs confirmed hosted acceptance and cleanup results plus final human review.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Match the existing SBOM promotion path: only image tags enter the scan matrix. This prevents a concurrent fallback-index promotion from deleting discovery metadata before image-owned referrer cleanup, which the live version-inventory assertion exposed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged registry writes and deletes across reusable workflows require human confirmation of the complete hosted acceptance run and verified cleanup.

No open issues 🎉

Review effort: Balanced

@toddysm
toddysm merged commit 28c7e13 into main Oct 27, 2026
19 checks passed
@toddysm
toddysm deleted the feature/mirror-history/e2e-163-25e59c6f branch October 27, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mirror-history: end-to-end validation (mirror -> promote -> delete -> re-run)

2 participants