Skip to content

fix: report unavailable Codex App probes as skipped - #1335

Merged
EtanHey merged 3 commits into
mainfrom
fix/harness-probe-app-skip-20261016
Oct 16, 2026
Merged

EtanHey merged 3 commits into
mainfrom
fix/harness-probe-app-skip-20261016

Conversation

@EtanHey

@EtanHey EtanHey commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Summary

When the Codex App is stopped and its existing control socket is absent, the probe records SKIP with the unavailable-context reason. A running app with no socket remains a failure, and a skipped harness still keeps the aggregate convergence gate non-green.

Verification

  • RED: the new adapter test failed because the process check did not exist; the wrapper test failed because it attempted Bridge verification after the native skip event.
  • GREEN: python3 -m pytest tests/test_harness_probe*.py -q — 64 passed.
  • ruff check, ruff format --check, and git diff --check on the changed files — passed.
  • Live App row — SKIP, reason “Codex App is not running; existing app context is unavailable”; probe process exit 0, aggregate command exit 1. Read-only check confirmed no Codex process and no control socket; no app was launched.
  • Local CodeRabbit found two issues; the UID-scoped process check and JSON event guard were fixed and verified.

Loop-record: implementer=codex/gpt-6-luna reviewer=none rounds=0/0 defects=1 role=worker
Reviewer route: Per the handoff, orc routes 2× Sol + Cursor. This worker did not invoke remote reviewers. BrainLayer AGENTS.md exclusions for Greptile/Bugbot are preserved.

Retirement policy follow-up

  • Orc lead review approved the two narrow purpose admissions in scripts/retirement_policy.json: 6512dc7e2f2ea2c69d7ba067ca0db43314949c134d94416f8542da99e5f3d2b1 covers the current-user Codex process-table subprocess.run check; d96c889adfe7312245aab3a1e9a2d5b4ada8a073764625f362237514ecdbea73 covers local subprocess.SubprocessError classification. No source changed.
  • Applied with scripts/retirement_policy_generate.py --write --purposes and the lead-approved two-key purpose map. python3 scripts/retirement_policy_generate.py --check passed.
  • Verification: python3 -m pytest tests/test_retirement_ast_portability.py -q (5 passed, 12 subtests passed); python3 -m pytest tests/test_harness_probe*.py -q (64 passed); git diff --check passed. The policy-only diff adds 20 lines and removes none.
  • Commit: a6063215ed8abb6141059c8a5d3f20b89b11b99d.

Main refresh

  • Merged origin/main at 462b97000cd90f0d148b6c548a30567944689a63 with merge commit 8116e0b4925c59f3ebb4d517d617c349cd13137b (parents: a6063215ed8abb6141059c8a5d3f20b89b11b99d, then 462b97000cd90f0d148b6c548a30567944689a63).
  • The sole conflict was tests/test_harness_probe_runtime.py; resolution keeps the App skip and running-without-socket failure tests and the parameterized Codex CLI/Claude originator and environment-preservation test. scripts/harness_probe.py auto-merged with the codex-cli-only CODEX_INTERNAL_ORIGINATOR_OVERRIDE removal; MCP endpoint/authorization filtering remains unchanged.
  • Regenerated with the documented policy generator and the already-approved purpose map; python3 scripts/retirement_policy_generate.py --check passed. There was no additional policy diff beyond the two approved entries from the preceding commit.
  • Verification: python3 -m pytest tests/test_harness_probe*.py -q (66 passed); python3 -m pytest tests/test_retirement_ast_portability.py -q (5 passed, 12 subtests passed); git diff --check passed.

— brainlayerCodex (worker) · codex/gpt-6-luna

Co-Authored-By: brainlayerCodex running gpt-6-luna <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: df79cda4-2043-48ff-8821-690617c8412f)

@EtanHey EtanHey added the size:M Tight-loop PR size: 151-400 hand-written lines added label Oct 16, 2026
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: a24e5fe6-c529-42d0-99da-a1b4a3e02c67
📥 Commits

Reviewing files that changed from the base of the PR and between 6339608 and 95c45ab.

📒 Files selected for processing: 4
  • scripts/harness_probe.py
  • scripts/harness_probe_native.py
  • tests/test_harness_probe_native.py
  • tests/test_harness_probe_runtime.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 9
  • GitHub Check: BrainBar renders no enrichment status
  • GitHub Check: global model retirement (installed + native)
  • GitHub Check: swift (macos-15)
  • GitHub Check: test (3.12)
  • GitHub Check: test (3.11)
  • GitHub Check: test (3.13)
  • GitHub Check: installed wheel imports (python 3.13)
  • GitHub Check: crash sweeps
  • GitHub Check: Analyze (swift)
🧰 Additional context used
🪛 ast-grep (0.45.3)
tests/test_harness_probe_runtime.py

[info] 134-139: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
{
"type": "harness.skip",
"reason": "Codex App is not running; existing app context is unavailable",
}
)
Note: [CWE-116] Improper Encoding or Escaping of Output.

(use-jsonify)

scripts/harness_probe_native.py

[error] 23-28: Command coming from incoming request
Context: subprocess.run(
["pgrep", "-x", "-U", str(os.getuid()), "Codex"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=5,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(subprocess-from-request)

[info] 217-222: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
{
"type": "harness.skip",
"reason": "Codex App is not running; existing app context is unavailable",
}
)
Note: [CWE-116] Improper Encoding or Escaping of Output.

(use-jsonify)

🔇 Additional comments: 4
scripts/harness_probe_native.py: 2

20-35: LGTM!

The static-analysis hint about command injection does not apply here. The pgrep argv is a fixed list. The only dynamic value is os.getuid(), and no request data reaches the command.

215-225: LGTM!

tests/test_harness_probe_native.py: 1

115-159: LGTM!

tests/test_harness_probe_runtime.py: 1

117-188: LGTM!

📝 Walkthrough

The native probe checks whether Codex is running when its control socket is missing. If it is not running, the probe emits a harness.skip event. The harness probe records successful skip events as SKIP results and marks all probe legs skipped.

Changes: Codex App probe skip handling

Layer / File(s) Summary
Check Codex state and emit skip event
scripts/harness_probe_native.py, tests/test_harness_probe_native.py
The native probe checks for a Codex process when the control socket is missing. It emits a skip event if no process is running. Tests cover process-check statuses and missing-socket outcomes.
Record skip event in harness results
scripts/harness_probe.py, tests/test_harness_probe_runtime.py
The harness probe records successful skip events as SKIP, preserves the event reason, and marks all probe legs skipped. Tests also confirm that a missing socket remains a failure when the app process exits with code 1.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant CodexApp
  participant NativeProbe
  participant HarnessProbe
  NativeProbe->>CodexApp: Check whether app process is running
  CodexApp-->>NativeProbe: Return process status
  NativeProbe-->>HarnessProbe: Emit harness.skip event
  HarnessProbe->>HarnessProbe: Record SKIP and mark probe legs SKIPPED
Loading
Merge Risk: ⚪ Minimal · up to 95c45

The absent-app skip and running-app failure paths remain distinct. No actionable issue remains before merge.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: reporting unavailable Codex App probes as skipped.
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 2
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the socket door
No Codex waits behind it there
A skip event hops into the stream
The probe marks each leg with care
Then bounds away through fields of green

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

BrainLayer ratchet

Every Value below was measured by this run. A row this machine cannot measure says n/a — <reason> instead of a number; baselines in Notes name their own machine, method and date and were not measured here.

Row Status Value (measured by this run) Method Notes
commit provenance 🟢 GREEN measured 8116e0b4925c == PR head · checkout 8116e0b4925c commit graph + live PR head · in-process · runner Which commit this whole table is about. On a pull_request event the checkout is GitHub's synthetic merge ref, whose sha is not on the PR — #759's table printed 13fa724278bf while that PR's head was 4632f979 — so this row names the PR-head parent instead, the sha a reviewer can actually see. The comparison sha is read live from repos/{owner}/{repo}/pulls/{n} when the table is collected, not taken from the event payload, because the payload cannot know the run has been overtaken. Residual window, stated rather than papered over: a push landing between that read and the comment being posted is not caught here — the run for that push refreshes the table.
baseline attestation 🟢 GREEN baseline f421d1a7c5e6 unchanged from base a6063215ed8a by git · the attestation is for main 462b97000cd9, not this checkout's base a6063215ed8a main attestation artifact via Actions API · in-process · runner What every comparison is measured AGAINST, and who says so. The baseline fields of tests/fixtures/sprint_gate/corpus.json (queries, latency_baseline_ms, thresholds) are compared to the ratchet-attestation artifact of the latest successful push or (no-input) workflow_dispatch run of ratchet-attest.yml on main, fetched through the Actions API — a PR run cannot write to another run's artifacts. A field that differs is RED unless that main run measured the new value. The calibrated socket collector can license p50/p95; every absent measured path stays locked, so missing collection never passes as permission for a hand edit. Boundary: the comparator is this PR's checkout of ci_ratchet_table.py, diff-reviewable, not tamper-proof.
provenance 🟢 GREEN stamped 8116e0b4925c == HEAD, tree clean wheel stamp · in-process · runner Sha half of #749 keg-mode provenance: a keg built from this wheel can answer __build_sha__. The helper-age and served-process predicates need a running BrainBar and are measured only by scripts/sprint_gate.py on an installed Mac. The sha here is the checkout's — the explicit PR head in ratchet.yml — because that is what publish.yml stamps at release time; the PR-head sha this table describes is the one in commit provenance above.
fallback replay debt ⚪ n/a n/a — no fallback queue on this machine: the pending memories live in ~/Gits/*/docs.local/decisions, and docs.local/ is gitignored, so a runner checkout has no copy of them to count docs.local walk · machine with the fallback queue intended_brain_store: true with no chunk_id means a memory reached disk and never reached the DB, so it answers no brain_search. Budget: 0. Any pending or unparseable file is a finding, never a band -- 122 of these sat from 2026-06-28 to 2026-09-05 because nothing counted them where a reader would look. Measured by walking the tree, so it is only ever measured on a machine that HAS the tree.
pre-push: delete-only push runs no suite 🟢 GREEN 0 suite starts · exit 0 real checkout hook · delete-only stdin · sentinel suite runner Bug SHA: 44ecbf5ba6f1ada3b965797559eee90c94b45595 (RED replay, 2026-10-06). Fix: #1084; measured at 8116e0b4925c59f3ebb4d517d617c349cd13137b. Ceiling: 0 suite starts; hook must exit 0 and announce the delete-only skip.
mapped bytes ⚪ n/a n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would socket · installed Mac Baseline 26.2 GB — installed Mac, socket, 2026-09-03, after R2 drained 15,070 → 0. Up from 16.8 GB because the drain left more vectors mapped under the same cap: the change is the drain, not a leak. Not measured by this run.
search p50/p95 ⚪ n/a n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would socket · installed Mac Margin p50: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin p95: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Calibrated on MacBook-Pro.local at 2026-09-01T08:42:22Z under active_sprint_load (tests/fixtures/sprint_gate/corpus.json). Not measured by this run.
idle CPU ⚪ n/a n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would ps sampling · installed Mac Ceiling: average CPU < 30% over a 60 s window (resource_budget in scripts/sprint_gate.py), ratified and kept as a hard budget. Margin daemon: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin helper: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin watcher: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Needs the BrainBar daemon, helper and watcher actually running. Not measured by this run.
signature_valid ⚪ n/a n/a — the macOS signature-parity job is trigger-gated and did not run on this PR: it touches no release or signing path (pyproject.toml, scripts/release-*, scripts/brainlayer-version-check.sh, publish.yml, ratchet.yml) and carries no ratchet:signatures label — a GitHub macOS runner bills at ~10× Linux minutes and rebuilds the keg venv from source codesign · installed keg scripts/release-verify-signatures.sh <keg> codesign-verifies every *.so/*.dylib under libexec/venv. The macOS parity job installs the published tap formula (etanhey/layers/brainlayer), so this row measures the release path — formula, published sdist and Homebrew's relocation — and not this PR's tree. Release-time baseline for the same keg on a different machine: 442 valid / 0 invalid — installed Mac (M4 Max), brew --prefix brainlayer 1.5.11, 2026-09-03.
BrainBar renders no enrichment status 🟢 GREEN 19 actual-view captures; active pending-store/replay retained macOS DEBUG binary · AppKit actual views · Vision OCR + icon RGBA comparison · private synthetic state Missing render capability/evidence is RED. Historical data is retained; this is source-build proof, not installed-app proof.
Watcher heartbeat freshness and recovery ⚪ n/a n/a — macOS native probe trigger-gated: no affected path or ratchet:heartbeat label private Python producer -> compiled Swift consumer Preserves genuine silence; independent agent stores cannot revive the watcher. Not a load or installed-UI measurement.
maintenance quiesce: services stay down under a reviving watchdog ⚪ n/a n/a — macOS replay trigger-gated: no affected path or ratchet:quiesce label real launchd · GitHub macOS runner · real maintenance + fleet-watchdog Fix: #1076 (bc6d690f4331387485678d0247e19e14e6ef8d97). Ceiling: 0 revivals; supervisor held, both resumed, no hold marker.
no cloud model call reachable anywhere 🟢 GREEN 492 source files; 428 installed imports; 3 native tests; wheel ea651e3efce3; 0 candidate attempts immutable installed wheel (default + dev) + AST source + actual native BrainBar Synthetic private fixtures; SDK absence gated on merged R10b. Exact-head candidate proof, not deployment.

🟢 GREEN measured, within budget · 🔴 RED measured, out of budget — a finding to clear before merge · ⚪ n/a not measurable on this machine, never guessed.

No RED rows.

Measured on Linux/x86_64 · measured 8116e0b4925c · PR head 8116e0b4925c · checkout 8116e0b4925c · run · updated 2026-10-16 06:24:05 UTC

Co-Authored-By: brainlayerCodex running gpt-6-luna <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 9244c155-72e3-4eff-9bb5-375db5a6d4f5)

…-skip-20261016

# Conflicts:
#	tests/test_harness_probe_runtime.py
@cursor

cursor Bot commented Oct 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 3b026668-4de3-4574-9a73-3e3d0c39a956)

@EtanHey
EtanHey merged commit 38aeafb into main Oct 16, 2026
26 checks passed
@EtanHey
EtanHey deleted the fix/harness-probe-app-skip-20261016 branch October 16, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M Tight-loop PR size: 151-400 hand-written lines added

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant