[EiffL]

Summary

• AGENTS.md is now the reference for coding agents, in about 240 lines:
  • commands, architecture, project rules, conventions and testing patterns;
  • Invariants and traps, grouped by area (CLI and convergence, storage, identity and classification, execution, sandbox, containers, compute, crate and viewer): the rules that are not visible from the code at the point where they would be broken;
  • Decided against: alternatives that were considered or built and then rejected, each with its reason.
• CLAUDE.md is reduced to @AGENTS.md, so Claude Code loads the same content.
• The previous 2,983-line CLAUDE.md is gone, along with its layer table, "rebuild in progress" framing and the history of how each decision was reached. The invariants and rejected alternatives that still apply are carried over without that framing.
• The four remaining mentions of rebuild layers in docstrings and comments are removed (engine/__init__.py, tests/conftest.py, tests/test_sandbox_wrap.py, tests/test_worker.py).

Two statements differ from the old file because the code disagreed with it: the front end is built by the custom hatch_build.py hook, and the eval workflow runs on dispatch and on every non-draft PR.

Testing

Text-only changes. ruff check src/ tests/ passes and tests/test_sandbox_wrap.py passes locally; the full suite was not run locally.

The identifiers named in the new sections were checked against src/. Behavioural claims marked measured are carried over from the old file, not re-measured.

🤖 Generated with Claude Code

[github-actions]

✅ Eval

Metric	Value
Outputs check	success
Agent run	success
Turns	8
Tool calls	6
Cost	$0.12
Agent wall time	0m39s
Model	claude-sonnet-5-5

lc status

  mode:    direct
  sandbox: landlock (fs: declared, network: allowed)
  crate:   up to date with the outputs

  · current  baseline/best_fit        4300526
  · current  baseline/hubble_diagram  4300526
  · current  baseline/residuals       4300526

3 current

Confusion & pain points (Claude analysis)

Confusion & pain points

• The run was essentially clean. There were no errored tool calls and no failed commands. The agent read the spec, wrote the scripts, filled in the spec, validated it, launched compute and materialized, all in 6 tool calls. The friction below is minor.

• The agent pinned a different astra version than the spec declares. It ran uvx astra-tools@0.2.18 validate against an astra.yaml that declares version: "0.0.14". The two numbers are different things (the spec format version and the tool release), but nothing in the trace shows where 0.2.18 came from. The agent seems to have guessed it, or taken it from the skill. If the harness expects a specific astra-tools release, the skill or the project scaffold should state it. Otherwise astra validate on PATH would avoid the uvx pin.

• The agent put the license in pyproject.toml with a blind sed. sed -i 's/^version = "0.0.1"/&\nlicense = "CC-BY-4.0"/' turns on RO-Crate maintenance, but only if the version line matches exactly. The agent never printed the file afterwards to check, and the trace doesn't show whether the edit took. The ro-crate-metadata.json check passed, which suggests it worked. A path-free way to declare the license (a scaffold prompt or documented lc setting) would remove the need for this hack.

• Only the baseline universe was executed. The l_bfgs_b, stat_only and none options were never exercised, though the agent's summary says so openly. A bug in those argparse branches, for example a choices mismatch with the spec's decision values, would not show up until later. This is a gap in how complete the verification was. It isn't a tooling fault.

• The agent made the whole setup a single shell command, which hides intermediate failures. The mutation of astra.yaml uses a Python string replace that fails silently when the anchor text doesn't match. The commit and lc compute launch ran in the same command, and the output was piped through tail. Validation passed and all three outputs were made, so the replacements did match. However, a spec-editing step that can silently match nothing is fragile, and a CLI helper for setting a recipe or inputs would make it safer.

Full trace: agent-trace artifact on this run.

[EiffL]

lgtm