Skip to content

Refuse a malformed spec instead of tracebacking (#200) - #267

Draft
EiffL wants to merge 1 commit into
mainfrom
fix/spec-load-error
Draft

EiffL wants to merge 1 commit into
mainfrom
fix/spec-load-error

Conversation

@EiffL

@EiffL EiffL commented Nov 1, 2026

Copy link
Copy Markdown
Member

Fixes #200. Paired with astra-project/astra-tools#122. Draft until that is released: this branch imports astra.helpers.SpecLoadError, so it needs the astra-tools pin bumped to that release (a last commit here). Until then CI fails on the import.

Summary

The ScannerError in #200 was one case of a wider problem. On main, any of these tracebacks out of lc status and lc materialize --check:

  • an astra.yaml or universe that is empty, not UTF-8, or a top-level list
  • any wrong-typed field that reaches ASTRA's semantic checks, e.g. outputs: 5 or decisions: [a]

lc's gate now asks ASTRA the way astra validate does:

  • Schema first, semantic only once it passes. The semantic checks assume the structure holds, and they crashed when both stages ran unconditionally. This also adds the universe schema check, which lc never ran, so a universe with decisions: [a] or an unknown key used to pass the gate.
  • A file that is not a spec reaches lc as one exception, astra's new SpecLoadError, which names the file (and the line and column for YAML). plan._validate refuses with it. lc run's read_spec refuses with it too, because the probe reads the inputs that file declares.

Nothing broader is caught: an exception from astra on a file that does load is astra's bug, and reporting it as "your spec does not validate" would blame the user for it.

Error: …/astra.yaml does not validate, so there is nothing to materialize from it:
  mapping values are not allowed here
  in "…/astra.yaml", line 8, column 22

This is an alternative to #252 and #258, which both catch yaml.YAMLError in _validate and fix the case in the issue but not the others above. The CLI test here is adapted from #258's.

Tests

  • test_plan.py: each kind of file that won't load, refused with the file's name; a structural error in the analysis or a universe, refused before the semantic stage.
  • test_run.py: read_spec on a malformed spec.
  • test_cli.py: lc status on the spec from the issue gives exit 1 and no traceback.
  • The 8 new tests fail on main.
  • A fuzz of plan.build, with a wrong-typed value in every field of a spec, universe and sub-analysis, crashes in 192 of 448 cases on main and in none on this branch.
  • Full suite against the astra branch: 1163 passed, 4 failed. The 4 are the rerun tests, which resolve the engine's pinned astra-tools 0.2.18 from PyPI and so cannot import SpecLoadError until the pin moves.

CLAUDE.md: the "Known violation, unfixed" paragraph becomes the invariant, and the gate's paragraph records the two-stage order and why it is load-bearing.

🤖 Generated with Claude Code

A spec that did not parse escaped `lc status` and `lc materialize
--check` as a raw ScannerError, and it was one case of several: an empty
or non-UTF-8 file, a top-level list, and any wrong-typed field that
reached ASTRA's semantic checks (`outputs: 5`, `decisions: [a]`) all
tracebacked the same way.

The gate now asks ASTRA the way `astra validate` does:

- Schema first, for the analysis and every universe, and the semantic
  checks only once that passes. They assume the structure holds and
  crashed on it. The universe schema was not checked at all before, so
  a universe with `decisions: [a]` or an unknown key passed the gate.
- A file that is not a spec at all reaches lc as astra's one
  SpecLoadError, naming the file (and the line and column for YAML).
  `_validate` refuses with it, and so does `lc run`'s read_spec, whose
  probe reads the inputs the file declares.

Nothing broader is caught: an exception from astra on a file that does
load is astra's bug, not the user's spec.

Requires the astra-tools release with SpecLoadError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A malformed astra.yaml tracebacks out of lc status and lc materialize --check

1 participant