Skip to content

[Metadata] Reference objectives per scheme, computed by a script #375

Closed

Description

@ocots

Agents: read the guide for agents in #310 first. Specification: this description, #306 and its comments.

Sub-issue of #306, part 1 of 5. Independent.

Scope

  • :reference_objective in the metadata: one entry per scheme at the default grid of the problem (value, source = :computed, grid_size, the configurations that converged and their objectives, their agreement, package versions); optionally an analytic value of the continuous problem with its source, shown as information only.
  • Computed entries are generated by scripts/reference_objectives.jl into one file per problem (ext/MetaData/references/<p>.toml), never edited by hand. Reference = best objective among the configurations of TestConfigs.CONFIGS that converged (tolerance 1e-8); a disagreement between configurations (local minima) is written and printed, not hidden.
  • Convergence report ([Tests] Report convergence instead of testing it; keep hard checks for breakage and model equivalence #360): gap of each solve to the entry of its grid and scheme; no entry, no gap.
  • Consistency tests: type of the field; every :valid problem has a reference; comparison rule (a problem valid in one modeling only needs a reference).

Done when

  • Every problem has a computed trapezoid reference; the PR shows the agreement table.
  • A cron run shows the gap column filled.
  • READMEs say how to (re)compute the references.

Refs #306, #310

Activity

  1. ocots commented on Nov 5, 2026

    @ocots
    MemberAuthor

    Taken by an agent; draft PR to follow.

  2. ocots commented on Nov 5, 2026

    @ocots
    MemberAuthor

    Done in #380 (merged).

    What was done

    • Reference objective per scheme at the default grid, for all 22 problems (trapezoidal rule): ext/MetaData/references/<problem>.toml, generated by tools/references/update.jl from the results of the convergence tests (cron artifacts or local test/reports), never edited by hand. Value = best converged objective; the spread between configurations is recorded, agreement = false above 1e-6.
    • OptimalControlProblems.reference_objective(problem; grid_size, scheme); generated metadata field :reference_objective.
    • Convergence report: grid size and scheme columns, objective at full precision, gap to the reference.
    • Consistency tests: a reference that no longer matches the default grid or parameters fails; the comparison rule fails for a problem without any reference.
    • Procedure: tools/references/README.md.

    Left to others

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions