[Dev] Per-modeling lists of problems, discovered automatically, with a status per modeling #303

Agents: read the guide for agents in #310 first (order, difficulty, how to claim an issue, working method). The specification is in this description and in the comments below.

Goal

Each modeling has its own list of problems, discovered automatically, and each problem has a status per modeling (:draft or :valid). A problem is provided to users in a modeling only when it is valid there, and consistent with every other modeling where it is valid. Nothing that can be computed is written by hand.

Part of #310. Lands on main on the current API; no release needed (0.6.0-beta will follow with #283).

Current state (code to change)

• src/OptimalControlProblems.jl
  • make_list_of_problems() (l. 49–69): list from readdir(ext/MetaData) minus a hard-coded problems_to_exclude = [:bioreactor, :cart_pendulum, :moonlander].
  • l. 73–112: for each problem, eval generates p(backend, args...; kwargs...) and p_s(...), which throw ExtensionError(weakdeps[T]); weakdeps = Dict(OptimalControlBackend => :OptimalControl, JuMPBackend => :JuMP).
  • Their docstrings are broken: a stray ``` line ([Doc] Typo in the docstrings #172) and $(TYPEDSIGNATURES) interpolated into a plain string, printed as DocStringExtensions.TypedMethodSignatures() ([Doc] Examples are not displayed correctly in the REPL #173).
• ext/JuMPModels.jl:25 and ext/OptimalControlModels.jl:9,14,21 include one file per problem of problems(), in JuMPModels/, OptimalControlModels/ and OptimalControlModels_s/.
• The three excluded problems are most likely excluded because of this rule: they have OptimalControl and JuMP files but no _s file, and the OptimalControl extension includes $(problem)_s.jl for every problem.
• Users of the list: both extensions; every test file (LIST_OF_PROBLEMS, eval(f), eval(Symbol(f, :_s))); docs/problems.jl:429; CTBenchmarks (__problems() = OptimalControlProblems.problems()).
• Docstrings of the 66 problem methods are hand-written; examples use N=100 instead of grid_size=100.

Design

Modelings

Three backends, each a dispatch tag, described by one table (the only place to edit to add a modeling, e.g. Corleone in #308):

• The _s variants become the OptimalControlScalarizedBackend modeling: beam(OptimalControlScalarizedBackend()) replaces beam_s(OptimalControlBackend()). No more p_s functions. If Description and problems #174 drops them, delete one directory and one table entry.
• Generate the per-problem functions with the existing @eval loop (no @generated: nothing depends on argument types).

Existence and status

• A problem exists if ext/MetaData/<p>.jl exists. It exists in a modeling if the file of that modeling exists.

• Its status is declared in the metadata, per modeling, for the modelings where it exists:

  :status => (OptimalControl=:valid, OptimalControlScalarized=:valid, JuMP=:draft)

  Status	Meaning	Tests ([Tests] Test layout per environment and consistency tests #304, [Tests] Crons per scope for the slow tests, with reports #305)
  :draft	written, not validated yet	run, failures reported but do not fail CI
  :valid	passes its own tests and is consistent with every other modeling where it is :valid	failures fail CI

• :notes (optional String): why a problem is a draft.

• The three excluded problems: remove the exclusion list, try them in each modeling; :valid where their tests pass, :draft with :notes otherwise.

API

problems()                                        # problems valid in at least one modeling
problems(JuMPBackend())                           # valid in JuMP
problems(OptimalControlBackend(), JuMPBackend())  # valid in both (comparison tests)
problems(JuMPBackend(); status=:draft)            # drafts in JuMP
problems(; status=:all)                           # every existing problem
status(:beam)                                     # (OptimalControl=:valid, …), from the metadata

Return sorted Vector{Symbol}.

Generic method of each problem (in src)

• Problem has no file for this modeling → CTBase.IncorrectArgument("beam is not available for JuMPBackend; see problems(JuMPBackend())").
• File present but extension not loaded → CTBase.ExtensionError(<weakdep>), as today.
• Calling a draft works but emits a warning (@warn "beam is a draft for JuMPBackend: …" with :notes), so drafts stay easy to develop on.

Generated docstrings

The 66 hand-written docstrings are replaced by docstrings generated from one template, per problem and per modeling:

• one sentence (first sentence of ext/Descriptions/<p>.md);
• a usage example for that modeling (OptimalControl: get the problem and solve; JuMP: get the model, set_optimizer, optimize!), in non-executed ```julia-repl fences, with grid_size, not N;
• the default parameters (from the metadata) and the status;
• a link to the online page of the problem.

The generic method gets a docstring that lists the modelings where the problem exists and their status. This closes #172 and #173.

Stale precompilation cache

The lists are computed with readdir at precompile time; adding a file to ext/JuMPModels/ does not invalidate the cache (the same is already true for ext/MetaData/). Make the directories precompilation dependencies (include_dependency), or compute availability at run time, and add a test that shows it. Check behaviour on Julia 1.10 and 1.12.

Out of scope

• Test layout per environment and the full consistency suite: [Tests] Test layout per environment and consistency tests #304. Here, adapt the existing tests minimally: loop over problems(backend) per modeling, comparisons over the intersection, no eval(Symbol(f, :_s)).
• Documentation rewrite: [Doc] Rewrite the documentation: getting started, user guide, problems, developers #307. Only keep docs/problems.jl building.
• CTBenchmarks: open a follow-up issue there (use problems(backend), drop _s names).

Done when

• No list of problems written by hand in src/, ext/, test/.
• Adding ext/MetaData/foo.jl and ext/JuMPModels/foo.jl with JuMP=:draft makes foo appear in problems(JuMPBackend(); status=:draft) and be tested, without touching src/; switching it to :valid makes it appear in problems(JuMPBackend()).
• Three backends; no p_s function left; OptimalControlModels_s/ renamed.
• bioreactor, cart_pendulum, moonlander are back, as :valid or :draft with notes.
• Error messages and draft warning tested.
• help?> beam renders correctly ([Doc] Typo in the docstrings #172, [Doc] Examples are not displayed correctly in the REPL #173).
• Precompilation cache test passes on Julia 1.10 and 1.12.
• Follow-up issue opened in CTBenchmarks.