Repository navigation
fix(config): refuse three configurations that ran to a meaningless result (#996, #997, #999) - #1001
Merged
Merged
Conversation
…sult (#996, #997, #999) - #996: `normalization ... = scale` under a Bayesian sampler (am, dream, hmc, mh, p_dream, pt) is refused when the configuration loads, naming the observables: scale is a profile, not a marginal. The sampler set is the registry family noise_profiling and linear_profiling are refused for, now read by all three through Configuration._bayesian_sampler. Every form is covered (whole fit, per observable, per experiment, either edition). - #997: any normalization (init, peak, zero, unit, floor) of a column scored by a count noise model is refused at load, naming the column, the normalization, the experiment and the noise model. Decided by a new NoiseModel.scores_counts property (NegBinomial sets it), not the class. scale on such a column keeps #990's no-additive-scale refusal, so it has one message. A normalized continuous column beside a raw count column still loads. - #999: a network-free action BNG2.pl cannot run (method nf with sample_times or no n_steps, as edition-2 experiments write; RuleMonkey) is refused wherever BNG2.pl would simulate it, naming the experiment and BNG2.pl's message: at load under bngl_backend = bionetgen and under job_type = check (which simulates with BNG2.pl whatever the backend); under auto, when the fit sets its models up, before any network generation, if bngsim is unavailable, cannot take the model, or lacks the engine, and after a bngsim set-up step fails and auto falls back. A configuration under auto still loads without bngsim for readers that simulate nothing. - Tests: tests/test_load_refusals_996_997_999.py, with oracles that do not go through PyBNF (a quadrature of the Gaussian likelihood over the scale, scipy.stats.nbinom and scipy.special, and BNG2.pl itself on each action). Existing tests that built the now-refused combinations are updated: test_network_free_bngsim_supported asserted that nf under BNG2.pl builds. - Docs: config_keys (normalization, experiment method, bngl_backend) and algorithms (check); changelog fragments 996, 997, 999.
… scale hint (#999, #996) Review follow-up on the load-time refusals. - #999: bng2pl_network_free_refusals now reads an action as BNG2.pl runs it: a trailing ';' and comment dropped (BNGLModel keeps both on a hand-written action), simulate_nf parsed, and the method matched as written ('NF' is not BNG2.pl's 'nf'). An NFsim action is refused in simulate_nf's order: continue, n_steps below 1, no t_end, t_end not after t_start, sample_times, no n_steps. Each refusal carries its cause, and the message states only the causes at hand; a hand-written action also gets the hint to write it BNG2.pl's way. Valid hand-written NF actions, with or without ';' or a comment, still load under bionetgen. - #999: the job_type = check remedy names the job that works: a one-generation de fit with no free parameters under bngsim. - #999: the reason auto is refused at set-up rather than at load no longer cites a PEtab export, which builds no Configuration; from_pybnf, which reloads a finished run's configuration, is the example. - #996: scale profiles one factor per series (experiment and column), so the refusal lists each series and asks for one free scale parameter per series; one shared parameter would be a different model. - Tests: each new refusal and each valid neighbour is checked against BNG2.pl itself; the per-series remedy scores 0 at the truth where one shared scale cannot; the review's tests are added unchanged (tests/test_load_refusals_review_1001.py).
…auto refusal test The bngsim-less CI leg sets PYBNF_NO_BNGSIM, which the unavailability reason reports before a missing module, so the not-installed case saw the other reason.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #996. Closes #997. Closes #999.
Each of these configurations ran to a result with no meaning. Each is now refused before anything runs, with a message that names the cause.
#996:
normalization = scaleunder a Bayesian samplerscaleprofiles a multiplicative scale out at each evaluation. That is a profile, not a marginal, so the draws were not posterior draws.noise_profilingandlinear_profilingwere already refused for samplers for this reason.Configuration._bayesian_sampler(), the registry family'sampler'(am, dream, hmc, mh, p_dream, pt), now decides this for all three refusals.observable: …, formula: c * ywithcfree) or an optimizer.c. The gap equals ½ log(2π/A), which moves with the prediction.#997:
normalizationon a column scored by a count familyNoiseModel.scores_countsproperty (True forNegBinomial) decides this, not the class name, and it coversinit,peak,zero,unitandfloor.scalekeeps fix(am): record every experiment's trajectory, normalized once; refuse what would be wrong (#990) #994's "no additive scale" refusal, so each case gets one message.neg_bincolumn still loads.-nbinom.logpmfand sums to 1;neg_bin, no scored column holds a non-integer, so nothing in the repository depends on it.#999: network-free experiments that BNG2.pl cannot run
grid.bng2pl_network_free_refusals, checked against BNG2.pl 2.9.3 itself):method nfwithoutn_steps, which covers the edition-2 time course (sample_times) and dose-response;nf, i.e. RuleMonkey. That is the BNG2.pl half of Nine loud failures that block a valid job or give the wrong cause: RuleMonkey experiments, one-time-point data, PEtab import and export cases, error messages that hide their reason, and a crash on deeply nested formulas #962 item 1.bngl_backend = bionetgenand underjob_type = check, which always runs BNG2.pl;autowhen the model would land on BNG2.pl: bngsim unavailable, bridge can't take it, missing engine, or a failed set-up that fell back.autois not refused at load. Whether it uses bngsim depends on the machine, and readers such as PEtab export simulate nothing.Sibling paths
param_scanactions-rresumejob_type = checkTests
tests/test_load_refusals_996_997_999.py, 55 tests.test_network_free_bngsim_supported::test_nf_under_bng2pl_buildsasserted the combination An edition-2 network-free experiment (method: nf) under bngl_backend = bionetgen fails every simulation, because BNG2.pl's NFsim refuses the sample_times PyBNF writes, and nothing refuses it at load #999 refuses, and its comment wrongly said BNG2.pl's NFsim honours the output points. It now asserts the refusal.neg_bin+ normalization oram+scale. They now assert the load-time refusal, andam's own checks are exercised on Gaussian columns.bng_command, and space-separated data files. Those will be filed separately.mainbuild.ruffandtools/changelog.py checkare clean.Review
The first independent review found no blocking defect and checked that nothing valid is refused:
check;It raised four findings, fixed in 47c21b6:
;or comment,simulate_nf,method=>"NF",n_steps=>0,continue=>1, not_end). Lines are now read as BNG2.pl runs them.scalehint suggested one sharedc. It now asks for one per profiled series.The reviewer's tests are committed (
tests/test_load_refusals_review_1001.py). A second check of 47c21b6 tried 24 hand-written network-free actions against BNG2.pl itself, and none that BNG2.pl runs is refused.b52d08c isolates one test from the bngsim-less CI leg's
PYBNF_NO_BNGSIM.