Repository navigation
Conversation
Resolve conflicts from main's plugin-driven output-column refactor. Main replaced the hardcoded output column lists with schema-derived OutputColumn objects, superseding the manual list edits on this branch: - utilities/finalize.py: _PARSED_COLS / QUANT_OUT_COLS / QUAL_OUT_COLS were removed in favor of columns passed in from the plugin's OUTPUT_COLUMNS. Took main's version; the "ord_text" entry added here is no longer needed. - plugin/one_shot/components.py: the literal possible_out_cols list was replaced by self.POSSIBLE_OUT_COLS. Took main's version for the same reason. - extraction/ghp/geothermal_heat_pump_schema.json5: combined both sides by keeping main's expanded summary guidance alongside the ord_text guidance from this branch. The ord_text column still reaches both CSV outputs because _out_cols_from_config() derives output columns from each schema's "required" array, where ord_text is already listed. Verified for the GHP, geothermal electricity, and water rights schemas. Also dropped the ord_text assertions from test_save_db_writes_csvs. That test auto-merged without a conflict but was left broken: it exercises COMPASSWindExtractor, which has no ord_text column, so save_db() would never emit it.
Per the decisions from the extraction output meeting: - Rename the column to "ordinance_text" in all one-shot schemas (GHP, geothermal electricity, water rights demo). - Change the semantics from "a single verbatim sentence, no ellipses" to "the complete relevant excerpt". The first sentence must still be the start of the ordinance text and must be reproduced in full, but an ellipsis may now be used after it to drop irrelevant passages. Everything outside the ellipses stays verbatim. Also add shared ORDINANCE_TEXT_PROMPT and EXPLANATION_PROMPT templates and wire them into the legacy decision-graph output nodes for wind, solar, small wind, and the shared extra-restriction graphs, so those plugins produce the same two fields the one-shot schemas do. The legacy min_dist/max_dist nodes are deliberately left alone: their results are merged with dict.update() over the main setback result, so emitting ordinance_text there would overwrite the primary excerpt. System-size classification nodes are also unchanged since they do not produce ordinance rows. No plugin declares the new legacy columns yet, so wind/solar/small_wind CSV output is unchanged by this commit. Note that there is no eval suite for wind or solar (only date extraction), so the quality of ordinance_text under the legacy multi-turn prompts is not yet measured.
Implements the output-format decisions from the extraction meeting, uniformly across schema-driven and legacy plugins. - Qualitative rows now carry value and units. Output columns no longer vary by row kind, so include_in_qual_output / include_in_quant_output are no longer consulted when writing. - The LLM explanation is now written to the CSV. _out_cols_from_config previously filtered it out. - summary is deprecated and no longer written. It stays in the one-shot schemas (so the model still reasons through it) but is dropped from the output via _DEPRECATED_OUT_COLS, and is removed outright from the legacy plugin column lists. - ordinance_text is added to the legacy plugin column lists, so the fields the graph prompts already request now reach the output. - Both CSVs are replaced by a single ordinances.csv. The quantitative flag is retained as a column so the two row kinds stay separable, and the run manifest now exposes one DATA_FILE key. This also fixes a pre-existing bug where QUAL_DATA_FILE pointed at the quantitative CSV. - ordinance_text is trimmed to 5000 characters on the way to disk, cut at a word boundary and marked with a trailing ellipsis. The RMP schema packed a summary and a justification into one field; that is split into ordinance_text (verbatim excerpt) and explanation (justification), and all 16 few-shot examples are rewritten so they demonstrate verbatim quoting rather than paraphrase. The water rights demo schema gains the explanation field it was missing. Breaking change for anyone reading qualitative_ordinances.csv or quantitative_ordinances.csv, including the validation notebooks under support/validation. There is no eval suite for wind, solar, or small wind, so the quality of ordinance_text and explanation under the legacy multi-turn prompts is still unmeasured.
Now that both kinds of row share one CSV, qualitative features get the literal string "str" in the units column. Selecting one kind of feature is then a plain column comparison, without needing the boolean quantitative flag. The sentinel is derived from the quantitative flag at write time rather than requested from the LLM. That keeps the invariant (quantitative is False) == (units == "str") true by construction, and avoids a failure mode where the model writes "string", "text", or an invented unit for a qualitative feature. Any such invented unit is overwritten. Applied in both _formatted_db and save_db so it holds no matter which path assembled the frame; the operation is idempotent.
The units sentinel ("str" on qualitative rows) already distinguishes the
two kinds of feature, so the boolean quantitative column is redundant in
the published file. It is removed from OUTPUT_COLUMNS for the one-shot
path and for wind, solar, and small wind.
The flag is still needed internally to derive the sentinel, so
_formatted_db now carries it alongside the output columns and save_db
projects it away just before writing. _formatted_db also tolerates
frames that arrive without the column, which previously could not happen
because it was always part of parsed_cols.
Also fixes a bug introduced when summary was deprecated:
ordinances_bool_index decides whether a row is an ordinance by checking
for a non-null value or summary. Qualitative rows have neither once
summary is gone, so they were being silently dropped from the output.
ordinance_text now counts as evidence too; summary stays in the list so
previously written per-jurisdiction CSVs still load.
Lint: trim_ordinance_text moves to compass.utilities.finalize (where it
was already applied) and is re-exported from compass.plugin.post_processing
for the step registry. That removes the function-level import that
import-outside-top-level flagged, which existed only to dodge a circular
import. Verified with `ruff check ./compass` and `ruff format --check
./compass` on ruff 0.16.2, matching the "latest" pin the CI lint job uses.
Reverses the summary deprecation. The four content columns now have
clearly separated jobs:
value the extracted answer; for a qualitative feature this
is the requirement itself, stated in full
summary prose restatement of the rule, carrying caveats and
conditions that value and units cannot hold
ordinance_text exact quotes copied from the source document
explanation the model's reasoning about its interpretation
summary no longer asks for quotes, since verbatim text is what
ordinance_text is for. It is also never blank: on qualitative rows the
value is copied into it at write time, in _normalize_qualitative_rows
alongside the units sentinel. Doing it there rather than in the prompt
keeps the two columns identical and saves the model writing the same
text twice.
Backward compatibility removed, since this is a single application that
never reads previous-generation output:
- "summary" is out of _ORD_CHECK_COLS; ordinance_text is the evidence
column that decides whether a row is an ordinance.
- The "column not in db.columns" guards in _formatted_db and the
qualitative labeling are gone. Every parser sets "quantitative" on
each row it emits, and every plugin that reaches this code declares
units, so the guards only masked bugs. The invariant is documented on
_formatted_db instead.
- _DEPRECATED_OUT_COLS is deleted.
Test fixtures that used summary as a row's only ordinance evidence now
use ordinance_text, matching what the parsers actually produce.
The previous commit had the copy backwards. The schemas ask the LLM for a null value on qualitative rows and put the requirement in summary, so copying value into summary blanked the one field that had the text. _normalize_qualitative_rows now copies summary into value instead, which matches what the schemas already request and keeps the model writing the requirement only once. Both columns end up populated on every row. The summary description is updated to say plainly that it is the primary output field for qualitative features, and that value is filled in from it when the output is written. Also widens the value column to object dtype before the copy. A numeric-only value column is float-typed, and writing qualitative text into it raised a pandas FutureWarning that becomes an error in a later release. Numeric values are unaffected.
Two items from Paul's review on #487. Examples now show summary and ordinance_text diverging (his request for "one or two examples where the sentence for the ordinance text extraction is completely different than the summary"). Every example summary used to be the ordinance_text wrapped in quotes, which demonstrated exactly the redundancy the two-field split is meant to remove. All 13 examples across the GHP, geothermal electricity, and water rights schemas are rewritten so summary reads as prose while ordinance_text stays verbatim, and several now show a multi-sentence excerpt joined with an ellipsis so the elision convention is demonstrated rather than only described. Examples that were missing explanation now have one. The water rights instruction block he and Copilot both flagged is fixed. It carried two contradictions: it still described ordinance_text as a single contiguous sentence with no ellipses, which is the semantics that was replaced everywhere else, and it told the model it could emit a row with null fields when a feature has no requirement even though ordinance_text is required and non-empty. The rule now matches the current definition and says plainly to omit the row.
Scopes the PR back to adding two columns, combining the outputs, and the qualitative value/units handling. rmp_schema.json (:85, :150, :164): summary is restored byte-for-byte, description and all 16 example values, so the RMP diff is now purely additive. Its two-section Summary/Justification format is left alone as the deliberate exception; ordinance_text and explanation are simply added beside it. common/base.py (:20) and the geothermal, GHP, and water rights schemas: the original summary wording is restored, including the per-schema lists of qualitative features, with only the "use direct excerpts/quotes" clause replaced. Summary now asks for the gist of the requirement with all of its specifics and details, and points at ordinance_text for verbatim wording. geothermal_schema.json (:88): the qualitative feature list (permitting, fencing, lighting, seismic monitoring, decommissioning, prohibitions) comes back with the restored description. geothermal_schema.json (:92): ordinance_text may now gather passages from anywhere in the document, before or after the primary sentence, joined with an ellipsis in document order. The first sentence must still be the one corresponding to the extracted value and summary. Applied to all four schemas and the legacy prompt. test_plugin_ordinances.py (:435): reverted. "summary" goes back into _ORD_CHECK_COLS, so a row carrying only a summary counts as an ordinance again and the fixtures no longer needed to change. That file and test_utilities_parsing.py are now identical to their pre-PR state.
Add a schema-driven one-shot plugin covering local data center siting ordinances, including the extraction schema and plugin config, and register the extractor so the `data_centers` technology is available in the plugin registry. The 34-feature controlled vocabulary, the feature boundary rules, and the units convention are derived from the COMPASS data center ordinance research workbook, which hand-codes 147 feature rows across 22 jurisdictions in 10 states.
There was a problem hiding this comment.
Pull request overview
Adds a new schema-driven one-shot extraction plugin for data center ordinances under compass.extraction, wiring it into the existing plugin registration/import flow so it can be resolved and executed like the other one-shot technologies.
Changes:
- Added a new
data_centersone-shot plugin configuration (YAML) and extraction schema (JSON). - Registered the new extractor in
compass.extractionand re-exported it from the top-levelcompasspackage. - Added a new
compass.extraction.data_centerspackage initializer that constructs the schema-based plugin viacreate_schema_based_one_shot_extraction_plugin.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| compass/extraction/data_centers/plugin_config.yaml | Defines retrieval/heuristic and extraction-system prompt for the data centers one-shot plugin |
| compass/extraction/data_centers/data_centers_schema.json | Introduces the data centers extraction schema and feature definitions/instructions |
| compass/extraction/data_centers/init.py | Registers the COMPASSDataCentersExtractor via the schema-based plugin factory |
| compass/extraction/init.py | Imports the data centers extractor to ensure plugin registration |
| compass/init.py | Re-exports the data centers extractor from the top-level package |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| "size_tiering": [ | ||
| "Data center ordinances frequently set different standards for different size classes of facility. When they do, emit one row per class and record the jurisdiction's own class name in data_center_type.", | ||
| "Where one standard applies to data centers generally, emit a single row and set data_center_type to the term the ordinance uses for the use, or null when it uses none.", | ||
| "Do not merge tiered standards into a single row, and do not map a local class name onto another jurisdiction's taxonomy." | ||
| ], |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #536 +/- ##
==========================================
+ Coverage 68.27% 69.26% +0.98%
==========================================
Files 79 81 +2
Lines 7976 8351 +375
Branches 829 880 +51
==========================================
+ Hits 5446 5784 +338
- Misses 2351 2376 +25
- Partials 179 191 +12
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The field description told the model to emit null where an ordinance
sets one standard for data centers generally, while the size_tiering
instructions told it to record the ordinance's own generic term for the
use. Align both on the latter, which matches the reference workbook:
none of its 147 feature rows leave the field blank, and the 17 untiered
jurisdictions record their own generic term ("Data Center", "Large Data
Center"). Null is now reserved for a provision that attaches no term to
the use at all.
| - database | ||
|
|
||
| collection_prompts: true | ||
| text_extraction_prompts: true |
There was a problem hiding this comment.
Did you find that including this improved extraction? If not, I would recommend leaving this out so that the extractor has access to unfiltered text
Add a normalize_data_center_types post-processing step that blanks generic labels and shortens accessory types, tighten the schema and system prompt guidance, narrow query templates, and remove text_extraction_prompts so the parser sees unfiltered collected text. Finalize now saves through the plugin's save_structured_data.
Jurisdiction CSVs with a FIPS column are matched on FIPS alone, and known local docs/URLs are keyed by string codes, falling back to keys without leading zeros.
Introduce PriorityCrawler, enabled via the priority_search option, which merges Google and DuckDuckGo results and follows LLM-ranked links from one shared queue. Replace search-result link expansion and viewer URL unwrapping with it, and log elapsed time for long-running search, retrieval, crawl, filtering, and extraction operations.
Take main's final quotes (#487) wording in files only the quotes branch touched, including the summary prompt in compass/common/base.py, which otherwise auto-merged back to a superseded version. Keep timing logs in main's refactored _run_collection_step, keep FIPS known-source lookup alongside the new rate tracking, and format priority search queries with _format_queries so {jurisdiction_website} works. Drop the local SerpAPIDuckDuckGoSearch adapter: nlr-elm 0.0.50 ships an adapter under the same name, which shadowed ours.
Update test doubles for the combined runtime: collection-step fakes gain priority_search, and the known-source FIPS fakes gain search_params for num_search_results_to_crawl.
The quotes branch had registered trim_ordinance_text as a post-processing step before dropping it; merging the branch in early kept that registration and its tests alongside the data center step. Remove them so post_processing.py only adds normalize_data_center_types. Also apply ruff format, give the priority-search runtime test placeholder Azure credentials so it does not depend on the local environment, and ignore the ELM loader docstring references that the COMPASSWebFileLoader subclass now inherits.
The data center plugin does not need it: normal runs already save through save_structured_data, which applies post_processing_steps. Keep the export test on that path only.
|
Superseded by a stack of three PRs that split this work: #606 (event logging and workflow map) → #607 (priority search crawler) → #608 (data center plugin). Stack: https://github.com/NatLabRockies/COMPASS/stack/609. This branch is kept for its full commit history. |
Adds the
data_centersone-shot plugin (schema + config) and registers the extractor, following the same shape as #522.The feature set is derived from the COMPASS DC Ordinance Research workbook (2026-08-16), which hand-codes 147 feature rows across 22 jurisdictions in 10 states. All 34 controlled terms from its "Feature List (DC)" tab are carried over as the schema's feature enum, so extracted rows can be compared directly against the hand-coded reference set.
Notes on the schema
data_center_typeoutput column. Data center ordinances routinely tier standards by facility size class — Birmingham sets separate residential setbacks for Hyperscale, Medium, and Micro. The schema adds adata_center_typefield so tiered rules emit one row per class, recorded in the jurisdiction's own vocabulary rather than a normalized taxonomy, matching how the workbook codes its Data Center Type column.$definitions. The workbook's feature descriptions carry explicit disambiguation ("Distinct from X, which…") for pairs a model will otherwise conflate: minimum lot size vs. maximum lot coverage vs. minimum vegetative cover; setback-residential vs. setback-property-line; buffer vs. equipment screening vs. fencing; and the three water features. These are encoded as IGNORE clauses.dBA day / night at receiving residential propertyfor a value of55 / 45. The schema requires positional correspondence between compound values and compound units, and instructs the model to keep such provisions in one row.$qualitative_featuresis limited to prohibitions/moratorium, on-site power technology, and decommissioning. District features are deliberately excluded, following the wind schema, since theirvaluecarries the district list.Verification
validate_plugin_configuration()at import;resolve_plugin("data_centers")resolves and yields 14 output columns.NOT_TECH_WORDS, whichKeywordBasedHeuristicstrips before counting matches.pytest tests/python/unit: 575 passed, 63 skipped.