Skip to content

Add data center one-shot extraction plugin - #536

Closed
rajeee wants to merge 23 commits into
mainfrom
add-data-center-extraction-plugin
Closed

rajeee wants to merge 23 commits into
mainfrom
add-data-center-extraction-plugin

Conversation

@rajeee

@rajeee rajeee commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Adds the data_centers one-shot plugin (schema + config) and registers the extractor, following the same shape as #522.

The feature set is derived from the COMPASS DC Ordinance Research workbook (2026-08-16), which hand-codes 147 feature rows across 22 jurisdictions in 10 states. All 34 controlled terms from its "Feature List (DC)" tab are carried over as the schema's feature enum, so extracted rows can be compared directly against the hand-coded reference set.

Notes on the schema

  • data_center_type output column. Data center ordinances routinely tier standards by facility size class — Birmingham sets separate residential setbacks for Hyperscale, Medium, and Micro. The schema adds a data_center_type field so tiered rules emit one row per class, recorded in the jurisdiction's own vocabulary rather than a normalized taxonomy, matching how the workbook codes its Data Center Type column.
  • Feature boundaries in $definitions. The workbook's feature descriptions carry explicit disambiguation ("Distinct from X, which…") for pairs a model will otherwise conflate: minimum lot size vs. maximum lot coverage vs. minimum vegetative cover; setback-residential vs. setback-property-line; buffer vs. equipment screening vs. fencing; and the three water features. These are encoded as IGNORE clauses.
  • Compound values. The workbook's Units column carries the measurement basis, not just the unit — dBA day / night at receiving residential property for a value of 55 / 45. The schema requires positional correspondence between compound values and compound units, and instructs the model to keep such provisions in one row.
  • $qualitative_features is limited to prohibitions/moratorium, on-site power technology, and decommissioning. District features are deliberately excluded, following the wind schema, since their value carries the district list.

Verification

  • Plugin registers and passes validate_plugin_configuration() at import; resolve_plugin("data_centers") resolves and yields 14 output columns.
  • Heuristic passes on data center ordinance text and rejects a look-alike decoy — "community center", "recreation center", "data collection", and "database" are in NOT_TECH_WORDS, which KeywordBasedHeuristic strips before counting matches.
  • pytest tests/python/unit: 575 passed, 63 skipped.

rajeee added 13 commits June 23, 2026 14:13
Resolve conflicts from main's plugin-driven output-column refactor.

Main replaced the hardcoded output column lists with schema-derived
OutputColumn objects, superseding the manual list edits on this branch:

- utilities/finalize.py: _PARSED_COLS / QUANT_OUT_COLS / QUAL_OUT_COLS
  were removed in favor of columns passed in from the plugin's
  OUTPUT_COLUMNS. Took main's version; the "ord_text" entry added here
  is no longer needed.
- plugin/one_shot/components.py: the literal possible_out_cols list was
  replaced by self.POSSIBLE_OUT_COLS. Took main's version for the same
  reason.
- extraction/ghp/geothermal_heat_pump_schema.json5: combined both sides
  by keeping main's expanded summary guidance alongside the ord_text
  guidance from this branch.

The ord_text column still reaches both CSV outputs because
_out_cols_from_config() derives output columns from each schema's
"required" array, where ord_text is already listed. Verified for the
GHP, geothermal electricity, and water rights schemas.

Also dropped the ord_text assertions from test_save_db_writes_csvs.
That test auto-merged without a conflict but was left broken: it
exercises COMPASSWindExtractor, which has no ord_text column, so
save_db() would never emit it.
Per the decisions from the extraction output meeting:

- Rename the column to "ordinance_text" in all one-shot schemas (GHP,
  geothermal electricity, water rights demo).
- Change the semantics from "a single verbatim sentence, no ellipses"
  to "the complete relevant excerpt". The first sentence must still be
  the start of the ordinance text and must be reproduced in full, but
  an ellipsis may now be used after it to drop irrelevant passages.
  Everything outside the ellipses stays verbatim.

Also add shared ORDINANCE_TEXT_PROMPT and EXPLANATION_PROMPT templates
and wire them into the legacy decision-graph output nodes for wind,
solar, small wind, and the shared extra-restriction graphs, so those
plugins produce the same two fields the one-shot schemas do.

The legacy min_dist/max_dist nodes are deliberately left alone: their
results are merged with dict.update() over the main setback result, so
emitting ordinance_text there would overwrite the primary excerpt.
System-size classification nodes are also unchanged since they do not
produce ordinance rows.

No plugin declares the new legacy columns yet, so wind/solar/small_wind
CSV output is unchanged by this commit. Note that there is no eval
suite for wind or solar (only date extraction), so the quality of
ordinance_text under the legacy multi-turn prompts is not yet measured.
Implements the output-format decisions from the extraction meeting,
uniformly across schema-driven and legacy plugins.

- Qualitative rows now carry value and units. Output columns no longer
  vary by row kind, so include_in_qual_output / include_in_quant_output
  are no longer consulted when writing.
- The LLM explanation is now written to the CSV. _out_cols_from_config
  previously filtered it out.
- summary is deprecated and no longer written. It stays in the one-shot
  schemas (so the model still reasons through it) but is dropped from
  the output via _DEPRECATED_OUT_COLS, and is removed outright from the
  legacy plugin column lists.
- ordinance_text is added to the legacy plugin column lists, so the
  fields the graph prompts already request now reach the output.
- Both CSVs are replaced by a single ordinances.csv. The quantitative
  flag is retained as a column so the two row kinds stay separable, and
  the run manifest now exposes one DATA_FILE key. This also fixes a
  pre-existing bug where QUAL_DATA_FILE pointed at the quantitative CSV.
- ordinance_text is trimmed to 5000 characters on the way to disk, cut
  at a word boundary and marked with a trailing ellipsis.

The RMP schema packed a summary and a justification into one field; that
is split into ordinance_text (verbatim excerpt) and explanation
(justification), and all 16 few-shot examples are rewritten so they
demonstrate verbatim quoting rather than paraphrase. The water rights
demo schema gains the explanation field it was missing.

Breaking change for anyone reading qualitative_ordinances.csv or
quantitative_ordinances.csv, including the validation notebooks under
support/validation. There is no eval suite for wind, solar, or small
wind, so the quality of ordinance_text and explanation under the legacy
multi-turn prompts is still unmeasured.
Now that both kinds of row share one CSV, qualitative features get the
literal string "str" in the units column. Selecting one kind of feature
is then a plain column comparison, without needing the boolean
quantitative flag.

The sentinel is derived from the quantitative flag at write time rather
than requested from the LLM. That keeps the invariant
(quantitative is False) == (units == "str") true by construction, and
avoids a failure mode where the model writes "string", "text", or an
invented unit for a qualitative feature. Any such invented unit is
overwritten.

Applied in both _formatted_db and save_db so it holds no matter which
path assembled the frame; the operation is idempotent.
The units sentinel ("str" on qualitative rows) already distinguishes the
two kinds of feature, so the boolean quantitative column is redundant in
the published file. It is removed from OUTPUT_COLUMNS for the one-shot
path and for wind, solar, and small wind.

The flag is still needed internally to derive the sentinel, so
_formatted_db now carries it alongside the output columns and save_db
projects it away just before writing. _formatted_db also tolerates
frames that arrive without the column, which previously could not happen
because it was always part of parsed_cols.

Also fixes a bug introduced when summary was deprecated:
ordinances_bool_index decides whether a row is an ordinance by checking
for a non-null value or summary. Qualitative rows have neither once
summary is gone, so they were being silently dropped from the output.
ordinance_text now counts as evidence too; summary stays in the list so
previously written per-jurisdiction CSVs still load.

Lint: trim_ordinance_text moves to compass.utilities.finalize (where it
was already applied) and is re-exported from compass.plugin.post_processing
for the step registry. That removes the function-level import that
import-outside-top-level flagged, which existed only to dodge a circular
import. Verified with `ruff check ./compass` and `ruff format --check
./compass` on ruff 0.16.2, matching the "latest" pin the CI lint job uses.
Reverses the summary deprecation. The four content columns now have
clearly separated jobs:

  value          the extracted answer; for a qualitative feature this
                 is the requirement itself, stated in full
  summary        prose restatement of the rule, carrying caveats and
                 conditions that value and units cannot hold
  ordinance_text exact quotes copied from the source document
  explanation    the model's reasoning about its interpretation

summary no longer asks for quotes, since verbatim text is what
ordinance_text is for. It is also never blank: on qualitative rows the
value is copied into it at write time, in _normalize_qualitative_rows
alongside the units sentinel. Doing it there rather than in the prompt
keeps the two columns identical and saves the model writing the same
text twice.

Backward compatibility removed, since this is a single application that
never reads previous-generation output:

- "summary" is out of _ORD_CHECK_COLS; ordinance_text is the evidence
  column that decides whether a row is an ordinance.
- The "column not in db.columns" guards in _formatted_db and the
  qualitative labeling are gone. Every parser sets "quantitative" on
  each row it emits, and every plugin that reaches this code declares
  units, so the guards only masked bugs. The invariant is documented on
  _formatted_db instead.
- _DEPRECATED_OUT_COLS is deleted.

Test fixtures that used summary as a row's only ordinance evidence now
use ordinance_text, matching what the parsers actually produce.
The previous commit had the copy backwards. The schemas ask the LLM for
a null value on qualitative rows and put the requirement in summary, so
copying value into summary blanked the one field that had the text.

_normalize_qualitative_rows now copies summary into value instead, which
matches what the schemas already request and keeps the model writing the
requirement only once. Both columns end up populated on every row.

The summary description is updated to say plainly that it is the primary
output field for qualitative features, and that value is filled in from
it when the output is written.

Also widens the value column to object dtype before the copy. A
numeric-only value column is float-typed, and writing qualitative text
into it raised a pandas FutureWarning that becomes an error in a later
release. Numeric values are unaffected.
Two items from Paul's review on #487.

Examples now show summary and ordinance_text diverging (his request for
"one or two examples where the sentence for the ordinance text
extraction is completely different than the summary"). Every example
summary used to be the ordinance_text wrapped in quotes, which
demonstrated exactly the redundancy the two-field split is meant to
remove. All 13 examples across the GHP, geothermal electricity, and
water rights schemas are rewritten so summary reads as prose while
ordinance_text stays verbatim, and several now show a multi-sentence
excerpt joined with an ellipsis so the elision convention is
demonstrated rather than only described. Examples that were missing
explanation now have one.

The water rights instruction block he and Copilot both flagged is
fixed. It carried two contradictions: it still described ordinance_text
as a single contiguous sentence with no ellipses, which is the
semantics that was replaced everywhere else, and it told the model it
could emit a row with null fields when a feature has no requirement
even though ordinance_text is required and non-empty. The rule now
matches the current definition and says plainly to omit the row.
Scopes the PR back to adding two columns, combining the outputs, and the
qualitative value/units handling.

rmp_schema.json (:85, :150, :164): summary is restored byte-for-byte,
description and all 16 example values, so the RMP diff is now purely
additive. Its two-section Summary/Justification format is left alone as
the deliberate exception; ordinance_text and explanation are simply
added beside it.

common/base.py (:20) and the geothermal, GHP, and water rights schemas:
the original summary wording is restored, including the per-schema lists
of qualitative features, with only the "use direct excerpts/quotes"
clause replaced. Summary now asks for the gist of the requirement with
all of its specifics and details, and points at ordinance_text for
verbatim wording.

geothermal_schema.json (:88): the qualitative feature list (permitting,
fencing, lighting, seismic monitoring, decommissioning, prohibitions)
comes back with the restored description.

geothermal_schema.json (:92): ordinance_text may now gather passages
from anywhere in the document, before or after the primary sentence,
joined with an ellipsis in document order. The first sentence must still
be the one corresponding to the extracted value and summary. Applied to
all four schemas and the legacy prompt.

test_plugin_ordinances.py (:435): reverted. "summary" goes back into
_ORD_CHECK_COLS, so a row carrying only a summary counts as an ordinance
again and the fixtures no longer needed to change. That file and
test_utilities_parsing.py are now identical to their pre-PR state.
Add a schema-driven one-shot plugin covering local data center siting
ordinances, including the extraction schema and plugin config, and
register the extractor so the `data_centers` technology is available in
the plugin registry.

The 34-feature controlled vocabulary, the feature boundary rules, and
the units convention are derived from the COMPASS data center ordinance
research workbook, which hand-codes 147 feature rows across 22
jurisdictions in 10 states.
Copilot AI lite review requested due to automatic review settings August 25, 2026 19:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new schema-driven one-shot extraction plugin for data center ordinances under compass.extraction, wiring it into the existing plugin registration/import flow so it can be resolved and executed like the other one-shot technologies.

Changes:

  • Added a new data_centers one-shot plugin configuration (YAML) and extraction schema (JSON).
  • Registered the new extractor in compass.extraction and re-exported it from the top-level compass package.
  • Added a new compass.extraction.data_centers package initializer that constructs the schema-based plugin via create_schema_based_one_shot_extraction_plugin.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
compass/extraction/data_centers/plugin_config.yaml Defines retrieval/heuristic and extraction-system prompt for the data centers one-shot plugin
compass/extraction/data_centers/data_centers_schema.json Introduces the data centers extraction schema and feature definitions/instructions
compass/extraction/data_centers/init.py Registers the COMPASSDataCentersExtractor via the schema-based plugin factory
compass/extraction/init.py Imports the data centers extractor to ensure plugin registration
compass/init.py Re-exports the data centers extractor from the top-level package

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +246 to +250
"size_tiering": [
"Data center ordinances frequently set different standards for different size classes of facility. When they do, emit one row per class and record the jurisdiction's own class name in data_center_type.",
"Where one standard applies to data centers generally, emit a single row and set data_center_type to the term the ordinance uses for the use, or null when it uses none.",
"Do not merge tiered standards into a single row, and do not map a local class name onto another jurisdiction's taxonomy."
],
@codecov-commenter

codecov-commenter commented Aug 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.53608% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.26%. Comparing base (fc0b46b) to head (d18109e).

Files with missing lines Patch % Lines
compass/web/priority_crawl.py 85.82% 26 Missing and 10 partials ⚠️
compass/scripts/download.py 25.00% 18 Missing ⚠️
compass/utilities/timing.py 90.90% 1 Missing and 2 partials ⚠️
compass/pipeline/coordinator.py 75.00% 1 Missing and 1 partial ⚠️
compass/web/file_loader.py 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #536      +/-   ##
==========================================
+ Coverage   68.27%   69.26%   +0.98%     
==========================================
  Files          79       81       +2     
  Lines        7976     8351     +375     
  Branches      829      880      +51     
==========================================
+ Hits         5446     5784     +338     
- Misses       2351     2376      +25     
- Partials      179      191      +12     
Flag Coverage Δ
unittests 69.26% <84.53%> (+0.98%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The field description told the model to emit null where an ordinance
sets one standard for data centers generally, while the size_tiering
instructions told it to record the ordinance's own generic term for the
use. Align both on the latter, which matches the reference workbook:
none of its 147 feature rows leave the field blank, and the 17 untiered
jurisdictions record their own generic term ("Data Center", "Large Data
Center"). Null is now reserved for a provision that attaches no term to
the use at all.
@rajeee
rajeee marked this pull request as draft August 25, 2026 21:49
- database

collection_prompts: true
text_extraction_prompts: true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you find that including this improved extraction? If not, I would recommend leaving this out so that the extractor has access to unfiltered text

Add a normalize_data_center_types post-processing step that blanks
generic labels and shortens accessory types, tighten the schema and
system prompt guidance, narrow query templates, and remove
text_extraction_prompts so the parser sees unfiltered collected text.
Finalize now saves through the plugin's save_structured_data.
Jurisdiction CSVs with a FIPS column are matched on FIPS alone, and
known local docs/URLs are keyed by string codes, falling back to keys
without leading zeros.
Introduce PriorityCrawler, enabled via the priority_search option,
which merges Google and DuckDuckGo results and follows LLM-ranked links
from one shared queue. Replace search-result link expansion and viewer
URL unwrapping with it, and log elapsed time for long-running search,
retrieval, crawl, filtering, and extraction operations.
Take main's final quotes (#487) wording in files only the quotes branch
touched, including the summary prompt in compass/common/base.py, which
otherwise auto-merged back to a superseded version.

Keep timing logs in main's refactored _run_collection_step, keep FIPS
known-source lookup alongside the new rate tracking, and format priority
search queries with _format_queries so {jurisdiction_website} works.

Drop the local SerpAPIDuckDuckGoSearch adapter: nlr-elm 0.0.50 ships an
adapter under the same name, which shadowed ours.
Update test doubles for the combined runtime: collection-step fakes
gain priority_search, and the known-source FIPS fakes gain
search_params for num_search_results_to_crawl.
The quotes branch had registered trim_ordinance_text as a
post-processing step before dropping it; merging the branch in early
kept that registration and its tests alongside the data center step.
Remove them so post_processing.py only adds normalize_data_center_types.

Also apply ruff format, give the priority-search runtime test
placeholder Azure credentials so it does not depend on the local
environment, and ignore the ELM loader docstring references that the
COMPASSWebFileLoader subclass now inherits.
The data center plugin does not need it: normal runs already save through
save_structured_data, which applies post_processing_steps. Keep the export
test on that path only.
@rajeee

rajeee commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by a stack of three PRs that split this work: #606 (event logging and workflow map) → #607 (priority search crawler) → #608 (data center plugin). Stack: https://github.com/NatLabRockies/COMPASS/stack/609. This branch is kept for its full commit history.

@rajeee rajeee closed this Nov 9, 2026

This branch was previously deployed

2 inactive deployments
pypi — d18109e2 Deployed Oct 7, 2026 by rajeee via Build, check, and optionally upload wheels #341
codecov — d18109e2 Deployed Oct 7, 2026 by rajeee via Codecov #514
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants