[gmechali]

config.json is now described by pydantic models (ImportConfig, InputFileConfig in stats/config.py) and validated in one place, parse_config(). pydantic 2.9.2 is already a dependency.

Scope: this models every field config.json accepts today, as it is today, including fields we plan to remove (filename, importName, format, dataDownloadUrl, referenceColumns/columnsToResolve, groupStatVarsByProperty). No documented field is added or dropped here (only the internal, write-only _import_name / _dir_import_names keys are gone). Follow-up PRs will start dropping fields from the model, one at a time, so each removal is its own small, clear change.

Changes

• Unknown keys are still rejected (extra="forbid"). Models are frozen, with camelCase aliases.
• Every problem is reported at once, as one INVALID_CONFIGURATION error. Existing messages are kept (unknown keys with the allowed keys, entities block, format, reference columns); new checks read Invalid '<key>' in 'inputFiles' entry for '<pattern>': <reason>.
• Config keeps its getters and per-file lookup; for_file() and patterns() replace private accessors, and config.data is gone.
• Multi-import runs merge the validated models; workers receive the model instead of re-parsing JSON. The write-only _import_name / _dir_import_names keys are removed.
• groupStatVarsByProperty keeps its lenient parsing, with a deprecation TODO.
• Pins annotated-types and typing-extensions (the installed versions). config.md gets a short "Validation" section.

Configs that passed before and now fail

1. Wrong JSON types (e.g. ignoreColumns: "x", columnMappings: [...], null for a list).
2. Duplicate pattern (the last entry used to win silently).
3. Entities entry without rowEntityType (was an AssertionError).
4. includeInputSubdirs that is not a boolean or a boolean string: true/false and the documented "true"/"false" still work, but "false" now means false (it meant true), and values like "maybe" are rejected.
5. Unsupported importType or ** in a file name now fails when the config is read.
6. Non-list dataDownloadUrl in input-dir mode.

Verification

• New output-level cases in test_config.py for each change above, for error rendering (unknown keys grouped per object, nested locations), and for which inputFiles entry applies to a file. e2e goldens unchanged. Lint clean; full suite passes.
• All 19 UN configs and 5 e2e configs validate.
• UN 10-import subset vs master: identical output, no perf change.

[codacy-production]

Not up to standards ⛔

🔴 Issues 13 minor

Alerts:
⚠ 13 issues (≤ 0 issues of at least minor severity)

Results:
13 new issues

Category	Results
Documentation	8 minor
CodeStyle	5 minor

View in Codacy

🟢 Metrics 35 complexity

Metric	Results
Complexity	35

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[gemini-code-assist]

Code Review

This pull request refactors the configuration parsing and validation logic by adopting Pydantic models (InputFileConfig and ImportConfig) in simple/stats/config.py. This replaces manual dictionary key validation, improves error reporting, and updates the runner, validation, and test suites to integrate with the new Pydantic-based configuration. The review feedback suggests two improvements: checking if value is already an instance of ImportType in _check_import_type to ensure robustness during model copy/update operations, and adding defensive checks when accessing e["ctx"]["error"] in _render_errors to prevent potential KeyError exceptions.

[clincoln8]

Thanks!