Skip to content

✨ Give Graph.metadata a schema - #181

Open
elinscott wants to merge 10 commits into
scinode:mainfrom
elinscott:declared-metadata-keys
Open

elinscott wants to merge 10 commits into
scinode:mainfrom
elinscott:declared-metadata-keys

Conversation

@elinscott

@elinscott elinscott commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #180 (removes graph_type); review the commits after it.

TL;DR

from node_graph import Graph
from node_graph.graph import GraphMetadata
from pydantic import ConfigDict

class StrictMetadata(GraphMetadata, total=False):
    __pydantic_config__ = ConfigDict(extra="forbid")
    my_key: int

class StrictGraph(Graph):
    _metadata_schema = StrictMetadata

StrictGraph(name="g", metadata={"my_key": 1})        # fine
StrictGraph(name="g", metadata={"my_key": "one"})    # ValueError: my_key: Input should be a valid integer
StrictGraph(name="g", metadata={"typo_key": 1})      # ValueError: typo_key: Extra inputs are not permitted

Problem

Graph.metadata has no schema: any key a caller passes at construction is silently accepted, stored, and round-tripped, whether or not anything downstream ever reads it.

  • Fine for Graph itself, whose own reserved keys are few and well known.
  • Bad for downstream aiida-workgraph: WorkGraph also uses metadata to carry AiiDA's own process-launch keys (label, description, ...), and inheriting node-graph's schema-less dict meant an AiiDA user writing the obvious WorkGraph(name, metadata={'label': ...}) got silently the wrong metadata — accepted without complaint, stored as graph bookkeeping, and never properly passed on as AiiDA metadata (metadata means two different things, and the constructor accepts the wrong one silently aiidateam/aiida-workgraph#812).

node-graph isn't at fault for not knowing what label means to AiiDA, but without a schema that declares which keys Graph itself reserves, a downstream subclass has no way to distinguish "this key is mine" from "this key is a typo" either, so it can't validate an extended metadata schema.

Changes

The schema

GraphMetadata is a total=False TypedDict naming the keys Graph writes for its own serialization bookkeeping, and what each one holds. It carries extra="allow", so a bare Graph keeps behaving as it does today for callers who stash arbitrary keys — but a declared key still has to hold the right thing.

from typing import Any, Dict, TypedDict
from pydantic import ConfigDict
from node_graph import Graph

class GraphMetadata(TypedDict, total=False):
    __pydantic_config__ = ConfigDict(extra="allow", strict=True)

    graph_class: Dict[str, Any]
    definition: Dict[str, Any]

Graph(name="g", metadata={"anything": 1})                # ✅ unknown key, kept
Graph(name="g", metadata={"graph_class": "not a dict"})  # ❌ ValueError

strict=True refuses type coercion, not just unknown keys: a declared field holding a value of the wrong type is rejected rather than converted (a bool field given "yes" or 1, an int field given a numeral spelled as a string).

Narrowing in a subclass

A subclass declares its own schema by inheriting a wider TypedDict from GraphMetadata and pointing _metadata_schema at it. extra="forbid" on that TypedDict is what turns a typo into an error; the schema and the strictness travel together, so there is one thing to declare rather than two.

from typing import Optional
from pydantic import ConfigDict
from node_graph import Graph
from node_graph.graph import GraphMetadata

# Simplified for illustration: aiida-workgraph's real WorkGraphMetadata composes
# GraphMetadata with a second base, EngineLaunchMetadata, carrying AiiDA's own
# process-launch keys (see aiida-workgraph's `unified-metadata` branch).
class WorkGraphMetadata(GraphMetadata, total=False):
    __pydantic_config__ = ConfigDict(extra="forbid")
    pk: Optional[int]

class WorkGraph(Graph):
    _metadata_schema = WorkGraphMetadata

Validated at the boundaries, not on every mutation

metadata is a plain dict — no custom dict class, no setter that checks, nothing to keep in step. Graph.validate_metadata() runs a cached TypeAdapter and returns a validated copy, and it is called at three seams: construction, from_dict(), and get_metadata(), whose result to_dict() writes. So a stray key set after construction is not refused at the line that sets it; it is refused the moment the graph is serialized.

ng = StrictGraph(name="g", metadata={"my_key": 1})  # continuing StrictGraph from the TL;DR above
ng.metadata["typo_key"] = 1      # quiet (but would be flagged by typechecking)
ng.to_dict()                     # ❌ ValueError: typo_key: Extra inputs are not permitted

Graph.metadata is new here too: a plain property over _metadata, so downstream code and subclasses stop reaching for a private name.

Reconstruction validates like construction

from_dict() runs the same check, naming the graph it could not load. For a validating subclass this is a deliberate break: a serialized graph carrying a key outside the current declared set (schema drift, or a stray key from an older version) now raises on load instead of loading silently, and affected data needs that key stripped before it will load again.

clean = StrictGraph(name="g", metadata={"my_key": 1})
data = clean.to_dict()
data["metadata"]["worker_name"] = "localhost"   # written by no current code
StrictGraph.from_dict(data)   # ❌ ValueError: Invalid metadata for graph 'g'. Valid keys: [...]. worker_name: Extra inputs are not permitted

Upstream

Paired with aiidateam/aiida-workgraph's unified-metadata branch, which composes GraphMetadata with AiiDA's own process-launch port names (introspected from WorkGraphEngine's spec) and forbids extras. That PR implements the wiring half of #812; this is the schema primitive it's built on.

elinscott and others added 3 commits August 26, 2026 18:39
Nothing in node-graph or aiida-workgraph ever read self.graph_type;
its only observed value was the "NORMAL" default. Drop the
constructor argument, the attribute, and its slot in
get_metadata()/to_dict(); from_dict() now discards a stored
graph_type key instead of round-tripping it, so old serialized
graphs still load.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A downstream package (aiida-workgraph) needs to know which metadata
keys a Graph subclass already reserves for its own serialization
bookkeeping, so it can validate its own metadata schema without
guessing or hand-coding node-graph's internals (aiidateam/aiida-workgraph#812).

- Add `Graph._declared_metadata_keys`, a frozenset class attribute
  (`{"graph_class", "definition"}` on the base class) that subclasses
  extend by union. `definition` is the key `utils/graph.py` writes
  when a graph is built from a `@task.graph`-decorated function.
- Add `Graph._validate_metadata_keys`, an opt-in flag (default
  `False`). When a subclass sets it `True`, a `metadata=` key outside
  `_declared_metadata_keys` raises `ValueError` at construction,
  naming the valid set. Base `Graph` stays permissive, matching
  today's behavior for callers who stash arbitrary keys in
  `metadata`.
- Add `_skip_metadata_validation`, an internal constructor flag
  `from_dict()` sets when reconstructing a graph. Metadata keys an
  older version wrote (or a subclass's declared set has since
  dropped) still load; only fresh, caller-driven construction
  validates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`_skip_metadata_validation` let from_dict() silently reload metadata
keys outside a validating subclass's declared set; from_dict() now
validates exactly like fresh construction, so stale keys raise instead
of loading.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.78%. Comparing base (8d85e61) to head (488da15).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #181      +/-   ##
==========================================
+ Coverage   89.68%   89.78%   +0.10%     
==========================================
  Files          81       81              
  Lines        8984     9073      +89     
==========================================
+ Hits         8057     8146      +89     
  Misses        927      927              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@elinscott
elinscott marked this pull request as draft August 26, 2026 17:09
elinscott and others added 4 commits August 26, 2026 19:24
The key-name registry accepted a declared key holding any value, and
accepted any key added to `_metadata` after construction, so both
reached `to_dict()` unremarked.

- Add `GraphMetadata`, a `total=False` TypedDict naming `graph_class`
  and `definition`, carrying `extra="allow"` so a bare `Graph` still
  keeps any other key a caller stashes in `metadata`.
- Point the new `Graph._metadata_schema` attribute at it. A subclass
  narrows by inheriting a wider TypedDict from `GraphMetadata` and
  overriding `__pydantic_config__` with `extra="forbid"`.
- Add `Graph.validate_metadata()`, returning a validated copy from a
  cached `TypeAdapter`, and call it at three seams: `__init__`,
  `from_dict()` (naming the graph in the error) and `get_metadata()`,
  whose result `to_dict()` writes.
- Remove `_declared_metadata_keys`, `_validate_metadata_keys` and
  `_validate_metadata`; `_metadata` is a plain dict between seams.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reaching a graph's metadata meant touching `_metadata`, a private name,
which downstream subclasses were already doing.

- Add a `Graph.metadata` property, reading and writing `_metadata`
  itself: a plain dict, unchecked on mutation and on whole-dict
  reassignment, with `_metadata_schema` applied at the seams as before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`utils/graph.py` reached into `_metadata` to record a built graph's
callable identity.

- Use the `Graph.metadata` property instead; same dict, public name.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GraphMetadata accepted values pydantic would coerce into a declared field's type (a bool field given "yes" or 1, an int field given a numeral spelled as a string) instead of refusing them, and each validation seam raised a bare pydantic.ValidationError naming neither the graph nor the schema's valid keys.

- Add strict=True to GraphMetadata.__pydantic_config__, refusing type coercion alongside the existing extra check.
- Add MetadataValidationError (a ValueError) naming the graph and listing _metadata_schema's valid keys; validate_metadata raises it instead of the raw pydantic error, threaded through __init__, get_metadata, and from_dict.
- Update tests/test_graph.py's pytest.raises expectations from pydantic.ValidationError to ValueError to match, and the from_dict message assertion to the new wording.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@elinscott elinscott changed the title ✨ Add Graph._declared_metadata_keys & opt-in metadata key validation ✨ Give Graph.metadata a schema: a TypedDict validated at the boundaries Aug 27, 2026
@elinscott elinscott changed the title ✨ Give Graph.metadata a schema: a TypedDict validated at the boundaries ✨ Give Graph.metadata a schema Aug 27, 2026
elinscott and others added 3 commits August 27, 2026 11:07
Three tests each declared the same StrictMetadata/StrictGraph pair
inline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A module-level lru_cache stood between a subclass and its own
validator; the adapter now appears on each Graph subclass at
class definition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@elinscott
elinscott marked this pull request as ready for review August 27, 2026 09:49
elinscott added a commit to elinscott/node-graph that referenced this pull request Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant