Skip to content

feat(nvidia): export Auto Ontology terms as an Ossie ontology and mapping - #590

Merged
jbonofre merged 1 commit into
apache:mainfrom
liofnv:feat/nvidia-ontology-export
Oct 16, 2026
Merged

jbonofre merged 1 commit into
apache:mainfrom
liofnv:feat/nvidia-ontology-export

Conversation

@liofnv

@liofnv liofnv commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Builds on #589. Please review only the last commit (4e9d27d); the first commit is #589, and this diff shrinks to that commit once #589 merges.

Auto Ontology's terms are business concepts, its attributes are properties of those concepts, and its foreign keys link one concept to another. The converter wrote only the semantic model, so an exported Auto Ontology model showed no concepts, relationships or mappings. The import command can now also write an Ossie ontology and a mapping document that populates it from the semantic model:

uv run ossie-nvidia-auto-ontology import \
  --input tpch.auto_ontology.yaml \
  --output tpch.semantic_model.yaml \
  --ontology-output tpch.ontology.yaml \
  --mapping-output tpch.mapping.yaml

The same is available as convert_auto_ontology_to_ossie_ontology. The mapping references both other documents by name and, from the CLI, by a path relative to itself.

How the ontology is read off the semantic model

Semantic model Ontology Mapping
Dataset Entity concept, named in PascalCase One concept mapping
Primary key, else the first unique key identify_by Object mappings: one expression for a single-field key, referent mappings otherwise
Field with a portable datatype ManyToOne relationship to the built-in value type, or OneToOne when the field alone identifies the concept Link mapping from the field
Relationship ManyToOne relationship to the referenced concept Link mapping to the referenced concept's objects

A key made of a link and a field, such as a TPC-H line item's order and line number, is mapped through nested referent mappings, as in the specification's own OrderLineItem example.

Nothing is guessed:

  • A field whose type is unknown, or has no built-in value type (Time, Opaque), is left out.
  • A concept whose dataset has no usable key is declared but not mapped.
  • Relationships are verbalized from their field names ({Customer} has Account Balance {Decimal}), because Auto Ontology records no verbalizations.

Export changes the mapping needs

  • Names. Auto Ontology keeps term names unique across every database it catalogs, so it renames a term to Name (database.schema.table) when another database already uses the name. An Ossie document is its own namespace, so the dataset is exported as Name wherever that is free in the document. The term keeps its full name on the way back.
  • Key columns. Auto Ontology gives a foreign-key column no attribute. A key or relationship column that no field exposes is now exported as a field, so the mapping can reference it. It is marked so it does not become an attribute on the way back.

Testing

  • uv run pytest in converters/nvidia: 118 passed, including 13 new tests (10 in tests/test_ontology.py).
  • The three documents the CLI writes pass validation/validate.py against the core, ontology and mapping schemas (test_cli_writes_three_documents_that_pass_official_validation).
  • The generated ontology parses with converters/ontology's OssieParser.
  • A live export of SNOWFLAKE_SAMPLE_DATA.TPCH_SF1 gives 8 concepts and 64 relationships. 6 of the 8 concepts are mapped: LINEITEM and PARTSUPP have no key, because Snowflake declares none and their keys are composite. With the TPC-H primary keys declared, all 8 are mapped.

AI disclosure

This contribution was prepared with AI assistance (Claude Code), per the ASF Generative Tooling Guidance.

@liofnv

liofnv commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

@jbonofre

@liofnv
liofnv force-pushed the feat/nvidia-ontology-export branch from 4e9d27d to 678c075 Compare October 16, 2026 18:08
@jbonofre
jbonofre self-requested a review October 16, 2026 19:13
jbonofre
jbonofre previously approved these changes Oct 16, 2026

@jbonofre jbonofre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @liofnv for this PR!

The ontology and standalone mapping report (ontology.py) is very clean, well-aligned with the ontology/mapping specifications, and the round-trip handling for database-qualified term names and key_column fields is well thought out. Great test coverage with the schema validator as well.

Two minor observations for potential follow-up:

  1. In _identity() (ontology.py), checking set(relationship["from_columns]") <= set(key) in addition to column in relationship["from_columns"] would prevent a composite foreign key from being treated as an identifying link when only a subset of its columns is part of the dataset's key.
  2. In _referent_mappings(), tracking a visited set of concepts when recursing through identifying link would guard against RecursionError if a schema ever has cyclic identifying relationships.

LGTM 👍

Let's merge #589 first once CI completed, then I will merge this one.

…ping

Auto Ontology's terms are business concepts, its attributes are properties
of those concepts, and its foreign keys link one concept to another, but the
converter wrote only the semantic model. The import command now also writes
an Ossie ontology and a mapping document that populates it from that model
(--ontology-output, --mapping-output; convert_auto_ontology_to_ossie_ontology).

Each dataset becomes an entity concept identified by its primary key, or its
first unique key. Each field with a portable datatype becomes a relationship
to the built-in value type, and each relationship a many-to-one link to the
referenced concept. A key made of a link and a field is mapped through
nested referent mappings, as in the specification's own examples. A concept
whose dataset has no usable key is declared but not mapped.

Two export changes make that possible:

- A term Auto Ontology renamed `Name (database.schema.table)` to keep names
  unique across databases is exported as `Name` where that is free in the
  document, and keeps its full name on the way back.
- A key or relationship column that no field exposes, such as a foreign-key
  column Auto Ontology gives no attribute, is exported as a field, so the
  mapping can reference it. It is marked so it does not become an attribute
  on the way back.
@jbonofre
jbonofre force-pushed the feat/nvidia-ontology-export branch from 678c075 to e95f71f Compare October 16, 2026 19:32
@jbonofre
jbonofre merged commit 64a1ae4 into apache:main Oct 16, 2026
8 checks passed
liofnv added a commit to NVIDIA/auto-ontology that referenced this pull request Oct 16, 2026
The ontology export merged upstream in apache/ossie#590, so the converter no longer comes from a fork.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants