Repository navigation
Make public header components acyclic, and forbid new cycles - #8478
Open
Amaury Chamayou (achamayou) wants to merge 6 commits into
Open
Amaury Chamayou (achamayou) wants to merge 6 commits into
Amaury Chamayou (achamayou) wants to merge 6 commits into
Conversation
The source dependency checker only scanned src/, so public headers under include/ccf could still include each other in cycles between components, and could include private headers with angle brackets. Break the three cycles by moving declarations down, without renaming anything: the base64 declarations to ccf/ds/base64.h, the BlitSerialiser<Sha256Hash> specialisation next to its primary template, and the DID and Feed aliases to ccf/pal/uvm_endorsements.h. Assign endpoint.h, endpoint_context.h and common_auth_policies.h to endpoints, tx.h to kv, and ccf/indexing/strategies to its own component. Keep ccf/tx.h providing the typed KV maps, and stop ccf/research/create_tx_claims_digest.h from including the private kv/kv_types.h. The checker now treats public headers as their own layer: they must only include public headers, and their components must not form a cycle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Four new changelog entries lack the required #8478 pull-request reference.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Extends dependency enforcement to public headers and removes existing component cycles without changing runtime behavior.
Changes:
- Adds public-header dependency and private-include validation.
- Relocates declarations and adds compatibility includes.
- Updates release notes and checker documentation.
Custom instructions used: .github/copilot-instructions.md, .github/instructions/reviewing.instructions.md, .github/instructions/changelog.instructions.md, and the formatting/testing skills.
| File | Description |
|---|---|
.github/skills/formatting-and-linting/SKILL.md |
Updates include-check coverage. |
CHANGELOG.md |
Documents header and dependency changes. |
include/ccf/crypto/base64.h |
Forwards to the new DS declaration header. |
include/ccf/crypto/sha256_hash.h |
Removes the KV dependency. |
include/ccf/ds/base64.h |
Adds relocated base64 declarations. |
include/ccf/ds/json.h |
Uses the lower-layer base64 header. |
include/ccf/kv/serialisers/blit_serialiser.h |
Relocates the SHA-256 specialization. |
include/ccf/pal/uvm_endorsements.h |
Owns DID and Feed aliases. |
include/ccf/research/create_tx_claims_digest.h |
Replaces a private include. |
include/ccf/service/tables/uvm_endorsements.h |
Imports aliases from PAL. |
include/ccf/tx.h |
Makes required and compatibility includes explicit. |
scripts/check-source-dependencies.py |
Checks public-header layering and cycles. |
scripts/includes-checks.sh |
Describes expanded dependency checks. |
scripts/source-dependencies.json |
Adds public component ownership overrides. |
src/node/internal_tables_access.h |
Adds its direct table dependency. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ccf/tx.h used to provide ccf/kv/unit_value.h transitively, through ccf/service/map.h. Include it with the other typed KV containers, so that ccf/tx.h and its includers only lose the ccf::Service* aliases, and list ccf::ServiceUnit among them in the changelog. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
8.x is a major release, so public headers no longer need to stay source compatible. Move tx.h to ccf/kv/, and endpoint.h, endpoint_context.h and common_auth_policies.h to ccf/endpoints/, which removes the ownership overrides from the public header dependency check. Declare the base64 functions in ccf::ds, remove ccf/crypto/base64.h, and stop ccf/kv/tx.h from including the typed KV maps for its includers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5506285a45b24ad086e34660c090a7ab
| include_path, path, root, quoted, generated_headers | ||
| ) | ||
| if included_file is None: | ||
| if is_internal(include_path): |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Motivation
Follow-up to #8476, for #3517.
#8476 made the
src/component graph acyclic, butcheck-source-dependencies.pyonly scanssrc/. Underinclude/ccf/, three groups of public header components still include each other in cycles. For example,ccf/ds/json.hincludesccf/crypto/base64.h, andccf/crypto/sha256_hash.hincludesccf/service/map.hfor a single serialiser, so the lowest-level headers pull in the whole typed KV API. Nothing stops new cycles either.ccf/research/create_tx_claims_digest.halso included the private<kv/kv_types.h>, which is not installed, so applications could not use it.includes-checks.shonly checks quoted includes, so it missed this.This targets 8.x, a major release, so the fix does not preserve source compatibility: headers move to the component that owns them, without forwarding headers or aliases.
The resulting public header layering, and the includes this PR removes
Each node is a public header component, a directory under
include/ccf/. Top-level headers are left out: an arrow means that a component's headers include another's, directly or through top-level headers. As in #8476, an arrow is omitted when a longer path implies it.The dashed red arrows are the includes this PR removes. Three pointed up the layering and closed the cycles between
crypto,ds,kv,palandservice, and one reached into a private header.Implementation summary
Public headers are checked as their own layer, not as part of the
src/component of the same name. A component's interface can legitimately be used by components that its implementation depends on:ccf/node_context.hexposes the indexing interface, whilesrc/indexingdepends onnode. The checker now also requires that:include/ccf/is a component, and each top-level header is its own. There are no exceptions or ownership overrides.Since public headers cannot include private ones and the
src/policy is acyclic, no cycle can span the two layers, so the whole include graph between components is acyclic.Headers that belonged to a component but lived at the top level move into it:
ccf/tx.hccf/kv/tx.hccf/endpoint.hccf/endpoints/endpoint.hccf/endpoint_context.hccf/endpoints/endpoint_context.hccf/common_auth_policies.hccf/endpoints/common_auth_policies.hccf/crypto/base64.hccf/ds/base64.hThe remaining cycles are broken as follows:
ds -> crypto): the functions are nowccf::ds::b64_from_raw,b64url_from_raw,raw_from_b64andraw_from_b64url, declared inccf/ds/base64.h.json.hmust keep encodingstd::vector<uint8_t>as base64, sodshas to own the declaration. The implementation stays insrc/crypto/base64.cpp, as it uses the OpenSSL wrappers, whichsrc/dsmay not depend on.BlitSerialiser<Sha256Hash>(crypto -> service): the specialisation moves next to its primary template inccf/kv/serialisers/blit_serialiser.h, soccf/crypto/sha256_hash.hno longer includesccf/service/map.h.pal -> service):ccf/pal/uvm_endorsements.honly used theccf::DIDandccf::Feedaliases ofstd::stringfromccf/service/tables/uvm_endorsements.h. It now usesstd::stringand no longer includes the table header, which is unchanged.indexing -> node_context.h -> indexing):ccf/indexing/strategies/seqnos_by_key_bucketed.hforward-declaresccf::AbstractNodeContext, which it only takes by reference, instead of includingccf/node_context.h.create_tx_claims_digest.h: includesccf/claims_digest.handccf/kv/tx.hinstead of the private header.ccf/kv/tx.hincludes what it uses, and no longer provides the typed KV maps. It gainsccf/kv/abstract_handle.h, which it always needed: it destroys astd::unique_ptr<AbstractHandle>of a forward-declared type, and only compiled because the full type arrived transitively.Two changes under
src/follow from the moves:NetworkTables::js_endpointsis removed. It was unused, and was the only reasonsrc/serviceincludedendpoint.h, which is now inendpoints, a component that depends onservice.rustmay depend onendpointsinsource-dependencies.json, assrc/rust/app_bridge.cppincludes the auth policies header.The two scans share their include parsing, and report source and public violations together, with one include per edge of a cycle. On
main, the checker reports:Validated locally:
unittests pass, as do thee2e_logging,e2e_basic_rustandprogrammability_and_jwte2e tests. The rest of the e2e suite is left to CI.includes-checks.shpasses without overrides, and reported theservice -> endpointsandrust -> endpointsdependencies described above before they were resolved.--fail-on-warning.Safety and compatibility
Runtime behaviour does not change. Every moved declaration and definition is otherwise unchanged, so there is no wire, ledger, KV or JSON format change, and no effect on mixed-version operation or recovery.
Sha256Hashkeys are still serialised as hex, and byte vectors as base64 in JSON. The removedNetworkTables::js_endpointsmember was never read; thepublic:ccf.gov.endpointstable is unaffected.This is a source-breaking change for C++ applications, intended for 8.0 only and not for backport to 7.x. Applications must:
ccf::crypto::base64 calls toccf::ds::, andccf/service/map.h,ccf/node_context.horccf/service/tables/uvm_endorsements.hdirectly where they obtained them transitively throughccf/kv/tx.h,ccf/crypto/sha256_hash.h,seqnos_by_key_bucketed.horccf/pal/uvm_endorsements.h.The 7.x to 8.x migration guide lists these replacements, and the change description links to it.
ccf/app_interface.hstill provides the endpoint registry, authentication policies and transaction types; the samples needed only the include renames.Losing transitive includes cannot silently change behaviour here. The headers that some includers lose contain only two specialisations,
fmt::formatter<ccf::ByteVector>andBlitSerialiser<Sha256Hash>, and each is in the header that declares its type or its primary template. Code that can name the type still sees the specialisation; anything else fails to compile.