Skip to content
gregl83Public

About

πŸŸ©β–³ Bazel Aspect for Computing Output Hashes

Topics

Resources

Stars

5 stars

Watchers

1 watching

Forks

Latest commit

Β 

History

106 Commits

Folders and files

Repository files navigation

Build Release MIT licensed

bazel-paq

Know which Bazel build artifacts changed.

bazel-paq adds a .paq content hash beside each generated directory or file. Your deployment or upload tooling can compare these fingerprints across builds and transfer only changed artifacts.

Hashing is powered by paq, which averaged 30.0 ms to hash the Go programming language repository (157 MB across 14,490 files) in its published standalone benchmark.

No changes to individual build rules are needed. Add a module and a build configuration; the aspect downloads paq automatically.

bazel-bin/service/
β”œβ”€β”€ assets/
β”œβ”€β”€ assets.paq
β”œβ”€β”€ server
β”œβ”€β”€ server.paq
β”œβ”€β”€ config.json
└── config.json.paq

Change a configuration file, and its hash changes. Leave the service binary unchanged, and its hash stays the same. Your tooling can upload only what changed.

Each .paq contains one JSON string: a BLAKE3-based fingerprint. Directory outputs get one recursive hash, stored outside the directory. Multiple outputs from one target each get their own hash.

Quick Start

Add the module to MODULE.bazel:

bazel_dep(name = "bazel_paq", version = "2.0.0")

Add a build configuration to .bazelrc:

build:paq --aspects=@bazel_paq//:defs.bzl%paq_aspect
build:paq --output_groups=+paq_files

Build your targets:

bazel build --config=paq //...

To run without a .bazelrc configuration:

bazel build //... --aspects=@bazel_paq//:defs.bzl%paq_aspect --output_groups=+paq_files

Example Workspace

The example workspace demonstrates configuration files, infrastructure templates, a Python service, and a Rust executable. It includes directory outputs, directory and file symlinks, empty artifacts, and a target with multiple outputs.

From a checkout of this repository:

cd example
bazel build --config=paq //...

See its build output for the complete layout. The tree at the top of this README is a simplified illustration.

Aspect Output

The aspect hashes generated artifacts exposed through DefaultInfo.files and returns their sidecars in the paq_files output group.

Output artifact Hash file What is hashed
Directory artifact <directory-name>.paq, beside the directory Recursive contents and relative paths
Regular or executable file <filename>.paq File contents
Symlink to a directory <link-name>.paq Referenced directory tree
Symlink to a file <link-name>.paq Referenced file contents
Source file or source symlink None Skipped
  • Independent outputs: each generated artifact gets its own hash. A directory artifact gets one hash for its tree, rather than a hash for each child.
  • Followed symlinks: paq 2.0.0 runs with --follow. Referents must be available through declared outputs or dependencies. Broken links and cycles fail the build. A link and its referent have the same fingerprint.
  • Shared artifacts: filegroups and other forwarding rules reuse hashes from their dependencies, including across packages.
  • Content fingerprints: hidden entries and empty artifacts are included. Permissions, ownership, timestamps, and other filesystem metadata are excluded.

The .paq suffix is reserved for aspect-generated hashes. Build rules must not produce conflicting paths.

Bazel may omit empty subdirectories when materializing directory artifacts in a sandbox or from a cache. Use an archive output when preserving those empty subdirectories matters.

Your deployment system chooses how to group artifacts and act on changes. bazel-paq supplies the fingerprints as part of the build. Consumers track artifact membership and removals using the current build's output inventory, rather than assuming every file left in bazel-bin is current.

Warning

Only consume hashes from a successful build of the paq_files output group. Check that bazel build --config=paq ... exits with code 0 before using its hashes. A failed or cancelled build can leave older .paq files in place; their presence does not indicate success.

Verify a Hash

With paq 2.0.0 installed:

paq --follow bazel-bin/service/server
cat bazel-bin/service/server.paq

The printed fingerprint should match the JSON string in the .paq file. Use the same paq version as the aspect; its fingerprints are not raw b3sum checksums.

Upgrading from v1

paq v2 fingerprints are incompatible with v1. Regenerate your stored baselines. Targets with multiple outputs now receive one adjacent hash per artifact instead of a shared .paq file. Update consumers accordingly and start with a clean output tree to avoid discovering obsolete hash files.

Development

Before releasing changes, run the manual release tests.

Run the test suite:

bazel test //tests/... --test_output=all

Verify hashes against standalone paq --follow runs, directory and file changes, symlink failures, local execution, and disk-cache restoration in a temporary consumer workspace:

python tests/integration_test.py

CI runs this check on Linux, macOS, and Windows. On Windows, set BAZEL_SH to your Bash executable and enable symlink creation, as in the CI configuration.

Live comparisons cover nested and empty directories, hidden entries, empty and binary files, executables, hard-link outputs, unusual filenames, and directory/file symlinks (including chains, empty referents, and links inside directories). They also check content changes, link retargeting, metadata independence, and cache restoration. Broken links, cycles, and failures after a successful build run as separate tests, checking both Bazel and standalone paq.

POSIX checks cover unreadable directories/files and Bazel's rejection of FIFOs, sockets, and device links inside directory outputs. These report skips on Windows; permission checks also skip when running as root. They do not create privileged device nodes or exercise Windows ACLs. Source-artifact exclusions and graph behavior remain covered by bazel test.

Run a single integration case by name, for example:

python tests/integration_test.py IntegrationTests.test_broken_file_link

License

MIT

About

πŸŸ©β–³ Bazel Aspect for Computing Output Hashes

Topics

Resources

Stars

5 stars

Watchers

1 watching

Forks

Releases

Used by

Contributors

Languages