Skip to content

Preserve LLVM archive alternatives and add feature changelog - #878

Merged
helly25 merged 2 commits into
bazel-contrib:masterfrom
helly25:fix/distribution-helper-changelog
Oct 1, 2026
Merged

helly25 merged 2 commits into
bazel-contrib:masterfrom
helly25:fix/distribution-helper-changelog

Conversation

@helly25

@helly25 helly25 commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Fix the remaining xz/zstd mismatch in the single-release distribution helper and add a history-backed changelog.

AG;DR

Fix

#870 changed the Python updater to retain all archive formats and made the toolchain selector exclude zstd until Bazel can extract LLVM's large-window archives. The shell helper still had #836's old filter, which discarded xz whenever the same archive was available as zstd.

  • Retain xz, zstd, and gzip archives and their individual checksums in extra_distributions.sh.
  • Leave extraction compatibility decisions in the toolchain selector; this does not enable zstd selection.
  • Correct the README's stale zstd-preference claim, updater scope, and dependencies.

Changelog

Add CHANGELOG.md with release headings and bullets, newest first. Historical feature introductions are checked against the first containing release tags; current changes are grouped under the version heading 1.11.0. Distinguish earlier functionality from later enhancements, including Linux versus macOS ThinLTO, initial versus expanded dynamic-libstdc++ support, and MSan introduction versus correctness fixes.

Regression coverage

  • Execute the real shell helper against offline release metadata, allowing only the metadata request and rejecting unexpected archive downloads.
  • Cover stable and prerelease versions, both LLVM naming schemes, xz/zstd/gzip alternatives and individual formats, exact checksum preservation, unrelated-asset filtering, and no-match errors.
  • Extend the Python updater test to preserve all three formats and checksums.
  • Run both script suites in a Linux/macOS CI job and include it in required_checks_done.needs.

Verification

  • Demonstrated the new shell regression fails before the fix in all four stable/prerelease and naming-scheme combinations, specifically because xz is missing.
  • python3 -m unittest discover -s utils -p '*_test.py' -v: all 8 tests pass on macOS.
  • USE_BAZEL_VERSION=9.0.0 bazel test //... --test_output=errors --nocache_test_results: all 21 tests executed and passed on the rebased head.
  • bash -n for both shell entry points.
  • Repository lint/push hooks, git diff --check, and YAML validation that the required-check gate includes every declared job and has no unknown dependencies.

Fresh CI: Tests run 36820892858. Protected squash auto-merge is enabled; this is the final PR in the #873 → #875 → #876 → #877 → #878 sequence. The changelog heading remains ## 1.11.0.

helly25 added a commit that referenced this pull request Oct 1, 2026
# Enable linker-managed ThinLTO on macOS

Fix thin_lto feature on MacOS.

# AG;DR

## Problem

`--features=thin_lto` was silently ineffective on macOS: rules_cc only
defines the ThinLTO feature on its non-Darwin path. Explicit
`-flto=thin` compile and link flags work with Apple ld, but the
toolchain did not provide them.

Simply exposing the literal C++ `thin_lto` feature on Darwin is not
sufficient. It activates Bazel's distributed indexing/backend path,
which requires start/end-lib support and ELF indexing options that Apple
ld does not provide.

## Fix

- Translate configuration-level `--features=thin_lto` into
linker-managed ThinLTO on Darwin, applying `-flto=thin` to compilation
and linking without activating ELF indexing actions.
- Respect explicit `--features=-thin_lto`, including when both enable
and disable requests are present.
- Keep target and execution configurations independent: build tools use
`--host_features=thin_lto`.
- Leave Linux's existing distributed ThinLTO behavior unchanged.
- Document the macOS configuration-level scope: rule-local `features`
attributes do not select this behavior. LLVM 23 still needs the existing
native-linker override on macOS.

## Regression coverage

- Analyze C compilation, C++ compilation, and linking; assert the flags
are present only when enabled and no ELF-style indexing/backend actions
are introduced.
- Build and run a C/C++ cross-translation-unit probe with LLVM 23.1.2
and Apple ld. It contains an unresolved call that disappears only after
optimization through both static libraries, so an ordinary non-LTO build
cannot pass.
- Verify the same probe fails with the expected undefined symbol when
ThinLTO is disabled, and when only target features are enabled for an
execution-configured tool.
- Verify `--host_features=thin_lto` enables the tool independently of
disabled target ThinLTO.
- Add a required macOS CI job for Bazel 9/Bzlmod and Bazel 8/WORKSPACE,
including the corresponding `required_checks_done.needs` entry.

## Local verification

On Apple Silicon, repeated after rebasing onto merged #876 (`3764511`):

- Bazel 9.0.0 / Bzlmod / LLVM 23.1.2: full
`run_darwin_thin_lto_tests.sh` passed (six cached test results, both
expected-failure checks, and the host-feature build).
- Bazel 8.6.0 / WORKSPACE / LLVM 23.1.2: the same full regression script
passed (six cached test results, both expected-failure checks, and the
host-feature build).
- #876's feature-matrix fixture: `--features=thin_lto //:selection_test`
passed with Bazel 9 and Apple ld.
- Root Bazel 9.0.0 `bazel test //... --nocache_test_results`: all 20
tests executed and passed.
- `trunk check --no-fix` and the commit/push hooks passed.

The Linux distributed ThinLTO fixture also passed in the existing
container after #876's signed-feature API update. The rebase leaves the
ThinLTO patch unchanged (`git range-diff` confirms this).

## CI cleanup follow-up during sequential merging

- [#875's post-merge Ubuntu/Bazel 9
job](https://github.com/bazel-contrib/toolchains_llvm/actions/runs/36803938052/job/110184071511)
passed its native tests, then failed deleting the shared repo contents
cache before the WebAssembly phase: `Directory not empty`.
- Its log shows separate root-workspace and tests-workspace Bazel
servers, with an orphan Java process still alive after cleanup. Change
the initial banner from `bazel version` to client-only `bazel
--version`, removing that unnecessary idle root server. [Idle servers
can run repo-cache garbage
collection](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryOptions.java),
making this a likely source of the cleanup race. Compiler/linker
configuration and cache-cleanup error handling are unchanged.
- An isolated local probe verified that the old `version` command starts
a server, while standalone `--version` succeeds on Bazel 8.6.0 and 9.0.0
without creating an output root. The probe server was shut down; `bash
-n`, `git diff --check`, and the commit/push hooks passed.

Current head: `0d84b16` (unchanged ThinLTO patch plus the CI banner
fix). Fresh CI: [Tests run
36810263471](https://github.com/bazel-contrib/toolchains_llvm/actions/runs/36810263471).
Protected squash auto-merge is enabled; #878 will be rebased only after
this PR merges.
Keep xz, zstd, and gzip checksums in the extra-distributions helper, leaving archive compatibility decisions to the toolchain selector. Exercise the actual shell helper with offline release fixtures, extend the updater regression, and run both suites in required Linux/macOS CI.

Correct stale updater documentation and add a release-heading changelog separating verified historical feature introductions from the planned 1.11.0 additions. Stacked on PR bazel-contrib#877.
@helly25
helly25 force-pushed the fix/distribution-helper-changelog branch from 3f458d4 to 4f94d11 Compare October 1, 2026 05:40
@helly25
helly25 enabled auto-merge (squash) October 1, 2026 05:40
@helly25
helly25 merged commit ea8c191 into bazel-contrib:master Oct 1, 2026
97 checks passed
@helly25
helly25 deleted the fix/distribution-helper-changelog branch October 1, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant