[mattiasbrand]

Summary:

Make the Buck CUDA runtime link its AOTI shims the way the CMake build already does.

A compiled AOTInductor shared library does not contain the tensor runtime. When the CUDA backend loads it, the library resolves its aoti_torch_* shim calls from the host process by name. Nothing in the host code calls most of these functions directly, so a static link without whole-archive drops them from common_shims_slim and loading fails with unresolved symbols. CMake already links aoti_common_shims_slim with --whole-archive; this sets link_whole = True on the Buck target to match, along with supports_python_dlopen = True like the other force-linked AOTI and CUDA targets. An executable that loads an AOTI library also has to export these symbols, as CMake does with -Wl,--export-dynamic. That includes a Buck python_binary that uses the Python bindings, because the extension links into its native executable.

backends/aoti/tests:test_common_shims_slim_link looks every shim up by name in a statically linked test binary that never calls them. It fails for every shim when link_whole is removed. test_common_shims_slim cannot catch this, because it calls the shims directly and that keeps them linked.

backends/cuda/runtime/shims/tensor_attribute.cpp defined aoti_torch_get_device_type and aoti_torch_device_type_cuda for ETensor, which the CUDA backend no longer uses. common_shims_slim defines the same extern "C" symbols for SlimTensor, so once both are force-linked they collide. CMake already leaves this file out of the CUDA shim sources. This deletes the file and its header and removes them from the Buck target.

executorch_pybindings(use_cuda = True) compiles code that calls the CUDA runtime directly but did not declare the CUDA dependency, so such a target could not link. The macro now adds it when use_cuda is set, together with //executorch/extension/cuda:cuda_allocator, which pybindings_cuda.h includes and calls. Previously that dependency was only met indirectly through the AOTI CUDA backend.

//executorch/extension/pybindings:cuda_lib builds these bindings with the AOTI CUDA backend. LibTorch defines the same aoti_torch_* symbols with a different tensor handle ABI, so it cannot link into the same binary. extension/pybindings/test:test_pybindings_cuda_lib exports a CUDA-delegated program with external weights, runs it through cuda_lib in a separate LibTorch-free binary, and compares the output with eager PyTorch. Without link_whole, or without -Wl,--export-dynamic on that binary, it fails with undefined symbol: aoti_torch_dtype_float32. pytest.ini and pytest-windows.ini leave it out of the pytest runs, because it needs a GPU and the Buck-built runner.

Differential Revision: D129530257

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24643

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job, 4 Unrelated Failures

As of commit f0fdaa7 with merge base e0015e1 ():

CANCELLED JOB - The following job was cancelled. Please retry:

• Test CUDA Builds / test-muse-glimmer-cuda-e2e-kquant-17gb-dflash-image-batching / linux-job (gh)
  Could not load credentials from any providers

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Core AI / backend-tester (operators, coreai_w8a8) / test-coreai_w8a8-operators (gh) (trunk failure)
  AssertionError: 24 tests failed; at most 23 allowed
• Test CUDA Builds / check-all-cuda-builds (gh) (trunk failure)
• Test CUDA Builds / test-executorch-cuda-build-12.6 / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test Vulkan Backend (specialized runners) / test-model-vulkan-e2e / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: mattiasbrand / name: Mattias Brand (4b522bd)

[meta-codesync]

@mattiasbrand has exported this pull request. If you are a Meta employee, you can view the originating Diff in D129530257.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shoumikhin]

@claude /pr-review

[claude]

Claude finished @shoumikhin's task in 16m 12s —— View job

---

No blockers found, but common_shims_slim is the only force-linked target missing supports_python_dlopen, the use_cuda pybindings path still lacks a direct cuda_allocator dependency, and nothing in CI exercises the fix.

Detailed review

PR Review: #24643

Summary

This PR force-links common_shims_slim in Buck so that compiled AOTI libraries can bind its aoti_torch_* shims. It also deletes the ETensor tensor_attribute shims that now collide with it, and declares the CUDA dependency for executorch_pybindings(use_cuda = True). Nothing is confirmed broken. Two items below are one-line build fixes, one is an open question about matching CMake's symbol export, and there is no test or test plan.

Infrastructure

1. common_shims_slim is the only link_whole library without supports_python_dlopen = True (backends/aoti/targets.bzl:103-104)

   • Every other force-linked target sets both flags:
     • in this file: common_shims, delegate_handle, aoti_common, aoti_common_slim (:53-54, :71-72, :89-90, :127-128);
     • in backends/cuda/runtime/targets.bzl: cuda_platform, runtime_shims, cuda_backend (:19-20, :55-56, :113-114).
   • This is the target where the flag is most likely to matter. Most of its symbols are only bound at load time, and the consumer this PR targets, executorch_pybindings(use_cuda = True), is a Python extension.
   • Add the flag, or say why this target is different.
   • The mismatch is confirmed. What Buck's Python native linking does without the flag is not: the prelude isn't in this repo. Fix this →

2. Buck reproduces only the whole-archive half of the CMake setup (backends/aoti/targets.bzl:103-104). Low confidence, please confirm.

   • CMake does two things:
     • It whole-archives aoti_common_shims_slim into the shared library aoti_cuda_shims (backends/cuda/CMakeLists.txt:263, :316-321).
     • It adds -Wl,--export-dynamic as a PUBLIC link option on aoti_cuda_shims and aoti_cuda_backend (:295-297, :406-408).
   • No .bzl, BUCK, or TARGETS file in the repo sets --export-dynamic or -rdynamic.
   • The AOTI .so is opened with RTLD_LAZY | RTLD_LOCAL (backends/cuda/runtime/platform/platform.cpp:102). Its undefined aoti_torch_* references can only bind to symbols the host exports, and AOTI_SHIM_EXPORT is empty outside Windows (backends/aoti/export.h:25).
   • Result: a statically linked cxx_binary or cpp_unittest that depends on cuda_backend may still fail to load a model, unless the fbcode toolchain exports executable symbols by default.
     • Shared objects such as the pybindings extension are not affected.
     • The same gap already applies to the force-linked runtime_shims.
   • Separately, the new lint-ignore reason says AOTI libraries resolve this API "with dlsym". They never call dlsym. The dynamic loader binds their undefined references, which is why exporting the symbols matters. Fix this → (rewords the comment only)

3. use_cuda = True still leaves out a direct dependency of the code it turns on (shim_et/xplat/executorch/extension/pybindings/pybindings.bzl:82)

   • With -DEXECUTORCH_PYBINDINGS_CUDA, extension/pybindings/pybindings_cuda.h:17 includes executorch/extension/cuda/cuda_allocator.h, and :30 calls CudaAllocator::instance().
   • The macro adds only //executorch/extension/cuda:caller_stream, which does not export cuda_allocator (extension/cuda/targets.bzl:28-57).
   • It builds today only because a cuda_backend in cppdeps re-exports it through slimtensor → storage (backends/cuda/runtime/targets.bzl:125, backends/aoti/slim/core/targets.bzl:23).
   • A use_cuda extension without the AOTI CUDA backend fails on the missing header. One example is an extension that only uses TensorRT through caller_stream (extension/cuda/targets.bzl:23-24).
   • This is the same kind of undeclared dependency the PR fixes on line 85. Add //executorch/extension/cuda:cuda_allocator next to caller_stream. Fix this →

Testing

• Nothing in CI exercises these changes, and the PR has no test plan.
  • The failure only shows up when a compiled AOTI library is loaded, and no Buck test does that. There is no load_library or dlopen call in backends/cuda/runtime/test/ or backends/aoti/tests/.
  • test_common_shims_slim calls aoti_torch_get_device_type directly (backends/aoti/tests/test_common_shims_slim.cpp:417). That keeps the symbol linked whether or not link_whole is set, so the test cannot catch this bug.
  • No target in the repo passes use_cuda = True, so the pybindings.bzl change is never built here.
  • As a result, removing link_whole again would pass CI.
  • CONTRIBUTING.md:345-347 asks for tests with bug fixes, and :403-408 asks for repeatable test instructions.
  • Please add a statically linked Buck test that loads an AOTI library. If that isn't practical, put the internal command that reproduced the failure and verified the fix in the PR description.

Recommendation

Needs Discussion

No change is confirmed broken. But CI can't exercise any of this, so correctness depends on the author's internal verification. Please say:

• how the Buck path was tested;
• whether common_shims_slim needs supports_python_dlopen (item 1);
• whether it needs executable symbol export (item 2).

The cuda_allocator dependency (item 3) is a one-line follow-up and does not block.

Review steps

• Gather context (diff, CLAUDE.md, CONTRIBUTING.md, review checklist)
• Run three independent reviewers over all 5 changed files
• Consolidate findings
• Fact-check each finding against the code
• Post review
• Not done: building the Buck targets. This runner has no fbcode toolchain, Buck prelude, or GPU, so the runtime effects in items 1 and 2 could not be checked.

[shoumikhin]

@Gasoonjia could you take a look? The idea looks right to me: it matches the CMake whole-archive link, and nothing else in the CUDA backend uses the deleted file. A few things before it's ready:

1. common_shims_slim is now link_whole but does not set supports_python_dlopen = True, unlike every other link_whole target in these two files.
2. The new lint-ignore comment says AOTI libraries find these functions with dlsym. They don't. The host uses dlsym to find the model's entry points, and the dynamic loader binds the model's aoti_torch_* calls when it loads.
3. There is no test plan, and no CI job loads a compiled AOTI library through Buck, so removing link_whole again would still pass.

[Gasoonjia]

thanks! Can you add buck target for pybinding test also to test it?

[Gasoonjia]

Thanks for your work! Do you mind to enable the existing cuda pybinding tests with valid buck target (e.g. https://github.com/pytorch/executorch/blob/main/extension/pybindings/test/test_pybindings_cuda_dlpack.py) , instead of creating new tests?