[Wheel Variants] List of use cases

[konstin]

In the wheel variant design, it's often hard to motivate what complexities we need and what we can keep simple. In this issue, I want to collect use cases to reference when designing features and the scope the eventual PEP. Ideally, we can include a shortened version of this list in the motivation of the PEP and a longer one in the appendix.

Please feel free to edit this Post. When you add a use case, please write a comment with what you added so the change gets visibility.

Important Use Cases

These are cases that projects/users have been asking for, cases that are currently causing problems, where wheel tags are insufficient.

torch: GPU For CUDA, we require a minimum driver version, i.e. a greater-than-equal relation with driver version (See https://github.com/Slicer/light-the-torch/blob/33397cbe45d07b51ad8ee76b004571a4c236e37f/light_the_torch/_cb.py#L150-L213). This is different than https://pytorch.org/get-started/locally/, where the CUDA major.minor version is modeled: torch ships with CUDA, so it’s actually independent of the CUDA install. There are however cases where users need to select a specific CUDA version, for example if there are regressions in a later version (need for a manual override). For hosts without a GPU, we should install the CPU wheel. Open Question: Can there be more than one CUDA version in a Python process?

torch: compute capability Each torch wheel is compiled against a range of microarchitectures (sm). The host has a single sm (again assuming you're not mixing GPUs), and only wheels that contain the relevant sm can be installed. By having a list of microarchitectures that the wheel supports and having the microarchitecture of the host, we can ensure only a compatible wheel gets installed (not operator, just matching from a list). We can use this to split the large torch wheel into smaller wheels that contain less or only one microarchitecture.

x86_64 microarchitecture levels There are four microarchitecture levels of x86_64: x86_64_v1 (or x86_64 baseline), x86_64_v2, x86_64_v3 and x86_64_v4. For example, v4 includes support for AVX512 that is not yet widespread but can speed up some hashing functions significantly. Example:

• pysimdjson: simdjson gets about 8% faster on AVX512 (Basic AVX-512 implementation (icelake or better) simdjson/simdjson#1813 (comment)).
• Pillow gets a 3x to 7x speedup from using SIMD: https://python-pillow.github.io/pillow-perf/

Pillow benchmarks, the bars are baseline/SSE4/AVX2.

• File size reductions: https://discuss.python.org/t/wheelnext-wheel-variants-an-update-and-a-request-for-feedback/102383/125

OpenMP OpenMP is a shared-memory multiprocessing API. There are multiple implementation such as the GNU OpenMP, Intel OpenMP, LLVM OpenMP and Microsoft OpenMP. Intel OpenMP has better performance on Intel chips than GNU OpenMP [TODO: Benchmarks for Python code]. GNU OpenMP is often used with GCC. OpenMP also can be used through a system component, e.g. /lib/x86_64-linux-gnu/libgomp.so.1 for GNU OpenMP on Ubuntu, can be vendored, or can be in a separate wheel. Currently, it is neither possible to have builds for different OpenMP implementations for the same platform, nor is it possible to take the installed OpenMP implementation into consideration for resolution nor installation.

BLAS Similar to OpenMP, BLAS is a specification for linear algebra operations such as matrix multiplication, which are used in many scientific and machine learning applications, including numpy and scipy. There are several implementations, such as OpenBLAS and Intel MKL. Different from OpenMP, there is no system component, but there is desire to have BLAS in a wheel. The desired functionality is in this case not running detection code, but declaring a dependency on a BLAS wheel depending on which wheel is installed, independent of the platform.

Variants

torch nvidia-cuda packages torch depends on a number of CUDA packages, but only for GPU backend and CUDA major version specific. These are packages such as nvidia-cublas-cu12 and nvidia-cuda-runtime-cu12. To have consistent metadata between wheels, we need a way, such as a marker, to select the nvidia-*-cu12 packages only if a CUDA 12 variant was selected.

Solved

This section largely describes the existing wheel tags and what they solve. Features that are currently in the wheel tag may have been part of variants if variants existed earlier, while we know about their solution and its rollout, so they are good reference examples.

libc On Linux, extension modules can be built against glibc Python or against musl Python. The by far more popular one, glibc, is forward compatible with newer versions and uses the manylinux_x_y tag, i.e. it works like a semver caret (or a >=, considering that glibc is perpetually 2.x).

Python ABI Python extension modules can use libpython symbols in two different ways: requiring a specific Python minor version (cp310-cp310) or requiring a Python major with a minimum minor (cp310-abi). Those are similar to a semver tilde and semver caret respectively.

Universal2 and multiple libc A wheel can be compatible with both x86_64 and aarch64 macs. A statically linked binary in a wheel can support both manylinux and musllinux. The first one can be expressed either by listing both compatible tags (macosx_10_12_x86_64.macosx_11_0_arm64) or by using the special universal2 architecture (macosx_10_12_universal2). For statically linked binaries, we can tag them with both the glibc and musl tags (x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl).

[konstin]

@rgommers You mentioned having a BLAS wheel for NumPy and SciPy in #7 (comment), could you share more about which variant would depend on which BLAS package?

[emmatyping-nv]

re x86_64 microarchitecture levels, I would definitely point to pypa/manylinux#1725. While manylinux allows a description of the right libc, some Linux distros are moving to making v2 the minimum while others aren't, and that is a problem for providing portable manylinux wheels.

For examples of libraries that could take advantage of higher x86_64 versions would be pysimdjson, OpenBLAS used by numpy/scipy, or Pillow-simd.

---

Similar to x86_64 versions, ARM versions and extensions matter a lot. For example, we publish wheels for some projects for both server aarch64 and Jetson aarch64. The instruction sets on the CPU side are rather different but share an aarch64 wheel platform tag. Having variants to express these differences would allow a lot of flexibility we don't currently have.

[konstin]

For the mentioned libraries, do you have links to e.g. benchmarks that show that the difference that the microarchitecture levels make?

Are there reasons these libraries aren't using runtime feature detection? I was trying to find benchmarks for Rust crates, but those I found were using runtime feature detection for SSE and AVX features.

[emmatyping-nv]

Well, first of all, I would say that even projects that use runtime detection could benefit. Runtime detection introduces a lot of overhead, and binary size for each SIMD variant. For pysimdjson, simdjson has AVX-512 benchmarks in this PR: simdjson/simdjson#1813. For Pillow-SIMD, there's https://python-pillow.github.io/pillow-perf/ for AVX benchmarks.

Are there reasons these libraries aren't using runtime feature detection?

I think anything deployed today probably uses runtime detection. That doesn't mean variants aren't useful. Avoiding runtime detection can improve inlining and other optimizations.

[mgorny]

I think the key use case for x86_64 level is specifying -march flags rather than using specific instruction sets directly.

[konstin]

Thanks for the links!

[DEKHTIARJonathan]

I think the key use case for x86_64 level is specifying -march flags rather than using specific instruction sets directly.

To be honest - the case is whatever people want to use :D Wheel Variant allows both. I think we should avoid this kind of statement, it tends to reduce the scope of what are trying to achieve.

Even if I don't disagree with you that v2/v3/v4 is probably of much greater interest. I don't see what good it makes to "reduce the scope" artificially on -march rather than also emphasizing that -mavx512fp16 (just to give one example) is perfectly supported if people so wishes.

My point being: we should focus on stating the possible more than trying to "decide of a winner". Even if at the end of the day, you're probably right and that using directly instructions is probably a bad idea for most packages.

[rgommers]

@rgommers You mentioned having a BLAS wheel for NumPy and SciPy in #7 (comment), could you share more about which variant would depend on which BLAS package?

Sure. This is the current status:

• macOS:
  • for macOS >=14, wheels depend on the OS-provided Apple Accelerate
  • for macOS <14, wheels vendor OpenBLAS (hopefully will depend on scipy-openblas64 wheels in the future)
• Linux: vendors OpenBLAS
• Windows:
  • i686 (32-bit Python) doesn't depend on any BLAS library
  • x86-64 and arm64: vendors OpenBLAS

Nothing depends on MKL, performance-wise it may be desirable to have such a build variant on x86-64 Linux in the future.

[rgommers]

some Linux distros are moving to making v2 the minimum while others aren't, and that is a problem for providing portable manylinux wheels.

There's a good chance that NumPy will follow suit and drop support for hardware that can't support x86-64-v2: numpy/numpy#27851.

I think anything deployed today probably uses runtime detection. That doesn't mean variants aren't useful. Avoiding runtime detection can improve inlining and other optimizations.

Not just that, it's that the complexity of runtime dispatching is extremely high, so only a few projects use it. Everyone else with an interest in SIMD just drops the idea of using SIMD at all when they look at that complexity (e.g., scipy and scikit-learn), or puts it up as a separate package on PyPI (pillow-simd).

SimSIMD is also a nice example, it can be O(100x) faster than SciPy's distance metrics (scipy/scipy#19454) which are implemented in reasonably well-written C/C++.

[konstin]

@rgommers For that listing, what variants properties and Requires-Dist would packages use? I'm trying to cast this into a motivating example, but I don't have enough background with the core of the scientific stack, so I'd need help to answer why variants are needed here.

[rgommers]

@konstin I'm not sure I understood your exact request, what "listing" exactly? Are you asking for scipy, scikit-learn et al. what SIMD and/or BLAS variants I expect each of them would want to ship?

[konstin]

I wanted to refer to #54 (comment), basically how we translate this to Requires-Dist with variant markers and wheel with variant labels.

[rgommers]

I'm not sure I can write a full pyproject.toml addition here for numpy or scipy + blas-provider. We should probably do that as an exercise; for now I'll point at what @mgorny has done so far for x86-64 and aarch64 providers at https://github.com/wheelnext/numpy/blob/wheel-variants/pyproject.toml#L62. Adding a BLAS provider to that raises the question of exactly what combos we would want to build in practice. But let's say that the top addition is to add a manylinux MKL wheel with x86-64-v4, since it doesn't exist yet and that's the most performant x86-64 build we can get. Then the choice is between vendoring MKL and depending on it. The latter is more interesting. I'll also pretend #18 (comment) is settled and use a none namespace here.

With all that, we'd have to add this:

[project]
name = 'scipy'
dependencies = [
    # This goes here, right? We have a rollout problem with older `pip` versions choking on `variant_properties`
    mkl==2025.2.0; platform_system == "Linux" and platform_machine == "x86_64" and "none :: blas :: mkl" in variant_properties'
    # Accelerate is provided by macOS itself
    # OpenBLAS we provide continue to vendor; if we want an unvendored version on Linux x86-64 and macOS arm64 it'd be:
    scipy-openblas32>=0.3.30.0.2,<0.3.31; platform_system == "Linux" and platform_machine == "x86_64" and "none :: blas :: openblas" in variant_properties'
    scipy-openblas32>=0.3.30.0.2,<0.3.31; platform_system == "Darwin" and platform_machine == "arm64" and "none :: blas :: openblas" in variant_properties'

[variant.default-priorities]
namespace = ["x86_64", "aarch64"]

[variant.providers.blas]
requires = ["provider-variant-blas"]
enable-if = "platform_machine == 'x86_64' or platform_machine == 'AMD64'"
plugin-api = "provider_variant_blas.plugin:BlasPlugin"
plugin-type = "build-only"

# See https://github.com/wheelnext/numpy/blob/wheel-variants/pyproject.toml#L62
# for how to add x86-64 and aarch64 providers

This probably raises as many questions as it answers, and hence is best moved to a separate issue I suspect. To make the use case itself more compelling we probably also need some benchmarks, comparing the scipy benchmark suite between:

• baseline wheel
• x86-64-v4 variant wheel
• MKL variant wheel
• x86-64-v4 + MKL variant wheel

[konstin]

I added OpenMP and BLAS as example. Please proofread and let me know if anything is wrong!