[mcremon-meta]

Summary:

This diff makes the quantizer in frontends/fused_quant/quantizer/ usable by
any backend:

• get_core_aten_quantizers(config, weighted_config) in
  quantizer/quantizer.py: the op quantizers every backend can share, with the
  contract below. arg1_is_scalar is the node filter its scalar add/sub/mul
  variants use.
• FusedQuantQuantizer now composes with other PT2E quantizers on a
  first-claim-wins basis. Its annotations set _annotated (the flag other
  quantizers check), and a fused_quant_owned node flag lets fuse() skip
  nodes that another quantizer claimed.
• FusedQuantQuantizer.fuse first repairs the convert_pt2e output: it
  canonicalizes torchao quantize_affine / dequantize_affine packets to
  .default and refreshes meta["val"], both of which fusion depends on.

The contract of get_core_aten_quantizers:

• Only core ATen ops. Every op it quantizes is core ATen
  (torch.Tag.core): add, sub, mul, bmm, relu, max_pool2d_with_indices,
  avg_pool2d, convolution, sigmoid, tanh, hardtanh, gelu and native_layer_norm.
  It does not cover every core ATen op, and more can be added.
• Each quantizer always sees its op. Export's decompositions keep core
  ATen ops, so a model's use of these ops reaches the quantizer without
  preserved_ops() keeping anything whole.
• Portable. Whatever no backend claims decomposes to quantize/dequantize
  plus the same core ATen ops, all of which have portable kernels.
• Qparams come from the caller. The package builds no QuantizationConfig
  of its own: callers build theirs with TorchAO or reuse their backend's.
  weighted_config applies to convolution, config to everything else. The
  only change it makes is moving the per-channel weight axis to 1 for
  transposed convolution. Grouped transposed convolution uses config
  instead.
• Non-core ops belong to the backend. A backend that wants an op such as
  linear or hardswish kept whole adds its own OpQuantizer, and
  preserved_ops() keeps it out of decomposition. Non-core ops that many
  backends share, such as linear, can move into the shared list later.

Differential Revision: D129194623

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24769

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 1ec7bad with merge base e0015e1 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Test Vulkan Backend (specialized runners) / test-model-vulkan-e2e / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@mcremon-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D129194623.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.