Skip to content

Move expression analysis and rewrite tooling into vortex-expr - #10474

Closed
robert3005 wants to merge 1 commit into
rk/split-array-04-aggregatesfrom
rk/split-array-05-expr
Closed

robert3005 wants to merge 1 commit into
rk/split-array-04-aggregatesfrom
rk/split-array-05-expr

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

Summary

PR 5 of the stack that splits vortex-array into smaller crates.

Nothing in vortex-array calls vortex_array::expr::{analysis, forms}, the partition_bound* transforms or the replace transform. Their only users are scan planning in vortex-layout and one replace call in vortex-datafusion.

Changes

  • These items move to a new vortex-expr crate:
    • vortex_expr::analysis: annotations, labeling, field-access analysis and referenced_field_paths. Its items are also re-exported at the crate root, as they were before.
    • vortex_expr::forms::conjuncts.
    • vortex_expr::transform: partition_bound, partition_bound_annotations, BoundPartitionedExpr and replace.
  • find_between stays in vortex-array because BoundExpression::optimize uses it. The now-internal vortex_array::expr::transform module becomes private.
  • One optimizer test used replace. It now builds the replaced expression directly.
  • The umbrella crate's vortex::expr re-exports both vortex_array::expr and vortex_expr, so vortex::expr::transform::replace and friends keep their paths.

API Changes

vortex_array::expr::{analysis, forms, transform::*} and the analysis items re-exported at the root of vortex_array::expr move to vortex_expr. Paths through vortex::expr::… are unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j


Generated by Claude Code

@robert3005 robert3005 added the changelog/break A breaking API change label Oct 14, 2026 — with Claude
@codspeed

codspeed Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 2 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 16 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 2 regressed benchmarks
✅ 2168 untouched benchmarks
⏩ 37 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ WallTime threshold_neon[8] 2.8 µs 4.1 µs -30.33%
❌ WallTime take_u16_nullable_from_65536_avx512[1048576] 380.5 µs 450.5 µs -15.55%
⚡ WallTime filter_neon[8] 2.2 µs 1.1 µs ×2.1
⚡ ⚠️ Simulation density_sweep_dense_runs[0.001] 46.8 µs 29.5 µs +58.79%
⚡ ⚠️ Simulation filter_powerlaw_by_mostly_true[250000] 144.6 µs 103.4 µs +39.83%
⚡ WallTime threshold_avx2[8] 2.3 µs 2 µs +18.23%
⚡ Simulation random_i8[0.01] 123.4 µs 108 µs +14.26%
⚡ WallTime filter_neon[80] 3.6 µs 3.2 µs +11.83%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/split-array-05-expr (276f15b) with rk/split-array-04-aggregates (33fb97e)

Open in CodSpeed

Footnotes

  1. 37 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

`vortex_array::expr::{analysis, forms}` and the `partition_bound*` and
`replace` transforms are only used by scan planning in vortex-layout (and one
`replace` call in vortex-datafusion); nothing in vortex-array calls them. They
move to a new `vortex-expr` crate:

- `vortex_expr::analysis` (annotations, labeling, field-access analysis,
  `referenced_field_paths`), also re-exported at the crate root as before
- `vortex_expr::forms::conjuncts`
- `vortex_expr::transform::{partition_bound, partition_bound_annotations,
  BoundPartitionedExpr, replace}`

`find_between` stays in vortex-array because `BoundExpression::optimize`
uses it. The now internal `vortex_array::expr::transform` module becomes
private, and the one optimizer test that used `replace` builds the replaced
expression directly.

The umbrella crate's `vortex::expr` module re-exports both
`vortex_array::expr` and `vortex_expr`, so `vortex::expr::transform::replace`
and friends keep their paths.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j
@robert3005
robert3005 force-pushed the rk/split-array-04-aggregates branch from f6708e9 to 33fb97e Compare October 14, 2026 03:07
@robert3005
robert3005 force-pushed the rk/split-array-05-expr branch from 2bde4df to 276f15b Compare October 14, 2026 03:07

Copy link
Copy Markdown
Contributor Author

The CodSpeed Performance Analysis check is red. Its headline number swings with no relevant code change: earlier runs on this PR reported +12.99% and −9.77%, and this head shows "2 benchmarks regressed".

The two regressions it flags (density_sweep_dense_runs and filter_powerlaw_by_mostly_true) are filter kernels. This PR doesn't touch them; it only moves expression-analysis code, which those benchmarks never run. CodSpeed also warns that the runs it compared came from different runtime environments, so I'm treating this as runner noise and not changing code for it.

The earlier Rust (docs) failure (a broken intra-doc link) is fixed on the current head.


Generated by Claude Code

@robert3005 robert3005 closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants