Repository navigation
DuckDB 2.0 (v2.0-cyanoptera head) - #10513
joseph-isaacs wants to merge 2 commits into
Conversation
Polar Signals Profiling ResultsLatest Run
Previous Runs (3)
Powered by Polar Signals Cloud |
Benchmarks: String Encoding 📖Commits: PR vortex / vortex-file-compressed / ms (1.003x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Merging this PR will degrade performance by 28.33%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ |
Simulation | density_sweep_dense_runs[0.9] |
13.3 µs | 51.9 µs | -74.42% |
| ❌ |
Simulation | random_i128[0.01] |
4.9 µs | 6 µs | -17.88% |
| ❌ | WallTime | bitpack_blocked_compress_avx2 |
6.7 µs | 7.6 µs | -12.21% |
| ❌ |
Simulation | test_repeated_scan_scalar_at[int_str] |
441.9 µs | 491.6 µs | -10.1% |
| ⚡ | Simulation | test_scan_filter[int] |
3.5 ms | 3 ms | +14.03% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/relaxed-goldberg-03ljyd (3078838) with develop (52a1d15)1
Footnotes
Benchmarks: Compression 📖Commits: PR vortex / vortex-file-compressed / ns (0.987x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / ratio (0.989x ➖, 0↑ 0↓)
vortex / parquet / ns (1.003x ➖, 0↑ 0↓)
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
vortex / arrow-ipc / ns (0.992x ➖, 0↑ 0↓)
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
|
|
CodSpeed Performance Analysis reports "1 benchmark regressed" on both Generated by Claude Code |
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
vortex / arrow-ipc / ns (1.087x ➖, 1↑ 3↓)
random-access / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
random-access / parquet / ns (0.956x ➖, 2↑ 0↓)
random-access / lance / ns (0.996x ➖, 0↑ 0↓)
|
Benchmarks: Random Access (S3) 📖Commits: PR How to read Verdict and Engines
random-access / vortex-file-compressed / ns (0.901x ➖, 2↑ 0↓)
random-access / parquet / ns (1.002x ➖, 0↑ 0↓)
random-access / lance / ns (1.130x ➖, 1↑ 6↓)
|
Squash of #9141 (DuckDB 2.0) merged with develop, plus: - Bump DEFAULT_DUCKDB_VERSION to 8588363bf15e090fcbbfe2a68ec98b10c6866cb1, the head of DuckDB's v2.0-cyanoptera release branch. - Build httpfs with DuckDB's pinned vcpkg for Linux R2 builds; drop it from the macOS universal build. build.rs only links httpfs when VCPKG_TOOLCHAIN_PATH is set. - Read DynamicFilterData::lock (now an annotated_shared_mutex) under annotated_shared_lock. - Keep develop's i8 decimal narrowing alongside #9141's zeroing of invalid decimal slots in convert/vector.rs. Co-authored-by: Mikhail Kot <mikhail@spiraldb.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1DozjdAzpYiyEykgbhPi Signed-off-by: Claude <noreply@anthropic.com>
b4a7dbf to
eb261db
Compare
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.039x ➖, 1↑ 2↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.702x ✅, 8↑ 0↓)
duckdb / parquet / ns (1.310x ❌, 0↑ 9↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.001x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.945x ➖, 5↑ 4↓)
duckdb / parquet / ns (1.010x ➖, 4↑ 3↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.974x ➖, 12↑ 7↓)
duckdb / parquet / ns (1.225x ❌, 4↑ 30↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.086x ➖, 19↑ 47↓)
duckdb / parquet / ns (1.260x ❌, 8↑ 78↓)
No file size changes detected. |
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.878x ✅, 9↑ 2↓)
duckdb / parquet / ns (1.031x ➖, 3↑ 11↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.107x ➖, 1↑ 6↓)
datafusion / parquet / ns (1.251x ➖, 0↑ 9↓)
duckdb / vortex-file-compressed / ns (0.885x ➖, 5↑ 1↓)
duckdb / parquet / ns (0.653x ✅, 15↑ 0↓)
|
Regenerated with the sqllogictest runner's --complete against DuckDB 8588363b. The plan changes come from DuckDB's optimizer (for example avg is no longer rewritten to sum/count, and duplicate pushed-down scan filters are deduplicated); the set of Vortex scans and pushed-down filters is unchanged. TPC-DS q9's last bucket differs by one unit in the 16th significant digit on both Parquet and Vortex, from a change in DuckDB's floating-point summation order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013h1DozjdAzpYiyEykgbhPi Signed-off-by: Claude <noreply@anthropic.com>
Summary
Builds on #9141 (DuckDB 2.0 support), merged with the current
developand bumped to the latest DuckDB 2.0 release-branch commit. DuckDB has not taggedv2.0.0yet, so this pins the head ofv2.0-cyanoptera. #9141's commits are squashed into one commit, with Mikhail Kot as co-author, so that DCO passes.Changes
develop:vortex-duckdb/src/convert/vector.rs: keeps both sides. Duckdb 2.0 #9141 zeroes invalid decimal slots, duckdb: i16 Decimal on duckdb side may fit in Vortex's i8 decimal buffer #10155 narrows small decimals into ani8buffer, and thei8path now zeroes invalid slots as well.vortex-duckdb/src/convert/expr.rs: uses the importedvortex_ensure_eq!where the merge left an unimportedvortex_ensure!.DEFAULT_DUCKDB_VERSIONbumped from561522aea03e400bd20adc64fcc63e78b8721f3f(2026-09-30) to8588363bf15e090fcbbfe2a68ec98b10c6866cb1(2026-10-15,v2.0-cyanopterahead). The old commit is an ancestor, 2,315 commits behind.cpp/table_filter.cpp:DynamicFilterData::lockis now anannotated_shared_mutex, so it is read underannotated_shared_lock, as DuckDB does.httpfsis now an out-of-tree DuckDB extension with vcpkg dependencies, and DuckDB refuses to build it withoutVCPKG_TOOLCHAIN_PATH:scripts/duckdb-r2-prepare.sh: on Linux, bootstraps DuckDB's pinned vcpkg (make setup-vcpkg) and keepshttpfs. The macOS universal build dropshttpfs(and the hand-built universal OpenSSL), since vcpkg can't produce universal binaries and the S3 test only runs on Linux.vortex-duckdb/build.rs: when building DuckDB from source, linkshttpfsonly ifVCPKG_TOOLCHAIN_PATHis set.--completeagainst8588363b:EXPLAINplan snapshots changed because of DuckDB optimizer changes. For example,avgis no longer rewritten tosum/count, and duplicate pushed-down scan filters are now deduplicated.FILTERnodes (525) is unchanged, and every scan filter is still pushed down.…512→…511) on both Parquet and Vortex. DuckDB now sums the floats in a different order.Testing
8588363b:duckdb-bench tpch --formats parquet,vortex --opt scale-factor=0.1 -i 1ran all 22 queries on both formats. All DuckDB SQL logic tests pass.httpfsbuilt with vcpkg) and DCO. The exception is CodSpeed, which flags benchmarks in crates this PR doesn't touch.action/bench-sqloneb261dbc: no suite showed a confirmed regression. Note that DuckDB's own Parquet reader also changes in this bump, so the Parquet control rows are not a stable baseline for the DuckDB attribution.🤖 Generated with Claude Code
https://claude.ai/code/session_013h1DozjdAzpYiyEykgbhPi