do not merge: Enable PlanV2 by default - #9008
Conversation
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
…n-default Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Merging this PR will not alter performance
Comparing Footnotes
|
…ecute Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
…n-default Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.022x ➖, 0↑ 1↓)
datafusion / vortex-compact (1.009x ➖, 0↑ 0↓)
datafusion / parquet (1.003x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (1.005x ➖, 1↑ 0↓)
duckdb / vortex-compact (1.001x ➖, 2↑ 0↓)
duckdb / parquet (1.043x ➖, 0↑ 4↓)
duckdb / duckdb (1.000x ➖, 0↑ 0↓)
File Size Changes (34 files changed, +0.0% overall, 18↑ 16↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.078x ➖, 0↑ 2↓)
datafusion / vortex-compact (1.013x ➖, 0↑ 0↓)
datafusion / parquet (1.021x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.075x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.017x ➖, 1↑ 0↓)
duckdb / parquet (1.032x ➖, 0↑ 0↓)
No file size changes detected. |
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Benchmarks: PolarSignals Profiling 📖Vortex (geomean): 1.081x ➖ How to read Verdict and Engines
datafusion / vortex-file-compressed (1.081x ➖, 0↑ 4↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.000x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.005x ➖, 0↑ 2↓)
datafusion / parquet (1.000x ➖, 2↑ 2↓)
duckdb / vortex-file-compressed (1.002x ➖, 0↑ 2↓)
duckdb / vortex-compact (1.005x ➖, 0↑ 2↓)
duckdb / parquet (0.993x ➖, 1↑ 0↓)
duckdb / duckdb (0.999x ➖, 0↑ 0↓)
No file size changes detected. |
…n-default Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> # Conflicts: # vortex-layout/src/scan/plan.rs
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.033x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.145x ➖, 0↑ 1↓)
datafusion / parquet (0.922x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed (0.988x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.025x ➖, 0↑ 0↓)
duckdb / parquet (0.988x ➖, 0↑ 0↓)
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed (0.997x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.998x ➖, 0↑ 0↓)
duckdb / parquet (0.997x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.902x ➖, 8↑ 0↓)
datafusion / vortex-compact (0.917x ➖, 7↑ 0↓)
datafusion / parquet (1.013x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.968x ➖, 2↑ 0↓)
duckdb / vortex-compact (0.990x ➖, 0↑ 1↓)
duckdb / parquet (1.050x ➖, 0↑ 6↓)
duckdb / duckdb (0.991x ➖, 0↑ 0↓)
File Size Changes (64 files changed, +0.1% overall, 48↑ 16↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.950x ➖, 1↑ 1↓)
datafusion / parquet (0.932x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed (0.969x ➖, 1↑ 0↓)
duckdb / parquet (0.976x ➖, 0↑ 0↓)
duckdb / duckdb (0.968x ➖, 0↑ 0↓)
File Size Changes (201 files changed, -0.0% overall, 96↑ 105↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.096x ➖, 0↑ 2↓)
datafusion / vortex-compact (1.132x ➖, 0↑ 3↓)
datafusion / parquet (1.240x ➖, 0↑ 8↓)
duckdb / vortex-file-compressed (1.071x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.050x ➖, 0↑ 0↓)
duckdb / parquet (1.201x ➖, 0↑ 4↓)
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.929x ➖, 14↑ 0↓)
datafusion / parquet (0.971x ➖, 6↑ 1↓)
duckdb / vortex-file-compressed (1.015x ➖, 2↑ 3↓)
duckdb / parquet (1.007x ➖, 0↑ 0↓)
duckdb / duckdb (1.011x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: Appian on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.032x ➖, 0↑ 0↓)
datafusion / parquet (1.021x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.024x ➖, 0↑ 0↓)
duckdb / parquet (1.021x ➖, 0↑ 0↓)
duckdb / duckdb (1.002x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.025x ➖, 0↑ 1↓)
datafusion / vortex-compact (1.089x ➖, 0↑ 2↓)
datafusion / parquet (1.156x ➖, 0↑ 4↓)
duckdb / vortex-file-compressed (0.796x ➖, 3↑ 0↓)
duckdb / vortex-compact (0.788x ➖, 4↑ 0↓)
duckdb / parquet (1.194x ➖, 0↑ 3↓)
|
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
…n-default Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Benchmarks: Vortex queries 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.050x ➖, 0↑ 0↓)
datafusion / parquet (1.029x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.061x ➖, 0↑ 0↓)
duckdb / parquet (1.013x ➖, 0↑ 0↓)
No file size changes detected. |
…ecute Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
…n-default Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Summary
apply_expr(...).optimize()path whenVORTEX_SCAN_IMPLis unsetplan,v1,legacy, andlayout-readercompatibility opt-outsscan::plan_v2This is stacked on #9005 (
ji/planning-old-execute).Checks
RUSTC_WRAPPER= cargo test -p vortex-layout --all-features --no-runRUSTC_WRAPPER= cargo test -p vortex-layout --all-features --lib(198 passed with PlanV2 selected by default)VORTEX_SCAN_IMPL=plan RUSTC_WRAPPER= cargo test -p vortex-layout scan_impl --all-featurescargo +nightly fmt --allgit diff --check