ci: add focused PR benchmark labels - #9018
Conversation
89eebfa to
77a7e5d
Compare
|
note that we probably dont want all of these labels, but I feel like a bit more than what we have on develop now would be nice? |
|
actually Im going to make them all disjoint, lets see how that works |
77a7e5d to
9cbc6da
Compare
2d3d02c to
c5173aa
Compare
Merging this PR will improve performance by 16.9%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | WallTime | cuda/bitpacked_u8/unpack/3bw[100M] |
350.6 µs | 299.9 µs | +16.9% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/benchmark-labels (e332145) with develop (6609288)
Footnotes
-
3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
Let's use bench-random-access, ra isn't straight obvious |
c5173aa to
0b4eb99
Compare
0b4eb99 to
dac3fb7
Compare
| # v4 (Postgres) ingest -- REQUIRED (see develop-bench.yml rationale). Empty records: | ||
| # post-ingest.py --postgres upserts the commit row only. Gated on the | ||
| # ingest-role ARN var (the assume-role input that MUST exist for OIDC to | ||
| # succeed). |
There was a problem hiding this comment.
Probably need a action/bench-gpu-compress label for GPU compress?
There was a problem hiding this comment.
yeah I am going to run this once this merges:
gh label edit "action/benchmark-gpu-compress" --name "action/bench-gpu-compress" \
--description "Run only the GPU compression benchmark on this PR"
gh label delete action/benchmark --yes
gh label delete action/benchmark-sql --yes
gh label delete action/benchmark-sql-full --yes
| iterations=10, | ||
| runs={ | ||
| "pr": DEFAULT, | ||
| "pr-compact": COMPACT, |
There was a problem hiding this comment.
pr and pr-compact SQL runs overwrite each other's PR comment
| "pr-compact": COMPACT, | ||
| "pr-full": FULL_PR, | ||
| "develop": FULL_LOCAL, | ||
| }, |
There was a problem hiding this comment.
is there any label reaching pr full?
There was a problem hiding this comment.
no since I think we should just keep everything disjoint, if someone wants to run more benchmarks I think they should just add all 4 (or 5 for gpu) labels.
4bbe37b to
fb5f8aa
Compare
| cat comment.md >> "$GITHUB_STEP_SUMMARY" | ||
| - name: Comment PR | ||
| if: github.event.pull_request.head.repo.fork == false | ||
| if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false |
There was a problem hiding this comment.
i think this is a qol change
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
fb5f8aa to
e332145
Compare
3 participants
Adds four mutually exclusive PR benchmark triggers:
action/bench-raaction/bench-compressaction/bench-sql(no Vortex Compact)action/bench-sql-compact(only Vortex Compact)TODO after merge: delete the legacy
action/benchmark*labels, which remain temporarily for the currentdevelopdispatcher.