chore: bump the actions-all group across 1 directory with 14 updates - #21
chore: bump the actions-all group across 1 directory with 14 updates#21dependabot[bot] wants to merge 3 commits into
Conversation
LabelsThe following labels could not be found: Please fix the above issues or remove invalid values from |
dependabot
Bot
added
the
dependencies
dependabot
Bot
added
the
dependencies
dependabot
Bot
changed the title
c6fad67 to
717c5cb
Compare
Bumps the actions-all group with 14 updates in the / directory: | Package | From | To | | --- | --- | --- | | [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `4.2.0` | `9.0.0` | | [actions/checkout](https://github.com/actions/checkout) | `4` | `7` | | [anthropics/claude-code-action](https://github.com/anthropics/claude-code-action) | `1.0.169` | `1.0.183` | | [github/codeql-action/init](https://github.com/github/codeql-action) | `3.35.5` | `4.37.3` | | [github/codeql-action/analyze](https://github.com/github/codeql-action) | `3.35.5` | `4.37.3` | | [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) | `3` | `4` | | [docker/login-action](https://github.com/docker/login-action) | `3` | `4` | | [docker/build-push-action](https://github.com/docker/build-push-action) | `6` | `7` | | [actions/setup-python](https://github.com/actions/setup-python) | `5` | `7` | | [actions/cache](https://github.com/actions/cache) | `5.0.5` | `6.1.0` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4.6.2` | `7.0.1` | | [actions/setup-node](https://github.com/actions/setup-node) | `4.4.0` | `7.0.0` | | [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.1` | | [actions/download-artifact](https://github.com/actions/download-artifact) | `4.3.0` | `8.0.1` | Updates `astral-sh/setup-uv` from 4.2.0 to 9.0.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@38f3f10...c771a70) Updates `actions/checkout` from 4 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Commits](actions/checkout@v4...v7) Updates `anthropics/claude-code-action` from 1.0.169 to 1.0.183 - [Release notes](https://github.com/anthropics/claude-code-action/releases) - [Commits](anthropics/claude-code-action@37b464c...be7b93b) Updates `github/codeql-action/init` from 3.35.5 to 4.37.3 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@458d36d...e4fba86) Updates `github/codeql-action/analyze` from 3.35.5 to 4.37.3 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@458d36d...e4fba86) Updates `docker/setup-buildx-action` from 3 to 4 - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](docker/setup-buildx-action@v3...v4) Updates `docker/login-action` from 3 to 4 - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@v3...v4) Updates `docker/build-push-action` from 6 to 7 - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](docker/build-push-action@v6...v7) Updates `actions/setup-python` from 5 to 7 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v7) Updates `actions/cache` from 5.0.5 to 6.1.0 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@27d5ce7...55cc834) Updates `actions/upload-artifact` from 4.6.2 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v4.6.2...043fb46) Updates `actions/setup-node` from 4.4.0 to 7.0.0 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@49933ea...8207627) Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.1 - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@ed0c539...ba38be9) Updates `actions/download-artifact` from 4.3.0 to 8.0.1 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@d3f86a1...3e5f45b) --- updated-dependencies: - dependency-name: actions/cache dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: actions/checkout dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: actions/download-artifact dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: actions/setup-node dependency-version: 6.4.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: actions/setup-python dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: anthropics/claude-code-action dependency-version: 1.0.171 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions-all - dependency-name: astral-sh/setup-uv dependency-version: 8.3.2 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: docker/build-push-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: docker/login-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: docker/setup-buildx-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: github/codeql-action/analyze dependency-version: 4.37.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: github/codeql-action/init dependency-version: 4.37.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-all - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions-all ... Signed-off-by: dependabot[bot] <support@github.com>
717c5cb to
efa0268
Compare
uipreliga
left a comment
uipreliga
left a comment
There was a problem hiding this comment.
Review: chore: bump the actions-all group across 1 directory with 14 updates
PR #21 by @dependabot · dependabot/github_actions/actions-all-5a53a122b0 → main · OPEN · reviewed against 1591548 · all 8 axes · 2026-07-27T20:41Z
Change class: complex — 14 GitHub Actions bumps, 9 crossing a major version, on the release/publish path and inside the published composite action (action.yml), so correctness requires checking each action's breaking changes rather than reading the diff
A clean, high-discipline dependency-bump PR on an exceptionally healthy codebase (perfect scores on type safety, architecture, and error handling; zero critical or high findings in the Python surface), where the only real risks live entirely in the CI/publishing layer: the published composite action's unparameterized astral-sh/setup-uv bump to v9.0.0 silently rebinds the graded toolchain to whatever uv the consumer's repo pins (plus an uninvited unpruned cache write and a new node24 runner floor) and ships to every @v0 consumer on the next release, while a pre-existing two-tier pinning convention leaves five mutable-tag refs — including the push-to-main docker build that shares an image with the SHA-pinned release.yml path — outside the repo's own SHA-pin policy with no mechanical guard; bottom line: merge-ready once action.yml:83 gets an explicit with: block, with the pinning convergence and an actionlint/CE-rule guard as fast follow-ups.
Summary
| Axis | Score | 🔴 | 🟠 | 🟡 | 🔵 | Top Issue |
|---|---|---|---|---|---|---|
| 1. Code Quality & Style | 9.4 / 10 | 0 | 0 | 1 | 1 | Two-tier pinning convention: 5 bumped uses: refs left on mutable major tags while the other 55 repo refs are SHA-pinned (incl. the docker buildcache producer/consumer pair), with no guard test enforcing the convention |
| 2. Type Safety | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 3. Test Health | 9.9 / 10 | 0 | 0 | 0 | 1 | No workflow linter runs anywhere in CI, so a 9-file / 50-line workflow edit lands with zero mechanical validation |
| 4. Security | 9.9 / 10 | 0 | 0 | 0 | 1 | Security-rationale comments attached to the two bumped refs in claude-pr-review.yml no longer match the pinned versions' behavior |
| 5. Architecture & Design | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 6. Error Handling & Resilience | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 7. API Surface & Maintainability | 9.9 / 10 | 0 | 0 | 0 | 1 | CI tutorial's consumer-facing gate recipe left stale by these bumps — it still teaches action majors that no longer match any workflow in the repo, with no parity guard |
| 8. Evaluation Harness Quality | 9 / 10 | 0 | 1 | 0 | 0 | Published composite action's setup-uv step passes no with:, so the v4.2.0→v9.0.0 bump silently rebinds defaults (cache/prune-cache, uv-version resolution from the consumer's pyproject, PATH, Node runtime) for every @v0 consumer, with no opt-out input and no doc note |
Overall Score: 9.8 / 10 · Weakest Axis: Evaluation Harness Quality at 9 / 10
Totals: 🔴 0 · 🟠 1 · 🟡 1 · 🔵 4 across 8 axes.
Blockers
- [Axis 8] Published composite action's setup-uv step passes no
with:, so the v4.2.0→v9.0.0 bump silently rebinds defaults (cache/prune-cache, uv-version resolution from the consumer's pyproject, PATH, Node runtime) for every@v0consumer, with no opt-out input and no doc note (action.yml:83) —action.yml:83isuses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0with nowith:block, so all four of these defaults changed under it (read from the two pinned action.yml blobs at SHA38f3f10…= v4.2.0 andc771a70…= v9.0.0): version: v4default: "latest"→ v9default: ""("Defaults to the version in pyproject.toml or 'latest'"). v9 resolves it fromworking-directory(v9 default${{ github.workspace }}= the CONSUMER's repo) viagetWorkspaceCandidates()→uv.toml, thenpyproject.toml→pyproject.tool?.uv?.["required-version"](setup-uvsrc/version/version-request-resolver.ts+src/version/file-parser.ts). A consumer repo that pins an old uv now gets THAT uv for the next step'suv tool install "coder-eval==$CE_VERSION"(action.yml:94).enable-cache:"false"→"auto"(= true on GitHub-hosted). The gate now restores AND saves a uv cache entry into the consumer's 10 GB repo cache uninvited, keyed on the consumer's own files (cache-dependency-globnow includes**/pyproject.toml,**/uv.lock,**/*requirements*.txt).prune-cache:"true"→"false"(v9.0.0's headline breaking change) — that cache entry is unpruned.- Runtime
using: "node20"→using: "node24"(v7.0.0 breaking change), requiring Actions Runner ≥ 2.327.1; on an older self-hosted runner the gate's FIRST step hard-fails.
Blast radius: per CLAUDE.md,release.ymlmaintains action.yml'sversion:default *plus the movingv<major>tag, so everyUiPath/coder_eval@v0consumer (README.md:105, docs/CI_GATE.md:22) picks this up with no coder-eval version change to blame. The harness's own coverage cannot catch it: the dogfood job (pr-checks.yml:856 uses: ./) runs in coder_eval's workspace, which has nouv.tomland no[tool.uv] required-version/uvrequirement inpyproject.toml, so it still resolveslatestexactly as v4 did — green dogfood, broken consumer. Fix: make the published gate's toolchain a property of coder-eval, not of the consumer's repo — addwith:on action.yml:83 pinning the previous behavior explicitly (version: "latest",enable-cache: "false"; orenable-cache: "true"+prune-cache: "true"` if caching is wanted), and state the Runner ≥ 2.327.1 floor in docs/CI_GATE.md.
Non-blocking, but please consider before merge
- [Axis 1] Two-tier pinning convention: 5 bumped
uses:refs left on mutable major tags while the other 55 repo refs are SHA-pinned (incl. the docker buildcache producer/consumer pair), with no guard test enforcing the convention (.github/workflows/docker-publish.yml:68) — This PR bumped everyuses:ref in the repo, but did so in two different styles. 55 of the 60 third-party refs (per tmp/code-review-260727-1341/automated/uses-matrix.txt) are full-SHA + trailing version comment, e.g..github/workflows/docker-publish.yml:54: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1. Five refs the PR touched stayed on mutable tags:docker-publish.yml:68: uses: docker/setup-buildx-action@v4,:71: uses: docker/login-action@v4,:78: uses: docker/build-push-action@v7,docs.yml:37: - uses: actions/checkout@v7,:38: - uses: actions/setup-python@v7. The worst part is the divergent duplicate: docker-publish.yml:68/71/78 and release.yml:382/387/396 run the same three actions to build the same image from the samefile: docker/Dockerfileinto the same ghcrcoder-eval-agentrepo, yet release.yml is SHA-pinned (docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0,docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1,docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0) while docker-publish.yml is not. The two build paths can therefore silently run different action code, and nothing mechanically flags it:tests/test_action_version_pin.pyonly asserts action.yml'sdefault: "…" # <-- kept in synccoder-eval version, andtests/test_pr_review_workflow.py:107only matchesstartswith("actions/checkout@"). Fix: SHA-pin those 5 refs to the same commits already used elsewhere for the docker trio (bb05f3f…/abd2ef4…/53b7df9…) so both build paths are byte-identical, and add a guard — either an actionlint/CI grep asserting every non-localuses:matches@[0-9a-f]{40}with a trailing# v…comment, or atests/lint/-style test in the CE00n family walking.github/workflows/*.yml+action.yml. Otherwise state explicitly in a comment why docker-publish.yml is exempt.
Nits
-
[Axis 1] Version-comment spacing drift on two lines this PR rewrote (one space before
#vs two everywhere else) (.github/workflows/pr-checks.yml:170) — Every other SHA-pinned ref in the repo separates the ref from its version comment with two spaces, e.g.pr-checks.yml:237: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1. The two lines in theno-uipath-extrajob use one:pr-checks.yml:170: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1andpr-checks.yml:173: uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0. The drift is pre-existing but both lines were rewritten by this PR, so it is free to fix: add the second space to match the surrounding 55 refs. Purely cosmetic — noted, not blocking. -
[Axis 3] No workflow linter runs anywhere in CI, so a 9-file / 50-line workflow edit lands with zero mechanical validation (
.github/workflows/pr-checks.yml:84) —grep -rn "actionlint\|zizmor\|yamllint" .github/ Makefile .pre-commit-config.yaml pyproject.tomlreturns no matches — the CI-config surface has no syntax or action-metadata gate at all, while the Python surface has ruff + pyright + a full CE001+ custom-lint suite (.github/workflows/pr-checks.yml:84:run: .venv/bin/pytest tests/test_custom_lint.py -v --tb=short --no-header -p no:warnings). That asymmetry is why this diff's 14 major bumps get no automated check on whether a new major still accepts the inputs the surrounding unchanged steps pass (e.g. actions/setup-python v7.0.0's release notes list "Remove the pip-install input" — harmless here, sincegrep -rn "pip-install" .github/workflows/ action.ymlis empty, but nothing would have told the author that). actionlint's popular-actions database knows the input sets for actions/checkout, setup-python, setup-node and cache, so it is the cheapest gate for exactly this class of change. Add anactionlintstep to the quality-gate job beside line 84 (pin the binary by version + sha256 as the osv-scanner step at.github/workflows/pr-checks.yml:109-122already does). Note when wiring it that running actionlint overaction.ymlproduces false "jobs section is missing" / "unexpected key" syntax-check noise (automated/actionlint.txt lines foraction.yml:1:1,:2:1,:3:1,:16:1,:71:1,:79:1) becauseaction.ymlis a composite action, not a workflow — scope the glob to.github/workflows/only. -
[Axis 4] Security-rationale comments attached to the two bumped refs in claude-pr-review.yml no longer match the pinned versions' behavior (
.github/workflows/claude-pr-review.yml:56) — Refresh the two comment blocks that justify this job's containment design; the mitigations themselves are correct and should stay, only their stated mechanism/pointer is stale. -
.github/workflows/claude-pr-review.yml:55-58says of the checkout bumped on line 51 tov7.0.1: "Default-true persistence writes the write-scoped token to disk as anhttp.extraheader" in.git/config, and the same claim is repeated at :139-141 ("GITHUB_TOKEN would be persisted to .git/config by checkout") and in the docstring oftests/test_pr_review_workflow.py::test_checkout_does_not_persist_credentials. As ofactions/checkoutv6.0.0 ("Persist creds to a separate file", actions/checkout#2286 — v6-beta notes: "store the credentials under$RUNNER_TEMPinstead of directly in the local git config"), a persisted token lands in aRUNNER_TEMPfile referenced by anincludeIf, not in.git/config.persist-credentials: false(:60) is still required and still right — the token would still be on the filesystem, just elsewhere — so the risk here is only that a future reader concludes.git/configis the sole exfil path and relaxes the setting. Reword to "persisted to disk (under RUNNER_TEMP since checkout v6) and readable by the allowlisted Read tool". -
.github/workflows/claude-pr-review.yml:86points at "see action.yml lines 299-309" for the Bedrock env forwarding in the action bumped on line 91 tov1.0.183. At that exact pinned commit (be7b93b1907a4abad570368f3c74b6fe3807510b) the forwarding is at lines 333 (CLAUDE_CODE_USE_BEDROCK: ${{ inputs.use_bedrock == 'true' && '1' || '' }}) and 342 (AWS_BEARER_TOKEN_BEDROCK: ${{ env.AWS_BEARER_TOKEN_BEDROCK }}). Prefer naming the step over hardcoding upstream line numbers, which drift on every bump.
(For the record, the containment surface itself was verified intact at the new pin: use_bedrock, include_comments_by_actor, exclude_comments_by_actor, and track_progress all still exist as inputs at that commit, so no silently-dropped input reopens the planted-comment vector.) CVSS:3.1/AV:L/AC:H/PR:H/UI:R/S:U/C:L/I:N/A:N
4. [Axis 7] CI tutorial's consumer-facing gate recipe left stale by these bumps — it still teaches action majors that no longer match any workflow in the repo, with no parity guard (.github/workflows/docs.yml:37) — This PR bumped the docs-publishing workflow itself — docs.yml:37 - uses: actions/checkout@v7 and docs.yml:38 - uses: actions/setup-python@v7 — but not the hand-rolled CI recipe that workflow publishes. docs/tutorials/02-ci-pipeline.md is the canonical "wire coder-eval into CI" surface and still shows the pre-bump majors of five of the exact actions bumped here: line 84 - uses: actions/checkout@v4, line 86 - uses: actions/setup-python@v5, line 90 - uses: astral-sh/setup-uv@v4, line 129 uses: actions/upload-artifact@v4, line 168 - uses: actions/setup-node@v4. The sharpest inconsistency is line 90: the shipped composite action now runs setup-uv v9 (action.yml:83) while the tutorial's equivalent step tells users v4 — and v9's enable-cache: auto / node24 defaults are materially different from v4's. Nothing is broken (the old major tags still resolve), so this is informational: refresh the five refs in the tutorial, or add one sentence stating the example deliberately pins majors and is not kept in lockstep with .github/workflows/. README.md:158's actions/setup-node reference is version-less and needs no change. Worth pairing with a docs/workflow parity check in the deferred actionlint harness item (.claude/harness-candidates.md:92-98) so the next grouped bump surfaces the drift mechanically instead of relying on a reviewer.
What's Missing
⚠️ Not produced. The dedicated What's-Missing synthesis pass failed to complete
(reviewer-side quota), so this section is incomplete, not clean — read it as
unmeasured rather than as "nothing missing." Partial coverage survives in Priority
Actions #2 and #5 below, and in the docs-drift item under Nits.
Harness & Lint Improvements
⚠️ Not produced. The dedicated Harness & Lint synthesis pass failed to complete
(reviewer-side quota), so this section is incomplete, not clean. Partial coverage
survives in Priority Action #4 (anactionlintstep + atests/lint/CE00n rule
asserting every non-localuses:is SHA-pinned) and in the workflow-linter item
under Nits.
Top 5 Priority Actions
- Pin the published gate's toolchain explicitly at
action.yml:83— add awith:block (version: "latest",enable-cache: "false", orenable-cache: "true"+prune-cache: "true") so setup-uv v9's defaults can no longer resolve uv from the consumer'suv.toml/[tool.uv] required-versionand hand a different uv touv tool install "coder-eval==$CE_VERSION"(action.yml:94), which is the one change here that can alter a graded run's harness/toolchain — and therefore a task's score or final_status — for byte-identical agent output. - Close the dogfood blind spot that let this through:
pr-checks.yml:856'suses: ./runs in coder_eval's own workspace (nouv.toml, no[tool.uv] required-version), so it hits setup-uv'slatestfallback exactly as v4 did — add a consumer-shaped fixture job that checks out a workspace pinning an old uv and asserts the gate still installs and runs the expected coder-eval version, and document the new Actions Runner >= 2.327.1 (node24) floor indocs/CI_GATE.md. - Converge the divergent docker build paths by SHA-pinning
.github/workflows/docker-publish.yml:68/71/78to the same commitsrelease.yml:382/387/396already uses (bb05f3f5519dd87d3ba754cc423b652a5edd6d2c# v4.2.0,abd2ef45e78c5afb21d64d4ca52ee8550d9572c7# v4.5.1,53b7df96c91f9c12dcc8a07bcb9ccacbed38856a# v7.3.0), plusdocs.yml:37/38, since docker-publish.yml runs on every push tomainwithpackages: writeand bothUV_INDEX_UIPATH_*build secrets in scope while building the samedocker/Dockerfileinto the same ghcrcoder-eval-agentrepo. - Make the pinning convention mechanical instead of conventional: add an
actionlintstep beside the custom-lint gate at.github/workflows/pr-checks.yml:84(pinned by version + sha256 like the osv-scanner step at :109-122, glob scoped to.github/workflows/so compositeaction.ymldoesn't produce false 'jobs section is missing' noise) and atests/lint/-style CE00n rule asserting every non-localuses:matches@[0-9a-f]{40}with a trailing# v…comment — todaytests/test_action_version_pin.pyonly checks action.yml's coder-eval version andtests/test_pr_review_workflow.py:107only matchesstartswith("actions/checkout@"). - Refresh the docs and rationale comments this bump left stale: update the five action majors in
docs/tutorials/02-ci-pipeline.md(lines 84, 86, 90, 129, 168 — line 90's setup-uv@v4most sharply contradicts the shipped v9 ataction.yml:83) or state the example deliberately pins majors, and correct.github/workflows/claude-pr-review.yml:55-58(and the duplicate at :139-141 plustests/test_pr_review_workflow.py::test_checkout_does_not_persist_credentials) to say credentials persist underRUNNER_TEMPsince checkout v6 rather than in.git/config, and replace the hardcoded 'action.yml lines 299-309' pointer at :86 with the step name; the one-space-vs-two version-comment drift atpr-checks.yml:170/173is free to fix in the same pass.
Stats: 0 🔴 · 1 🟠 · 1 🟡 · 4 🔵 across 8 axes reviewed.
uipreliga
left a comment
uipreliga
left a comment
There was a problem hiding this comment.
Review: chore: bump the actions-all group across 1 directory with 14 updates
PR #21 by @dependabot · dependabot/github_actions/actions-all-5a53a122b0 → main · OPEN · reviewed against 1591548 · all 8 axes · 2026-07-27T20:41Z
Supersedes the earlier review comment on this PR. That one was posted with the
What's Missing and Harness & Lint Improvements passes incomplete (they had failed
to run). Both are now complete and included below. Scores and findings are unchanged
— the two runs produced identical per-axis counts.
Change class: complex — 14 GitHub Actions bumps, 9 crossing a major version, on the release/publish path and inside the published composite action (action.yml), so correctness requires checking each action's breaking changes rather than reading the diff
The Python core is in excellent shape — type safety, architecture, and error handling are clean at 10/10 with zero critical or high findings across seven of eight axes — and the entire residual risk sits in the CI/CD surface: one high-severity unparameterized astral-sh/setup-uv v4→v9 bump in the published composite action (action.yml:83) that silently hands the gate's toolchain resolution, cache behavior, and Node runtime floor to every external @v0 consumer while the dogfood job stays green, plus a handful of unpinned workflow refs and stale docs; bottom line, merge-ready once the action's toolchain is explicitly pinned, because nothing here touches evaluation correctness but the gate itself can break under consumers with no coder-eval change to blame.
Summary
| Axis | Score | 🔴 | 🟠 | 🟡 | 🔵 | Top Issue |
|---|---|---|---|---|---|---|
| 1. Code Quality & Style | 9.4 / 10 | 0 | 0 | 1 | 1 | 5 of 60 non-local uses: refs remain on mutable major tags (incl. write-credentialed jobs and the docker buildcache producer/consumer pair) while 55 are SHA-pinned, with no guard test enforcing the convention |
| 2. Type Safety | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 3. Test Health | 9.9 / 10 | 0 | 0 | 0 | 1 | No workflow linter runs anywhere in CI, so a 9-file / 50-line workflow edit lands with zero mechanical validation |
| 4. Security | 9.9 / 10 | 0 | 0 | 0 | 1 | Security-rationale comments attached to the two bumped refs in claude-pr-review.yml no longer match the pinned versions' behavior |
| 5. Architecture & Design | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 6. Error Handling & Resilience | 10 / 10 | 0 | 0 | 0 | 0 | — |
| 7. API Surface & Maintainability | 9.9 / 10 | 0 | 0 | 0 | 1 | CI tutorial's consumer-facing snippet is left 3–5 majors behind the pins this PR sets, with no parity guard |
| 8. Evaluation Harness Quality | 9 / 10 | 0 | 1 | 0 | 0 | action.yml:83 setup-uv v4.2.0 -> v9.0.0 with no with: block silently rebinds defaults (cache/prune-cache, uv-version resolution from consumer's config, PATH mode, Node runtime) for every external @v0 consumer, with no opt-out input and no doc note |
Overall Score: 9.8 / 10 · Weakest Axis: Evaluation Harness Quality at 9 / 10
Totals: 🔴 0 · 🟠 1 · 🟡 1 · 🔵 4 across 8 axes.
Blockers
- [Axis 8] action.yml:83 setup-uv v4.2.0 -> v9.0.0 with no
with:block silently rebinds defaults (cache/prune-cache, uv-version resolution from consumer's config, PATH mode, Node runtime) for every external@v0consumer, with no opt-out input and no doc note (action.yml:83) —action.yml:83isuses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0with nowith:block, so all four of these defaults changed under it (read from the two pinned action.yml blobs at SHA38f3f10…= v4.2.0 andc771a70…= v9.0.0): version: v4default: "latest"→ v9default: ""("Defaults to the version in pyproject.toml or 'latest'"). v9 resolves it fromworking-directory(v9 default${{ github.workspace }}= the CONSUMER's repo) viagetWorkspaceCandidates()→uv.toml, thenpyproject.toml→pyproject.tool?.uv?.["required-version"](setup-uvsrc/version/version-request-resolver.ts+src/version/file-parser.ts). A consumer repo that pins an old uv now gets THAT uv for the next step'suv tool install "coder-eval==$CE_VERSION"(action.yml:94).enable-cache:"false"→"auto"(= true on GitHub-hosted). The gate now restores AND saves a uv cache entry into the consumer's 10 GB repo cache uninvited, keyed on the consumer's own files (cache-dependency-globnow includes**/pyproject.toml,**/uv.lock,**/*requirements*.txt).prune-cache:"true"→"false"(v9.0.0's headline breaking change) — that cache entry is unpruned.- Runtime
using: "node20"→using: "node24"(v7.0.0 breaking change), requiring Actions Runner ≥ 2.327.1; on an older self-hosted runner the gate's FIRST step hard-fails.
Blast radius: per CLAUDE.md,release.ymlmaintains action.yml'sversion:default *plus the movingv<major>tag, so everyUiPath/coder_eval@v0consumer (README.md:105, docs/CI_GATE.md:22) picks this up with no coder-eval version change to blame. The harness's own coverage cannot catch it: the dogfood job (pr-checks.yml:856 uses: ./) runs in coder_eval's workspace, which has nouv.tomland no[tool.uv] required-version/uvrequirement inpyproject.toml, so it still resolveslatestexactly as v4 did — green dogfood, broken consumer. Fix: make the published gate's toolchain a property of coder-eval, not of the consumer's repo — addwith:on action.yml:83 pinning the previous behavior explicitly (version: "latest",enable-cache: "false"; orenable-cache: "true"+prune-cache: "true"` if caching is wanted), and state the Runner ≥ 2.327.1 floor in docs/CI_GATE.md.
Non-blocking, but please consider before merge
- [Axis 1] 5 of 60 non-local
uses:refs remain on mutable major tags (incl. write-credentialed jobs and the docker buildcache producer/consumer pair) while 55 are SHA-pinned, with no guard test enforcing the convention (.github/workflows/docker-publish.yml:68) — This PR bumped everyuses:ref in the repo, but did so in two different styles. 55 of the 60 third-party refs (per tmp/code-review-260727-1341/automated/uses-matrix.txt) are full-SHA + trailing version comment, e.g..github/workflows/docker-publish.yml:54: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1. Five refs the PR touched stayed on mutable tags:docker-publish.yml:68: uses: docker/setup-buildx-action@v4,:71: uses: docker/login-action@v4,:78: uses: docker/build-push-action@v7,docs.yml:37: - uses: actions/checkout@v7,:38: - uses: actions/setup-python@v7. The worst part is the divergent duplicate: docker-publish.yml:68/71/78 and release.yml:382/387/396 run the same three actions to build the same image from the samefile: docker/Dockerfileinto the same ghcrcoder-eval-agentrepo, yet release.yml is SHA-pinned (docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0,docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1,docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0) while docker-publish.yml is not. The two build paths can therefore silently run different action code, and nothing mechanically flags it:tests/test_action_version_pin.pyonly asserts action.yml'sdefault: "…" # <-- kept in synccoder-eval version, andtests/test_pr_review_workflow.py:107only matchesstartswith("actions/checkout@"). Fix: SHA-pin those 5 refs to the same commits already used elsewhere for the docker trio (bb05f3f…/abd2ef4…/53b7df9…) so both build paths are byte-identical, and add a guard — either an actionlint/CI grep asserting every non-localuses:matches@[0-9a-f]{40}with a trailing# v…comment, or atests/lint/-style test in the CE00n family walking.github/workflows/*.yml+action.yml. Otherwise state explicitly in a comment why docker-publish.yml is exempt.
Nits
-
[Axis 1] Version-comment spacing drift on two lines this PR rewrote (one space before
#vs two everywhere else) (.github/workflows/pr-checks.yml:170) — Every other SHA-pinned ref in the repo separates the ref from its version comment with two spaces, e.g.pr-checks.yml:237: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1. The two lines in theno-uipath-extrajob use one:pr-checks.yml:170: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1andpr-checks.yml:173: uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0. The drift is pre-existing but both lines were rewritten by this PR, so it is free to fix: add the second space to match the surrounding 55 refs. Purely cosmetic — noted, not blocking. -
[Axis 3] No workflow linter runs anywhere in CI, so a 9-file / 50-line workflow edit lands with zero mechanical validation (
.github/workflows/pr-checks.yml:84) —grep -rn "actionlint\|zizmor\|yamllint" .github/ Makefile .pre-commit-config.yaml pyproject.tomlreturns no matches — the CI-config surface has no syntax or action-metadata gate at all, while the Python surface has ruff + pyright + a full CE001+ custom-lint suite (.github/workflows/pr-checks.yml:84:run: .venv/bin/pytest tests/test_custom_lint.py -v --tb=short --no-header -p no:warnings). That asymmetry is why this diff's 14 major bumps get no automated check on whether a new major still accepts the inputs the surrounding unchanged steps pass (e.g. actions/setup-python v7.0.0's release notes list "Remove the pip-install input" — harmless here, sincegrep -rn "pip-install" .github/workflows/ action.ymlis empty, but nothing would have told the author that). actionlint's popular-actions database knows the input sets for actions/checkout, setup-python, setup-node and cache, so it is the cheapest gate for exactly this class of change. Add anactionlintstep to the quality-gate job beside line 84 (pin the binary by version + sha256 as the osv-scanner step at.github/workflows/pr-checks.yml:109-122already does). Note when wiring it that running actionlint overaction.ymlproduces false "jobs section is missing" / "unexpected key" syntax-check noise (automated/actionlint.txt lines foraction.yml:1:1,:2:1,:3:1,:16:1,:71:1,:79:1) becauseaction.ymlis a composite action, not a workflow — scope the glob to.github/workflows/only. -
[Axis 4] Security-rationale comments attached to the two bumped refs in claude-pr-review.yml no longer match the pinned versions' behavior (
.github/workflows/claude-pr-review.yml:56) — Refresh the two comment blocks that justify this job's containment design; the mitigations themselves are correct and should stay, only their stated mechanism/pointer is stale. -
.github/workflows/claude-pr-review.yml:55-58says of the checkout bumped on line 51 tov7.0.1: "Default-true persistence writes the write-scoped token to disk as anhttp.extraheader" in.git/config, and the same claim is repeated at :139-141 ("GITHUB_TOKEN would be persisted to .git/config by checkout") and in the docstring oftests/test_pr_review_workflow.py::test_checkout_does_not_persist_credentials. As ofactions/checkoutv6.0.0 ("Persist creds to a separate file", actions/checkout#2286 — v6-beta notes: "store the credentials under$RUNNER_TEMPinstead of directly in the local git config"), a persisted token lands in aRUNNER_TEMPfile referenced by anincludeIf, not in.git/config.persist-credentials: false(:60) is still required and still right — the token would still be on the filesystem, just elsewhere — so the risk here is only that a future reader concludes.git/configis the sole exfil path and relaxes the setting. Reword to "persisted to disk (under RUNNER_TEMP since checkout v6) and readable by the allowlisted Read tool". -
.github/workflows/claude-pr-review.yml:86points at "see action.yml lines 299-309" for the Bedrock env forwarding in the action bumped on line 91 tov1.0.183. At that exact pinned commit (be7b93b1907a4abad570368f3c74b6fe3807510b) the forwarding is at lines 333 (CLAUDE_CODE_USE_BEDROCK: ${{ inputs.use_bedrock == 'true' && '1' || '' }}) and 342 (AWS_BEARER_TOKEN_BEDROCK: ${{ env.AWS_BEARER_TOKEN_BEDROCK }}). Prefer naming the step over hardcoding upstream line numbers, which drift on every bump.
(For the record, the containment surface itself was verified intact at the new pin: use_bedrock, include_comments_by_actor, exclude_comments_by_actor, and track_progress all still exist as inputs at that commit, so no silently-dropped input reopens the planted-comment vector.) CVSS:3.1/AV:L/AC:H/PR:H/UI:R/S:U/C:L/I:N/A:N
4. [Axis 7] CI tutorial's consumer-facing snippet is left 3–5 majors behind the pins this PR sets, with no parity guard (.github/workflows/docs.yml:37) — This PR bumped the docs-publishing workflow itself — docs.yml:37 - uses: actions/checkout@v7 and docs.yml:38 - uses: actions/setup-python@v7 — but not the hand-rolled CI recipe that workflow publishes. docs/tutorials/02-ci-pipeline.md is the canonical "wire coder-eval into CI" surface and still shows the pre-bump majors of five of the exact actions bumped here: line 84 - uses: actions/checkout@v4, line 86 - uses: actions/setup-python@v5, line 90 - uses: astral-sh/setup-uv@v4, line 129 uses: actions/upload-artifact@v4, line 168 - uses: actions/setup-node@v4. The sharpest inconsistency is line 90: the shipped composite action now runs setup-uv v9 (action.yml:83) while the tutorial's equivalent step tells users v4 — and v9's enable-cache: auto / node24 defaults are materially different from v4's. Nothing is broken (the old major tags still resolve), so this is informational: refresh the five refs in the tutorial, or add one sentence stating the example deliberately pins majors and is not kept in lockstep with .github/workflows/. README.md:158's actions/setup-node reference is version-less and needs no change. Worth pairing with a docs/workflow parity check in the deferred actionlint harness item (.claude/harness-candidates.md:92-98) so the next grouped bump surfaces the drift mechanically instead of relying on a reviewer.
What's Missing
Parallel paths:
- 🟡 The two GHCR build paths were bumped in opposite styles instead of converged:
docker-publish.yml:68/71/78went floating@v4/@v4/@v7whilerelease.yml:382/387/396went pinnedbb05f3f… # v4.2.0/abd2ef4… # v4.5.1/53b7df9… # v7.3.0— samefile: docker/Dockerfile, sameghcr.io/<owner>/coder-eval-agentimage, same build secrets. The PR should have pinned docker-publish.yml to the identical three SHAs so both paths run byte-identical action code. (trigger: .github/workflows/docker-publish.yml) (restates: Axis 1: 5 of 60 non-localuses:refs remain on mutable major tags) - 🟠
astral-sh/setup-uvwas bumped to v9.0.0 at three sites but only two were kept parameterized:release.yml:96andpublish-testpypi.yml:55both carrywith: enable-cache: true, whileaction.yml:83— the one site that executes in a consumer's workspace — has nowith:block at all, so the v9 default flips (version: ""resolved from the consumer'spyproject.toml/uv.toml,enable-cache: auto,prune-cache: false, node24) land only on the externally-published path. The parallel-site edit that pins the prior behavior on action.yml was not made. (trigger: action.yml) (restates: Axis 8: action.yml:83 setup-uv v4.2.0 -> v9.0.0 with nowith:block silently rebinds defaults) - 🔵
docs.yml:37/38are the only twoactions/checkout/actions/setup-pythonrefs in the repo left floating (@v7/@v7) after this PR; the other 12 checkout refs and 10 setup-python refs across the same 9 files all got3d3c42e5… # v7.0.1/5fda3b95… # v7.0.0. The bump touched the file, so the pin was free to apply there too. (trigger: .github/workflows/docs.yml) (restates: Axis 1: 5 of 60 non-localuses:refs remain on mutable major tags) - 🔵 The container toolchain was not moved with the CI toolchain: CI/action now install uv via
setup-uv@… # v9.0.0, whiledocker/Dockerfile:40(the image this workflow builds and publishes ascoder-eval-agent:latest) still installs uv from an unversionedhttps://astral.sh/uv/install.sh. The agent image and the harness can therefore run different uv majors, and nothing — no pin, no comment, no test — keeps the two in step. (trigger: .github/workflows/docker-publish.yml)
Tests:
- 🟡 No test encodes the SHA-pin convention this PR mostly follows:
tests/test_action_version_pin.pyonly asserts action.yml'scoder-eval==<version>default matches pyproject, andtests/test_pr_review_workflow.py:107only matchesstartswith("actions/checkout@"). A ~15-line test walking.github/workflows/*.yml+action.ymland asserting every non-localuses:matches@[0-9a-f]{40}with a trailing# v…comment (plus an explicit exemption list) would have failed this PR on all 5 floating refs. (trigger: .github/workflows/docker-publish.yml) (restates: Axis 1: 5 of 60 non-localuses:refs remain on mutable major tags) - 🟠 The published action's only coverage is structurally blind to the setup-uv bump:
pr-checks.yml:856runsuses: ./withversion: localinside coder_eval's own workspace, which has nouv.tomland no[tool.uv] required-version, so v9 still resolveslatestexactly as v4 did and theuv tool install "coder-eval==$CE_VERSION"PyPI branch is never taken. Missing: a second dogfood invocation withworking-directory(or a fixture checkout) containing arequired-versionpin, which is the case that actually regresses for@v0consumers. (trigger: action.yml) (restates: Axis 8: action.yml:83 setup-uv v4.2.0 -> v9.0.0 with nowith:block silently rebinds defaults) - 🔵
tests/test_pr_review_workflow.py::test_checkout_does_not_persist_credentialsstill documents the pre-v6 mechanism ("writes the write-scoped GITHUB_TOKEN into .git/config") for the checkout ref this PR bumped to v7.0.1; the assertion is still correct but its stated rationale is not, and no test covers the RUNNER_TEMP/includeIflocation the token now takes. The guard test should have been updated in the same commit as the ref it guards. (trigger: .github/workflows/claude-pr-review.yml) (restates: Axis 4: Security-rationale comments attached to the two bumped refs in claude-pr-review.yml are stale) - 🔵 A 9-file / 14-action-bump change to the CI surface merged with zero mechanical validation —
grep -rn "actionlint\|zizmor\|yamllint"over.github/,Makefile,.pre-commit-config.yaml,pyproject.tomlreturns nothing — while the Python surface has ruff + pyright + the CE001+ suite atpr-checks.yml:84. Anactionlintstep (scoped to.github/workflows/only;action.ymlis a composite and produces false "jobs section is missing" noise) is the missing gate for exactly this diff class. (trigger: .github/workflows/pr-checks.yml) (restates: Axis 3: No workflow linter runs anywhere in CI)
Downstream consumers:
- 🟡 The release artifact hand-off crossed two majors on both ends in one shot with no round-trip exercised anywhere:
release.yml:241producesdistwithupload-artifactv4.6.2 -> v7.0.1 andrelease.yml:429consumes it in a separate job withdownload-artifactv4.3.0 -> v8.0.1. Every other upload-artifact site (pr-checks.yml:129/147/321/500/673/751/829/887) is upload-only, so PR CI proves nothing about the pairing; the first real download happens mid-release, immediately before the PyPI publish step. (trigger: .github/workflows/release.yml) - 🟡 The registry buildcache has a cross-workflow producer/consumer contract that this PR left straddling two pin styles:
docker-publish.yml:78(build-push-action@v7, floating) is the sole writer (cache-to: …:buildcache,mode=max) andrelease.yml:396(build-push-action@53b7df96… # v7.3.0, pinned) is a reader-only consumer whose comment explicitly says "Read the shared buildcache docker-publish.yml writes". Cache-format compatibility for the release build is now a function of a mutable tag on the writer side. (trigger: .github/workflows/docker-publish.yml) (restates: Axis 1: 5 of 60 non-localuses:refs remain on mutable major tags) - 🟡 Nothing downstream of the composite action was updated to announce its behavior change:
docs/CI_GATE.md(which advertisesuses: UiPath/coder_eval@v0at line 22 and documents theversioninput at line 43) gains no note about the new Actions Runner >= 2.327.1 (node24) floor, the uninvited uv cache write into the consumer's repo cache, or uv resolution from the consumer'spyproject.toml; and the CHANGELOG entryrelease.ymlgenerates is a Python-package semver that carries no signal about the gate's toolchain.@v0consumers get this force-moved onto them (release.yml:229-230) with nothing to attribute a breakage to. (trigger: action.yml) (restates: Axis 8: action.yml:83 setup-uv v4.2.0 -> v9.0.0 with nowith:block silently rebinds defaults)
Display & mapping dicts:
- 🔵 The version-to-docs mapping was not extended:
docs/tutorials/02-ci-pipeline.mdstill showscheckout@v4(:84),setup-python@v5(:86),setup-uv@v4(:90),upload-artifact@v4(:129),setup-node@v4(:168) — the pre-bump major of five of the actions changed here — anddocs/CI_GATE.md:128similarly carriesmikepenz/action-junit-report@v5. Either refresh them or add one line stating the examples deliberately pin majors and are not kept in lockstep with.github/workflows/. (trigger: .github/workflows/docs.yml) (restates: Axis 7: CI tutorial's consumer-facing snippet is left 3–5 majors behind) - 🔵
.github/dependabot.yml'sgithub-actionsblock states the repo policy in a comment — "No ignore block — keep SHA pins current for patch and minor releases too, otherwise SHA pinning fossilizes the actions in place" — but Dependabot rewrites a floating tag as another floating tag, so the 5 refs it just moved to@v4/@v4/@v7/@v7/@v7can never acquire a pin. Neither the policy comment nor any exemption list was updated to record that these 5 are permanently outside it. (trigger: .github/workflows/docker-publish.yml) (restates: Axis 1: 5 of 60 non-localuses:refs remain on mutable major tags) - 🔵 The pin-comment format is a de-facto mapping (
<40-hex>+ two spaces +# vX.Y.Z) and two lines this PR rewrote don't match it:pr-checks.yml:170and:173use a single space before#where the other 53 pinned refs use two. Free to normalize on lines the diff already touches. (trigger: .github/workflows/pr-checks.yml) (restates: Axis 1: Version-comment spacing drift on two lines this PR rewrote)
Daily/nightly:
- 🟠 The PR states nothing about the production publish path, and none of it runs on PRs:
release.ymlisworkflow_dispatch-only and crossed majors on six actions at once (checkout v7, setup-python v7, setup-uv v9, upload-artifact v7, download-artifact v8, docker v4/v4/v7) on the job that does OIDC Trusted Publishing to public PyPI, pushes the versioned GHCR image, and force-moves thev<major>action tag;publish-testpypi.yml(the designated rehearsal) exercises neither the artifact hand-off nor the docker path. The workflow's own header documents the mitigation — a PRERELEASE dispatch from a non-main branch, which the ADO nightly infra then pins viacoderEvalVersion— so the missing step is dispatchingrelease.ymloff this branch (andpublish-testpypi.yml) before merge, and saying so in the PR. (trigger: .github/workflows/release.yml) - 🟡
docker-publish.ymlruns on every push tomain, so the merge commit itself is the first-ever execution ofsetup-buildx@v4+login@v4+build-push@v7— withpackages: writeand theUV_INDEX_UIPATH_USERNAME/UV_INDEX_UIPATH_PASSWORDbuild secrets in scope, and writing the shared:buildcachetag that the release build later consumes. A failure or a cache-format change here silently degradescoder-eval-agent:latest(what downstream agent runs pull) with no pre-merge signal and no note in the PR. (trigger: .github/workflows/docker-publish.yml) - 🔵
codeql.ymlcarries a weekly cron (0 6 * * 1) plussecurity-events: write, and itsinit/analyzecrossed v3.35.5 -> v4.37.3 withqueries: security-and-quality. It does at least run onpull_request, so this PR's own CodeQL run is the evidence that v4 accepts the config — the gap is that the PR does not cite it, and the workflow has noworkflow_dispatchtrigger, so if the scheduled run later regresses there is no manual re-run path and no required check that would go red. (trigger: .github/workflows/codeql.yml)
Harness & Lint Improvements
Static checks (lint / type):
- [ce-lint] CE026 (class 1) —
uses:pin form. New whole-tree rule wired as a@pytest.mark.linttest class intests/test_custom_lint.py(the CE027–CE031 pattern, not aBaseRule— it walks YAML text, not one.pyAST at a time), with the walker helper intests/lint/workflow_pins.py. Statement: every non-localuses:ref in.github/workflows/*.ymlandaction.ymlmust match^uses: [^./].*@[0-9a-f]{40} # v\S+$— a 40-hex commit SHA, exactly two spaces, and a trailing# v<version>comment. Local refs (./,./.github/...) are exempt; any deliberate exception needs anEXEMPTentry with a reason (mirrors CE030's exemption mechanism).CE026is the id already earmarked for this pattern in.claude/harness-candidates.md; it is free (implemented rules stop at CE031), while CE032/CE033 are earmarked for the workflow-heredoc rules, so subsequent new rules below claim CE034+. Runs offline inmake lint, so it gates before any network-dependent job. Prevents: A1/A3/A4/A5 medium — the 5 floating refs (docker-publish.yml:68/71/78docker/setup-buildx-action@v4/docker/login-action@v4/docker/build-push-action@v7,docs.yml:37/38actions/checkout@v7/actions/setup-python@v7) against 55 SHA-pinned siblings, with no guard test (tests/test_action_version_pin.pyonly checks action.yml's coder-eval version;tests/test_pr_review_workflow.py:107onlystartswith("actions/checkout@")). The two-space clause also catches A1 low — the one-space drift onpr-checks.yml:170and:173. It additionally closes the Dependabot hole documented in.github/dependabot.yml(a floating tag is rewritten as another floating tag, so those refs never acquire a pin). - [ce-lint] CE026 (class 2) — cross-file SHA agreement. Second violation class in the same rule: an action referenced from more than one file must resolve to the same SHA everywhere, unless listed in an
EXEMPTmap with a reason. Implementation is a dict-of-sets over the refs CE026 class 1 already parsed (action_name -> {sha}), failing whenlen(shas) > 1. Keeps the two docker build paths byte-identical by construction rather than by reviewer vigilance. Prevents: A1/A3/A4/A5 medium, specifically the divergent-duplicate half:docker-publish.yml:68/71/78andrelease.yml:382/387/396run the same three docker actions to build the samedocker/Dockerfileinto the same ghcrcoder-eval-agentrepo, but only release.yml is pinned (bb05f3f5…v4.2.0,abd2ef45…v4.5.1,53b7df96…v7.3.0). The verify pass confirmed this divergence is pre-existing and survived a PR that touched both sides — exactly the shape a mechanical check exists for. - [ce-lint] CE034 — published composite action must not inherit upstream defaults. New whole-tree lint test class scoped to
action.yml(and any future published composite action): everyuses:step in a published composite action must carry awith:block that explicitly sets the upstream inputs that determine toolchain behavior — for the setup-family actions that means at minimumversion:, plusenable-cache:/prune-cache:forastral-sh/setup-uvandcache:foractions/setup-python/setup-node. Encode the required-input set as a small{action_name: frozenset[str]}table so it is auditable and extensible; an unknown action defaults to "must declareversion:". Rationale to put in the rule docstring:action.ymlexecutes inside the consumer's workspace, so an unparameterizeduses:silently re-binds behavior to whatever upstream defaults ship next, for everyUiPath/coder_eval@v0consumer. Prevents: A8 high (merged A2/A5/A6/A7) —action.yml:83astral-sh/setup-uv@c771a70e… # v9.0.0with nowith:block, which flipped four defaults under the published gate:version"latest"→""(now resolved from the consumer'suv.toml/[tool.uv] required-versionvia the newworking-directory: ${{ github.workspace }}input),enable-cache"false"→"auto",prune-cache"true"→"false", andusing: node20→node24. The rule would have failed the PR atmake lint, before the movingv<major>tag carried it to consumers. - [ce-lint] CE035 — doc ↔ workflow action-version parity. New doc-surface lint test class in the CE028/CE030 family (helper in
tests/lint/doc_action_parity.py): for anyuses: <action>@vNappearing inREADME.mdordocs/**/*.md,Nmust be ≥ the major pinned for that action in.github/workflows/*.ymloraction.yml(the major is read from the trailing# v…comment CE026 already mandates — the two rules compose). Escape hatch: an inline<!-- action-version-exempt: <reason> -->above the fence, for examples that deliberately show an older major. This is the same SSOT-parity shape the repo already enforces for env vars (CE027), docs indexes (CE028), and schema fields (CE030). Prevents: A7 low (merged A3/A8) —docs/tutorials/02-ci-pipeline.mdstill showsactions/checkout@v4(:84),actions/setup-python@v5(:86),astral-sh/setup-uv@v4(:90),actions/upload-artifact@v4(:129),actions/setup-node@v4(:168) while the shipped gate runs setup-uv v9 (action.yml:83) — a 5-major gap on the canonical "wire coder-eval into CI" surface, where v9'senable-cache: auto/ node24 behavior is materially different from what the tutorial describes. - [ce-lint] CE036 — checkout credential hygiene in privileged/untrusted jobs. New workflow-YAML lint test class: any
actions/checkoutstep in a job that declares a writepermissions:scope, or lives in a workflow triggered bypull_request_target/issue_comment/workflow_run, must setpersist-credentials: false(or carry anEXEMPTentry naming the step that needs the on-disk token). Parses the workflow withyaml.safe_loadand walksjobs.*.steps, so it is robust to comment rewording. This turns a hand-written rationale comment into an executable invariant. Prevents: A4 low — the security-rationale comments atclaude-pr-review.yml:55-58and:139-141(and the docstring oftests/test_pr_review_workflow.py::test_checkout_does_not_persist_credentials) still assert the token lands in.git/config, which stopped being true atactions/checkoutv6.0.0 (credentials moved under$RUNNER_TEMPbehind anincludeIf). The stated risk is that a future reader concludes.git/configis the sole exfil path and relaxespersist-credentials: false; with CE036 that relaxation failsmake lintno matter how stale the prose is. - [ce-lint] CE037 — no hardcoded upstream line-number citations in CI config comments. Cheap regex class (can ride inside CE026's file walk): comments in
.github/workflows/*.ymlandaction.ymlmay not cite line numbers in a third-party file — forbid\bline[s]?\s+\d+(\s*[-–]\s*\d+)?\bin a comment that also names a.yml/.yaml/.ts/.jspath, and require naming the step/input instead. Upstream line numbers drift on every SHA bump and nothing revalidates them. Prevents: A4 low, item 2 —claude-pr-review.yml:86says "see action.yml lines 299-309" for the Bedrock env forwarding, but at the newly pinnedbe7b93b1…(v1.0.183) that forwarding is at lines 333 and 342. Pure prose rot that a 10-line regex makes impossible to reintroduce.
Harness improvements (not statically reachable):
- Add
actionlint(+ optionallyzizmor) over.github/workflows/**to the quality-gate job, beside the custom-lint step at.github/workflows/pr-checks.yml:84, with the binary pinned by version + sha256 exactly as the osv-scanner step atpr-checks.yml:109-122already does. Scope the glob to.github/workflows/only — running it overaction.ymlproduces spurious "jobs section is missing" / "unexpected key" errors because a composite action is not a workflow. Start non-blocking (annotations only) for one cycle, then flip to blocking. This is the already-deferred item at.claude/harness-candidates.md:92-98; CE026 above is the offline pin gate, actionlint is the semantic one, and they are complementary rather than redundant. Why not static: actionlint's value here is its bundled shellcheck pass overrun:bodies and its popular-actions database of upstream input sets and deprecations — data that lives outside this repo and is refreshed with the binary. Noruff/pyright/CE AST rule can reproduce it (tests/lint/runner.py::check_pathswalks only*.pyundersrc/), so it must be an external pinned binary invoked as a CI/makestep. Prevents: A3 low — the whole CI-config surface has zero mechanical validation today (grep -rn "actionlint\|zizmor\|yamllint" .github/ Makefile .pre-commit-config.yaml pyproject.tomlis empty) while the Python surface has ruff + pyright + CE001+. It is the generic net for "a new major dropped an input the surrounding unchanged steps still pass" — the class behind A8 high and the 14 major bumps in this PR. - Consumer-simulation job for the published composite action. The existing dogfood step (
pr-checks.yml:856 uses: ./) runs in coder_eval's own workspace, which has nouv.tomland no[tool.uv] required-version, so it provably resolves setup-uv'sversiontolatestexactly as before the bump: green dogfood, changed consumer. Add a second matrix leg that checks out a synthetic minimal consumer fixture (tests/fixtures/ci-gate-consumer/— apyproject.tomldeclaring an old[tool.uv] required-version, nouv.lock), invokes the action from there, and asserts the resolveduv --versionandcoder-eval --versionare what coder-eval intends rather than what the consumer's files dictate. Also assert no unexpected cache entry is written. Why not static: The defect is invisible inaction.yml's text — the file is byte-identical in the passing and failing cases. It only manifests from the consumer's workspace contents at action runtime, so it needs a live runner, a realuv tool install, and network egress; no offline lint can reach it. CE034 above blocks the unparameterizeduses:; this job is what proves the parameterization actually produces the intended toolchain. Prevents: A8 high (merged A2/A5/A6/A7) —action.yml:83setup-uv v4.2.0 → v9.0.0, whereversionis now resolved from the consumer'spyproject.toml/uv.tomlviaworking-directory: ${{ github.workspace }}. Generally: closes the "our dogfood job is not a consumer" blind spot for every future change to the published gate. - Upstream action-metadata diff on grouped Dependabot bumps. Add a workflow (triggered on PRs touching
.github/workflows/**oraction.yml) that, for every changeduses:ref, fetches the upstreamaction.ymlat both the old and new SHAs and posts a diff ofinputs:(names +default:values +deprecationMessage) andruns.using. Comment it on the PR so default flips are visible in review instead of requiring the reviewer to go read upstream sources by hand. Why not static: Requires network fetches of two upstream blobs per ref; it is inherently a CI job with GitHub API access, not an offline gate inmake verify. (Vendoring the metadata to make it offline would just move the staleness problem into this repo.) Prevents: A8 high — every one of the four setup-uv default flips (version"latest"→"",enable-cache"false"→"auto",prune-cache"true"→"false",usingnode20→node24, plus the 7-entrycache-dependency-globand the newworking-directoryinput) is a one-line entry in that diff. It also surfaces A4's class:actions/checkoutv6's credential-persistence change would have shown up as a metadata/behavior delta rather than silently invalidating the rationale comments atclaude-pr-review.yml:55-58. - Runner-floor guard for the published gate. Resolve the
runs.using:runtime of every action pinned inaction.yml, compute the implied minimum Actions Runner version (node24 ⇒ ≥ 2.327.1), and fail if it exceeds the floor documented indocs/CI_GATE.md. Ship it as a scheduled/low-frequency job rather than a per-PR blocker, and pair it with an explicit "Requirements" line indocs/CI_GATE.md(todaygrep -n "2.327\|node24\|Runner" docs/CI_GATE.md README.mdreturns nothing). Why not static: The runtime of a pinned third-party action is recorded in itsaction.yml, not ours, so determining it needs a network fetch at the pinned SHA. Only the docs half (a documented floor exists at all) is locally checkable. Prevents: A8 high, node24 sub-claim — on an Actions Runner older than 2.327.1 the gate's first step hard-fails for@v0consumers, with no coder-eval release note to attribute it to (the semver bumprelease.ymlpublishes is a Python-package version and carries no signal that the action's CI toolchain changed). - Record the outcome back into
.claude/harness-candidates.mdand close the loop on CE026. The SHA-pinning rule has now been proposed by two independent reviews (the 260701-1954 entry and this one) and observed as a live defect in a write-credentialed, secrets-scoped workflow (docker-publish.ymlruns on every push tomainwithpackages: writeand bothUV_INDEX_UIPATH_*build secrets in scope). Promote CE026 out of the candidates file in this cycle rather than deferring it a third time, and add a short "CI-config surface" section toCLAUDE.md's lint guidance noting that.github/workflows/**+action.ymlare in scope for CE rules — the current text reads as if onlysrc/**/*.pyis lintable, which is plausibly why the gap persisted. Why not static: This is a process/documentation item about which gates exist, not a defect a gate can detect. Prevents: Recurrence of the entire A1/A3/A4/A5 cluster: every workflow finding in this review, and in the 2026-07-24 review before it, was caught by a human reading YAML.
Top 5 Priority Actions
- Add an explicit
with:block toaction.yml:83pinningversion: "latest"andenable-cache: "false"(orenable-cache: "true"+prune-cache: "true"), so the published gate's uv toolchain is a property of coder-eval rather than of the consumer'suv.toml/[tool.uv] required-version— today a consumer pinning an old uv silently supplies the uv that runsuv tool install "coder-eval==$CE_VERSION"at action.yml:94, which can change a gate's pass/fail outcome for identical agent output, and the dogfood job (.github/workflows/pr-checks.yml:856 uses: ./) provably cannot catch it. - Document the Actions Runner >= 2.327.1 floor in
docs/CI_GATE.md(setup-uv v9 movedusing:from node20 to node24), since on an older self-hosted runner the gate's very first step now hard-fails before any evaluation runs. - SHA-pin the three floating docker refs at
.github/workflows/docker-publish.yml:68/71/78to the commitsrelease.yml:382/387/396already uses (bb05f3f5…v4.2.0,abd2ef45…v4.5.1,53b7df96…v7.3.0) — both paths build the samedocker/Dockerfileinto the same ghcrcoder-eval-agentrepo, yet only this one runs on every push tomainwithpackages: writeand theUV_INDEX_UIPATH_*secrets in scope while resolving mutable tags. - Add a mechanical guard for the SHA-pin convention plus workflow linting next to the custom-lint step at
.github/workflows/pr-checks.yml:84— atests/lint/-style CE00n rule asserting every non-localuses:in.github/workflows/*.ymlandaction.ymlmatches@[0-9a-f]{40}with a trailing# v…, and anactionlintstep scoped to.github/workflows/only (it emits false syntax errors on the compositeaction.yml) — today the CI-config surface has zero validation while the Python surface has ruff, pyright, and CE001+. - Refresh the stale consumer-facing and rationale text left behind by these bumps: the five action majors in
docs/tutorials/02-ci-pipeline.md:84/86/90/129/168(notably setup-uv@v4versus the shippedaction.yml:83v9), the.git/configcredential-persistence claim at.github/workflows/claude-pr-review.yml:55-58and:139-141(checkout v6+ persists underRUNNER_TEMP, sopersist-credentials: falseis still required), the hardcoded upstream line pointer atclaude-pr-review.yml:86, and the one-space comment drift at.github/workflows/pr-checks.yml:170/173.
Stats: 0 🔴 · 1 🟠 · 1 🟡 · 4 🔵 across 8 axes reviewed.
2 participants
Bumps the actions-all group with 14 updates in the / directory:
4.2.09.0.0471.0.1691.0.1833.35.54.37.33.35.54.37.3343467575.0.56.1.04.6.27.0.14.4.07.0.01.13.01.14.14.3.08.0.1Updates
astral-sh/setup-uvfrom 4.2.0 to 9.0.0Release notes
Sourced from astral-sh/setup-uv's releases.
... (truncated)
Commits
c771a70chore(deps): roll up Dependabot updates (#970)2f537cachore: update known checksums for 0.11.30 (#968)2269552Speed up version client by partial response reads (#807)47a7f4fChangeprune-cachedefault tofalse(#967)71966efchore(deps): roll up Dependabot updates (#962)f12b1f0fix: fall back to distribution ID when os-release has no version field (#961)ecd24ddchore: update known checksums for 0.11.29 (#960)6a19136docs: update version references to v8.3.2 (#949)11f9893chore: roll up Dependabot updates (#948)f798556docs: update version references to v8.3.1 (#946)Updates
actions/checkoutfrom 4 to 7Release notes
Sourced from actions/checkout's releases.
... (truncated)
Commits
3d3c42eprep v7.0.1 release (#2531)2880268escape values passed to --unset (#2530)12cd223trim only ascii whitespace for branch (#2521)62661c4skip running unsafe pr check if input is default (#2518)e8d4307Bump the minor-actions-dependencies group with 2 updates (#2499)631c942eslint 9 (#2474)4f1f4aeBump actions/upload-artifact from 4 to 7 (#2476)ba09753Bump actions/checkout from 6 to 7 (#2488)b9e0990Bump docker/login-action from 3.3.0 to 4.2.0 (#2479)e8cb398Bump docker/build-push-action from 6.5.0 to 7.2.0 (#2478)Updates
anthropics/claude-code-actionfrom 1.0.169 to 1.0.183Release notes
Sourced from anthropics/claude-code-action's releases.
... (truncated)
Commits
be7b93bchore: bump Claude Code to 2.1.220 and Agent SDK to 0.3.220e0cf66dchore: bump Claude Code to 2.1.219 and Agent SDK to 0.3.21944423bdchore: bump Claude Code to 2.1.218 and Agent SDK to 0.3.218b00a341fix: share one exchanged WIF credential across spawned Claude processes (#1407)fa7e2f0chore: bump Claude Code to 2.1.217 and Agent SDK to 0.3.217b76a077chore: bump Claude Code to 2.1.216 and Agent SDK to 0.3.216af0559echore: bump Claude Code to 2.1.215 and Agent SDK to 0.3.2153553f84chore: bump Claude Code to 2.1.214 and Agent SDK to 0.3.214700e7f8chore: bump Claude Code to 2.1.212 and Agent SDK to 0.3.2123e807ecfix: handle null comment/review author from deleted accounts (#1490)Updates
github/codeql-action/initfrom 3.35.5 to 4.37.3Release notes
Sourced from github/codeql-action/init's releases.
... (truncated)
Changelog
Sourced from github/codeql-action/init's changelog.
... (truncated)
Commits
e4fba86Merge pull request #4031 from github/update-v4.37.3-72f6a9da0fb50ab5Update changelog for v4.37.372f6a9dMerge pull request #4030 from github/mbg/fix/no-proxy3b5ee58Use defaultrequestoptions instead ofundefinedbfb6be4Merge pull request #4028 from github/mergeback/v4.37.2-to-main-e0647621526ab84Rebuildd6217b9Update changelog and version after v4.37.2e064762Merge pull request #4027 from github/update-v4.37.2-385bcdc5ae0faed8Add a couple of change notes73aad0eUpdate changelog for v4.37.2Updates
github/codeql-action/analyzefrom 3.35.5 to 4.37.3Release notes
Sourced from github/codeql-action/analyze's releases.
... (truncated)
Changelog
Sourced from github/codeql-action/analyze's changelog.
... (truncated)
Commits
e4fba86Merge pull request #4031 from github/update-v4.37.3-72f6a9da0fb50ab5Update changelog for v4.37.372f6a9dMerge pull request #4030 from github/mbg/fix/no-proxy3b5ee58Use defaultrequestoptions instead ofundefinedbfb6be4Merge pull request #4028 from github/mergeback/v4.37.2-to-main-e0647621526ab84Rebuildd6217b9Update changelog and version after v4.37.2e064762Merge pull request #4027 from github/update-v4.37.2-385bcdc5ae0faed8Add a couple of change notes73aad0eUpdate changelog for v4.37.2Updates
docker/setup-buildx-actionfrom 3 to 4Release notes
Sourced from docker/setup-buildx-action's releases.