fix(mutmut): make scripts/ test-loader module names match mutmut's path-derived keys - #264
Merged
Merged
Conversation
…th-derived keys
Rename (Fowler, Rename Function/Variable) every tests_py/scripts/*.py
module handle that loaded its scripts/ subject under a synthetic or bare
name (e.g. "_cortex_launcher_deps", "generate_pip_constraints", bare
`import wiki_backfill_ids`) to the dotted path mutmut derives from the
file's location ("scripts.launcher_deps", "scripts.generate_pip_constraints",
"scripts.wiki_backfill_ids", ...).
Root cause (read from mutmut's own source, mutmut/mutation/trampoline.py +
__main__.py::_check_test_to_mutant_associations): the trampoline records
`orig_func.__module__` at test-run time, which is whatever name
spec_from_file_location/module_from_spec gave the loaded module. Mutant
keys are computed independently, purely from the mutated file's path
relative to repo root ("scripts/launcher_deps.py" -> "scripts.launcher_deps").
A synthetic test-loader name never matches, so mutmut aborted every scoped
run before scoring anything ("Stopping early... none match any mutant key").
Excludes tests_py/scripts/test_check_doc_claims.py, already fixed by #236
(precedent for this exact fix) and owned by the concurrent #235 worktree.
Also widens [tool.mutmut] also_copy (uv.lock, requirements/, pyproject.toml)
so scripts/generate_pip_constraints.py's test — which reads all three
directly — mutates cleanly instead of every mutant raising
FileNotFoundError inside mutants/ (both changes are needed together for a
trustworthy tally on that module).
Verified: real mutant tallies now produced for both AC-named modules
(previously an early abort with zero usable result):
scripts/launcher_deps.py: 213 mutants, 128 killed, 85 survived
scripts/generate_pip_constraints.py: 243 mutants, 95 killed, 65 survived, 83 no-tests
Survivor backlog filed as #263 (same split precedent as #236 -> #235):
this PR fixes the attribution SEAM, not the pre-existing coverage gaps it
now makes visible.
tests_py/scripts/: 291 passed before and after (0 test-code behavior
change — only the module handle string changed).
Closes #262 (attribution mechanism fix; see #263 for the surfaced backlog)
Co-Authored-By: Claude <noreply@anthropic.com>
Issue #262 AC 2: "launcher_deps.py remains loadable by path at runtime (the launcher's pre-dependency bootstrap must not regress) — assert it, don't assume it." The prior commit renamed this file's OWN test-loader handle from a synthetic name to "scripts.launcher_deps" (so mutmut can attribute mutants); that rename does not touch launcher_deps.py or launcher.py themselves, but the assertion was still missing. test_launcher_deps_stays_bare_importable_by_launcher_py spawns a real `python3 scripts/launcher.py` subprocess (no pytest sys.path, no test loader involved) and asserts it reaches the plain "Usage: ..." message rather than a traceback — proving launcher.py's own bare `import launcher_deps` (module-level, executed before main() runs) still resolves. Verified the test actually catches a regression: temporarily renamed launcher.py's import target to a nonexistent module, watched this test fail with the ModuleNotFoundError surfaced in stderr, then reverted. tests_py/scripts/test_launcher_deps.py: 33 passed (32 before + this one). Co-Authored-By: Claude <noreply@anthropic.com>
test_launcher_deps_stays_bare_importable_by_launcher_py (two commits prior) adds one collected test on top of main's count. Rebased onto main (which had independently merged #250/#253/#252 in the interim, advancing the baseline from 6439 to 6549); resolved the six suite-size files' conflicts on their merits (structure from main, count recomputed) rather than taking either side's stale number, per coding-standards.md's rule that the advertised count is a measured absolute, not a delta arithmetic on a side's claim. Verified against a live collection on the rebased tree: pytest --collect-only -q -> 6550 tests collected python scripts/check_doc_claims.py --test-count 6550 -> doc claims OK python scripts/generate_repo_badges.py --check --test-count 6550 -> badges OK (5 checked) Co-Authored-By: Claude <noreply@anthropic.com>
cdeust
force-pushed
the
fix-mutation-scripts-attribution-262
branch
from
July 29, 2026 23:26
7b926e0 to
16d296c
Compare
This was referenced Jul 30, 2026
cdeust
added a commit
that referenced
this pull request
Jul 30, 2026
…nt_sets.py mutation-testable (#262) (#280) * fix(mutmut): dot the last scripts/ test-loader gap and make pip_constraint_sets.py mutation-testable (#262) test_typecheck_env_parity.py loaded scripts/pip_constraint_sets.py under the bare name "pip_constraint_sets" instead of the path-derived "scripts.pip_constraint_sets" mutmut keys mutant trampolines on — #264 fixed this for 14 sibling scripts/ test files but missed this one, so a scoped mutation run on it aborted with "Stopping early ... none match any mutant key" (found by code-reviewer post-merge on #264). Dotting the loader alone was not sufficient: ConstraintSet's three methods (path/command/header_lines) carried its entire logic, and mutmut's mutation generator categorically excludes the body of any @dataclass-decorated class (confirmed empirically: 0 mutants for the whole file with the methods in place, vs 58 after extracting them to free functions constraint_path/constraint_command/constraint_header_lines with ConstraintSet left as pure data). Every call site across generate_pip_constraints.py and its two test files is updated to match; the existing 301-test tests_py/scripts/ suite passes unchanged, proving the refactor is behavior-preserving. A dedicated tests_py/scripts/test_pip_constraint_sets.py directly exercises the three extracted functions (neither existing test file called them for real: everything went through compose()/render(), which every test either mocks or short-circuits before reaching). The scoped mutation run (tests_py/scripts/test_typecheck_env_parity.py + test_pip_constraint_sets.py against scripts/pip_constraint_sets.py) now produces real attribution: 58 mutants, 58 killed, 0 survived, 0 no-tests. Boy-scout finds in the same sweep (grepped every spec_from_file_location and sys.path-plus-bare-import in tests_py/, 21 call sites verified): - tests_py/scripts/test_generate_repo_badges.py bare-imported scripts/badge_render.py (`import badge_render` after a sys.path insert) instead of loading it under "scripts.badge_render" — same defect class, now fixed with its own dotted load, independent of generate_repo_badges.py's own bare import of the same file. - Two ruff E501 violations introduced by this diff's own line-length growth, fixed in the same commit (§14). - generate_repo_badges.py's RepoBadge class has the identical @dataclass-exclusion gap (confirmed: 234 mutants total, 0 attributed to RepoBadge.path()/.render()) — out of this PR's blast radius (a separate root-cause fix of comparable size; mixing it in here would violate §15.1's "don't mix the enabling refactor with the fix"); reported with evidence rather than filed, per this task's no-new- issues instruction. Resynced the 6 suite-count sites + badge SVG to the measured absolute (6560, +10 for test_pip_constraint_sets.py) via generate_repo_badges.py and manual doc edits; both check_doc_claims.py --test-count 6560 and generate_repo_badges.py --check --test-count 6560 pass. Full suite: 6560 passed, 117 subtests passed (167.83s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u * fix(mutmut): add .github to [tool.mutmut] also_copy for scripts/pip_constraint_sets.py scoped runs tests_py/scripts/test_typecheck_env_parity.py reads .github/workflows/ci.yml directly (cross-checking CONTRIBUTING.md's documented type-check command against it). Running scripts/mutation_check.sh with this test in the selection failed collection under every mutant with FileNotFoundError (mutants/.github/workflows/ci.yml missing) instead of scoring the suite -- found while producing the real mutant tally for the previous commit's pip_constraint_sets.py fix. .github wasn't in also_copy for any prior scoped run (json_native.py, launcher_deps.py, generate_pip_constraints.py all read files already listed), so this is additive and does not change behavior for any other scoped run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #262: mutation testing could not attribute mutants for any
scripts/module becausetests_py/scripts/*.pytests loaded theirsubjects under synthetic or bare module names
(
_cortex_launcher_deps,generate_pip_constraints, bareimport wiki_backfill_ids, ...), while mutmut keys mutant trampolineson the path-derived dotted name (
scripts.launcher_deps,scripts.generate_pip_constraints, ...) it computes purely from eachmutated file's path relative to the repo root. Every scoped run aborted
early: "Stopping early... tests recorded trampoline hits but none match
any mutant key."
Root cause traced by reading mutmut's own source
(
mutmut/mutation/trampoline.py'srecord_trampoline_hit(f"{orig_func.__module__}...")vs
mutmut/utils/format_utils.py::get_mutant_name, which derives the keyfrom the file path alone).
__module__is bound to whatever nameimportlib.util.spec_from_file_location/module_from_specgave theloaded module — a test-only artifact completely disconnected from the
path-derived key mutmut expects.
Precedent already existed in this repo: PR #236 fixed the identical
defect for
test_check_doc_claims.pyalone and filed the survivorbacklog it surfaced as #235, rather than folding a much larger
test-writing effort into the attribution-fix PR. This PR applies the
same fix to every other affected file (
test_check_doc_claims.pyexcluded — already fixed, and owned by the concurrent
#235worktree),and follows the same split for the backlog it surfaces: filed as #263.
Changes (3 commits)
fix(mutmut)— rename every affected test's module-loader handleto the dotted
scripts.<file>name mutmut expects (14 files); widen[tool.mutmut] also_copy(uv.lock,requirements/,pyproject.toml)so
generate_pip_constraints.py's test — which reads all threedirectly — mutates cleanly instead of every mutant raising
FileNotFoundError.test— new regression test(
test_launcher_deps_stays_bare_importable_by_launcher_py) provingAC 2:
launcher_deps.pystays loadable by a bareimportat realruntime (a genuine
python3 scripts/launcher.pysubprocess, not atest-loader path). Verified it actually catches a regression
(temporarily broke the import, watched it fail, reverted).
docs— resync the advertised test count across README.md,CONTRIBUTING.md, CLAUDE.md, docs/ASSURANCE-CASE.md,
.bestpractices.json, and the regeneratedassets/badge-tests.svg.Rebased onto
mainafter it independently merged json_native.py: 14 surviving mutants under the repository's own committed mutmut scope #250/pyright gate resolves a different tree-sitter-language-pack than the documented dev install #253/normalize_date_to_iso silently re-anchors an unresolvable timezone abbreviation to UTC #252(baseline moved 6439 -> 6549 while this branch was in flight);
resolved the six suite-size files' conflicts on their merits —
main's structure, count recomputed from a live collection on the
rebased tree (6550 = main's 6549 + this PR's 1 new test), not a
delta applied to either side's stale number.
Verification (acceptance criteria from #262)
an early abort with zero usable result):
launcher_deps.pybare-importability asserted, not assumed (newregression test above).
also_copywidened withuv.lock/requirements//pyproject.toml.#236->#235precedent) — this PR fixes the attribution seam, notthe pre-existing coverage gaps it now makes measurable for the first
time.
Local, on the rebased tree (head 16d296c), quoted for reviewer sanity:
pytest tests_py/-> 6550 passed, 117 subtests passed in 161s.pytest tests_py/scripts/-> 301 passed, 117 subtests passed.ruff check+ruff format --checkon every touched.pyfile: clean.scripts/check_doc_claims.py --test-count 6550-> OK.scripts/generate_repo_badges.py --check --test-count 6550-> OK (5checked).
scripts/check_marketplace_pins.py-> OK.Completion Ledger
tests_py/scripts/*.pyfilestests_py/scripts/full run: 301 passed before AND after the rename commit (0 test-code behavior change)[tool.mutmut] also_copygenerate_pip_constraints.pyscoped mutation run produces a real 95/65/83 tally instead ofFileNotFoundErrorabortstest_launcher_deps_stays_bare_importable_by_launcher_pylauncher.pyimport, then pass against the real onecheck_doc_claims.py --test-count 6550+generate_repo_badges.py --check --test-count 6550both green; full suite collects exactly 6550Boy-scout (§14)
ruff formatflagged one line-length issue introduced by my own edit intest_setup_py_backend_skip.py— fixed in the same commit (formatterauto-fix, verified
ruff format --checkclean afterward).files by recomputing the count from a live
pytest --collect-only -qon the rebased tree rather than taking either side's stale number, and
corrected the reconciliation commit's own message/date to describe
what actually happened (was previously drafted against the pre-rebase
6440 number before the conflict was known).
Deferred (§14.3 — outside this PR's blast radius, filed with acceptance criteria)
(plus 83 uncovered "no tests" mutants in
generate_pip_constraints.py).Triaging/killing them is a test-writing effort against orchestration
and subprocess-heavy code, independent of the attribution-seam fix this
PR makes.
🤖 Generated with Claude Code