fix(v4): ETWFE reference-period normalization (#724) + Stata jwdid/csdid parity

[igerber]

Summary

Fixes issue #724 and anchors both staggered estimators to their canonical Stata implementations. Eleven local review rounds surfaced further identification defects in the same code, including a P0; those are fixed here too.

The #724 fix. With control_group="never_treated" the saturated ETWFE design emitted every cohort×time indicator including the reference period — collinear with the absorbed unit FE — and left generic QR rank detection to remove whichever column it picked. On the real mpdta panel that silently removed ATT(2004, 2004) and ATT(2006, 2007), returning 5 of 7 post-treatment cells. The design now omits ref(g) explicitly, per-cohort and weight-aware.

The pre-fix survivors were not merely incomplete — they were measured against the wrong baseline. ATT(2007,2007) computed by hand against references 2006/2005/2004/2003 gives −0.0261 / −0.0571 / −0.0599 / −0.0294; the library returned −0.0571, a correct 2×2 DiD against a silently chosen 2005.

Not confined to never_treated: any cohort whose cell block spans its own cohort dummy hits it, which anticipation > 0 makes reachable on the default path.

Stata parity arm. Anchors ETWFE and CallawaySantAnna against jwdid / csdid on the committed mpdta panel (pinned SHA-256, so CI needs neither Stata nor network). This replaces a test that validated nothing: test_wooldridge.py asserted ETWFE == CS within 5e-3, false on real data (0.0171 at (2007,2007)), passing only because load_mpdta() was silently serving a synthetic effect-homogeneous DGP when its source 404'd (issue #722). Stata's own two implementations reproduce the same disagreement.

Identification defects found while reviewing the above:

• P0 — partial rank loss returned mislabeled contrasts as ATTs. When some treatment columns were dropped, survivors were re-identified as whatever contrast the reduced design supports and kept their ATT(g,t) labels. Measured with true effects 1.0 and 3.0: ATT(3,5) = −1.9393 (wrong sign) inside overall_att = +0.7703 against a true ~1.8, with no warning under rank_deficient_action="silent". Pre-existing and worse at base.
• Within-cohort support connectivity — per-period support is necessary but not sufficient; identification runs through the unit FE. A cohort whose units split into disjoint observation windows had a genuine post cell dropped. OLS-only; nonlinear paths use explicit dummies and are unaffected.
• A fit no longer returns a NaN overall ATT as success (point estimate only — NaN SE stays legitimate where only the variance is unidentifiable).
• Invalid survey metadata could hide in deleted rows. NaN / negative / Inf weights, fractional fweight, missing PSU/strata confined to a cohort that exclusion removed produced a confident finite estimate with a warning blaming a missing pre-treatment period. The full design is now validated pre-exclusion.
• survey_design= + unidentified-cohort exclusion is refused — deleting rows drops their PSUs/strata from the TSL variance and from df_survey = n_PSU − n_strata (measured 22 → 14, finite SE, no diagnostic).
• rank_deficient_action="error" was silently downgraded to a warning for skipped cells; all three modes now dispatch.

Behavior changes (please read before merging)

Deliberate capability regression: all-eventually-treated panels (no never-treated group) now raise. They previously returned finite, silently wrong ATTs — on a true-1.5 DGP: ATT(3,8) = −0.0096, ATT(5,9) = −0.2393, overall_att = 1.0023. W2025 §5.4 defines the fix (last cohort as control, columns dropped deliberately rather than by QR); the library already applies this to the cohort-trend column but not to the cells. Tracked at High priority — landing it restores these fits.

Four existing tests asserted only that such values were finite, which is what let a P0 live in a tested path; they now assert the refusal with the measured numbers in their docstrings.

Ledger rows M-123 / M-124 and the CHANGELOG record all user-visible changes.

Methodology references (required if estimator / math changes)

• Method name(s): WooldridgeDiD (ETWFE); CallawaySantAnna (parity coverage only)
• Paper / source link(s): Wooldridge (2025), Econometrics Journal — Eq. 6.1/6.4 (reference-period normalization), §5.4 (all-eventually-treated), §6.1 (reference-period sensitivity), §8 (cohort trends). Lumley (2004) §3.4 (survey domain estimation). Stata jwdid (Rios-Avila) and csdid — SSC.
• Any intentional deviations from the source (and why):
  • Deviation from W2025 §5.4 — the last-cohort normalization is applied to trend columns but not cohort×time cells, so all-eventually-treated panels raise. Labeled in REGISTRY, tracked in TODO.
  • Deviation from Stata jwdid — hc1 SEs are uniformly below jwdid's (anti-conservative): ~0.1% at G=500, ~2.8% at G=20. The mechanism is not identified — it tracks sqrt(G/(G-1)) but sits above it, and solve_ols already applies full CR1 — so it is recorded as measured, not diagnosed, and pinned rather than "corrected" with an unverified factor. The never-treated arm shows the same factor over a different cell count, ruling out explanations keyed to the comparison set. CallawaySantAnna shows no such gap.
  • Unbalanced-panel reference falls back to a cohort's latest observed pre-period, a different finite-sample contrast from g−1 (legitimate per §6.1; documented).
  • Every refusal added here is classified terminal vs stopgap in TODO.md, with the stopgaps carrying what "done" means.

Validation

• Tests added/updated: tests/test_wooldridge.py, tests/test_etwfe_cs_stata_parity.py, tests/test_methodology_wooldridge.py, tests/test_v4_matrix.py
• Backtest / simulation / notebook evidence: Stata golden regenerated with local StataSE 19 — verified zero numeric or key-set change across csdid/jwdid/jwdid_never; only meta.ssc_versions moved (drdid gained a checksum identifier where it previously recorded "unknown"). docs/tutorials/16_wooldridge_etwfe.ipynb re-validated with pytest --nbmake. 6992 tests pass across every WooldridgeDiD/SurveyDesign-touching suite; mypy 2.3.0 clean; black/ruff clean.
• Known coverage gap: CI pins the full-panel G=500 SE ratio only; the few-cluster ratios quoted in REGISTRY were measured ad hoc and are explicitly labeled as trend-shape rather than gated. The subsample ladder that would pin them is a named required artifact on the SE-derivation TODO row — it is the same instrument that derivation needs.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

[github-actions]

Overall assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive summary

• Affected estimator: WooldridgeDiD; CallawaySantAnna changes are parity coverage only.
• Per-cohort reference omission matches Wooldridge (2025) Eq. 6.1/6.4.
• Rank-loss and NaN inference paths now fail closed and use safe_inference().
• Remaining methodology deviations are explicitly documented and tracked.
• Static review completed; tests could not run because pytest/NumPy/pandas/SciPy are unavailable.

Methodology

Documented Stata HC1 SE deviation

• Severity: P3 — informational
• Impact: ETWFE HC1 SEs are up to approximately 2.8% below jwdid with few clusters.
• Concrete fix: No merge action required. The deviation is labeled in docs/methodology/REGISTRY.md:L1994 and derivation work is tracked in TODO.md:L26.

Documented all-eventually-treated limitation

• Severity: P3 — informational
• Impact: Such panels now raise instead of returning mislabeled contrasts. This safely reduces capability until W2025 §5.4 last-cohort normalization is implemented.
• Concrete fix: No merge action required. The deviation is documented at docs/methodology/REGISTRY.md:L2080 and tracked at TODO.md:L33.

The primary normalization itself is methodologically sound: diff_diff/wooldridge.py:L612-L667 implements the per-cohort reference described in docs/methodology/REGISTRY.md:L1962-L1968.

Code Quality

No findings. All changed inference construction uses safe_inference(), and complete-cell/finite-overall gates cover OLS, logit, and Poisson at diff_diff/wooldridge.py:L2228-L2229, L2556-L2557, and L2812-L2813.

Performance

Duplicate survey resolution

• Severity: P3 — informational, tracked
• Impact: Supported survey fits repeat validation and array construction.
• Concrete fix: Reuse the early resolved survey tuple across fitters as specified in TODO.md:L34. Relevant calls are diff_diff/wooldridge.py:L1375, L1740, L2386, and L2640.

Maintainability

No findings. The changed interaction-matrix return contract is propagated to its callers, and diagnostics are centralized.

Tech Debt

Deferred estimator capabilities are explicitly tracked

• Severity: P3 — informational
• Impact: Survey-domain estimation, anticipation-only results, component-aware support, and last-cohort normalization remain unavailable but fail closed.
• Concrete fix: Follow the scoped work and acceptance tests in TODO.md:L28-L33; no PR-blocking action required.

Security

No findings. No credential, private-key, or secret patterns were detected in the changed sources or fixtures.

Documentation/Tests

Few-cluster Stata parity is not regression-pinned

• Severity: P3 — informational, tracked
• Impact: CI pins the G=500 SE ratio, while the more material G≈20–110 behavior is documented but not fixture-backed.
• Concrete fix: Add the parameterized Stata subsample ladder specified in TODO.md:L26. The current full-panel assertion is at tests/test_etwfe_cs_stata_parity.py:L286-L304.

The committed mpdta fixture checksum matches its expected SHA-256. Runtime tests were not executable in the supplied environment because the Python testing/scientific dependencies are absent.