Repository navigation
Conversation
…ve least squares Frank-Wolfe with a budget of inner_max_iter steps stopped short of the simplex least-squares optimum on ordinary factor-model panels, so the treated fit failed closed, placebos were dropped as failed, and the nested V search ranked V by where truncated solves stopped. The problem reduces exactly to one non-negative least squares, solved here by Lawson-Hanson's active set, which terminates finitely and, being diff-diff's own, returns the same vertex at a tie under every SciPy release. The CWZ conformal proxy and rank_control_units' synthetic_weight column solve the same problem and use the same solve. Fixes igerber#838
…-execute Tutorial 25 on top of igerber#841 Two cv-infeasibility warnings (in-space placebo and leave-one-out) still named inner_max_iter as the budget not to raise; with the exact inner solve the only budget left is n_starts. The donor-order canonicalization comment and its test described the tie-break as Frank-Wolfe's; the active set also breaks an exact gradient tie by position (the first tied column enters), so the canonicalization is still needed and the comments now say so. Tutorial 25 is re-executed on top of igerber#841 (treated unit in every placebo pool): the constant-effect confidence set prints [3.47, 20.45] and the nested V search keeps its uniform start exactly. The prose quotes neither number; the drift bands hold.
…s and flag a step-guard exhaustion instead of raising inner_max_iter and inner_min_decrease have had no effect since the donor weights moved to an exact non-negative least squares. Keeping them as silent no-ops hides that from a caller who tunes them; the repo precedent for an inert constructor parameter (SyntheticDiD lambda_reg, ledger M-001) is a warn-and-discard shim. Both now default to None, emit a DeprecationWarning when passed, and are stored as None (get_params and set_params see None, so a probe re-init warns once). Ledger rows M-148 and M-149 (deprecated 4.0, removed 5.0; 4.0 is the locked ladder's next release), migration-guide rows and prose, the design spec's phase-5 cell, and the naming guard's token predicate are updated together. The fit snapshot, _inner_solve_W, _v_starts, _outer_solve_V, _outer_solve_V_cv, _cwz_proxy_fit and the conformal grid helpers drop the dead budget plumbing (including the proxy's unused warm start), and the convenience function's docstring no longer lists the two names. _nnls_lawson_hanson no longer raises RuntimeError when it exhausts its step guard (50 steps per donor, not reached in practice). It returns the current feasible iterate flagged converged=False, _sc_weight_nnls threads the flag through (return_convergence=True), and _inner_solve_W and the conformal proxy report it, so the fail-closed paths that were kept (non-converged treated fit, dropped placebo, indeterminate grid point) handle it instead of a bare exception escaping from the middle of a nested V search or a confidence-interval grid. The post-fit inner-solve warning names the step guard instead of an iteration count. Tests: the budget-exhaustion test asserts a feasible flagged iterate rather than a raise; a new test monkeypatches the solve to exhaust and checks the flag reaches the fit warning, the fail-closed placebo and the conformal proxy; a new test pins the deprecation (warning text, discard to None, set_params, identical fit to the default, no warning on a default construction). Every inner_min_decrease=1e-3 in the suites, the tutorial-25 kwarg lock and the notebook's fit call drop the inert argument (the notebook's outputs are unchanged by it). Docs: REGISTRY inner-solve line and the three placebo failure notes, the llms-full signature, the changelog fragment (Deprecated section; the step-guard contract), and the resolved TODO row.
…ed on a non-finite difference, and apply the ridge per passive set _sc_weight_nnls conditioned the problem by sqrt(mean(Z**2)) on the raw donor-minus-treated block. For finite inputs beyond about 1e154 (below about 1e-154) that overflowed to inf (underflowed to 0), the division erased the fitting block, and the active set returned an arbitrary vertex flagged converged: Y = [[0, 1e200, 9e199]] gave [1, 0] instead of [0.1, 0.9], and an exact-match panel through SyntheticControl(standardize="none") put weight 1 on the wrong donor with _fit_converged still true. The scale is now computed from the max-normalized block (an RMS of O(1) numbers), so any finite magnitude gives the same weights; a non-finite Z (a difference that itself overflows) fails closed as uniform weights flagged converged=False, and a non-finite result does the same. zeta > 0 stacked a dense J x J identity under Z and carried it through every passive-set least squares (800 MB at 10,000 controls in rank_control_units). _nnls_lawson_hanson takes the ridge as a scalar and applies it inside each passive-set solve (an identity sized to the passive set) and in the gradient; the result equals the stacked form to 1e-12. The treated-fit non-convergence messages (in-space / in-time placebo, leave-one-out, the diagnostic-report reason text and the in_space_placebo docstring) attributed every non-converged fit to the outer V search and prescribed its knobs; they now name the inner step guard as the other cause and scope the remedy to the outer search. The conformal grid docstring no longer describes a warm-started Frank-Wolfe refit, and the REGISTRY donor-order Note names the active set's positional tie-break. Tests: scale invariance at 1e-300 .. 1e300 (same weights, KKT, converged); fail-closed on an overflowing difference; ridge equals the stacked identity; an exact-match fit at a 1e150 outcome scale keeps its weight, zero ATT and converged flag.
…rmal equations When the ridge dwarfs the fitting block (rank_control_units with a large lambda_reg, or a tiny outcome scale), the NNLS coefficients are O(1/ridge^2) and an SVD of the stacked [M; ridge I] system loses them: the information sits in O(1/ridge) entries of U computed to O(eps) absolute. Y = [[0, 2e-14, 1.8e-14]] with zeta = 1 returned [0.4986, 0.5014] as converged where the optimum is [0.5, 0.5] to 1e-28. With the ridge at or above the block's Frobenius norm the Gram matrix is well conditioned (cond <= 2), so the passive-set solve now uses (M'M + ridge^2 I) z = M'd there and keeps the stacked least squares otherwise; at the branch point the two agree to 1e-12. SyntheticControl and the conformal proxy run with zeta = 0 and are unaffected. The comment above test_outer_v_nonconvergence_warning no longer describes a loose inner tolerance that no longer exists.
…o heterogeneous donor scales are solved exactly A single global conditioning scale hid small donors behind a large one: with A = [[1e16, 1, 2]] and b = 1.5 the two O(1) donors fell below the active set's tolerance after the block was normalized by the 1e16 column, and the solve returned w = (0, 1, 0) (residual -0.5) as converged. Each donor column of Z is now scaled to unit norm first (B = Z D, w = D v, the sum constraint becoming (D1)'v = 1, that row itself scaled to a unit maximum), which keeps the reduction exact for any scale while giving every column its own rounding level; the active set's entering test is per column. The entering column is still chosen on the gradient in the original units (pivot_scale), so the vertex an exact-fit tie returns is the one the un-scaled reduction returned: Tutorial 25 and its drift bands are unchanged. The ridge for zeta > 0 becomes a per-column vector. A donor identical to the target gets weight 1 directly (zeta = 0). Tests: the heterogeneous-scale problem is solved to a zero residual at scales 1 and 1e+-200, as is the CWZ proxy on the same donors; an end-to-end standardize="none" fit with a 1e12 donor beside two O(1) donors reproduces the treated pre-path exactly. The tie-vertex, scale-invariance, ridge and KKT tests pass unchanged.
… and scope its magnitude claim to the float64 range Subnormal data (entries near 1e-320) makes a reciprocal column norm overflow, and entries near 1e308 make a column norm itself overflow; a NaN or inf then reaches the active set, whose comparisons are all False, so it returned the zero iterate as converged and the wrapper substituted uniform weights under that flag. Every intermediate (column norms, their reciprocals, the equilibrated block, the scale, the ridge vector and the constraint row) is now checked finite before the active set runs, and the active set itself refuses a non-finite problem; both paths return uniform weights flagged converged=False so the estimator's fail-closed paths (non-converged fit warning, skipped diagnostics) fire. The docstring and REGISTRY no longer claim finite inputs of any magnitude: the guarantee is the normal float64 range (column norms and reciprocals between about 1e-300 and 1e300), with the regimes outside it named as fail-closed. Tests: a subnormal panel and an overflowing column norm both fail closed; a NaN problem fails closed inside the active set; an end-to-end fit on subnormal outcomes reports a non-converged fit with the warning.
…rs beside a small one Two nearly opposing 1e8 donors next to an O(1) donor, with a 1e-8 wrinkle in a second predictor: the candidate optima differ in objective by about 1e-17, below the rounding of the 1e8 entries (ulp 1.5e-8), so a float64 solver can only choose among them by rounding. The test pins the contract that matters: the returned simplex vector is flagged converged, its objective is no worse than the alternative vertex evaluated in the same arithmetic, and its residual is within one ulp of the data.
…d exact solve rank_control_units() solves its simplex least squares exactly and then zeroes weights below 1e-6 and renormalizes for readability, so the column is the optimum up to that cleanup, not to the last digit (a donor whose optimal weight is below the threshold is dropped). The docstring, the solver comment and the test's name and docstring now say so; the test's assertion is unchanged (no optimal weight on its panel falls below the threshold).
…presenting its fallback as the solution rank_control_units() called the simplex solve without its convergence flag, so a fail-closed return (a non-finite intermediate or an exhausted step guard, neither reached in practice) came back as uniform weights that were then thresholded and reported as the informational synthetic_weight column. The call now requests the flag; a non-converged solve leaves the column NaN and emits a UserWarning saying so, while quality_score and the ranking (which never used the column) are unaffected. Regression test monkeypatches the solve to fail closed. Tutorial 25 said synthetic control identifies the counterfactual path even where the weights are not identified. At an exact pre-period fit it does not: donors on the same optimal face can differ post-period, so the solver's tie-break can move the synthetic path and the ATT. The caveat and the closing checklist now say that identification rests on the factor-model assumptions and a long pre-period, and point at the robustness diagnostics for tie-break sensitivity. Markdown only; the notebook's outputs are unchanged.
…instead of routing them to the missing-data fallback The uniform synthetic_weight column for an unbalanced panel is keyed on a non-finite pre-period block, so an infinite outcome took the same branch and was presented as a valid diagnostic (and could leave the ranking scores non-finite). The pre-period pivot is now checked for infinities right after it is built and raises ValueError naming the outcome column; NaN keeps its documented meaning of a missing cell. Parametrized test for an infinite treated and an infinite control outcome. Tutorial 25's tie-sensitivity caveat pointed at "sparse-subset diagnostics below" that the notebook does not run; it now points at the leave-one-out diagnostic below and names sparse_synthetic_control().
…he ranking The guard scanned the whole pre-period pivot, so an infinite outcome in a unit listed in exclude_units raised although that unit cannot reach any output. The check now runs after the treated list and the non-excluded control candidates are resolved and covers only those columns, on the valid pre-periods. Test: an excluded control with an infinite outcome neither raises nor appears, and the ranking columns stay finite. Two prep tests still passed the deprecated inner_min_decrease to SyntheticControl and emitted DeprecationWarnings; the kwarg is dropped (the prep suite now runs clean with DeprecationWarning as an error).
…rgence in the warning _sc_weight_nnls reports converged=False either when the active set exhausts its step guard or when the conditioned problem leaves the float64 range (an overflowing column norm or reciprocal, a non-finite intermediate), but the fit-time warning named only the step guard. The warning and the _inner_solve_W, conformal proxy and in_space_placebo docstrings now name both; the fail-closed behavior is unchanged.
igerber
force-pushed
the
fix/sc-exact-simplex-weights
branch
from
October 16, 2026 18:18
0f0b5ac to
bcb063b
Compare
…ing, in the opposing-large-donors case test_sc_weight_nnls_opposing_large_donors_beside_a_small_one asserted that the returned weights' objective was no worse than the vertex (0, 1e-8, 1 - 1e-8). Both candidates sit at the rounding floor of the 1e8 entries (objectives 8.1e-17 vs 1.0e-16 on Accelerate, 1.5e-16 vs 1.0e-16 on OpenBLAS), so the ordering is decided by BLAS rounding and the test failed on every x86 CI leg while passing on macOS and arm. The test now asserts what the case was pinned for: a converged simplex vector whose residual is within the rounding of the 1e8 entries (two ulps, since 1e8 * (w1 - w0) carries the rounding of two O(0.5) weights).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #838.
Problem
SyntheticControlsolved its donor weights,min_W ||V^½ (X1 − X0 W)||²over the simplex, by Frank-Wolfe with a budget ofinner_max_iter=10000steps. Frank-Wolfe converges sublinearly when the optimum lies on a face of the simplex, the usual case for synthetic control weights, and on ordinary factor-model panels it stopped on that budget. The treated fit then warned "did not converge" and the diagnostics that need a converged fit were skipped (placebo_p_valueNaN); placebos whose refit hit the budget were dropped from the reference set; and the nestedVsearch rankedVonly among the solves that met their stopping rule. In #838's reproducer the placebo test failed closed or lost placebos on 40, 40 and 33 of 40 panels, and at the full defaults the nested search excluded 46%, 25% and 0.5% of its solves fromVranking.Fix
The problem reduces exactly to one non-negative least squares (the derivation is in #838 and in
_sc_weight_nnls's docstring): withZ = V^½ (X0 − X1 1'),u = argmin_{u ≥ 0} ||[Z / s; 1'] u − [0; 1]||andW = u / 1'u, for any scales > 0. An active-set method solves it in finitely many steps, so there is no budget to exhaust.utils._sc_weight_nnls(Y, zeta):_sc_weight_fw's uncentered problem (intercept=False),zeta² ||w||² + ||A w − b||² / Non the simplex, solved exactly.zeta > 0stacks√N ζ IunderZ.utils._nnls_lawson_hanson(M, d, max_iter): Lawson and Hanson's active set (1974, ch. 23), with their guard against an entering column that rounding leaves without positive weight. It is ~40 lines of NumPy rather than a call toscipy.optimize.nnls; see SciPy versions. Pastmax_iter(50 per donor) it raisesRuntimeErrorrather than return a truncated solve; no handler in these modules catches it, and no test or run below raised it._inner_solve_W, which everyVevaluation and every placebo, leave-one-out, in-time and sparse refit goes through, calls_sc_weight_nnlsand always reports converged.conformal._cwz_proxy_fit,zeta=0), andprep.rank_control_units's informationalsynthetic_weightcolumn (zeta = sqrt(lambda_reg / N), which stopped atmax_iter=1000). A panel with a missing pre-period outcome keeps the uniform column it had.inner_max_iter/inner_min_decreaseas the remedy, now name the outer search's knobs (optimizer_options['maxiter'],n_starts), or none where only the inner solve could fail.synthdidparity) is untouched.Choices
The four from my comment on #838:
Won a face of the simplex fits them exactly. The active set returns a vertex of that face (at mostk + 1donors); RSynth'sipoptends to a point inside it, and Frank-Wolfe stopped wherever its budget left it. REGISTRY records this as a**Note (deviation from R)**. I added no selection rule. I tried the minimum-normW(a 1e-10 relative ridge) on Tutorial 25:W(V)becomes continuous, the nested search then drivesVto(0.017, 2e-12, 0.983), a corner where one predictor's weight is negligible, and the ATT to 6.91. A TODO row records the question; Abadie and L'Hour (2021) is the published rule.inner_max_iter,inner_min_decreaseare accepted and validated but no longer used, and their docstrings say so. Deprecating them is a TODO row._inner_solve_Wto report a truncated solve (_truncate_inner_solves). Keeping or removing the paths is in the same TODO row.SciPy versions
At a tie, the vertex an active set returns depends on its pivoting, and
scipy.optimize.nnlshas been reimplemented several times. With it, Tutorial 25's 21 placebo RMSPE ratios took five different sets over SciPy 1.11 to 1.18, where main's Frank-Wolfe ratios agree to 2e-5, and under 1.15 the constant-effect confidence set was unbounded. With_nnls_lawson_hansonthe ratios agree to the sixth decimal under 1.11 to 1.18.Tutorial 25 under each release,
DIFF_DIFF_BACKEND=python, Python 3.11 (1.17.1 on 3.12, 1.18.1 on 3.14). A letter names one set of the 21 placebo RMSPE ratios, compared at six decimals; the brackets are the constant-effect set as the tutorial prints it.scipy.optimize.nnlsA* is A with one ratio 1.5e-5 away (1.119307 → 1.119322). A′ and B′ each move one placebo under SciPy 1.10; that comes from the outer
Vsearch, not the inner solve, on main as on this branch.The constant set's printed endpoints move by one grid step between environments on both branches (main: 4.16 in the first seven rows, 4.05 in the last two). That is older than this PR: the tutorial prints the first and last accepted points of the 200-point display grid that
_invert_sharp_nulllays between the exact endpoints, and at those two points a placebo ties the treated unit, so rounding decides whether they show as accepted. The exact endpoints ineffect_confidence_setdo not depend on that tie. I left it alone here.Behavioral changes
Estimates move by the former solver's gap. On [Bug]: SyntheticControl's Frank-Wolfe stops on its budget at the default inner_max_iter; in_space_placebo() then fails closed or drops placebos on 33-40 of 40 panels #838's panels the treated pre-RMSPE was a median 1e-4 above the optimum (relative); it now equals it to 7e-16. At a tie the move can be large, because Frank-Wolfe's stopping point and the vertex are different exact fits. Tutorial 25 is such a case, and its text and drift bands are updated:
The tutorial's prose still holds (every in-time placebo below 1 in absolute value, the leave-one-out bound, the conformal claims); the numbers quoted in it are updated.
Conformal intervals are narrower and still cover above nominal. Over 50 simulated panels (6 donors, 22 periods, constant effect 6; 300 period intervals at nominal 90%, inverted on a fine grid), coverage goes from 0.970 to 0.953 and the median width from 0.515 to 0.450. No grid point failed to converge on either branch.
One existing test now inverts on a fine grid:
test_conformal_confidence_intervals_recover_true_constant_effect. The default grid puts 100 points on±0.5·|estimate|, a step of about 0.06 there, against intervals about 0.2 wide, so each interval's endpoints sit up to a step inside the accepted set. On that grid this branch brackets the true effect in 4 of 6 periods (the test asks for 5); onbounds=(5, 7), n_grid=401it brackets 5 of 6, and main 6 of 6 on both grids. A TODO row records the grid floor.R parity. At R's
V(Basque Tier 1), the weights matchSynthto 4.5e-6 (9.6e-5 on main); the test's tolerance goes from1e-3to2e-5. At the defaults (nestedV,seed=0), the pre-RMSPE is 0.0888 againstSynth's 0.0942 and the ATT −0.5842 against −0.5799, with weights 0.8331 (Cataluña) and 0.1669 (Madrid).Methodology references (required if estimator / math changes)
Synth: at an exact-fit tie the reportedWis the active set's vertex whereipoptends to an interior point (REGISTRY Note, above).Validation
test_methodology_synthetic_control.py:_sc_weight_nnlsmeets the simplex QP's KKT conditions (3 factor panels,zeta0 and 0.3) and returns a vertex at an exact-fit tie;_nnls_lawson_hansonequalsscipy.optimize.nnlswhere the optimum is unique, meets KKT on tall, square and wide matrices with a duplicated column, skips a column that rounding lets in, and raises rather than truncate; on factor panels where the budgeted solve stopped, the inner solve is exact,in_space_placebo()keeps every placebo, and the conformal proxy converges.test_prep.py:synthetic_weightis the exact simplex optimum, and stays uniform on a panel with a missing pre-period outcome._truncate_inner_solves(the solve itself cannot fail); the CWZ proxy oracle is held to SLSQP atatol=1e-7with a one-step budget (was1e-2with 200 000 steps); Basque Tier 1 atatol=2e-5(was1e-3); the conformal-interval test on a fine grid (below); Tutorial 25's drift bands; the naming guard'szetaallowlist._sc_weight_nnlsand_nnls_lawson_hansondo not exist on main.test_inner_solve_is_exact_where_a_budgeted_solve_stopped(the budgeted solve does not converge);test_in_space_placebo_keeps_every_placebo_on_a_factor_panel[0]and[1](the treated fit warns "did not converge", and 6 of 19 placebos are dropped);test_conformal_proxy_converges_on_a_factor_panel;test_cwz_proxy_fit_matches_scipy_simplex_ls;test_synthetic_weight_is_the_exact_simplex_optimum(main'ssynthetic_weightobjective 8.9e-11 against the optimum's 3.1e-14); andtest_basque_tier1_custom_v_parityatatol=2e-5.test_top_donor_weights(Tutorial 25's weights) andtest_inner_nonconvergence_warning(the warning no longer names Frank-Wolfe).inner_max_iterno longer acts): no "did not converge" warning, no NaN p-value and no dropped placebo on any of the 120 panels, and the treated pre-RMSPE above the exact optimum by at most 6.7e-16 (relative). The nested count from my comment: no solve excluded fromVranking on any of the 30 fits.DIFF_DIFF_BACKEND=python.pytest -n 4 -m ''(slow tests included) overtest_methodology_synthetic_control.py,test_t25_synthetic_control_policy_drift.py,test_prep.py,test_utils.py,test_methodology_sdid.py,test_diagnostic_report.py,test_business_report.py,test_practitioner.py,test_rust_backend.py,test_aliases.py,test_v4_matrix.py,test_v4_wrapper_shims.py,test_naming_guard.py,test_changelog_fragments.py,test_guides.py,test_doc_snippets.py,test_doc_deps_integrity.py,test_docs_ia.pyandtest_tracking_files.py: 2324 passed, 221 skipped, 2 failed.test_naming_guard.py::test_allowlists_are_reachable: this PR removed the lastzeta=fromsynthetic_control.py, which left itszetaallowlist entry with nothing to cover. The entry is removed; the docs, changelog, prep, guides and naming-guard files then pass (899 passed, 7 skipped).test_rust_backend.py::TestClusterVcovDeterminism::test_raw_kernel_noncontiguous_ids_bit_identicalfails on main too here: that class has noskipif(not HAS_RUST_BACKEND)and importsdiff_diff._rust_backend, which is not built here.ruff check,black --checkandmypyon the changed modules: clean.changelog_compile.py check: OK.synthetic_weighttests (50, selected with-k "t25 or nnls or lawson or exact or synthetic_weight or factor_panel"over the three files) pass under SciPy 1.10.1, 1.11.4, 1.12.0, 1.13.1, 1.14.1, 1.15.3, 1.16.3 (Python 3.11) and 1.17.1 (Python 3.12), NumPy as in the table above.Security / privacy
Changelog
changelog.d/20261006-sc-exact-simplex-weights.md,### Fixedand### Behavioral Changes. It states which results move and by how much (Tutorial 25, Basque, the conformal coverage), the vertex at ties, and thatinner_max_iter/inner_min_decreaseno longer act.