Remove HermitCrab's analysis-cascade memo

[johnml1135]

Retargeted to master. The conformance suite (#480) is deferred until after this release, so this PR no longer stacks on it. Its suite-side commit (removing the harness's --no-memoization flag along with the memo) is held back and will be added to #480 after this merges.

Quick summary

Removes HermitCrab's opt-in analysis-cascade memo; the sequential rule cascade
and MergeEquivalentAnalyses still work exactly as before. Measured as a
package (#490), it cost 28-32% and ~29 GB on Sena for no net speedup on FLEx's
parallel path. Pruning (#474 landed; #491/#494/#519 open) is untouched and is
where the real parse-time wins come from.

Where to look

• AnalysisMergeKey (AnalysisStratumRule.cs L226-309) still distinguishes non-head count and
  per-rule unapplication counts -- pinned by AnalysisMergeKeyTests.cs and MergeEquivalentAnalysesTests.cs.
• Memo plumbing is gone (AnalysisScope.cs, AnalysisStateKey.cs, MemoizedCombinationRuleCascade.cs
  deleted) -- maxDegreeOfParallelism and the sequential cascade are unchanged public API.
• Review guidance dropped the memo-key re-audit language (docs/review/hermitcrab.md and others).

Validation

• dotnet build (MSBUILDDISABLENODEREUSE=1) -- succeeded, 0 warnings, 0 errors.
• dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests --no-build -- 89 passed, 0 failed.
• dotnet csharpier check . -- 713 files checked, clean.
• pwsh -NoProfile -File scripts/comment-hygiene.ps1 -BaseRef origin/master -- clean.
• git diff origin/master..HEAD --stat -- 16 files, +196/-1520; production diff unchanged, only
  AnalysisMergeKeyTests.cs added.
• ./local_check.sh was not run as such; the commands above cover its steps individually.

---

Reading this a year from now

The memo only ran when maxDegreeOfParallelism == 1 and tracing was off. FLEx uses
the default parallelism, so production never executed it. #490's ledger measured
the optimization round as a package: no net speedup, and cross-word memo sharing
was 28-32% slower on Sena and cost ~29 GB of memory. The gains keep coming from
search-space pruning that doesn't need a replay cache: #474 (landed), and #491,
#494, and #519 (open at the time of writing). Those shrink the work instead of
caching it, and they apply on the parallel path FLEx actually uses. The memo was
~1,400 lines of subtle code (ReplayOnto grafting, arrival-count bookkeeping)
whose correctness depended on an unenforced key invariant (#490, "narrowing that
key breaks correctness with no test failure").

What stays, and why

• The public Morpher(..., maxDegreeOfParallelism) parameter and the sequential
  cascade. They are API and still useful without the memo.
• Fix bug in MergeEquivalentAnalyses #493's MergeEquivalentAnalyses fix. It keyed on AnalysisStateKey; that key
  now lives as an internal AnalysisMergeKey in AnalysisStratumRule, covering the
  same dimensions (shape, stratum, syntactic and realizational feature structures,
  non-head count, per-rule unapplication counts).

Prior measurements (not reproduced by this review round)

These are earlier author measurements from the memo-removal work, not commands run
as part of this pass's validation, and no reproducible artifact for them is checked
into the tree:

• Spot check on 5 real grammars (Sena, Amharic, Indonesian, Mbugwe, Aweti), 5 words
  each, all with at least one parse: analysis multisets (gloss, allomorph identity,
  root index, feature structures) were reported identical between master memo-off
  and this branch in both default and sequential modes, 25/25 match.
• On the 14 words where master memo-on also completed, it was reported no faster
  than this branch's sequential mode (e.g. 3,286 ms memo-on vs 3,122 ms without it;
  412 vs 350 ms; the rest within noise).
• Full-solution run reported as 1,003 passed / 3 skipped / 0 failed.

🤖 Generated with Claude Code

---

This change is 

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.00000% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.19%. Comparing base (a2f80e2) to head (e723ade).

Files with missing lines	Patch %	Lines
...chine.Morphology.HermitCrab/AnalysisStratumRule.cs	74.69%	11 Missing and 10 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #522      +/-   ##
==========================================
- Coverage   74.36%   74.19%   -0.17%     
==========================================
  Files         456      453       -3     
  Lines       38325    38157     -168     
  Branches     5263     5240      -23     
==========================================
- Hits        28499    28312     -187     
- Misses       8669     8684      +15     
- Partials     1157     1161       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[claude]

Minor: F1. The merge key drops FinalTemplateState. #491 (b240b7b) added it to AnalysisStateKey, so the PR's "covering the same dimensions" is not accurate. Two analyses that differ only in that field used to stay separate. Now the second becomes an Alternative of the first. Every stratum output is cleared to None (L172-178), so this is probably sound, but Word.ValueEquals still compares the field. Please confirm it is intended and pin it with a test in AnalysisMergeKeyTests.

[claude]

F1 still applies at e723ade, and my earlier reason for calling it probably sound was wrong on this base. Stratum outputs are not cleared: FinalTemplateState is reset only on entry to a blocking stratum (AnalysisStratumRule.cs:135-136). So a merged NonTemplate canonical carries into a non-blocking stratum below, and AnalysisAffixTemplateRule.cs:47 then blocks a final template that the merged-away None alternative would have let through. The PR body still says the key covers the same dimensions as AnalysisStateKey. Unverified: this environment would not let me write a reproduction.

[claude]

Minor: F2. The deleted ParseWord_MemoOnMatchesMemoOff_* tests compared maxDegreeOfParallelism: 1 against the parallel default. That still checks something real: the sequential CombinationRuleCascade added at AnalysisStratumRule.cs:51. Now only the compounding grammar checks that equivalence. Affix templates, a Linear stratum and Simultaneous epenthesis (buibui == 1) lose it, against docs/review/machine-tests.md:19. Keep them, renamed SingleThreaded_MatchesParallel_*, without the hit-counter asserts.

[claude]

F2 still applies at e723ade: the rewritten MorpherTests.cs still deletes the four ParseWord_MemoOnMatchesMemoOff_* tests. Only ParseWord_SingleThreaded_MatchesParallel_WithCompounding still compares sequential output with parallel output, so affix templates, the Linear stratum and Simultaneous epenthesis are no longer checked.

[claude]

FYI: F4. Order invariance is no longer tested directly. The PR deleted AnalysisStateKeyTests.Equals_IsInvariantOverUnapplicationOrder_ForEqualMultisets, and order invariance is the property the XOR multiset hash exists for. A change to a rolling combine at AnalysisStratumRule.cs:272 would still pass both remaining tests and would quietly stop merging.

[claude]

F4 still applies at e723ade: AnalysisMergeKeyTests.cs has two tests, one for non-head count and one for unapplication count. Neither unapplies rules in a different order.

[claude]

FYI: F6. A plain constructor now changes its argument: it freezes word.SyntacticFeatureStruct, which Word.FreezeImpl leaves unfrozen. Before, the PinAndKey name and its doc comment said so. A one-line comment here would keep that warning.

[claude]

F6 still applies at e723ade: the AnalysisMergeKey constructor still freezes its argument's SyntacticFeatureStruct (AnalysisStratumRule.cs:255), and no comment says so.

[claude]

Review: approve with fixes

1. Verdict: approve with fixes. Removing the memo is clean, and production parsing is unchanged apart from F1.
2. Most important: F1, AnalysisStratumRule.cs:245. AnalysisMergeKey leaves out FinalTemplateState, which AnalysisStateKey had included since Filter final templates in analysis #491. The PR body says the key covers the same dimensions. This is probably harmless, because stratum outputs are cleared to None, but it is not stated anywhere and no test pins it.
3. Counts: Critical 0, Important 2 (F1, F2), Low 4 (F3-F6).
4. Ran: dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests -c Release at 8313ba0: 587 passed, 0 failed. That includes EveryConformanceFixturePassesSelfCheck, which runs without pathological fixtures. I could not run ./local_check.sh --agent-strict or the conformance CLI with --include-pathological; the environment did not permit either.
5. Not verified: whether F1 changes the analysis or alternative set on any real grammar, and how fast the budgeted fixture runs (F5). The strict comment-hygiene scan was not run.

API / framework / parity: Morpher.MemoHits and the other counters were internal. The public API keeps maxDegreeOfParallelism, and its semantics are unchanged. The only doc change is the MergeEquivalentAnalyses summary text. The conformance CLI loses the --no-memoization flag. Target frameworks, packages and machine.py parity: none verified changed.

Findings: F1 new; F2 new; F3 new; F4 new; F5 new, unverified; F6 new.

Reviewed at 8313ba0