report: savings show $0 for every rule when a repo has no PR review history, even though check finds violations written in the window #520

Summary

report prices every rule at $0 when a repository has no PR review history, even though check finds violations that were written during the reporting window. The published page reads as "Taskless does nothing for us", which is what the user concluded.

What I did (following the recipes)

First-time onboarding on a public C#/.NET repository (NoMercy-Entertainment/nomercy-media-server), following agent onboard step by step:

1. agent ok, info --json, agent onboard, then asked the scripted question. The user said yes.
2. agent route and detect --json, then mined every source the recipe lists: TODOs, CLAUDE.md, PR review comments over 90 days, and commit history.
3. Wrote six local ast-grep rules (CSharp). All of them pass validate and verify.
4. check found 114 violations in 64 files.
5. onboard --mark-complete, then agent report.
6. Filled .taskless/.tmp-report.json exactly as ## Counting surfaced says: surfaced.prs / surfaced.reviewComments count only PR review comments in the window that ask for the rule's change.
   • Window: 2026-07-17 to 2026-10-15. 131 merged PRs, 12 comments in total (3 inline, all from the CodeQL bot). None of them asked for any of the six rules' changes.
   • So every rule got surfaced: {prs: 0, reviewComments: 0}, and history.patterns was {ruleShaped: 0, judgment: 0}. Those are the honest values, and the recipe says "Zeros are honest when there was nothing to mine".
   • estimates: contextTokens: 120000, callsPerCatch: 3.
7. report --from .taskless/.tmp-report.json published successfully.

Expected

A report that shows what the rules would actually have prevented. The onboard recipe's goal is to publish "a Taskless report showing what they catch", and the CLI already measures the current violations itself.

When review history is empty, the value model should fall back to evidence the CLI can measure without help. One example is how many of today's findings were introduced inside the window, using git blame on each finding's line. At a minimum, the page should never price a repo at $0 next to 114 current findings without explaining why.

Actual

The CLI summary:

Found in the code today: 114 problems in 64 files.
Raised in review from 2026-07-17 to 2026-10-15: 0 comments across 0 pull requests.

The published page shows $0 and 0 tokens saved for every rule and in total. The headline says "... built from 3 review comments." The embedded report data:

"perMonth":{"agentCatches":{"cli":0,"cloud":0},"commentsPrevented":{"cli":0,"cloud":0}}

The page's cost function computes tokens as agentCatches * callsPerCatch * contextTokens. With agentCatches at 0, the estimates inputs (and the sliders on the page) can't move the number off zero. The 114 findings the CLI measured never enter the calculation.

What the rules would actually have caught

I measured this afterwards, outside Taskless, by running git blame on each of the 114 findings' lines:

That's about 22 catches a month. With the report's own inputs (3 calls per catch, 120k context tokens) it comes to roughly 8M tokens a month, not 0. Blame dates a line's last change, so reformat commits can inflate the count; treat it as an upper bound. The order of magnitude still holds.

Why this will keep happening

• In some repos, review happens before a PR is opened. PR bodies here say things like "Reviewed twice with one fix round before opening". Some repos have agents writing most changes: 68 of 131 PRs here were agent-assisted. Either way, PR review comments are close to empty and the report always shows $0.
• The facts schema has no honest workaround. surfaced is defined strictly as review comments, so putting blame-derived counts there would publish a number labeled as something it isn't. I didn't do that. The agent has no place to put the evidence it does have.

Suggested fixes

1. Have report compute introductions inside the window from git blame of check findings, and feed them into perMonth.agentCatches. The CLI already has both the findings and the window.
2. Or add a facts-schema field for introductions measured from history, kept separate from surfaced, so an agent can supply them with the right label.
3. When every value input is zero, have the page say value is unmeasured and explain why, rather than showing $0.
4. Optionally, the onboard recipe could name commit history (fix-up commits) as a mining source. In this repo it held the real signal: at least four separate commits fixing missing license headers. PR comments held none.

Environment

• @taskless/cli-nightly@0.13.0-20261016004912x80f3934 (report topic v2, onboard topic v7)
• Linux (WSL2, kernel 6.6), Node.js v24.14.0
• Agent: Claude Code