Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions docs/decisions/implemented/2026-10-10-hybrid-sql-bind-order.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Bind hybrid candidates in SQL placeholder order

[中文](2026-10-10-hybrid-sql-bind-order.zh-CN.md)

**Status:** implemented
**Approved:** explicit
**Relates to:** [Borrow Float32 scoring payloads](2026-10-09-borrow-float32-scoring.md)

## Problem

Hybrid retrieval repeats lexical candidate IDs in `WHERE` and `ORDER BY`. The
second list was bound before event-time and scope values, although its SQL
placeholders follow those filters. Valid FTS hits could disappear; a controlled
window-only example also admitted out-of-window records. Production prevalence
is not measured. Empty-result parity cannot validate this path.

## Decision

The second FTS ID list follows all event-time and scope parameters and precedes
the row limit. SQL predicates, candidate bounds, priority expression, score
weights, vector representation and public APIs are unchanged. Filtering remains
in SQL rather than a compensating post-query pass.

The [store contracts](../../../tests/core/store/hybrid-bindings.test.ts) exercise
positive FTS hits without filters, each time bound, both bounds, multi-key scope
and their conjunction. They check inclusivity, missing fields, exclusions,
repeatable ordering, public second-pass context, and FTS-only/vector-only,
no-hit hybrid and forced-candidate controls. The
[scoring parity contract](../../../tests/core/store/vector-scoring.test.ts)
compares positive-FTS hybrid results, not just the no-hit fallback.

## Alternatives considered

- Disable lexical priority or force semantic-only retrieval: avoids the defect by
changing candidate behavior rather than repairing it.
- Reorder SQL clauses or filter after scoring: does not respect SQL syntax or
filter-before-limit semantics.
- Replace the complete query with named-parameter infrastructure: larger change
than the misplaced list requires. The regression matrix protects this seam.
- Time the broken hybrid path against the repair: different eligible outputs
make that comparison unsuitable as an optimization benefit.

## Consequences

The bounded reproducer fails six positive cases before repair and passes after
repair; existing fallback/forced paths keep their constraints. A context test
uses `qppThreshold: 1` to request expansion explicitly rather than assuming the
maximum result limit overrides QPP's valid early stop.

Only after correctness is restored does the
[context-level observation](../../experiments/retrieval/hybrid-context-scoring-2026-10-10.md)
compare decoded arrays with borrowed Float32 scoring on the same repaired query.
It records synthetic store-level latency and exact output parity, not real
LoCoMo/Qwen, RPC/harness, deployment or worst-case gains.
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# 按 SQL 占位符顺序绑定 hybrid 候选

[English](2026-10-10-hybrid-sql-bind-order.md)

**Status:** implemented
**Approved:** explicit
**Relates to:** [借用 Float32 打分载荷](2026-10-09-borrow-float32-scoring.zh-CN.md)

## 问题

Hybrid 在 `WHERE` 与 `ORDER BY` 中重复使用词法候选 ID。第二组 ID 绑定在时间和 scope
参数之前,但对应占位符在这些过滤条件之后。有效 FTS 命中可能消失;受控的时间窗口用例
也出现了窗口外记录。尚未量化生产影响范围,空结果相等不能证明路径正确。

## 决策

第二组 FTS ID 绑定在所有时间/scope 参数之后、行数上限之前。SQL 谓词、候选边界、词法
优先表达式、打分权重、向量表示及公开 API 不变。保留 SQL 过滤,不以评分后的补偿过滤修复。

[Store 契约](../../../tests/core/store/hybrid-bindings.test.ts)覆盖正向 FTS 命中,分别检查
无过滤、单侧/双侧时间界、多键 scope 及组合;验证边界包含、字段缺失、排除条件、重复排序
和公开二次检索上下文,并保留 FTS-only、vector-only、无命中 hybrid 回退及 forced-candidate
对照。[打分等价契约](../../../tests/core/store/vector-scoring.test.ts)直接比较正命中的 hybrid
结果,不再仅以无 FTS 命中的回退路径作对照。

## 考虑过的替代方案

- 关闭词法优先或强制只走语义检索:以改变候选行为规避缺陷,而非修复绑定。
- 调换 SQL 子句或评分后过滤:不符合 SQL 语法或先过滤再限额的契约。
- 为整个查询引入命名参数设施:超出移动错位列表所需范围;回归矩阵保护当前调用边界。
- 比较错误路径和修复后的耗时:可检索输出不同,不能作为优化收益。

## 后果

修复前六个正向用例失败,修复后通过;回退与 forced 路径仍执行相同约束。
上下文用例显式设 `qppThreshold: 1` 请求扩展,不把最大结果数误当作覆盖 QPP 合理早停的要求。

正确性恢复后,[上下文级观察](../../experiments/retrieval/hybrid-context-scoring-2026-10-10.md)
在同一个已修复查询上比较普通数组解码与 Float32 借用打分。结果只是合成负载的 store 级
耗时和精确输出等价证据,不代表真实 LoCoMo/Qwen、RPC/harness、部署或最坏耗时收益。
7 changes: 7 additions & 0 deletions docs/design/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -1906,6 +1906,13 @@ Search signals are separated by purpose:
- regular expression only as an advanced/debug fallback over a bounded candidate
set or raw session subset.

Structured scope conjunctions and inclusive event-time bounds constrain the SQL
candidate pool before scoring and limiting. A missing event time fails a supplied
bound. Hybrid lexical priority does not exempt candidates from these filters:
its repeated `ORDER BY` memory IDs are bound after every `WHERE` value and before
`LIMIT`, in SQL placeholder order. Forced candidates and the no-FTS-hit vector
fallback retain the same filtering contract.

On the FTS5-only path, after the Active Graph budget selects the ranked evidence,
adjacent candidates with exactly equal `combinedScore` are ordered by IDF-weighted
coverage of the query's word terms in their statements and bounded evidence
Expand Down
18 changes: 1 addition & 17 deletions docs/design/temporary-todo.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# NMG active TODO list

**Authority:** working queue, not design specification or implementation history.
**Updated:** 2026-10-09
**Updated:** 2026-10-10

Durable behavior belongs in the owning design or operating document; current
implementation evidence belongs in
Expand Down Expand Up @@ -256,22 +256,6 @@ duplicates that a SimHash pre-filter recalls (or shows the gap is already
covered), with a decision recorded either way — no speculative index until the
recall gap is real.

## 8. Repair hybrid FTS-hit SQL parameter ordering

- [ ] In `searchWithVector`, bind the repeated FTS IDs for `candidateOrder`
after the event-time and scope parameters, matching their SQL placeholder
order. Add positive FTS-hit regressions with event-time and scope filters.

**Observed:** on base `7c18f6f`, eight tier-2 records matching `scoring corpus`
produce eight FTS IDs but no hybrid results (`maxTier: 3`); a query with no FTS
hits returns positive vector results. Debugger inspection and the SQL/bind list
show that ORDER BY IDs currently occupy event-time filter slots. This predates
and is not repaired by the Float32 scoring-view optimization.

**Done when:** hybrid queries with positive FTS candidates retain eligible
results and deterministic ordering, including explicit time/scope constraints,
with SQL parameters in placeholder order. Remove this item after repair.

## Explicitly deferred — not missing current work

These options return to the active checklist only after their prerequisite is
Expand Down
Loading
Loading