2.3.0 — the index train: local-first recall index, incremental sweep, stemming + alias expansion
Build and Push Docker Image / build (push) Successful in 14s
Build and Push Docker Image / build (push) Successful in 14s
One schema-bump release (recall-index schema 3, entity-index schema 2; old recall indexes rebuild automatically, no vault migration): - Local-first recall index: the live BM25 index lives in the machine state dir (keyed by endpoint hash); capture's upkeep is a zero-network atomic file write, no advisory lock — replacing the O(vault) GET/PUT-whole-index round trip per write. The vault copy is a snapshot (sweep + session-end) used to seed fresh machines / catch up after another client swept (ECHO_RECALL_SYNC_HOURS, default 24). The read path never PUTs the vault. - Incremental sweep: entity + recall meta carry 16-char content hashes; `sweep --fast` fetches only new/gone/hash-missing notes plus a rotating weekday shard (--all-shards forces everything); deletions drop from both indexes; index-only so no --apply gate. `load` auto-runs it past ECHO_FAST_SWEEP_DAYS (7); doctor reports index freshness. Obsidian-side edits now reach the indexes without manual maintenance. - Stemming + alias expansion: echo_stem.py (conservative Porter-lite, unit- tested families + over-stemming guards) applied at index AND query time; a query fuzzy-matching an entity folds its title/alias vocabulary into the BM25 query at half weight — expansion can only boost docs containing the terms. capture hashes the note's FINAL content (auto-link reordered first). Eval gold set 8 -> 14 queries (6 paraphrases): recall@5/MRR 1.00/1.00 vs keyword baseline 0.75/0.79. +3 unit tests, +10 e2e; test harnesses now isolate ECHO_STATE_DIR. All seven suites green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -135,6 +135,16 @@ QUERIES = [
|
||||
("contract renewal Q3", {"resources/companies/vantage-systems.md"}, False),
|
||||
("renegotiation kickoff", {"journal/daily/2026-06-20.md"}, True),
|
||||
("graph neighbourhood expansion", {"resources/concepts/graph-expansion.md"}, False),
|
||||
# ---- paraphrase set (2.3): morphological variants + rephrasings the pre-2.3
|
||||
# purely-lexical index missed — the stemming gain is measured, not assumed.
|
||||
("renewing enterprise contracts", {"resources/companies/vantage-systems.md"}, False),
|
||||
("navigational cleanup", {"projects/active/website-redesign.md",
|
||||
"projects/archived/old-website.md"}, False),
|
||||
("preferring uv for tooling", {"_agent/memory/semantic/tooling-preferences.md"}, False),
|
||||
("penalties in the SLA",
|
||||
{"_agent/sessions/2026-06-20-1000-vantage-contract-review.md"}, True),
|
||||
("saturated term frequencies", {"resources/concepts/bm25-ranking.md"}, False),
|
||||
("expanded linked neighbourhoods", {"resources/concepts/graph-expansion.md"}, False),
|
||||
]
|
||||
FRESHNESS_QUERY = "marketing site navigation"
|
||||
FRESHNESS_TOP1 = "projects/active/website-redesign.md"
|
||||
|
||||
Reference in New Issue
Block a user