Files
echo/eval/results/latest.json
T
Jason Stedwell b4e923c2e7
Build and Push Docker Image / build (push) Successful in 14s
2.3.0 — the index train: local-first recall index, incremental sweep, stemming + alias expansion
One schema-bump release (recall-index schema 3, entity-index schema 2; old
recall indexes rebuild automatically, no vault migration):

- Local-first recall index: the live BM25 index lives in the machine state dir
  (keyed by endpoint hash); capture's upkeep is a zero-network atomic file
  write, no advisory lock — replacing the O(vault) GET/PUT-whole-index round
  trip per write. The vault copy is a snapshot (sweep + session-end) used to
  seed fresh machines / catch up after another client swept
  (ECHO_RECALL_SYNC_HOURS, default 24). The read path never PUTs the vault.
- Incremental sweep: entity + recall meta carry 16-char content hashes;
  `sweep --fast` fetches only new/gone/hash-missing notes plus a rotating
  weekday shard (--all-shards forces everything); deletions drop from both
  indexes; index-only so no --apply gate. `load` auto-runs it past
  ECHO_FAST_SWEEP_DAYS (7); doctor reports index freshness. Obsidian-side
  edits now reach the indexes without manual maintenance.
- Stemming + alias expansion: echo_stem.py (conservative Porter-lite, unit-
  tested families + over-stemming guards) applied at index AND query time;
  a query fuzzy-matching an entity folds its title/alias vocabulary into the
  BM25 query at half weight — expansion can only boost docs containing the
  terms. capture hashes the note's FINAL content (auto-link reordered first).

Eval gold set 8 -> 14 queries (6 paraphrases): recall@5/MRR 1.00/1.00 vs
keyword baseline 0.75/0.79. +3 unit tests, +10 e2e; test harnesses now isolate
ECHO_STATE_DIR. All seven suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 00:08:48 -05:00

46 lines
1.1 KiB
JSON

{
"version": "1.5.1",
"date": "2026-07-03",
"retrieval": {
"current (hybrid+priors, full corpus)": {
"precision_at_5": 0.243,
"recall_at_5": 1.0,
"mrr": 1.0,
"session_journal_queries_answered": "3/3",
"freshness_top1_correct": true
},
"baseline (keyword, entities only)": {
"precision_at_5": 0.186,
"recall_at_5": 0.75,
"mrr": 0.786,
"session_journal_queries_answered": "0/3",
"freshness_top1_correct": false
}
},
"dedup": {
"gate ON (v1.5.1 default)": {
"duplicate_notes_created": 0,
"legitimate_captures_blocked": 0,
"referring_attempts": 3,
"legit_attempts": 2
},
"gate OFF (pre-1.5, warn-only)": {
"duplicate_notes_created": 3,
"legitimate_captures_blocked": 0,
"referring_attempts": 3,
"legit_attempts": 2
}
},
"write_safety": {
"fault_scenarios": 4,
"silent_write_failures": 0,
"failures_surfaced_loud": 2
},
"durability": {
"writes_during_outage": 3,
"reported_queued": 3,
"landed_after_flush": 3,
"lost": 0,
"duplicated_on_reflush": 0
}
}