 Jason Stedwell
|
993abdc846
|
Retire ROADMAP-1.0; rewrite run_eval.py as current-version metrics harness
Knocks two items off ROADMAP-2.0 (§4 eval refresh, §5 roadmap retirement):
- run_eval.py: the stale 0.6-vs-0.7 A/B is replaced (old version in git
history) with a four-part current-version eval against the mock —
retrieval (hybrid+priors vs keyword/entities-only baseline: R@5 1.00
vs 0.75, MRR 1.00 vs 0.75, session/journal queries 2/2 vs 0/2,
freshness top-1 correct vs wrong), dedup (0 dupes gate-on vs 3
gate-off, 0 false blocks), write safety (0 silent failures across 4
fault scenarios), durability (3/3 offline writes queued+landed, 0
lost, 0 re-flush dupes). Results in eval/results/latest.json.
- README: metrics table published; repo-layout eval descriptor updated.
- eval/README: new harness documented; A/B framing marked historical.
- ROADMAP-1.0.md removed (fully shipped; story lives in the README
version table + git history); dangling docstring pointer fixed in
echo_concurrency.py; ROADMAP-2.0 checkboxes ticked.
All suites green; artifact rebuilt (1.5.1, docstring-only source change).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-03 13:30:01 -05:00 |
|