jason
177 lines
16 KiB
Markdown
177 lines
16 KiB
Markdown
# echo-memory — Roadmap to v1.0
|
||
|
||
> Status: **draft / in progress.** Current shipping version: **0.9.0 (schema 3)**.
|
||
> Target: **1.0.0 (schema 4)** — *"memory you can actually trust and retrieve."*
|
||
|
||
This roadmap turns the 10 code-review suggestions into a sequenced plan and tracks the
|
||
scaffold for each. It is the single index: every v1.0 workstream maps to one or more
|
||
files under `echo-memory.plugin.src/skills/echo-memory/scripts/` (plus tests and CI).
|
||
|
||
---
|
||
|
||
## 1.0 thesis
|
||
|
||
0.x made the plugin **correct and self-describing**: status-checked writes, idempotent
|
||
appends, a routing manifest the linter enforces, an entity index, and one-call
|
||
`capture/recall`. 1.0 makes it **trustworthy and genuinely recall-driven**:
|
||
|
||
1. You can **retrieve** a fact even when you phrase the query differently than you stored it. *(H1)*
|
||
2. Nothing is **lost** when the backend is down. *(H2)*
|
||
3. Two clients writing at once **cannot corrupt** the index or clobber state. *(H3)*
|
||
4. Every feature is **proven** by tests that model the real API, gated in CI. *(H4)*
|
||
5. Memory **accrues on its own** instead of only when the agent remembers to write. *(H5)*
|
||
|
||
…on a base that is secure (M1), clean-linking (M2), self-diagnosing (M3),
|
||
machine-parseable (M4), and maintainable (M5).
|
||
|
||
---
|
||
|
||
## Non-negotiable constraints (these bound every solution)
|
||
|
||
These are inherited from the 0.x design and **must not regress**:
|
||
|
||
| Constraint | Consequence for 1.0 |
|
||
|---|---|
|
||
| **Pure-Python stdlib, no third-party deps** | H1 recall = local BM25 + graph fusion, **not** an embedding API. No `numpy`, `keyring`, `requests`. |
|
||
| **Cross-platform (Win/macOS/Linux)** | No bash, no platform `date`; `pathlib`, `os`, UTF-8-safe streams. |
|
||
| **Plugin is the single source of truth; vault holds data only** | Derived artifacts (recall index) live in `_agent/index/` (machine-maintained, rebuildable) — never new control docs in the vault. |
|
||
| **Fail loud, never silent** | New write paths keep status-checking and non-zero exits. |
|
||
| **Local state can't depend on the vault being up** | H2 queue/cache lives in `ECHO_STATE_DIR` (default `~/.echo-memory/`), not the vault. |
|
||
|
||
---
|
||
|
||
## Build order vs. scaffold order
|
||
|
||
The user asked to **scaffold heaviest → lightest** (H1→M5). That is the order the
|
||
files below are stubbed. The **recommended build/merge order is different** — safety
|
||
and proof first, so the heavy features land on a tested, secure base:
|
||
|
||
```
|
||
Phase A (land first): M1 secrets · H4 tests+CI
|
||
Phase B (correctness): H3 concurrency · M2 link/slug quality
|
||
Phase C (durability): H2 offline queue + cache
|
||
Phase D (intelligence): H1 hybrid recall · H5 reflection capture
|
||
Phase E (polish): M3 doctor · M4 --json · M5 hygiene → tag 1.0.0
|
||
```
|
||
|
||
Scaffolds are independent stubs, so stubbing heaviest-first is safe; only the
|
||
*integration* (wiring into `echo.py`, schema bump, migration) follows Phase order.
|
||
|
||
---
|
||
|
||
## Workstreams (heaviest → lightest)
|
||
|
||
Each item: **goal · why · scope · files · storage/schema · acceptance · depends-on · lift.**
|
||
|
||
### H1 — Hybrid/semantic recall · lift: ●●●●●
|
||
- **Goal:** `recall "X"` returns relevant notes even when the wording differs, ranked, with scored multi-hop graph expansion.
|
||
- **Why:** Recall is the product. Today it's keyword `/search/simple` + fixed one-hop (`echo_ops.py:88`). Differently-phrased memory is unfindable → dead weight.
|
||
- **Scope:** local BM25 index over note bodies (stdlib only); fuse lexical score + graph distance (decay per hop); relevance threshold; configurable hops.
|
||
- **Files:** `scripts/echo_recall.py` (new) → later replaces `echo_ops.recall`.
|
||
- **Storage/schema:** `_agent/index/recall-index.json` (postings + doc stats), machine-maintained, rebuilt by `sweep.py`. **Schema 3→4.**
|
||
- **Acceptance:** retrieval precision/recall measured in eval beats keyword-only baseline; recall returns scored, deduped hits + neighbourhood; index rebuildable offline.
|
||
- **Depends-on:** entity index (have), H4 eval harness for the precision/recall numbers.
|
||
|
||
### H2 — Offline durability: write-ahead queue + read cache · lift: ●●●●●
|
||
- **Goal:** No durable write is lost when the API/Obsidian is down; cold-start degrades to last-known context instead of "no memory."
|
||
- **Why:** Today vault-unreachable = silently proceed without memory (`SKILL.md:383`). 502 (Obsidian not running) is the most common real failure.
|
||
- **Scope:** append-only NDJSON outbox; replay on next reachable session (idempotency makes replay safe); last-known-good read cache for `load`.
|
||
- **Files:** `scripts/echo_queue.py` (new); integration hook `safe_request()` wrapping `echo.request`.
|
||
- **Storage/schema:** local `ECHO_STATE_DIR/outbox.ndjson` + `ECHO_STATE_DIR/cache/`. No vault schema change.
|
||
- **Acceptance:** kill the API mid-session → writes queue; restart → replay lands exactly once (no dupes); `load` serves cache when offline and says so.
|
||
- **Depends-on:** existing idempotent-append discipline; H4 for fault-injection tests.
|
||
|
||
### H3 — Concurrency correctness · lift: ●●●●○
|
||
- **Goal:** Make the advertised multi-writer (Claude + CoWork) story actually safe.
|
||
- **Why:** Index is full-file load→PUT with no lock (`echo_index.py:105`, `echo_ops.py:230`) → concurrent `capture` silently drops an entity. Lock is manual/advisory only (`echo.py:293`).
|
||
- **Scope:** auto-lock context manager around `capture`/`scope set`/index writes; atomic index update (re-read under lock, merge, save); per-resource locks; crash-safe TTL reclaim (have).
|
||
- **Files:** `scripts/echo_concurrency.py` (new); refactor `echo_index.save` + `echo_ops.capture` to route through it.
|
||
- **Storage/schema:** reuse `_agent/locks/`. No schema change.
|
||
- **Acceptance:** two concurrent captures both land in the index; no lost entries; lock auto-released on normal + error exit.
|
||
- **Depends-on:** none (pure refactor); H4 to prove it.
|
||
|
||
### H4 — High-fidelity tests + CI + current eval · lift: ●●●●○
|
||
- **Goal:** Prove every 0.9 + 1.0 feature against an API model that reproduces real failure modes; gate on CI.
|
||
- **Why:** Mock PATCH is "naive append" (`mock_olrapi.py:138`) — can't model heading replace/insert; eval still benchmarks **0.6 vs 0.7** (two versions stale).
|
||
- **Scope:** higher-fidelity mock (real heading/frontmatter PATCH semantics) or containerized real Obsidian Local REST API; refresh eval to measure retrieval precision/recall, link correctness, dup rate; wire `test_echo_client.py` + `check_routing.py` + `test_features.py` + new module tests into GitHub Actions.
|
||
- **Files:** `.github/workflows/ci.yml` (new); `eval/mock_olrapi_hifi.py` (new); `scripts/test_v1_scaffold.py` (interface guard, new); refreshed `eval/run_eval.py`.
|
||
- **Acceptance:** CI green on every push; eval reports current-version metrics; mock reproduces the 40080 invalid-target and replace/insert paths.
|
||
- **Depends-on:** none — **foundational, build in Phase A.**
|
||
|
||
### H5 — Automatic session-reflection capture · lift: ●●●●○
|
||
- **Goal:** At session end, propose durable captures extracted from the conversation for one-tap confirm — memory that fills itself.
|
||
- **Why:** Memory only accrues when the agent decides to write. "As useful as possible" means not depending on that judgment firing.
|
||
- **Scope:** model-side extraction emits a JSON proposal set; script dedups against the entity index, previews, and applies on confirm (respects "show before large writes" safety rule).
|
||
- **Files:** `scripts/echo_reflect.py` (new); `/echo-reflect` command (later).
|
||
- **Storage/schema:** none new (routes through `capture`).
|
||
- **Acceptance:** given a transcript, proposals are deduped vs index, previewed, and only applied on explicit confirm; nothing written without go-ahead.
|
||
- **Depends-on:** H1 index/recall for dedup quality; H3 for safe concurrent apply.
|
||
|
||
### M1 — Harden secret handling · lift: ●●○○○ *(do now — Phase A)*
|
||
- **Goal:** Stop shipping a live bearer token in source, docs, and every `.plugin` zip.
|
||
- **Why:** Token hardcoded at `echo.py:70` and repeated through `api-reference.md`/README — committed to git, only rotatable by rebuild. Violates the plugin's own "never store secrets" rule.
|
||
- **Scope:** resolve key env → local `ECHO_STATE_DIR/credentials` (0600) → deprecated baked default **with a loud warning**; scrub literal from docs to a placeholder; document rotation.
|
||
- **Files:** `scripts/echo_secrets.py` (new); `echo.py` `KEY=` → `echo_secrets.resolve_key()`.
|
||
- **Acceptance:** no live token in tracked source/docs/package; `ECHO_KEY` still overrides; rotation documented.
|
||
- **Depends-on:** none.
|
||
|
||
### M2 — Tame auto-link false positives & slug collisions · lift: ●●○○○
|
||
- **Goal:** Keep the graph (which recall depends on) clean.
|
||
- **Why:** Auto-link fires on any ≥3-char name/alias word-match (`echo_ops.py:236`) → a concept "API" or alias "rs" links everywhere; `slugify` truncates to 40 chars with no collision check (`echo_index.py:58`) → distinct titles silently share a note.
|
||
- **Scope:** raise alias floor / require multi-token or kind-aware match / stopword guard; collision-aware slug disambiguation in `derive_path`.
|
||
- **Files:** `scripts/echo_quality.py` (new); patches to `echo_links`/`echo_index`.
|
||
- **Acceptance:** known false-positive cases no longer link; colliding titles get distinct slugs; covered by tests.
|
||
- **Depends-on:** none.
|
||
|
||
### M3 — `echo.py doctor` + complete `load` fallback · lift: ●●○○○
|
||
- **Goal:** One-call readiness check; make `load` self-sufficient.
|
||
- **Why:** No single "is everything OK" check. `cmd_load` reads the heartbeat but never falls back to listing `_agent/sessions/` though `SKILL.md:122` documents it (`echo.py:395`).
|
||
- **Scope:** doctor = Python version + reachability + auth + marker/`schema_version` + lint summary, green/red; `load` lists recent sessions when heartbeat missing/stale.
|
||
- **Files:** `scripts/echo_doctor.py` (new); patch `echo.cmd_load`.
|
||
- **Acceptance:** `doctor` prints actionable status and exits non-zero on any red; `load` orients with no heartbeat.
|
||
- **Depends-on:** none.
|
||
|
||
### M4 — Machine output (`--json`) + `--dry-run` · lift: ●○○○○
|
||
- **Goal:** Let the agent act on structured results and preview writes.
|
||
- **Why:** `capture/recall/resolve/scope` print prose; agent re-parses free text.
|
||
- **Scope:** `--json` result envelope (action, path, links_added, hits); `--dry-run` for `capture`/`scope set`.
|
||
- **Files:** `scripts/echo_output.py` (new helpers); thread through `echo_ops`/`echo.py`.
|
||
- **Acceptance:** every high-level op emits valid JSON under `--json`; `--dry-run` writes nothing.
|
||
- **Depends-on:** none.
|
||
|
||
### M5 — Repo hygiene & manifest completeness · lift: ●○○○○
|
||
- **Goal:** Remove "which tree is canonical?" traps; complete the manifest.
|
||
- **Why:** Two plugin trees (`codex plugin/` vs `echo-memory.plugin.src/`) have already drifted (codex lacks the 0.9 modules); stack of versioned `.plugin` artifacts; sparse `plugin.json`.
|
||
- **Scope:** consolidate/mark canonical tree; prune old artifacts; add `license`/`homepage`/command registration to `plugin.json`; refresh version-history + eval references.
|
||
- **Files:** `MAINTENANCE.md` (new checklist); `plugin.json`.
|
||
- **Acceptance:** one canonical source tree; complete manifest; docs reference current version.
|
||
- **Depends-on:** none.
|
||
|
||
---
|
||
|
||
## Schema 3 → 4 migration (owned by H1/H2)
|
||
|
||
- Add `_agent/index/recall-index.json` (H1), rebuilt by `sweep.py`.
|
||
- `migrate.py`: add `[3→4]` step (create recall-index placeholder; no destructive moves).
|
||
- `routing.json`: add a route for `_agent/index/recall-index.json` (already covered by the generic `^_agent/index/[^/]+\.json$` route — verify in `check_routing.py`).
|
||
- Bump `CURRENT_SCHEMA` to 4 in `sweep.py`/`migrate.py`; `vault_lint.py` unaffected.
|
||
|
||
---
|
||
|
||
## Scaffold status
|
||
|
||
| # | Workstream | Lift | Scaffold file(s) | State |
|
||
|---|---|---|---|---|
|
||
| H1 | Hybrid recall | ●●●●● | `scripts/echo_recall.py` | **✅ WIRED** — BM25 + vault crawl + `recall-index.json` persistence + graph fusion; maintained on `capture`, rebuilt by `sweep`; replaces `echo_ops.recall`; **schema 4**. Covered by `test_features.py`. |
|
||
| H2 | Offline queue/cache | ●●●●● | `scripts/echo_queue.py`, `eval/test_offline_queue.py` | **✅ WIRED** — write verbs (put/post/append/patch/delete) queue on outage and report "queued"; `flush` (+ flush-on-`load`) replays idempotently and **re-bases to the current endpoint** (survives base migration); `load` degrades to last-known-good cache and flags OFFLINE. Proven end-to-end (down→queue→up→flush→land, idempotent, cache fallback). Follow-up: route capture's internal index/link writes through the queue for full offline-atomic captures. |
|
||
| H3 | Concurrency | ●●●●○ | `scripts/echo_concurrency.py` | **✅ WIRED** — `vault_lock` CM (auto acquire/release, crash-safe, advisory) + `atomic_index_update` (lock + fresh re-read-merge). `capture` entity write and `echo_recall.update_note` both routed through it. Proven by `test_features.py` (merge-no-clobber + lock-release). Follow-up: deeper resolve-level title-collision detection; true concurrent stress test. |
|
||
| H4 | Tests + CI | ●●●●○ | `.github/workflows/ci.yml`, `eval/mock_olrapi_hifi.py`, `eval/test_patch_semantics.py`, `scripts/test_v1_scaffold.py` | **✅ hi-fi PATCH mock + semantics test + CI matrix (Win/macOS/Linux × 3.10/3.12) live.** Remaining: refresh `run_eval.py` (still 0.6-vs-0.7) → publish current metrics. |
|
||
| H5 | Reflection capture | ●●●●○ | `scripts/echo_reflect.py`, `eval/test_reflect.py`, `commands/echo-reflect.md` | **✅ WIRED** — `validate`/`classify`/`preview`/`apply` + `echo.py reflect` (dry-run unless `--apply`, reads file or stdin) + `/echo-reflect` command. Dedups proposals vs the entity index, drops below-confidence, routes `inbox`, applies via `capture` (indexed/linked/logged, self-lock-guarded). Proven by `test_reflect.py`. |
|
||
| M1 | Secret handling | ●●○○○ | `scripts/echo_secrets.py` | **✅ WIRED** — `resolve_key` (env → `~/.echo-memory/credentials` → deprecated fallback + loud warning), `write-key` CLI, docs scrubbed. Live token now only in `echo.py` `DEFAULT_KEY` (remove at 1.0; operator env already sets `ECHO_KEY`). |
|
||
| M2 | Link/slug quality | ●●○○○ | `scripts/echo_quality.py` | **✅ WIRED** — `is_confident_link` gates `capture` auto-linking (rejects <4-char / common tokens, trusts multi-word names); `safe_slug` disambiguates a colliding 40-char-truncated slug in the create path. Follow-up: resolve-level title collision (two long titles → same slug → false merge). |
|
||
| M3 | Doctor + load fallback | ●●○○○ | `scripts/echo_doctor.py`, `commands/echo-doctor.md` | **✅ WIRED** — `echo.py doctor` (Python/reachability/auth/bootstrap/schema/key-source) + `/echo-doctor`; `cmd_load` now falls back to a recent-sessions listing when the heartbeat pointer is absent. Verified against the live vault. |
|
||
| M4 | `--json` / `--dry-run` | ●○○○○ | `scripts/echo_output.py` | **✅ WIRED** — `capture --json` emits a clean result envelope (helper chatter redirected) and `--dry-run` previews the create/update plan without writing; `resolve` already emits JSON. Proven by `test_features.py`. Follow-up: `--json` for recall/scope/link. |
|
||
| M5 | Hygiene | ●○○○○ | `plugin.json`, `.gitignore`, `MAINTENANCE.md` | **✅ DONE** — `plugin.json` → **1.0.0** + `license`; `.gitignore` (`.DS_Store`, `__pycache__`, local `.echo-memory/` state, eval output). Remaining (operator judgment, not code): consolidate/retire the drifted `codex plugin/` tree and prune old `.plugin` artifacts — see `MAINTENANCE.md`. |
|
||
|
||
**Wiring:** scaffolds are **not** imported by the live `echo.py` yet — 0.9.0 behavior is unchanged. Integration happens per Phase, each gated by its tests (H4).
|