From 1881d2b105d0d92edb57a15f511cea9846c9db8b Mon Sep 17 00:00:00 2001 From: Jason Stedwell Date: Tue, 23 Jun 2026 21:19:00 -0500 Subject: [PATCH] documentation --- README.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 0f04686..152e5fa 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# echo-memory — v1.0.0 +# echo-memory — v1.2.0 Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`. @@ -6,7 +6,7 @@ Built for **Jason Stedwell** (Director of Technical Services / Systems Engineer, This repository (`jason/echo-v.05`) holds the plugin **source** (tracked tree at `echo-memory.plugin.src/`), the built `echo-memory.plugin` package artifact (rebuilt on each version bump), and a credential-free A/B `eval/` harness. -**0.9 in one line:** one-call `capture` routes and crosslinks a memory automatically, `recall` returns a topic plus its linked neighbourhood, and a machine-maintained entity index makes routing an alias-aware lookup — all on a cross-platform Python client with a linter-enforced routing manifest and graph-health checks. See the [version history](#version-history) for how it got here. +**1.2 in one line:** one-call `capture` routes and crosslinks a memory automatically, `recall` fuses BM25 over note bodies with graph expansion, and alias-aware entity resolution routes shortened names to the right note instead of spawning duplicates — all on a cross-platform, **connection-pooled** Python client (concurrent full-vault reads, so a sweep/lint pass that used to time out now finishes in under a second) with an offline write-ahead queue, a linter-enforced routing manifest, and graph-health checks. See the [version history](#version-history) for how it got here. --- @@ -57,7 +57,7 @@ echo-v.05/ └── echo-memory.plugin.src/ ← tracked source tree (the plugin) ├── .claude-plugin/plugin.json ← manifest (name, version, description) ├── README.md ← plugin-level README - ├── commands/ ← slash commands: echo-load, echo-save, echo-recall, echo-triage, echo-health, echo-sweep + ├── commands/ ← slash commands: echo-load, echo-save, echo-recall, echo-triage, echo-health, echo-sweep, echo-reflect, echo-doctor └── skills/echo-memory/ ├── SKILL.md ← operating procedure (authoritative) ├── references/ @@ -96,9 +96,9 @@ Executable logic ships under `skills/echo-memory/scripts/`; the agent prefers it | Tool | Purpose | |------|---------| -| `echo.py` | The validated client + high-level CLI. Low-level verbs `get/ls/map/search/put/post/append/patch/fm/bump/delete/lock/unlock/scope/load` inject auth, **check HTTP status** (non-zero exit on ≥400), retry transient 5xx, read-back-verify PUT, idempotent whole-line `append`, correct `::` heading targets. High-level ops **`capture/resolve/recall/link`** do the routing/linking for you (see below). | +| `echo.py` | The validated client + high-level CLI. The network layer is **keep-alive + connection-pooled** (one persistent connection per thread, reused across requests) with a concurrent `read_many()` bulk-GET (`ECHO_WORKERS`, default 8) — so full-vault passes that used to time out finish in under a second. Low-level verbs `get/ls/map/search/put/post/append/patch/fm/bump/delete/lock/unlock/scope/load` inject auth, **check HTTP status** (non-zero exit on ≥400), retry transient 5xx (and reconnect a stale pooled connection), read-back-verify PUT, idempotent whole-line `append`, correct `::` heading targets. High-level ops **`capture/resolve/recall/link`** do the routing/linking for you (see below). | | `capture / resolve / recall / link` | The input-reducing layer. **`capture "" --kind <k>`** routes via the entity index, stamps frontmatter, indexes, auto-links mentioned entities, and writes the Agent-Log line in one call. **`resolve "<mention>"`** → canonical path (alias-aware). **`recall "<query>"`** → matching notes + their one-hop linked neighbourhood. **`link A B`** → reciprocal `## Related` links. | -| `echo_index.py` | The **entity index** (`_agent/index/entities.json`): slug→{path, kind, title, aliases, last_seen}. Makes routing/resolve an O(1), alias-aware lookup and supplies the name→path map for linking and recall. Rebuilt automatically by `capture` and `sweep.py`. | +| `echo_index.py` | The **entity index** (`_agent/index/entities.json`): slug→{path, kind, title, aliases, last_seen}. Makes routing/resolve an O(1), alias-aware lookup and supplies the name→path map for linking and recall. `resolve()` matches on slug/title/alias; a `fuzzy_candidates()` "did-you-mean" fallback (1.2) surfaces near-matches for shortened names without auto-merging. Aliases are auto-derived from titles, learned from mentions on update, stored in note frontmatter, and folded back in by `sweep.py`. Rebuilt automatically by `capture` and `sweep.py`. | | `echo_links.py` / `echo_ops.py` | Cross-link primitives (parse/add `## Related`, bidirectional linking) and the high-level ops layer (capture/recall/resolve/link/agent-log). | | `routing.json` | The **canonical machine-readable** route manifest — one regex pattern per valid destination plus retired paths. The single source of truth for "what may be written where"; `vault_lint.py` enforces it against the vault, `check_routing.py` against the docs. | | `vault_lint.py` | Read-only invariant + **graph-health** checker (Vault Health). Tolerant frontmatter parsing, clock injected via `ECHO_TODAY`, exits `3` if the vault isn't bootstrapped. Checks scope-drift, **broken wikilinks, orphan notes, and entity-index drift**. | @@ -109,7 +109,7 @@ Executable logic ships under `skills/echo-memory/scripts/`; the agent prefers it ### Slash commands -`/echo-load` (cold-start read), `/echo-save <text>` (route + persist via `capture`), `/echo-recall <query>` (recall a topic's neighbourhood), `/echo-triage` (drain the inbox), `/echo-health` (run the linter), `/echo-sweep` (bring the vault up to spec) — explicit, reproducible entry points to the procedures below. +`/echo-load` (cold-start read), `/echo-save <text>` (route + persist via `capture`), `/echo-recall <query>` (recall a topic's neighbourhood), `/echo-triage` (drain the inbox), `/echo-health` (run the linter), `/echo-sweep` (bring the vault up to spec), `/echo-reflect` (extract→preview→apply durable items from the session), `/echo-doctor` (readiness check: Python, vault reachability, auth, bootstrap/schema, key source) — explicit, reproducible entry points to the procedures below. ### Concurrency (shared vault) @@ -365,6 +365,8 @@ If the API returns a connection error, timeout, or `502` (usually Obsidian / the | Version | Highlights | |---------|-----------| +| **1.2.0** | **Entity-resolution overhaul — fewer duplicate notes.** Resolution was exact-match only, so a shortened or expanded mention (e.g. "echo memory" for the `echo`-slugged active project) returned *no match* and the writer created a parallel note. Now `resolve()` matches on slug / title / **aliases**, with a `fuzzy_candidates()` "did-you-mean" fallback ranked by shared distinctive tokens (guarded by `echo_quality` so generic words like "data"/"api"/"the" can't drive a match). Aliases are **auto-derived from titles** (hyphen/space/case variants), **learned from mentions** on update, and stored in each note's `aliases:` frontmatter (the Obsidian-native home), then folded back into the entity index by `sweep` — so the graph gets better at resolution over time, while exact-match safety still prevents wrong auto-merges. Ships with the v1.1 connection-pool + concurrency work. 33 automated tests pass across Win/macOS/Linux; linter clean. See the [performance brief](echo-memory-performance-brief.html). | +| **1.1.0** | **Performance / network-layer rebuild — "from timing out to sub-second."** Full-vault scripts (`sweep.py`, `vault_lint.py`, recall rebuild) were making hundreds of *serial* requests, each opening a fresh TLS connection and re-reading every note 2–3×; on the constrained agent sandbox a single pass exceeded the ~120 s tool timeout and was killed mid-run, dropping the session/thread. Three structural fixes in `echo.py`: (1) **connection reuse (keep-alive)** — one persistent pooled connection per thread, reused across requests instead of a TCP+TLS handshake per file (**~4.5× faster per request**, the dominant win; benefits load/capture/recall for free); (2) **concurrent bulk reads** — new `read_many()` fans GETs across a thread pool (`ECHO_WORKERS`, default 8) for ~2× on full-vault scale; (3) **single-pass shared cache** — each note fetched *once* and reused across all passes (~2–3× fewer requests), eliminating `sweep`'s per-link-target re-fetch storm. Net effect: a full-vault pass that used to **time out now finishes in <1 s** (vault_lint 0.90 s, sweep plan 0.85 s on 186 notes); the practical change is fails → completes. Stdlib-only, no new dependencies. | | **1.0.0** | Schema 4. **"Memory you can trust and retrieve."** (H1) **Hybrid recall** — local BM25 over note bodies fused with decayed graph expansion (`echo_recall.py`, `_agent/index/recall-index.json`), maintained on `capture`, rebuilt by `sweep`; replaces keyword-only recall. (H2) **Offline durability** — write verbs queue to a local write-ahead outbox on an outage and report "queued"; `flush` (+ flush-on-`load`) replays idempotently and **re-bases to the current endpoint**; `load` degrades to a last-known-good read cache (`echo_queue.py`). (H3) **Concurrency** — `vault_lock` + `atomic_index_update` (lock + fresh re-read-merge) close the concurrent-`capture` entity-loss race (`echo_concurrency.py`). (H4) **Trust** — higher-fidelity PATCH mock + offline/reflect/patch-semantics suites gated in GitHub Actions across Win/macOS/Linux × Py 3.10/3.12. (H5) **Reflection capture** — `echo.py reflect` / `/echo-reflect` extract→dedup→preview→apply durable items (`echo_reflect.py`). (M1) **Secret hardening** — key resolves env → `~/.echo-memory/credentials` → deprecated fallback; `write-key` CLI; token scrubbed from docs (see [API-KEY-SETUP.md](API-KEY-SETUP.md)). (M2) confident auto-linking + slug-collision guard (`echo_quality.py`). (M3) `echo.py doctor` / `/echo-doctor` + heartbeat-less `load` fallback. (M4) `capture --json` / `--dry-run`. (M5) manifest → 1.0.0, `.gitignore`. | | **0.3.0** | Source promoted from zip-only to a tracked tree (`echo-memory.plugin.src/`); `.plugin` becomes a build artifact. All 7 skill-improvement items applied: search-first before writes, resilient daily Agent Log, `created:` semantics, project lifecycle + folder↔status rule, canonical HHMM session filenames, read-most-recent-N sessions, `source_notes` defined as backward links. | | **0.4.0** | Efficiency + robustness pass: parallel cold-start loading, idempotent POST (read-before-append), doc-map-before-first-PATCH, scoped `updated:` bump, Inbox Triage, Scope Switching, monthly Vault Health, Rules-vs-Observations split, formal deprecation of `decisions/by-project/`, heartbeat pointer. |