Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Jason Stedwell
|
5aace9e430 | ||
|
Jason Stedwell
|
ab1e514f78 | ||
|
Jason Stedwell
|
fb407d606c | ||
|
Jason Stedwell
|
55afdce70c | ||
|
Jason Stedwell
|
e7d86e14da | ||
|
Jason Stedwell
|
d27db4ae34 |
+7
-5
@@ -20,8 +20,10 @@ eval/results/*.json
|
||||
/echo-memory.config.json
|
||||
/dist/
|
||||
|
||||
# Per-user BAKED artifacts (build.py --bake-key, named echo-memory-<ver>-<label>.plugin)
|
||||
# carry a live vault bearer token — NEVER commit them. They belong in dist/ (above);
|
||||
# this also guards any that get built at the repo root. The version-only pointer
|
||||
# (echo-memory.plugin) and version-only artifacts (echo-memory-<ver>.plugin) stay tracked.
|
||||
echo-memory-*-*.plugin
|
||||
# Build artifacts (2.0 policy): the repo tracks ONLY the echo-memory.plugin pointer.
|
||||
# Versioned builds (echo-memory-<ver>.plugin) are published as Gitea release assets
|
||||
# on git.alwisp.com/jason/echo, one release per tag — not committed. Per-user BAKED
|
||||
# artifacts (echo-memory-<ver>-<label>.plugin) carry a live vault bearer token and
|
||||
# must NEVER be committed anywhere (they belong in dist/, also ignored above).
|
||||
*.plugin
|
||||
!echo-memory.plugin
|
||||
|
||||
@@ -1,5 +1,74 @@
|
||||
# Changelog
|
||||
|
||||
## 2.0.0
|
||||
|
||||
**Packaging & structure major — nothing about memory behavior changes.** The major
|
||||
version signals "reinstall, re-clone expectations": how the plugin is distributed
|
||||
and installed is what breaks.
|
||||
|
||||
### Removed — the legacy `commands/` directory (BREAKING for pre-skills clients)
|
||||
|
||||
The plugin is **skills-only**. The eight `/echo-*` entry points live exclusively at
|
||||
`skills/<name>/SKILL.md` (introduced 1.6.0, verified on desktop + CoWork
|
||||
2026-07-28, where they already took precedence). A Claude Code old enough to lack
|
||||
skills support loses the slash commands — install 1.6.0 from the release history
|
||||
instead.
|
||||
|
||||
### Changed — artifact policy (BREAKING for anyone pulling zips from the repo)
|
||||
|
||||
- The repo tracks **only** `echo-memory.plugin` (the current installable). The 15
|
||||
historical `echo-memory-<version>.plugin` zips are deleted from the tree (git
|
||||
history retains them).
|
||||
- Versioned builds are published as **Gitea releases** on
|
||||
`git.alwisp.com/jason/echo` — one release per `v<version>` tag, artifact
|
||||
attached, starting with `v2.0.0`.
|
||||
- `.gitignore` now ignores `*.plugin` except the pointer.
|
||||
|
||||
### Added — "packaging for other agent runtimes" README note
|
||||
|
||||
The retired Codex tree stays retired; the README now documents what a port needs
|
||||
(manifest shape, skill entry point, script invocation + hooks) and the rule that a
|
||||
port must be a **build target** from the canonical source tree, never a second
|
||||
hand-maintained copy.
|
||||
|
||||
## 1.6.0
|
||||
|
||||
### Changed — the eight slash commands migrated to the skills format
|
||||
|
||||
Each `commands/<name>.md` now has a `skills/<name>/SKILL.md` twin with the same
|
||||
invocation name and a byte-identical body — confirmed against the current docs
|
||||
that a same-name skill **takes precedence** over the legacy command, so both
|
||||
formats coexist safely in this release (deleting `commands/` is the 2.0
|
||||
packaging break). Done for the features, not the deprecation notice:
|
||||
|
||||
- **`allowed-tools` per skill** — the resolved `python3/python/py -3` script
|
||||
invocations (plus the CoWork `ls /sessions/*` fallback probe) are
|
||||
pre-authorized, so `/echo-load`, `/echo-health`, `/echo-recall` etc. stop
|
||||
prompting.
|
||||
- **`disable-model-invocation: true`** on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) — only the operator triggers them; the
|
||||
read-side skills stay model-invocable.
|
||||
- `argument-hint` / `$ARGUMENTS` carry over unchanged (same semantics in
|
||||
SKILL.md bodies).
|
||||
- The `$ECHO` path-resolution block stays per-skill: the skills format has no
|
||||
cross-skill snippet sharing (each skill directory is self-contained per the
|
||||
official docs), and the block is already the 2-line minimum.
|
||||
|
||||
### Fixed — lint blind spot: retired trees masked by the leaf-README route
|
||||
|
||||
`vault_lint.py` checked retired patterns only for paths matching **no** route,
|
||||
so the permissive `leaf-readme` route (`^(.+/)?README\.md$`) absolved any seed
|
||||
README inside a retired tree — `archive/` and `_agent/outputs/` survived the
|
||||
1.5.0 live cleanup unflagged. Retired patterns are now checked **first**: a
|
||||
path in a retired tree flags `retired-path` regardless of what else matches.
|
||||
New end-to-end test seeds `archive/notes/README.md` and asserts the flag
|
||||
(fails against the pre-fix linter, passes now).
|
||||
|
||||
### Added — plugin.json completeness
|
||||
|
||||
`homepage` + `repository` → `https://git.alwisp.com/jason/echo`; manifest →
|
||||
1.6.0.
|
||||
|
||||
## 1.5.1
|
||||
|
||||
### Fixed — duplicate gate over-firing on vault-common tokens and cross-kind names
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# echo-memory — v1.5.1
|
||||
# echo-memory — v2.0.0
|
||||
|
||||
Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`.
|
||||
|
||||
@@ -67,7 +67,8 @@ echo-v.05/
|
||||
├── docs/history/ ← historical inputs (e.g. the 1.5.0 frontmatter field report)
|
||||
├── build.py ← deterministic .plugin builder (--bake-key/--strip-key/--label/--outdir)
|
||||
├── echo-memory.plugin ← built, installable plugin (zip artifact, rebuilt on version bump)
|
||||
├── echo-memory-<version>.plugin ← versioned build artifacts (history; moves to Gitea releases in 2.0)
|
||||
│ — the ONLY tracked artifact; versioned builds are Gitea release
|
||||
│ assets (one release per v<version> tag), not committed (2.0 policy)
|
||||
├── dist/ ← per-user baked artifacts (secret-bearing) — GITIGNORED, never committed
|
||||
├── eval/ ← credential-free eval + test harness; not bundled
|
||||
│ ├── mock_olrapi.py ← deterministic mock of the REST API + fault injection
|
||||
@@ -80,8 +81,9 @@ echo-v.05/
|
||||
├── .claude-plugin/plugin.json ← manifest (name, version, description)
|
||||
├── README.md ← plugin-level README
|
||||
├── hooks/hooks.json ← session hooks: SessionStart auto-load · Stop reflection nudge
|
||||
├── commands/ ← slash commands (legacy format; skills-format migration = TODO-1.6):
|
||||
│ echo-load|save|recall|triage|health|sweep|reflect|doctor
|
||||
├── skills/echo-{load,save,recall,triage,health,sweep,reflect,doctor}/
|
||||
│ ← the eight slash commands as skills (skills-only since 2.0):
|
||||
│ per-skill allowed-tools; sweep/triage are operator-only
|
||||
└── skills/echo-memory/
|
||||
├── SKILL.md ← operating procedure (authoritative)
|
||||
├── references/
|
||||
@@ -120,6 +122,8 @@ echo-v.05/
|
||||
└── templates/ ← 8 canonical note templates
|
||||
```
|
||||
|
||||
**Packaging for other agent runtimes** (from the retired Codex tree, deleted 2026-07-03): if a port to another runtime (Codex, etc.) is ever wanted, implement it as a **build target** generated from the canonical `echo-memory.plugin.src/` tree (`build.py --target <runtime>` or similar) — never as a second hand-maintained source tree; that's how the original Codex copy drifted. A port needs three adaptations: the manifest shape (`.claude-plugin/plugin.json` → the runtime's equivalent), the skill entry point (`skills/*/SKILL.md` frontmatter + body conventions), and the script-invocation convention (`${CLAUDE_PLUGIN_ROOT}` resolution + the session-hook wiring in `hooks/hooks.json`). The Python toolchain itself is runtime-agnostic (pure stdlib, env/config-driven).
|
||||
|
||||
**Division of responsibility:** `SKILL.md` owns day-to-day *procedure* (loading order, search-first, triage, scope switching, PATCH rules) and points at the bundled tooling. `references/operating-contract.md` owns the durable, client-independent *principles, safety rules, and concurrency model*. `scripts/routing.json` is the machine-readable source of truth for routing; `references/routing-map.md` is its human-readable authority. The other references are the canonical layout, API, and bootstrap specs.
|
||||
|
||||
---
|
||||
@@ -417,6 +421,8 @@ From the credential-free harness (`eval/run_eval.py` against the deterministic m
|
||||
|
||||
| Version | Highlights |
|
||||
|---------|-----------|
|
||||
| **2.0.0** | **Packaging & structure major — memory behavior unchanged.** Skills-only: the legacy `commands/` directory is deleted (breaking for pre-skills clients; the 1.6.0-verified skills are the sole entry points). Artifact policy: the repo tracks only the `echo-memory.plugin` pointer; the 15 historical versioned zips leave the tree and versioned builds ship as **Gitea releases** (one per `v<version>` tag) from `v2.0.0` on; `.gitignore` blocks `*.plugin` except the pointer. README gains the "packaging for other agent runtimes" port note (build-target rule, never a second source tree). |
|
||||
| **1.6.0** | **Skills-format migration + lint blind-spot fix.** The eight slash commands gain `skills/<name>/SKILL.md` twins (same names, byte-identical bodies; a same-name skill takes precedence, so `commands/` coexists until its 2.0 deletion): per-skill `allowed-tools` end the permission prompts, `disable-model-invocation: true` makes `/echo-sweep` + `/echo-triage` operator-only. `vault_lint` checks retired patterns **before** routes, so a seed README inside a retired tree (`archive/…`) is flagged instead of being absolved by the leaf-README route (+ regression test). `plugin.json` gains `homepage`/`repository`. |
|
||||
| **1.5.1** | **Duplicate-gate precision.** Live use caught the gate blocking a decision note because it shared the single vault-common token "echo" with an archived project (min-normalized scoring gives any single-token entity a 1.0 match). New `gate_candidates()` blocks only on **same-kind** candidates (cross-kind name collisions — a decision titled after its project — warn instead), and a **lone shared token blocks only when unique to that entity** (vault df == 1); multi-token overlaps still block. Advisory warnings (`fuzzy_candidates`) unchanged; exit-76/`--merge-into`/`--force` unchanged. +5 tests; verified against the live vault both ways (false positive creates cleanly, true lookalike still gates). |
|
||||
| **1.5.0** | **Six-improvement pass: retention, routing, and self-firing memory.** (1) **Capture-update keeps the whole body** — the update path previously appended only the first line of a multi-line body (silent data loss); now the full body rides under the dated bullet. (2) **Frontmatter completeness** — capture stamps a kind-default `status` and kind-seeded `tags` (`--tags` enriches); `fm` is create-or-replace (missing keys are surgically inserted instead of 400-failing); `vault_lint` gains a kind-scoped `incomplete-frontmatter` check; `sweep --apply` backfills (fixes the 18/71-notes field-audit drift, one data source: `KIND_STATUS`/`KIND_REQUIRED_FM`). (3) **Pre-write duplicate gate** — a strong fuzzy candidate stops `capture` (exit 76 + candidates) before the duplicate exists; `--merge-into <slug>` updates the canonical entity, `--force` overrides. (4) **Recall corpus + ranking** — sessions and journal notes join the BM25 corpus (down-weighted); scores fuse freshness (half-life on `updated:`) and status (`active` boosted, `archived` demoted); hits show `updated:`/`status:`; `recall --json`; recall-index schema 2 (auto-rebuilds). (5) **Session hooks** — SessionStart auto-loads memory into context, Stop nudges reflection once per substantive session; fail-safe, CoWork-fallback-aware. (6) **One-tap triage** — `echo.py triage` lists the inbox structured, then classifies → previews → routes accepted items via `capture` with automatic processing-log audit lines; `--json` also lands on `link`/`scope show`. +22 mock end-to-end tests; all suites green. |
|
||||
| **1.4.2** | **CoWork sandbox path resilience.** In a remote CoWork sandbox `${CLAUDE_PLUGIN_ROOT}` can point at a host path the sandbox can't reach (`/var/folders/…`) while the plugin is mounted under `…/mnt/.remote-plugins/…`, so script invocations failed until the agent found the real path by hand. SKILL.md and all eight slash commands now resolve the scripts dir with a fallback — prefer `${CLAUDE_PLUGIN_ROOT}`, else locate the mounted copy under `/sessions/*/mnt/.remote-plugins/*/…` — reused via `$ECHO`/`$LINT`/`$SWEEP`/`$SDIR`. On a normal host the primary path always wins (no behaviour change). `echo-health`/`echo-sweep` `allowed-tools` broadened to match the resolved invocation. (The scripts never hardcoded a path — root cause is the harness env var — but the plugin now self-heals.) |
|
||||
|
||||
+15
-14
@@ -20,12 +20,12 @@ The repo root tracks every historical build (`echo-memory-0.6.0.plugin` …
|
||||
`echo-memory-1.5.1.plugin`) plus the `echo-memory.plugin` pointer. 2.0 changes the
|
||||
policy — **breaking for anyone who pulls artifacts straight from the repo**:
|
||||
|
||||
- [ ] Track only `echo-memory.plugin` (the current installable); delete the versioned
|
||||
zips from the tree.
|
||||
- [ ] Publish versioned builds as **Gitea releases** on `git.alwisp.com/jason/echo`
|
||||
instead (one release per tag, artifact attached).
|
||||
- [ ] `.gitignore`: `*.plugin` except the pointer.
|
||||
- [ ] Tag releases going forward (`v2.0.0`, …) so the release page is the version
|
||||
- [x] Track only `echo-memory.plugin` (the current installable); delete the versioned
|
||||
zips from the tree — **done 2.0.0** (15 zips removed; history keeps them).
|
||||
- [x] Publish versioned builds as **Gitea releases** on `git.alwisp.com/jason/echo`
|
||||
instead (one release per tag, artifact attached) — **v2.0.0 onward**.
|
||||
- [x] `.gitignore`: `*.plugin` except the pointer — **done 2.0.0**.
|
||||
- [x] Tag releases going forward (`v2.0.0`, …) so the release page is the version
|
||||
history for artifacts, and the README table stays the narrative history.
|
||||
|
||||
## 2. Complete the skills-format migration (finishes the 1.6 work)
|
||||
@@ -33,10 +33,11 @@ policy — **breaking for anyone who pulls artifacts straight from the repo**:
|
||||
1.6 migrates the eight slash commands to `skills/*/SKILL.md` with both formats
|
||||
coexisting (see `TODO-1.6.md`). 2.0 finishes it:
|
||||
|
||||
- [ ] **Delete the legacy `commands/` directory** — the packaging break that partly
|
||||
motivates the major bump.
|
||||
- [ ] Re-verify desktop + CoWork installs of the skills-only artifact.
|
||||
- [ ] Update SKILL.md / README / command docs that reference `commands/`.
|
||||
- [x] **Delete the legacy `commands/` directory** — **done 2.0.0** (1.6.0's install
|
||||
verification on both surfaces cleared the gate; skills had precedence anyway).
|
||||
- [ ] Re-verify desktop + CoWork installs of the skills-only artifact (operator step,
|
||||
post-2.0.0-install).
|
||||
- [x] Update SKILL.md / README / command docs that reference `commands/` — **done 2.0.0**.
|
||||
|
||||
## 3. Codex packaging (from MAINTENANCE › Canonical source tree)
|
||||
|
||||
@@ -47,9 +48,9 @@ regenerated after 2.0 if needed). What remains for 2.0:
|
||||
- [ ] If Codex support returns: implement it as a **build target** generated from the
|
||||
canonical `echo-memory.plugin.src/` tree (`build.py --target codex` or similar) —
|
||||
never as a second hand-maintained source tree (that's how the drift happened).
|
||||
- [ ] Otherwise: a short "packaging for other agent runtimes" note in the README
|
||||
- [x] Otherwise: a short "packaging for other agent runtimes" note in the README
|
||||
documenting what a Codex/other-runtime port needs (manifest shape, skill entry
|
||||
point, script invocation).
|
||||
point, script invocation) — **done 2.0.0** (README › after Repository layout).
|
||||
|
||||
## 4. Eval refresh — publish current-version metrics (from MAINTENANCE › Docs freshness; ROADMAP-1.0 H4 leftover) — ✅ DONE
|
||||
|
||||
@@ -67,8 +68,8 @@ the README. Re-run it per release and refresh the README table.
|
||||
## 5. Repo hygiene odds & ends
|
||||
|
||||
- [x] Retire `ROADMAP-1.0.md` (fully shipped; in git history) — **done 2026-07-03**.
|
||||
- [ ] Root README "Repository layout" section updated for the post-2.0 tree
|
||||
(no versioned zips, no codex tree, skills-only plugin).
|
||||
- [x] Root README "Repository layout" section updated for the post-2.0 tree
|
||||
(no versioned zips, no codex tree, skills-only plugin) — **done 2.0.0**.
|
||||
- [x] `echo-improvements-prompt.md` (the 1.5.0 field report) moved to `docs/history/`
|
||||
— **done 2026-07-03**.
|
||||
|
||||
|
||||
+30
-14
@@ -14,22 +14,36 @@ the official docs (code.claude.com/docs/en/skills.md, plugins.md):
|
||||
invocation name: `/echo-save` works identically. No deprecation date on `commands/`;
|
||||
this is future-proofing, not a fire.
|
||||
- **Do it for the features, not the notice:**
|
||||
- [ ] `allowed-tools` per skill — pre-authorize the resolved `python3 "$ECHO" …`
|
||||
invocations so `/echo-load`, `/echo-health`, `/echo-recall` stop prompting
|
||||
(1.4.2 hand-broadened two commands; the skill format does this properly).
|
||||
- [ ] `disable-model-invocation: true` on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) so only the operator triggers them; leave the
|
||||
read-side ones model-invocable.
|
||||
- [ ] De-duplicate the CoWork `$ECHO` path-resolution block currently copy-pasted
|
||||
into all eight command bodies — each skill folder can carry a shared snippet.
|
||||
- [ ] `argument-hint` carries over as-is; `$ARGUMENTS` substitution unchanged.
|
||||
- [ ] Verify a migrated build installs cleanly on desktop **and** in a CoWork session
|
||||
before deleting `commands/` (deleting the legacy dir is a 2.0 item —
|
||||
see `ROADMAP-2.0.md`; in 1.6 both may coexist).
|
||||
- [x] `allowed-tools` per skill — **done 1.6.0** (all eight skills pre-authorize
|
||||
the resolved `python3`/`python`/`py -3` invocations + the CoWork `ls` probe).
|
||||
- [x] `disable-model-invocation: true` on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) — **done 1.6.0**; read-side stays
|
||||
model-invocable.
|
||||
- [x] De-duplicate the CoWork `$ECHO` path-resolution block — **resolved 1.6.0 as
|
||||
not-supported**: per the official skills docs, skill directories are
|
||||
self-contained (no cross-skill snippet sharing), so the 2-line block stays
|
||||
per-skill. Docs confirmed same-name skill takes precedence over the legacy
|
||||
command, so coexistence is safe.
|
||||
- [x] `argument-hint` carries over as-is; `$ARGUMENTS` substitution unchanged —
|
||||
**done 1.6.0** (save/recall/reflect).
|
||||
- [x] Verify a migrated build installs cleanly on desktop **and** in a CoWork session
|
||||
before deleting `commands/` — **verified 2026-07-28** on the baked 1.6.0-jason
|
||||
artifact, all four checks pass: (1) all eight skills register and take
|
||||
precedence over the coexisting legacy commands; (2) `allowed-tools` present on
|
||||
load/health/recall; (3) `disable-model-invocation` confirmed on exactly
|
||||
sweep+triage; (4) CoWork `$ECHO` fallback resolves via
|
||||
`/sessions/*/mnt/.remote-plugins/*` and executes. **2.0's `commands/` deletion
|
||||
is unblocked.** *Known caveat, accepted (CLI is not a normal operating surface
|
||||
here): the `allowed-tools` globs match the literal script names
|
||||
(`*echo.py*`/`*vault_lint.py*`) while the bodies invoke via the resolved
|
||||
`"$ECHO"` variable — if the Claude Code CLI matches permissions before
|
||||
variable expansion, those skills could still prompt there. Re-confirm on the
|
||||
CLI if that surface ever matters.*
|
||||
|
||||
## 2. plugin.json completeness (from the old MAINTENANCE checklist)
|
||||
|
||||
- [ ] Add `homepage` / repository URL — now decided: `https://git.alwisp.com/jason/echo`.
|
||||
- [x] Add `homepage` / repository URL — **done 1.6.0**: both `homepage` and
|
||||
`repository` set to `https://git.alwisp.com/jason/echo`.
|
||||
|
||||
## 3. Vault follow-ups (noticed during the 1.5.0 dead-link cleanup)
|
||||
|
||||
@@ -46,7 +60,9 @@ they were unwrapped per instruction but repointing them would restore real graph
|
||||
|
||||
*(add items here as the new plugin gets real use)*
|
||||
|
||||
- [ ] **Lint blind spot: retired dirs masked by the leaf-README route** (found 2026-07-03).
|
||||
- [x] **Lint blind spot: retired dirs masked by the leaf-README route** (found
|
||||
2026-07-03; **fixed 1.6.0** — retired patterns now checked before routes, with
|
||||
the `archive/notes/README.md` regression test).
|
||||
`archive/` and `_agent/outputs/` are retired/unrouted pre-0.6 leftovers that survived
|
||||
in the live vault holding only their seed READMEs — and the permissive `leaf-readme`
|
||||
route (`^(.+/)?README\.md$`) matches first, so `vault_lint` never flagged them.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"""build.py — package the echo-memory plugin source into a .plugin artifact.
|
||||
|
||||
Zips the CONTENTS of echo-memory.plugin.src/ at the archive root (the layout the plugin
|
||||
loader expects: .claude-plugin/plugin.json, commands/, skills/ all at top level), excluding
|
||||
loader expects: .claude-plugin/plugin.json, hooks/, skills/ all at top level), excluding
|
||||
dev cruft. The version is read from the manifest, so the output is named automatically.
|
||||
|
||||
Usage:
|
||||
|
||||
@@ -0,0 +1,352 @@
|
||||
# echo-memory — Improvement Plans (post-1.5.1 review)
|
||||
|
||||
> Status: **planned, not started.** Source: full-code review 2026-07-28.
|
||||
> Nine improvements, each scoped for an independent build session. The tenth item
|
||||
> from that review — the **MCP server** — has its own detailed spec:
|
||||
> `docs/MCP-SERVER-SPEC.md`.
|
||||
>
|
||||
> Sequencing is at the bottom. Nothing here conflicts with `TODO-1.6.md` or
|
||||
> `ROADMAP-2.0.md`; release targets assume 1.6 (skills migration) ships first.
|
||||
|
||||
---
|
||||
|
||||
## 1. Token-budgeted load (`load --brief`) + parallel orientation reads
|
||||
|
||||
**Problem.** The SessionStart hook injects the *full text* of six files into every
|
||||
session. Live measurement: 15.9KB of hook context on a single cold start, and two of
|
||||
the six files grow without bound (operator-preferences `## Observations`,
|
||||
current-context `## Scope History`). Every session pays this tax before any work
|
||||
happens. Separately, `cmd_load` (echo.py) issues its six GETs serially even though
|
||||
`read_many()` exists.
|
||||
|
||||
**Design.**
|
||||
- New `echo.py load --brief` (and make the SessionStart hook use it):
|
||||
- marker → one line (`schema_version`, bootstrapped yes/no).
|
||||
- operator-preferences → `## Fact / Pattern` in full; `## Observations` capped at
|
||||
the last 10 lines with a `(+N older — /echo-load for full)` note.
|
||||
- current-context → `## Scope` + `scope_updated` + sessions-since (i.e. the
|
||||
`scope show` output); omit `## Scope History`.
|
||||
- heartbeat → the pointer line + only the `## Summary`/`## Outcomes` sections of
|
||||
the pointed-at session log (fetch it, extract those headings, cap ~30 lines).
|
||||
- today's daily note → `## Agent Log` lines only (or "absent").
|
||||
- inbox → count + age of oldest item, not the contents (that's all the reconcile
|
||||
needs; `/echo-triage --list` has the details).
|
||||
- Budget guard: after assembly, if the brief output still exceeds
|
||||
`ECHO_LOAD_BUDGET` chars (default ~8000), truncate lowest-priority sections first
|
||||
(observations → agent log → session summary) with explicit `(truncated)` markers.
|
||||
- Full `load` unchanged; `/echo-load` keeps using it.
|
||||
- Fetch all reads via `read_many()` (the 6 targets + the heartbeat-pointed session
|
||||
log + the sessions listing fallback) instead of the serial loop at
|
||||
`echo.py:640`. Cache-put still applies per path.
|
||||
|
||||
**Files.** `echo.py` (cmd_load), `echo_hook_session_start.py`, SKILL.md (document
|
||||
brief-vs-full), `eval/test_features.py`.
|
||||
|
||||
**Tests.** Mock vault with oversized preferences/history files → assert brief output
|
||||
under budget, sections present, truncation markers correct; hook emits brief.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** — highest leverage per line.
|
||||
|
||||
---
|
||||
|
||||
## 2. Local-first recall index
|
||||
|
||||
**Problem.** `echo_recall.update_note()` does GET-whole-index → add → PUT-whole-index
|
||||
against the vault, under the global advisory lock, on every capture and every corpus
|
||||
`put`. The index carries full BM25 postings for the whole corpus, so this round-trip
|
||||
is O(vault) network per write and is the emerging bottleneck + lock hot-spot as the
|
||||
vault grows past a few hundred notes.
|
||||
|
||||
**Design.**
|
||||
- The **live** recall index moves to the local state dir
|
||||
(`~/.echo-memory/recall-index.json`, honoring `ECHO_STATE_DIR`) keyed by endpoint
|
||||
(hash of `BASE`) so multiple vaults don't collide.
|
||||
- `update_note()` becomes a local read-modify-write (file lock via `os.O_EXCL`
|
||||
sidecar or atomic `os.replace`), **no vault round-trip, no advisory lock**.
|
||||
- The vault copy (`_agent/index/recall-index.json`) becomes a **snapshot**, written
|
||||
by `sweep.py` and at session end (piggyback on the heartbeat write / future
|
||||
`session-end` verb). It exists so a *fresh machine* can seed its local index
|
||||
without a full rebuild.
|
||||
- Freshness rule on recall: local index used when present; if the vault snapshot's
|
||||
embedded `built` stamp is newer than the local one (another client swept), pull it.
|
||||
Staleness across clients is tolerable — recall degrades gracefully and improvement
|
||||
#4 (incremental sweep) trues it up cheaply.
|
||||
- Schema bump: recall-index schema 3 adds `built` (ISO timestamp) + `endpoint_hash`.
|
||||
Old schema-2 vault copies are still readable as seeds.
|
||||
- The **entity index stays in the vault** (it is small, and it is the shared routing
|
||||
authority for concurrent clients) — only the BM25 index moves.
|
||||
|
||||
**Files.** `echo_recall.py` (load/save/update_note/rebuild), `echo_queue.py`
|
||||
(state-dir helpers shared), `sweep.py`, `echo.py` (cmd_put upkeep path), routing.json
|
||||
(the vault snapshot path is unchanged), README performance section.
|
||||
|
||||
**Tests.** Capture with vault mock → assert zero recall-index PUTs; sweep → snapshot
|
||||
written; fresh state dir + existing snapshot → seeded without rebuild; two-vault
|
||||
(endpoint) separation.
|
||||
|
||||
**Size/target.** Medium. **Pair with #4 in one index-focused minor** (they touch the
|
||||
same maintenance loop). Fine to land before or after 2.0.
|
||||
|
||||
---
|
||||
|
||||
## 3. Offline durability for the high-level ops (capture path)
|
||||
|
||||
**Problem.** The low-level verbs queue on outage via `safe_request`, but
|
||||
`_append_to_existing()` and `ensure_daily_log()` (echo_ops.py) call `echo.request`
|
||||
directly — so during an outage a raw `append` survives, while a `capture` **update**
|
||||
and its Agent Log line silently vanish. Inverted from the documented contract
|
||||
("capture is the default write").
|
||||
|
||||
**Design.** Queue *semantic* records, not byte-level requests, for the high-level
|
||||
path:
|
||||
- New outbox record type `{"op": "capture", "args": {...}}` alongside the existing
|
||||
raw records. On an unreachable vault, `capture` short-circuits: enqueue the full
|
||||
argument set (title/kind/body/tags/aliases/sources/date/domain/merge_into) and
|
||||
report `queued (offline): capture …`.
|
||||
- `flush()` replays `op:capture` records by calling `echo_ops.capture(...)` against
|
||||
the *current* index — so routing, the duplicate gate, aliasing, and linking all
|
||||
re-run with fresh state instead of replaying stale byte writes. A gate stop on
|
||||
replay (exit 76) keeps the record queued with a `needs_attention` flag and is
|
||||
surfaced by `load`/`flush` output ("1 queued capture needs --merge-into/--force").
|
||||
- Rationale for semantic-over-byte: a byte replay of `_append_to_existing`'s PATCH
|
||||
could target a heading that moved, and the create-vs-update decision made offline
|
||||
may be wrong by replay time. Re-running the op is the idempotent, correct unit
|
||||
(the dated-bullet idempotency key already prevents double-append).
|
||||
- `ensure_daily_log` stays best-effort but routes its POST/PATCH through
|
||||
`safe_request` so a standalone agent-log line survives an outage too.
|
||||
- reflect/triage `--apply` inherit this for free (they call capture).
|
||||
|
||||
**Files.** `echo_ops.py`, `echo_queue.py` (record schema v2 + replay dispatch),
|
||||
`eval/test_offline_queue.py`.
|
||||
|
||||
**Tests.** Fault-injected mock: capture-update offline → queued; flush replays via
|
||||
capture; gate-on-replay keeps record + flags; agent-log line queued; no duplicate
|
||||
bullets on double flush.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** — it's a correctness gap.
|
||||
|
||||
---
|
||||
|
||||
## 4. Incremental sweep via content hashes
|
||||
|
||||
**Problem.** Human edits in Obsidian and writes by other clients never touch the
|
||||
entity/recall indexes; drift is only corrected by a full sweep someone must remember
|
||||
to run. Full sweep re-reads the whole vault (fast now, but O(vault)).
|
||||
|
||||
**Design.**
|
||||
- Entity index entries and the recall-index snapshot gain a per-note
|
||||
`h` — SHA-1 of the note body (entity index schema 2; tolerated-absent for old
|
||||
entries).
|
||||
- `sweep.py --fast`: walk the listing (cheap), `read_many` only paths that are
|
||||
(a) new, (b) missing a stored hash, or (c) whose hash mismatches after fetch — to
|
||||
avoid fetching everything just to hash it, fast mode fetches only paths whose
|
||||
**listing is new/gone** plus a rotating shard (e.g. `hash(path) % 7 == weekday`)
|
||||
of existing notes, so the whole vault is verified over a week of fast sweeps while
|
||||
each run stays tiny. `--fast --all-shards` forces full verification.
|
||||
- Deletions: listing walk catches removed files → drop index + recall entries.
|
||||
- Auto-run: `load` (brief or full) runs `sweep --fast` opportunistically when the
|
||||
last fast-sweep stamp (state dir) is older than `ECHO_FAST_SWEEP_DAYS` (default 7),
|
||||
in the background-tolerant sense: bounded by the same read_many concurrency, and
|
||||
skipped entirely when offline.
|
||||
- `/echo-health` reports last fast/full sweep ages.
|
||||
|
||||
**Files.** `sweep.py`, `echo_index.py` (schema 2 + hash field), `echo_recall.py`
|
||||
(snapshot hash reuse), `echo.py` (load hook-in), `vault_lint.py` (report), migrate
|
||||
note (index schema is machine-owned; no vault migration needed).
|
||||
|
||||
**Tests.** Mock: edit a note out-of-band → fast sweep in that shard re-indexes it;
|
||||
delete → entry dropped; stamp gating; full sweep unchanged.
|
||||
|
||||
**Size/target.** Medium. **Ship with #2** (same schema-bump train).
|
||||
|
||||
---
|
||||
|
||||
## 5. Memory lifecycle — enforce forgetting
|
||||
|
||||
**Problem.** The vault only accretes. Working memory is "time-boxed" by convention
|
||||
only; Observations trimming is manual prose; stale-active detection only reports.
|
||||
Recall precision degrades as dead weight accumulates.
|
||||
|
||||
**Design.**
|
||||
- `capture --kind working` stamps `expires: <today + ECHO_WORKING_TTL_DAYS>`
|
||||
(default 14; `--ttl <days>` overrides). Template updated to carry the field.
|
||||
- `vault_lint.py` new checks: `expired-working` (past `expires`),
|
||||
`stale-observations` (Observations > 30 bullets) — advisory, like the rest.
|
||||
- `sweep.py --decay` (dry-run by default, `--apply` to write; **always
|
||||
preview-first**, honoring the operating contract):
|
||||
- expired working notes → `status: archived` + move body under a dated
|
||||
`## Archived` marker (never delete);
|
||||
- `projects/active/` untouched > 30 days → *propose* `on-hold` (folder move +
|
||||
status, the usual paired change) — apply only the ones confirmed;
|
||||
- Observations > 30 → trim oldest into a dated `_agent/memory/episodic/`
|
||||
overflow note (`observations-archive-YYYY.md` append), keeping the last 30
|
||||
in place. Nothing is lost, the hot file stays small (compounds with #1).
|
||||
- `/echo-health` output nudges: "run `sweep --decay` to review N aging items".
|
||||
|
||||
**Files.** `echo_ops.py` (`--ttl`/expires stamp), `echo_index.py`
|
||||
(KIND_REQUIRED_FM unchanged; working gains optional expires), `vault_lint.py`,
|
||||
`sweep.py`, scaffold working-memory template, SKILL.md (decay section),
|
||||
routing-map note.
|
||||
|
||||
**Tests.** Mock: expired note proposed+archived on apply; observations trim keeps
|
||||
last 30 and appends overflow; stale-active proposal list; dry-run writes nothing.
|
||||
|
||||
**Size/target.** Medium. Feature minor (1.7.x).
|
||||
|
||||
---
|
||||
|
||||
## 6. Contradiction / supersession handling
|
||||
|
||||
**Problem.** A new fact that contradicts a stored one lands beside it; both surface
|
||||
in recall with no signal about which is current. Biggest remaining trust gap in the
|
||||
write path.
|
||||
|
||||
**Design.**
|
||||
- **Write side:** `capture --supersedes "<slug-or-quoted-line>"`:
|
||||
- if it names an entity slug → the old entity gets `status: superseded` +
|
||||
a `superseded_by:` frontmatter path; new note carries `supersedes:` back-link
|
||||
in `## Related`. Recall's STATUS_FACTOR gains `superseded: 0.5`.
|
||||
- if it quotes a Fact/Pattern or Observations line in operator-preferences →
|
||||
the line is struck (`~~old~~ (superseded YYYY-MM-DD)`) and the new line
|
||||
appended, one PATCH each, in the same invocation.
|
||||
- **Detect side (deterministic assist, model decides):** `echo_reflect.classify`
|
||||
gains a `conflict scan` — for each proposal, run its distinctive tokens
|
||||
(`_sig_tokens`) against the Fact/Pattern + recent Observations lines; overlapping
|
||||
lines are attached as `_conflicts: [...]` and the preview row renders
|
||||
`update-or-conflict?` with the matched line(s). The model (which has the
|
||||
conversation) decides supersede-vs-append; the script only guarantees the
|
||||
question gets asked before the write.
|
||||
- Lint: `superseded_by` pointing at a missing note → violation.
|
||||
|
||||
**Files.** `echo_ops.py`, `echo_reflect.py`, `echo_recall.py` (STATUS_FACTOR),
|
||||
`echo.py` (flag plumbing), `vault_lint.py`, SKILL.md + operating-contract
|
||||
(supersession is additive: strike, never delete).
|
||||
|
||||
**Tests.** Mock: supersede entity → status/backlinks both sides; supersede line →
|
||||
strike + append idempotent; reflect preview flags a planted conflict; recall ranks
|
||||
superseded below active twin.
|
||||
|
||||
**Size/target.** Medium. Feature minor (1.7.x/1.8) — after #5 (shares the
|
||||
status-vocabulary touchpoints).
|
||||
|
||||
---
|
||||
|
||||
## 7. `session-end` bundle verb
|
||||
|
||||
**Problem.** Ending a session correctly is 4+ invocations (session-log PUT, Agent
|
||||
Log append, heartbeat PUT, optional scope set, optional reflect apply). Cost per
|
||||
session, and a partial failure leaves broken orientation state (log written,
|
||||
heartbeat stale).
|
||||
|
||||
**Design.**
|
||||
- `echo.py session-end <bundle.json> [--apply]`, bundle:
|
||||
```json
|
||||
{
|
||||
"slug": "echo-mcp-spec",
|
||||
"log_body": "<full session-log markdown, frontmatter included>",
|
||||
"agent_log_line": "- 2026-07-28: wrote MCP spec [[...]]",
|
||||
"scope": "optional new scope text",
|
||||
"reflect": [ { ...PROPOSAL_SCHEMA... } ]
|
||||
}
|
||||
```
|
||||
- Dry-run previews the whole plan (paths + reflect preview). `--apply`, in order,
|
||||
under **one** lock acquisition: PUT `_agent/sessions/<date>-<HHMM>-<slug>.md`
|
||||
(HHMM from `ECHO_NOW` env or wall clock, validated against the canonical regex) →
|
||||
agent-log line (ensure_daily_log) → reflect proposals via capture → scope set if
|
||||
given → heartbeat PUT **last** (it's the commit marker: a crash before it leaves
|
||||
the previous pointer intact, never a dangling one).
|
||||
- Reports per-step ok/queued/gated; offline → every step queues (rides on #3).
|
||||
- Stop hook reason text updated to name the one command; `/echo-reflect` docs point
|
||||
session logging at it. **Prerequisite for the MCP `echo_log_session` tool** —
|
||||
build this verb first so CLI and MCP share one implementation.
|
||||
|
||||
**Files.** new `echo_session.py`, `echo.py` (subcommand), `echo_hook_stop.py`
|
||||
(REASON text), SKILL.md (Session Logging section), commands/echo-reflect.md.
|
||||
|
||||
**Tests.** Mock: full bundle lands all five artifacts; heartbeat-last ordering
|
||||
(inject failure mid-way → heartbeat untouched); dry-run writes nothing; offline
|
||||
queues all steps.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** (and before the MCP build).
|
||||
|
||||
---
|
||||
|
||||
## 8. Recall robustness — stemming + alias query expansion
|
||||
|
||||
**Problem.** BM25 is purely lexical: "deploy"/"deployment"/"deployed" are distinct
|
||||
terms; a paraphrased query misses. The 8-query gold set can't see this class of miss.
|
||||
|
||||
**Design.**
|
||||
- **Stemming:** a light suffix-stripper (Porter-lite: s/es/ed/ing/ly/tion/ment/ness
|
||||
families, ~40 lines, pure stdlib) applied in `tokenize()` at **both** index and
|
||||
query time. Index schema bump (stemmed postings) → auto-rebuild on next
|
||||
recall/sweep, exactly like the 1→2 transition.
|
||||
- **Alias query expansion:** at recall time, run the query against
|
||||
`fuzzy_candidates()` (top 2, score ≥ 0.5); fold the matched entities' title +
|
||||
alias tokens into the BM25 query as **down-weighted expansion terms** (×0.5 idf
|
||||
contribution) so "the ECHO plugin" also scores `echo-memory`-alias vocabulary.
|
||||
Exact-match seeding stays as-is; expansion never *creates* hits ranked above
|
||||
genuine lexical matches without corpus support.
|
||||
- **Eval:** grow the gold set with ≥6 paraphrase queries (morphological variants +
|
||||
alias phrasings); publish before/after recall@5/MRR in the README table.
|
||||
|
||||
**Files.** `echo_recall.py` (tokenize, INDEX_SCHEMA, score, recall), new
|
||||
`echo_stem.py` (pure function, unit-testable), `eval/run_eval.py` + gold set.
|
||||
|
||||
**Tests.** Stemmer unit table (word → stem, incl. non-cases); paraphrase gold
|
||||
queries pass; schema-2 index discarded and rebuilt cleanly.
|
||||
|
||||
**Size/target.** Small-medium. Ride the **same index-schema train as #2/#4**.
|
||||
|
||||
---
|
||||
|
||||
## 9. Resolve-miss telemetry → alias suggestions
|
||||
|
||||
**Problem.** Every resolve that misses but ends in a confirmed candidate
|
||||
(`--merge-into`, or the operator picking a candidate) is a free mention→entity
|
||||
training pair, currently discarded.
|
||||
|
||||
**Design.**
|
||||
- `echo_index.resolve()` miss + `fuzzy_candidates` non-empty → append one NDJSON
|
||||
record to the local state dir (`~/.echo-memory/resolve-misses.ndjson`):
|
||||
`{mention, candidates:[slugs], ts?}` (no wall clock needed — session date via
|
||||
ECHO_TODAY). `capture --merge-into <slug>` appends the *confirmation* record
|
||||
`{mention, resolved: slug}` (this already learns the alias immediately — that
|
||||
path stays; the log covers the cases that never reach --merge-into).
|
||||
- `sweep.py` (and `/echo-health`) read the log: any `{mention → slug}` pair
|
||||
confirmed ≥2 times, or any mention that repeatedly fuzzy-matched a single
|
||||
candidate ≥3 times without confirmation, is **proposed** as an alias
|
||||
(`sweep --apply` writes it into the note's `aliases:` frontmatter + index, the
|
||||
existing fold-back path). Log entries consumed on apply; file capped (rotate at
|
||||
~500 lines).
|
||||
- Privacy note: mentions can contain operator phrasing — the log lives in the
|
||||
state dir (never the vault), same trust level as the offline queue.
|
||||
|
||||
**Files.** `echo_index.py` (log hooks), `echo_ops.py` (confirmation hook),
|
||||
`sweep.py` (suggest/apply), `vault_lint.py` or sweep report (surface count),
|
||||
SKILL.md one-liner.
|
||||
|
||||
**Tests.** Miss → record written; confirm ×2 → sweep proposes; apply folds alias +
|
||||
consumes log; cap/rotation.
|
||||
|
||||
**Size/target.** Small. Feature minor, pairs naturally with #8.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing (unified with TODO-1.6 and ROADMAP-2.0)
|
||||
|
||||
TODO-1.6 folds in as the opening train — the skills migration restructures the
|
||||
files everything below touches, so it goes first. ROADMAP-2.0 stays a **pure
|
||||
packaging major** (its own stated thesis) and is *not* folded into any feature
|
||||
train; it slots between the quick wins and the MCP server so the MCP work lands
|
||||
on the final packaging baseline (single command format, Gitea releases, one
|
||||
manifest-verification pass).
|
||||
|
||||
| Release | Items | Rationale |
|
||||
|---|---|---|
|
||||
| **1.6.0** | TODO-1.6: skills-format migration (+ per-skill `allowed-tools`, `disable-model-invocation` on write-heavy entries, `$ECHO` block dedupe) · lint blind-spot fix (retired-dir/leaf-README) · plugin.json `homepage` | Restructures the command/SKILL files later work edits; allowed-tools removes prompt friction for every verb added below. Vault link repointing (TODO-1.6 §3) is vault data, not plugin code — any session. |
|
||||
| **1.6.x quick wins** | #1 brief load · #3 offline capture · #7 session-end (+ return-not-print refactor, MCP spec Phase 0, if it fits) | Small, correctness/efficiency, no schema changes. #7 unblocks the MCP tool. Users get these without a reinstall — don't gate them behind 2.0. |
|
||||
| **2.0** | ROADMAP-2.0 unchanged: delete legacy `commands/` · artifact policy (Gitea releases, gitignore, tags) · Codex note · README layout | Packaging only, no feature riders — a bad feature must never force reverting a packaging break. |
|
||||
| **2.1** | MCP server (containerized, `echomcp.alwisp.com`) — `docs/MCP-SERVER-SPEC.md` | Needs #7 + Phase 0. Lands after 2.0 so its docs target one command format and the Gitea release channel exists. Deployed via CI + PORT, not the plugin manifest. |
|
||||
| **2.2 index train** | #2 local-first recall (CLI-path half) · #4 incremental sweep · #8 stemming+expansion | One schema-bump train; all touch the same index/sweep loop. The container already keeps its index warm in memory (spec §7), so #2 here covers the CLI fallback path; #4's fast sweep also becomes the container's background job. |
|
||||
| **2.3 memory train** | #5 lifecycle/decay · #6 supersession · #9 resolve-miss learning | Capability tier; #6 after #5 (status vocabulary). |
|
||||
@@ -0,0 +1,391 @@
|
||||
# echo-mcp — MCP Server Build Spec (containerized)
|
||||
|
||||
> Status: **spec for a future build session** (written 2026-07-28 against v1.5.1;
|
||||
> revised same day: **remote container architecture**, operator decision — heavy
|
||||
> lifting belongs in a deployed container, not on any one machine).
|
||||
> Companion plan for the other nine review items: `docs/IMPROVEMENT-PLANS.md`.
|
||||
>
|
||||
> **Prerequisites before starting this build:**
|
||||
> 1. The `session-end` verb (IMPROVEMENT-PLANS #7) — the MCP tool wraps it.
|
||||
> 2. Phase 0 below (the return-not-print refactor) — lands in the plugin tree
|
||||
> first and is independently shippable.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why an MCP server
|
||||
|
||||
Every ECHO operation today rides through Bash: resolve `$ECHO` (with the CoWork
|
||||
path-fallback snippet copy-pasted everywhere), quote a Python invocation, run
|
||||
it, re-parse stdout. Costs:
|
||||
|
||||
- **Tokens** — each memory op carries bash scaffolding + output re-parsing; a
|
||||
capture is ~3× the tokens of a typed tool call.
|
||||
- **Fragility** — quoting hazards, the `${CLAUDE_PLUGIN_ROOT}` sandbox mismatch
|
||||
(the whole 1.4.2 release), Windows `python3`-vs-`python` dispatch.
|
||||
- **Reach** — surfaces without a Python-capable shell can't use ECHO at all.
|
||||
- **Efficiency** — every CLI invocation is a cold Python start that re-reads the
|
||||
entity index from the vault over HTTPS.
|
||||
|
||||
What the server does **not** replace: SKILL.md remains the procedure authority
|
||||
(when to load, reconcile, search-first, third-person, etc.). The server replaces
|
||||
the *mechanics* — tools are the verbs, the skill is the discipline.
|
||||
|
||||
## 2. Architecture: remote container (decided)
|
||||
|
||||
A standalone Docker container on the Unraid box (ALPHA), published behind
|
||||
Cloudflare + NPM as **`echomcp.alwisp.com`**, speaking **MCP streamable HTTP
|
||||
(stateless JSON)** with bearer-token auth. Chosen over the earlier local-stdio
|
||||
draft for four reasons:
|
||||
|
||||
1. **The backend is already remote.** Clients round-trip to
|
||||
`echoapi.alwisp.com` today; a server co-located with the vault host turns
|
||||
every vault op into a LAN hop and collapses N client→vault calls into one
|
||||
client→server call.
|
||||
2. **Any device, no per-machine anything.** Desktop, CoWork sandbox, claude.ai
|
||||
(custom connector), mobile — same URL + token. The `${CLAUDE_PLUGIN_ROOT}`
|
||||
CoWork registration risk from the stdio draft disappears entirely.
|
||||
3. **Credentials consolidate server-side.** The vault key lives in the
|
||||
container env (PORT secret store); the image is credential-free; clients
|
||||
hold only an MCP bearer token. The per-user baked-key builds (1.4.x) become
|
||||
unnecessary for any surface that has MCP. Rotation = redeploy.
|
||||
4. **Room to grow.** The plugin's pure-stdlib constraint exists because its
|
||||
scripts run in arbitrary sandboxes. The container is ours: real SDK,
|
||||
SQLite/in-memory indexes that stay warm, background jobs, embeddings later —
|
||||
all without touching the plugin.
|
||||
|
||||
**What stays client-side (unchanged and shipped):** the skill + CLI path — it
|
||||
is the fallback when the server or network is down, the offline queue + read
|
||||
cache keep their per-machine role there, and the SessionStart/Stop **hooks stay
|
||||
CLI-based** (hooks run shell commands regardless of MCP).
|
||||
|
||||
**Cost acknowledged:** a second service to run and an exposed auth surface.
|
||||
Mitigations are existing infra: CF + NPM cert, long random bearer token, Kuma
|
||||
monitor, PORT deploy/rollback. Server down ⇒ exactly today's behavior.
|
||||
|
||||
## 3. Decisions (made — don't relitigate in the build session)
|
||||
|
||||
| Decision | Choice | Why |
|
||||
|---|---|---|
|
||||
| Language | **Python** | Imports the existing `echo_*.py` modules in-process — the entire ops layer is reused, not wrapped. |
|
||||
| SDK | **Official `mcp` Python SDK (FastMCP)** | The stdlib-only rule doesn't bind inside our own image. Hand-rolling the protocol (stdio draft) is no longer justified. |
|
||||
| Transport | **Streamable HTTP, stateless JSON** | Remote, multi-client, simplest to scale; SSE/sessions explicitly avoided. |
|
||||
| Auth | **Static bearer token** (`ECHO_MCP_TOKEN`), constant-time compare, 401 otherwise | Single-operator service; OAuth is overkill. Token from the PORT secret store. |
|
||||
| Vault credentials | Container env: `ECHO_BASE`, `ECHO_KEY`, `ECHO_OWNER` (SECRET: refs in the PORT template) | `echo_config.py` already resolves env-first — zero code change. Image ships no secrets. |
|
||||
| Server name / repo home | `echo-mcp`; lives in this repo under `mcp-server/` (own Dockerfile), deployed from `git.alwisp.com/jason/echo` CI | One canonical tree — the server vendors `skills/echo-memory/scripts/` into the image at build time; no second hand-maintained copy (the Codex-drift lesson). |
|
||||
| Tool prefix | `echo_` | Namespace safety next to other servers. |
|
||||
| Result shape | `structuredContent` (the `echo_output.envelope` dict) + a 1–3 line human text block | Envelope shape already exists. |
|
||||
| Statefulness | Stateless protocol; warm in-process caches | See §7. |
|
||||
| Version target | **2.1** (after the 1.6.x train and the 2.0 packaging major) | Needs #7 + Phase 0; docs then target the single post-2.0 command format. |
|
||||
| Local stdio variant | **Dropped for v1** (documented, not built) | The CLI/skill fallback covers the no-server case; two transports = two test matrices for little gain. Phase 0 keeps the door open — the `*_op` core is transport-agnostic. |
|
||||
| Result budgets | Every read tool takes an explicit size budget (`budget_chars` / `max_chars`) and truncates with a marker + "call X for more" | The single biggest token lever: one right-sized answer instead of follow-up fetches. |
|
||||
| Tool profiles | `ECHO_MCP_TOOLS=core\|full` env: `core` exposes only load/recall/capture/triage_inbox/log_session/health | Tool schemas cost context on every surface that lists them; the claude.ai connector doesn't need the escape hatches. Desktop registers `full`. |
|
||||
|
||||
## 4. Phase 0 — the return-not-print refactor (prerequisite, lands in the plugin)
|
||||
|
||||
Unchanged from the stdio draft, and still first: the ops layer prints results
|
||||
(even `--json` mode prints from inside `echo_ops.capture`) and returns exit
|
||||
codes; the server needs return values.
|
||||
|
||||
1. Each high-level op gets a **core function returning an envelope dict**,
|
||||
raising `echo.EchoError` on failure; CLI entry points become thin printing
|
||||
wrappers:
|
||||
|
||||
| Module | New core fn | CLI wrapper keeps |
|
||||
|---|---|---|
|
||||
| `echo_ops` | `capture_op`, `resolve_op`, `link_op` | `capture/resolve/link` |
|
||||
| `echo_recall` | `recall_op(query, limit)` | `recall` |
|
||||
| `echo_triage` | `list_op()` · `route_op(proposals, apply)` | `list_inbox/apply` |
|
||||
| `echo_reflect` | `apply_op(proposals, apply)` | `apply` |
|
||||
| `echo.py` | `load_op(brief)` · `scope_show_op/scope_set_op` · `doctor_op` | `cmd_*` |
|
||||
| `echo_session` (new, #7) | `session_end_op(bundle, apply)` | subcommand |
|
||||
|
||||
2. Special results are **data, not exit codes**: duplicate gate ⇒
|
||||
`{ok: false, action: "duplicate-gate", candidates}` (CLI maps to exit 76);
|
||||
offline queueing ⇒ `{ok: true, queued: true}`.
|
||||
3. Helper chatter (`ok: PUT …`) moves behind a `notify(msg)` callback
|
||||
defaulting to stderr; delete the `redirect_stdout` hack in `capture`.
|
||||
4. Tests: existing suites pass unchanged; new unit tests hit `*_op` directly
|
||||
against the mock and assert envelopes.
|
||||
|
||||
Shippable on its own as a 1.6.x refactor with zero behavior change.
|
||||
|
||||
## 5. Server application (`mcp-server/app.py`)
|
||||
|
||||
- **FastMCP** app, streamable HTTP, stateless. One `@mcp.tool` per tool in §6,
|
||||
each a thin adapter: validate → call the `*_op` core → wrap.
|
||||
- **Auth middleware**: require `Authorization: Bearer <ECHO_MCP_TOKEN>` on every
|
||||
request (constant-time compare); 401 with no detail otherwise. Additionally
|
||||
bind the app to the container interface only; exposure policy lives at
|
||||
NPM/Cloudflare.
|
||||
- **Result wrapper**:
|
||||
|
||||
```python
|
||||
def tool_result(env: dict, summary: str) -> ...:
|
||||
# content: [{type: "text", text: summary}] (1–3 lines, human)
|
||||
# structuredContent: env (echo_output envelope)
|
||||
# isError: not env.get("ok", True) — except the deliberate non-errors below
|
||||
```
|
||||
|
||||
- **Error map** (tool-level results, never protocol errors):
|
||||
|
||||
| Condition | Result |
|
||||
|---|---|
|
||||
| Vault unreachable on a **read** | `isError: true` — "vault unreachable (Obsidian/REST plugin likely down); proceed without memory, writes will queue server-side". |
|
||||
| Vault unreachable on a **write** | **Not an error**: `{ok: true, queued: true}` — the server-side outbox (§7) replays when the vault returns. |
|
||||
| 404 | `isError: true`, `{code: "not-found", path}`. |
|
||||
| Duplicate gate | **Not an error**: `{ok: false, action: "duplicate-gate", candidates}` + text naming the two resolutions (`merge_into` / `force`) — the model must decide, not blind-retry. |
|
||||
| Lock held | `{ok: false, action: "lock-held", holder}`. |
|
||||
| Misconfigured deployment (no `ECHO_BASE`/`ECHO_KEY`) | `isError: true` — "server deployment is missing vault credentials — operator: check the PORT template env". Startup also fails loudly (see §8 healthcheck). |
|
||||
| Anything else | `isError: true`, first line only; traceback to the container log. |
|
||||
|
||||
- **Validation**: FastMCP/pydantic handles types/enums/required; add
|
||||
`model_config = ConfigDict(extra="forbid")` so unknown params (model typos)
|
||||
fail with a field-naming message.
|
||||
|
||||
## 6. Tool surface (13 tools)
|
||||
|
||||
Same surface as the stdio draft **minus `echo_configure`** — with a remote
|
||||
server there is no per-machine config to import; credentials are
|
||||
deployment-side. Descriptions are written for the model: what it does, when to
|
||||
use it, what it returns, ≤3 sentences.
|
||||
|
||||
### 6.1 Orientation & read
|
||||
|
||||
- **`echo_load`** — cold-start orientation. `brief: bool = true`
|
||||
(IMPROVEMENT-PLANS #1 digest; `false` = full six-file dump). Returns
|
||||
`{sections: {marker, preferences, scope, last_session, today, inbox_count,
|
||||
inbox_oldest_days}, queued_flushed, offline}`. Also flushes the server-side
|
||||
outbox. `readOnlyHint: false` (flush), `openWorldHint: true`.
|
||||
- **`echo_recall`** — `query: str`, `limit: int = 6 (1–20)`,
|
||||
`include_linked: bool = true`, `budget_chars: int = 4000 (500–20000)`. Returns
|
||||
the `recall --json` shape (`primary` / `linked` with
|
||||
path/score/type/updated/status/excerpt) **packed to the budget**: the server
|
||||
allocates excerpt space by score, so the model gets the most relevant content
|
||||
in one right-sized answer instead of follow-up fetches. Description: truncated
|
||||
hits are marked — call `echo_get_note` for full content.
|
||||
`readOnlyHint: true`, `idempotentHint: true`.
|
||||
- **`echo_resolve`** — `mention: str` → match `{slug, path, kind, title,
|
||||
aliases}` or `{match: false, suggest_slug, candidates}`. "Call before creating
|
||||
any note by hand; `echo_capture` does this automatically." `readOnlyHint: true`.
|
||||
- **`echo_get_note`** — `path: str` (vault-relative; reject `..`/leading `/`;
|
||||
warn-not-block on paths outside `routing.json`); `section: str = null`
|
||||
(a heading name — return only that section, e.g. `Status`);
|
||||
`max_chars: int = 8000`. Returns `{path, content, frontmatter, truncated?}`.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_get_scope`** — `{scope, scope_updated, sessions_since}`; description
|
||||
tells the model to confirm scope with the operator when `sessions_since ≥ 3`.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_set_scope`** — `scope: str`; atomic switch (history + replace +
|
||||
stamp). `idempotentHint: true`.
|
||||
- **`echo_health`** — `deep: bool = false`. Shallow = doctor checks (vault
|
||||
reachability, auth, marker/schema, outbox depth). Deep = full `vault_lint`
|
||||
→ `{violations: [{check, path, detail}]}`. `readOnlyHint: true`.
|
||||
|
||||
### 6.2 Write
|
||||
|
||||
- **`echo_capture`** — the default write (route + frontmatter + index +
|
||||
auto-link + agent-log in one call). Params: `title` (req); `kind:
|
||||
enum[person, company, concept, reference, meeting, project, area, semantic,
|
||||
episodic, working, skill, decision]` (omit ⇒ inbox line); `body: str = ""`
|
||||
(markdown, inline); `tags/aliases/sources: str[]`; `status: str`;
|
||||
`date: YYYY-MM-DD` (meeting/decision); `domain: enum[business, personal,
|
||||
learning, systems] = business` (area); `merge_into: str` (slug);
|
||||
`force: bool = false`; `dry_run: bool = false`. Returns the capture envelope
|
||||
(`action: created|updated|inbox|duplicate-gate`, `path`, `links_added`,
|
||||
`near_duplicates?`, `candidates?`). Description spells out the gate contract
|
||||
(never blind-retry; `merge_into` or confirmed `force`). `destructiveHint:
|
||||
false`, `idempotentHint: true`.
|
||||
- **`echo_link`** — `a`, `b` (paths **or resolvable names** — server resolves
|
||||
via the index). Returns `{a, b, a_changed, b_changed}`. `idempotentHint: true`.
|
||||
- **`echo_append_note`** — `path`, `line`; whole-line idempotent append.
|
||||
- **`echo_patch_note`** — `path`, `operation: enum[append, prepend, replace]`,
|
||||
`target_type: enum[heading, frontmatter, block]`, `target`, `content`.
|
||||
Description carries the hard-won rules: heading targets are the full
|
||||
`::`-delimited path from the H1; on a 400 invalid-target the server fetches
|
||||
the document map and returns the actual heading list in the error — the worst
|
||||
silent-loss failure becomes self-correcting. `destructiveHint: true`.
|
||||
(Deliberately **no `echo_put_note` / `echo_delete_note`** in v1 — whole-file
|
||||
overwrite and deletion stay behind the CLI + operator explicitness; session
|
||||
logs go through `echo_log_session`.)
|
||||
- **`echo_triage_inbox`** — `proposals: object[] = []` (PROPOSAL_SCHEMA +
|
||||
optional `line`), `apply: bool = false`. Empty ⇒ structured listing; with
|
||||
proposals ⇒ preview, then route + processing-log audit on `apply: true`.
|
||||
"List → propose → preview → apply only after the operator confirms."
|
||||
- **`echo_reflect`** — `proposals: object[]` (req), `apply: bool = false`. Same
|
||||
preview/apply contract as the CLI; description restates: never apply without
|
||||
the operator's go-ahead, never invent memories.
|
||||
- **`echo_log_session`** — the session-end bundle (wraps `session_end_op`, #7):
|
||||
`slug`, `log_body`, `agent_log_line`, `scope?: str`, `reflect?: object[]`,
|
||||
`apply: bool = false`. Per-step results; heartbeat written last as the commit
|
||||
marker.
|
||||
|
||||
### 6.3 v1.1 tools (specced now, built after v1 ships)
|
||||
|
||||
- **`echo_rollup_data`** — `period: enum[week, month]`, `date: YYYY-MM-DD`.
|
||||
Assembles the rollup *digest data* in one call: open threads across
|
||||
`projects/active/` (each note's `## Status` + `updated:`), inbox items aging
|
||||
past 7 days, and the period's `## Scope History` entries. The model writes
|
||||
the prose; the server does the N fetches. Same division of labor as reflect.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_note_history`** — `path: str` → `{versions: [{id, ts, bytes,
|
||||
summary_line}]}` from the server's shadow write history (§7). `readOnlyHint:
|
||||
true`.
|
||||
- **`echo_restore_note`** — `path: str`, `version_id: str`,
|
||||
`confirm: bool = false` (preview diff unless confirmed). The undo for a bad
|
||||
`replace` PATCH or merge PUT. Description: operator confirmation required
|
||||
before calling with `confirm: true`. `destructiveHint: true`.
|
||||
|
||||
Deliberately absent from v1 (documented in the server README): sweep,
|
||||
bootstrap, migrate, lock/unlock (vault-wide maintenance stays operator-initiated
|
||||
via slash commands — routine sweeps run as background jobs instead), raw
|
||||
search/ls/map (recall/resolve/get_note cover reads; add only if transcripts
|
||||
show the need), and reflect *extraction* (only the model has the conversation —
|
||||
that division of labor is permanent, not a v1 scope cut).
|
||||
|
||||
## 7. Server internals — where the container earns its keep
|
||||
|
||||
### 7.1 v1
|
||||
|
||||
- **Warm entity index**: loaded once, invalidated on any index-writing tool and
|
||||
on a short TTL (`ECHO_MCP_INDEX_TTL`, default 60s) to pick up other clients'
|
||||
writes. Ends the per-invocation index re-read entirely.
|
||||
- **Recall index in memory (+ SQLite file at `/data`)**: the BM25 index lives
|
||||
in process memory, persisted to the container volume — recall answers in
|
||||
milliseconds with **zero** vault round-trips. This *supersedes the
|
||||
server-side half of IMPROVEMENT-PLANS #2*; #2's local-state-dir design still
|
||||
applies to the CLI fallback path.
|
||||
- **Note read cache**: short-TTL (30–60s) body cache for `get_note` and
|
||||
recall's neighbourhood expansion — repeated same-session reads of the same
|
||||
files stop hitting the vault at all.
|
||||
- **Per-path write serialization**: an internal per-path mutex serializes
|
||||
conflicting MCP-path writes — since all server-mediated writes flow through
|
||||
one process, the advisory-lock race window disappears for them. The server
|
||||
still takes the vault advisory lock around index updates to coordinate with
|
||||
CLI clients; idempotent appends stay as the last line of defense.
|
||||
- **Server-side outbox**: the existing `echo_queue` pointed at `/data`
|
||||
(`ECHO_STATE_DIR=/data`) — writes queue when the vault is down and flush on
|
||||
recovery/load. One queue at the always-on host instead of per-laptop.
|
||||
- **Concurrency**: FastMCP serves requests concurrently; guard the in-process
|
||||
caches with a plain `threading.Lock`; write integrity per the bullet above.
|
||||
- **Logging**: one line per call to stdout (container log): tool, ms,
|
||||
ok/queued/gated/error. Never log bodies or keys.
|
||||
|
||||
### 7.2 v1.1 container dividends (backlog, in priority order)
|
||||
|
||||
- **Nightly vault backup** — the vault currently has **no disaster-recovery
|
||||
story**; if the Obsidian host dies, memory is gone. A timer job walks the
|
||||
vault via `read_many`, writes a dated zip to `/data/backups/` (rotate 30),
|
||||
and reports the last-backup age in `/health`. Zero tokens; arguably the
|
||||
highest-value item in this spec.
|
||||
- **Shadow write history + undo** — before any PUT / PATCH-replace the server
|
||||
passes through, snapshot the prior body to `/data/history/` (content-
|
||||
addressed, per-path ring of ~20 versions). Backs the `echo_note_history` /
|
||||
`echo_restore_note` tools (§6.3). The vault's additive philosophy finally
|
||||
gets an undo for its residual risk: a bad `replace` or merge PUT.
|
||||
- **Background maintenance jobs** — a timer thread running `sweep --fast`
|
||||
(IMPROVEMENT-PLANS #4) hourly and the decay pass (#5) proposals weekly; both
|
||||
publish findings to `/health` and the vault-health note rather than
|
||||
auto-fixing.
|
||||
- **Ops alerting** — outbox stuck > N hours, vault unreachable > 1h, new lint
|
||||
violations after a background sweep ⇒ notify via the existing Unraid
|
||||
notification plumbing (Kuma already watches `/health` for liveness).
|
||||
Problems surface without anyone polling.
|
||||
- **Embeddings (explicitly deferred)**: the container is where an embedding
|
||||
recall tier would land later (model API or local), as an image upgrade —
|
||||
no plugin change. Noted so nobody bolts it into the plugin.
|
||||
|
||||
## 8. Container & deployment
|
||||
|
||||
- **Image**: `python:3.12-slim` (not alpine — no musl surprises), `pip install
|
||||
mcp` pinned, copy `skills/echo-memory/scripts/` + `mcp-server/` in.
|
||||
**Dockerfile must be legacy-format** — the git.alwisp.com CI runner has no
|
||||
BuildKit: no `# syntax=` line, no `RUN --mount`.
|
||||
- **Healthcheck**: probe `http://127.0.0.1:<port>/health` — **127.0.0.1, not
|
||||
localhost** (the ::1-vs-IPv4 lesson from cpas/memer/breedr). `/health`
|
||||
(unauthenticated, no vault data): `{ok, vault_reachable, outbox_depth,
|
||||
index_age_s}` — also the Kuma target.
|
||||
- **Env** (via the PORT template, secrets as `SECRET:` refs): `ECHO_BASE`,
|
||||
`ECHO_KEY`, `ECHO_OWNER`, `ECHO_MCP_TOKEN`, `ECHO_STATE_DIR=/data`,
|
||||
`ECHO_MCP_TOOLS=full` (or `core` — see §3 tool profiles), optional
|
||||
`ECHO_WORKERS/ECHO_TIMEOUT/ECHO_MCP_INDEX_TTL`. Volume: `/data`
|
||||
(outbox + recall index + backups + shadow history).
|
||||
- **Startup validation**: fail fast (non-zero exit) if `ECHO_BASE`/`ECHO_KEY`/
|
||||
`ECHO_MCP_TOKEN` are missing — a misdeployed container should crash-loop
|
||||
visibly, not serve errors.
|
||||
- **CI/deploy**: Gitea workflow on push-to-main (tags/releases do **not**
|
||||
autobuild on this host) builds the image and the PORT autodeploy webhook
|
||||
rolls it out; `deploy.unraid.yml` manifest in `mcp-server/`. Publish
|
||||
`echomcp.alwisp.com` via the usual CF + NPM flow; add the Kuma monitor.
|
||||
- **Vault adjacency (confirmed: Obsidian runs on ALPHA)**: the Local REST
|
||||
API's non-encrypted HTTP binding is enabled (2026-07-28) —
|
||||
**`ECHO_BASE=http://10.2.0.35:27123`** — so the container talks to Obsidian
|
||||
directly, never through the public `echoapi.alwisp.com` hairpin. All vault
|
||||
chatter behind a tool call stays host/LAN-local; vault ops stop depending on
|
||||
Cloudflare/DNS/NPM health entirely (the public chain is only needed for the
|
||||
`echomcp` ingress). Why HTTP not HTTPS here: the REST API's HTTPS port
|
||||
(27124) uses a self-signed cert, which `echo.py`'s default-verifying
|
||||
`HTTPSConnection` rejects — cleartext on the internal network beats adding an
|
||||
insecure-TLS flag to the client. **Exposure check (do once):** 27123 carries
|
||||
the vault bearer token in cleartext, so it must not be reachable from
|
||||
outside — confirm no router/firewall forward for 27123; the only public
|
||||
doors stay HTTPS 443 via NPM (`echoapi` for the CLI fallback path, which
|
||||
remains unchanged, and `echomcp` for this server).
|
||||
|
||||
## 9. Client registration & fallback
|
||||
|
||||
- **Claude Code / CoWork**: register as a remote MCP server —
|
||||
`https://echomcp.alwisp.com/mcp` with the `Authorization: Bearer` header
|
||||
(project or user scope). No plugin-manifest coupling; the plugin does **not**
|
||||
register the server.
|
||||
- **claude.ai**: custom connector with the same URL + token — ECHO memory from
|
||||
the browser/phone, a surface the plugin could never reach.
|
||||
- **SKILL.md** gains: *"When the `echo_*` MCP tools are available, prefer them
|
||||
for every operation they cover; the `$ECHO` CLI recipes are the fallback for
|
||||
hosts without the connector or when the server is unreachable."* Procedures
|
||||
unchanged.
|
||||
- **Fallback matrix**: server up ⇒ tools. Server down, machine has the plugin ⇒
|
||||
CLI path exactly as today (including its own offline queue). Both down ⇒
|
||||
today's "vault unreachable, proceed without memory".
|
||||
|
||||
## 10. Multi-user note (Andy / Bryan / Gretchen / Arsalan)
|
||||
|
||||
One container serves **one vault**. The image is credential-free, so additional
|
||||
users are additional deployments of the same image with their own env (their
|
||||
vault endpoint/key + their own `ECHO_MCP_TOKEN`), e.g. `echomcp-bryan` on
|
||||
another port/subdomain. Cheap, isolated, no code change. A single multi-tenant
|
||||
server (token → vault map) is deliberately out of scope — that's CHORUS's
|
||||
territory.
|
||||
|
||||
## 11. Testing & eval
|
||||
|
||||
1. **Protocol/integration tests** (`eval/test_mcp_server.py`): run the FastMCP
|
||||
app in-process (or via `httpx` test client) against `mock_olrapi`; assert
|
||||
initialize/tools-list schemas, auth (no token ⇒ 401; bad token ⇒ 401;
|
||||
`/health` open), envelope purity on every tool.
|
||||
2. **Tool behavior**: per tool — happy path, validation failure (unknown param
|
||||
rejected with field name), vault-down (writes ⇒ `queued: true`, reads ⇒
|
||||
actionable error), duplicate-gate flow (gate → `merge_into` retry succeeds).
|
||||
3. **Parity**: `capture` via MCP and via CLI against twin mock vaults produce
|
||||
byte-identical notes/index entries (guards Phase 0).
|
||||
4. **Container smoke** (CI): build image, start with mock env, `/health` green,
|
||||
one authed `tools/call` round-trip; healthcheck probes 127.0.0.1.
|
||||
5. **MCP Inspector** manual pass against the deployed container.
|
||||
6. **Eval set** (mcp-builder Phase 4): 10 read-only Q&A pairs against a seeded
|
||||
mock vault exercising recall→get_note→resolve chains; `eval/mcp_eval.xml`.
|
||||
|
||||
## 12. Build-session plan of record
|
||||
|
||||
| Phase | Deliverable | Est. size |
|
||||
|---|---|---|
|
||||
| 0 | Return-not-print refactor in the plugin tree (`*_op` cores); suites green | ~⅓ the session |
|
||||
| 1 | `mcp-server/`: FastMCP app, auth middleware, result wrapper, error map, `/health` | ~200 lines |
|
||||
| 2 | Read tools (load, recall, resolve, get_note, scope×2, health) | wiring |
|
||||
| 3 | Write tools (capture, link, append/patch, triage, reflect, log_session) | wiring + descriptions |
|
||||
| 4 | Warm caches + `/data` outbox + startup validation | small |
|
||||
| 5 | Dockerfile (legacy syntax) + CI workflow + PORT manifest + deploy + CF/NPM publish + Kuma | infra pass |
|
||||
| 6 | Tests (§11) + Inspector pass + eval XML; SKILL.md/README/CHANGELOG | ~⅓ the session |
|
||||
|
||||
Definition of done: Inspector lists 13 tools against `echomcp.alwisp.com`; a
|
||||
live session (desktop **and** CoWork) performs a full memory day — load →
|
||||
recall → capture → gate → merge_into → triage → log_session — without one Bash
|
||||
call; parity test green; container smoke test in CI; Kuma monitor green.
|
||||
@@ -0,0 +1,99 @@
|
||||
# ECHO Memory — Usage Guide
|
||||
|
||||
*Written for Bryan (and anyone else coming back to ECHO after a break). Current as of v1.5.1, July 2026.*
|
||||
|
||||
ECHO gives Claude persistent memory across sessions. Everything durable — people, companies, projects, decisions, preferences, session history — lives as markdown notes in an Obsidian vault, read and written over the Obsidian Local REST API. The plugin ships the whole toolchain (pure Python, stdlib only — works the same on Windows/macOS/Linux) plus the operating procedure Claude follows.
|
||||
|
||||
The short version of how to use it: **you mostly don't have to do anything.** Since v1.5 the system loads itself at session start and nudges itself to save at session end. The slash commands below are for when you want to drive it explicitly.
|
||||
|
||||
---
|
||||
|
||||
## One-time setup
|
||||
|
||||
If Jason handed you a **per-user baked plugin** (`echo-memory-1.x.x-bryan.plugin`), just install it. Your vault credentials are baked into the artifact — no config file, no key pasting, works on desktop and in every CoWork session. This is the normal path.
|
||||
|
||||
If you have the **generic plugin** instead, it will report `NOT CONFIGURED` on first use and ask for your config file. Install it once with:
|
||||
|
||||
```bash
|
||||
python3 echo.py config import <your-config.json>
|
||||
```
|
||||
|
||||
That writes `~/.claude/echo-memory/config.json` (owner, endpoint, API key). It survives plugin updates — one time per machine.
|
||||
|
||||
**Verify either way:** run `/echo-doctor`. Green across the board (Python, vault reachable, auth, bootstrapped, key source) means you're live.
|
||||
|
||||
---
|
||||
|
||||
## Day-to-day usage
|
||||
|
||||
### What happens automatically (v1.5+)
|
||||
|
||||
- **Session start** — a hook runs the cold-start load and injects your memory into context: your profile/preferences, current scope, the last session log, today's journal note, and inbox depth. You don't have to ask Claude to "load memory" anymore.
|
||||
- **Session end** — if the session was substantive and nothing was saved, a hook nudges (once) to run `/echo-reflect` and write the session log. Reflection always previews before writing — nothing lands without your OK.
|
||||
- **If the vault is unreachable**, writes queue to a local outbox and replay automatically next time it's up; reads fall back to a cached last-known-good. An outage degrades gracefully instead of erroring.
|
||||
|
||||
### The slash commands
|
||||
|
||||
| Command | When to use it |
|
||||
|---|---|
|
||||
| `/echo-load` | Manually load memory context. Rarely needed now — the session-start hook does this — but useful mid-session after a lot of vault writes, or if the hook didn't fire. |
|
||||
| `/echo-save <thing>` | Save something right now: "remember the Henagar tower password is in the vault", "log that we decided X". Routes it to the right note automatically. |
|
||||
| `/echo-recall <topic>` | Ask what memory knows about a topic. Returns ranked matches **plus their linked neighbourhood** — a project comes back with its people, decisions, and recent session mentions attached. |
|
||||
| `/echo-reflect` | End-of-session sweep: extracts durable facts, decisions, and new entities from the conversation, shows you the proposed writes, applies on confirm. The main way memory grows. |
|
||||
| `/echo-triage` | Empty the inbox: classifies each quick capture, previews destinations, routes accepted items with an audit trail. Run when load reports the inbox is getting deep. |
|
||||
| `/echo-doctor` | Readiness check — Python, vault reachability, auth, bootstrap/schema version, where the key came from. First stop when anything seems off. |
|
||||
| `/echo-health` | Full vault lint: routing violations, broken wikilinks, orphan notes, stale scope, incomplete frontmatter, index drift. Run occasionally or when things feel inconsistent. |
|
||||
| `/echo-sweep` | Bring the vault up to the current plugin spec after an upgrade — rebuilds the entity index, backfills frontmatter, fixes one-way links. Dry-run by default; **run this once right after updating from an old version.** |
|
||||
|
||||
### A typical session
|
||||
|
||||
1. Start a conversation — memory loads itself. Claude knows who you are, what's active, and what happened last session.
|
||||
2. Work normally. Say "save that" / "remember this" whenever something durable comes up (or `/echo-save` explicitly). Not sure where something belongs? Save it anyway — it lands in the inbox and `/echo-triage` sorts it later.
|
||||
3. At the end, accept the reflection nudge (or run `/echo-reflect` yourself). Confirm the preview. Done — next session picks up from here.
|
||||
|
||||
---
|
||||
|
||||
## What the memory system can do (current capabilities)
|
||||
|
||||
**Smart, deduplicating capture.** One `capture` call routes content to its canonical home via an entity index, stamps complete frontmatter (status, tags, timestamps), cross-links every entity it mentions, and logs the write. Names resolve through aliases and fuzzy matching, so "echo memory" finds the `echo` project instead of spawning a duplicate note. If a new title strongly resembles an existing **same-kind** entity, a pre-write duplicate gate *stops* the write and shows you the candidates — you choose merge or create. Duplicates get blocked before they exist, not cleaned up after.
|
||||
|
||||
**Real retrieval, not keyword grep.** Recall fuses BM25 full-text search with graph expansion along note links, over entities *and* session logs *and* journal entries. Ranking is freshness- and status-aware: recently updated and `active` notes float up, `archived` sinks. You get a topic's connected neighbourhood, not one isolated file.
|
||||
|
||||
**Structured memory model.** Working (transient) / episodic (what happened, when) / semantic (durable facts and preferences) memory layers, plus a current-context scope, per-session logs, an append-only journal with rollups, and PARA-style projects/areas/resources/decisions. A machine-readable routing manifest defines what may be written where, and the linter enforces it.
|
||||
|
||||
**Reflection.** `/echo-reflect` turns a conversation into memory: extract → classify → dedup against the index → confidence-filter → preview → apply. Low-confidence items go to the inbox instead of polluting the graph.
|
||||
|
||||
**Durability and safety.** Every write is HTTP-status-checked and read-back-verified (a failed write fails loudly, never silently). Appends are idempotent — retries can't double-write. Offline writes queue and replay. A cooperative advisory lock keeps Claude Code and CoWork from trampling each other on the shared vault.
|
||||
|
||||
**Self-maintaining.** The vault bootstraps itself from an empty Obsidian vault, migrates its own schema on version bumps, and `/echo-sweep` + `/echo-health` keep the graph honest (index rebuilds, link symmetry, frontmatter backfill, orphan/broken-link detection).
|
||||
|
||||
**Fast.** Connection pooling + concurrent reads mean full-vault operations that used to time out in the sandbox now finish in under a second.
|
||||
|
||||
---
|
||||
|
||||
## What's new since 0.7 (your last consistent version)
|
||||
|
||||
You left off right after the toolchain got its executable spine (`echo.py`, routing manifest, linter). Since then, in rough order of what you'll actually notice:
|
||||
|
||||
- **You don't route anything by hand anymore.** 0.7 was "pick the path, write with the right verb." Now `capture`/`recall`/`resolve`/`link` do routing, frontmatter, indexing, and cross-linking in one call — and in practice you just talk to Claude and it uses them.
|
||||
- **Memory loads and saves itself** (v1.5 hooks). No more remembering to run the loading procedure or write the session log.
|
||||
- **Recall is actually good now** (v0.9–1.5): entity index + hybrid BM25/graph search + freshness/status ranking, spanning sessions and journal too.
|
||||
- **Duplicates get blocked at write time** (v1.5 gate) instead of accumulating.
|
||||
- **Offline resilience** (v1.0): vault down ≠ data lost; writes queue and replay.
|
||||
- **Everything is pure Python** (v0.8): no bash, works identically on Windows.
|
||||
- **Credentials moved out of the plugin** (v1.3–1.4): generic builds prompt once per machine; your baked build carries them invisibly. Treat a baked `.plugin` file like a password — it contains your vault key. Don't share or commit it.
|
||||
|
||||
**After installing the new version, run `/echo-doctor`, then `/echo-sweep` once** to bring your vault up to the current schema (it dry-runs first and shows the plan).
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Fix |
|
||||
|---|---|
|
||||
| `NOT CONFIGURED` banner / doctor shows red config | Generic build with no config on this machine — `echo.py config import <file>`, or ask Jason for your baked build. |
|
||||
| Vault unreachable | Check that Obsidian + the Local REST API plugin are running on the backend and the endpoint is reachable. Meanwhile writes queue safely and replay on reconnect. |
|
||||
| Save blocked with "duplicate gate" (exit 76) | Not an error — it found a likely-existing entity. Review the candidates: merge into the existing note, or force-create if it's genuinely distinct. |
|
||||
| Memory feels stale or inconsistent | `/echo-health` to see what's off, `/echo-sweep` to repair index/links/frontmatter. |
|
||||
| Inbox keeps getting mentioned at load | `/echo-triage` — one pass empties it. |
|
||||
| Anything else weird | `/echo-doctor` first, then ask Jason. |
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,6 +1,8 @@
|
||||
{
|
||||
"name": "echo-memory",
|
||||
"version": "1.5.1",
|
||||
"version": "2.0.0",
|
||||
"homepage": "https://git.alwisp.com/jason/echo",
|
||||
"repository": "https://git.alwisp.com/jason/echo",
|
||||
"description": "Persistent memory via the ECHO Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /echo-* commands.",
|
||||
"author": {
|
||||
"name": "Jason Stedwell"
|
||||
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
---
|
||||
name: echo-doctor
|
||||
description: Check ECHO readiness — Python, vault reachability, auth, bootstrap/schema, and key source
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run a one-call readiness check before relying on memory.
|
||||
+2
-1
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: echo-health
|
||||
description: Run the ECHO vault-health linter and summarize any invariant violations
|
||||
allowed-tools: Bash(python3 *vault_lint.py*), Bash(python *vault_lint.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
allowed-tools: Bash(python3 *vault_lint.py*), Bash(python *vault_lint.py*), Bash(py -3 *vault_lint.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
---
|
||||
|
||||
Run the bundled, read-only vault linter and report findings. Pass the conversation's current date (`ECHO_TODAY`) so stale/aging math uses the same clock the agent writes with:
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
---
|
||||
name: echo-load
|
||||
description: Load ECHO memory — cold-start context read (profile, scope, latest session, today, inbox)
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to load memory now. Run the cold-start **Loading procedure** from `SKILL.md`: the six orientation reads (marker, operator-preferences, current-context, heartbeat, today's daily note, inbox) in **one call**, then do the load-time **reconcile** (inbox-depth + scope-drift) and surface it in a single line.
|
||||
@@ -165,14 +165,16 @@ def main() -> int:
|
||||
# this shared cache instead of issuing a fresh GET per file per section.
|
||||
texts = echo.read_many(md_files)
|
||||
|
||||
# Path membership + retired-path detection
|
||||
# Path membership + retired-path detection. Retired patterns are checked FIRST:
|
||||
# a file inside a retired tree is flagged even when a permissive route (the
|
||||
# leaf-README rule) would otherwise match it — previously `archive/x/README.md`
|
||||
# was silently absolved by `leaf-readme` and retired dirs survived unflagged.
|
||||
for path in all_files:
|
||||
if routes and not any(rx.match(path) for _, rx in routes):
|
||||
replacement = next((repl for rx, repl in retired if rx.match(path)), None)
|
||||
if replacement is not None:
|
||||
flag("retired-path", f"{path}: retired location — should be {replacement}")
|
||||
else:
|
||||
flag("unknown-path", f"{path}: matches no route in routing.json")
|
||||
replacement = next((repl for rx, repl in retired if rx.match(path)), None)
|
||||
if replacement is not None:
|
||||
flag("retired-path", f"{path}: retired location — should be {replacement}")
|
||||
elif routes and not any(rx.match(path) for _, rx in routes):
|
||||
flag("unknown-path", f"{path}: matches no route in routing.json")
|
||||
|
||||
# Per-note frontmatter checks
|
||||
template_re = re.compile(r"(^|/)(templates/|.*-template\.md$)")
|
||||
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-recall
|
||||
description: Recall a topic from ECHO memory — matching notes plus their linked neighbourhood
|
||||
argument-hint: "[topic, person, or project]"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to recall context for:
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-reflect
|
||||
description: Reflect on this session — extract durable memories and propose them for one-tap capture
|
||||
argument-hint: "[optional focus, e.g. 'just decisions']"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run **session reflection** (H5). Scan this conversation for
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-save
|
||||
description: Save to ECHO memory — route content to its canonical home (search-first, idempotent)
|
||||
argument-hint: "[what to remember]"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to persist this to the ECHO vault:
|
||||
+3
-1
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-sweep
|
||||
description: Bring the ECHO vault up to the current feature spec (entity index + cross-links)
|
||||
allowed-tools: Bash(python3 *sweep.py*), Bash(python *sweep.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
allowed-tools: Bash(python3 *sweep.py*), Bash(python *sweep.py*), Bash(py -3 *sweep.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to bring the vault up to the current spec. Run the sweep **dry-run first**, show the operator the plan, and only `--apply` on their go-ahead:
|
||||
+3
@@ -1,5 +1,8 @@
|
||||
---
|
||||
name: echo-triage
|
||||
description: Triage the ECHO inbox — route aging captures to their canonical homes and log the moves
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run **one-tap Inbox Triage** via the `triage` verb (it reuses the reflect pipeline: classify against the entity index → preview → apply via `capture`, and writes the audit log for you).
|
||||
@@ -298,6 +298,14 @@ def main():
|
||||
check("v1.5 sweep backfills the kind tag",
|
||||
"tags: [company]" in drift or '<fm:tags=["company"]>' in drift, drift)
|
||||
|
||||
# 16b. lint blind spot (1.6.0): a seed README inside a retired tree must be
|
||||
# flagged retired-path — previously the permissive leaf-readme route matched
|
||||
# first and absolved it, so retired dirs holding only a README went unflagged.
|
||||
h.seed("archive/notes/README.md", "This folder holds archived notes.\n")
|
||||
r = h.echo(LINT)
|
||||
check("v1.6 lint flags a README inside a retired tree",
|
||||
"archive/notes/README.md: retired location" in r.stdout, r.stdout[-800:])
|
||||
|
||||
# 17. hooks: stop-hook nudges once on a substantive session, then stays quiet.
|
||||
import tempfile
|
||||
tdir = Path(tempfile.mkdtemp())
|
||||
|
||||
Reference in New Issue
Block a user