Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 446cd9a0ff | |||
| 1deef7299e | |||
| 5aace9e430 | |||
| ab1e514f78 | |||
| fb407d606c | |||
| 55afdce70c | |||
| e7d86e14da | |||
| d27db4ae34 |
+7
-5
@@ -20,8 +20,10 @@ eval/results/*.json
|
||||
/echo-memory.config.json
|
||||
/dist/
|
||||
|
||||
# Per-user BAKED artifacts (build.py --bake-key, named echo-memory-<ver>-<label>.plugin)
|
||||
# carry a live vault bearer token — NEVER commit them. They belong in dist/ (above);
|
||||
# this also guards any that get built at the repo root. The version-only pointer
|
||||
# (echo-memory.plugin) and version-only artifacts (echo-memory-<ver>.plugin) stay tracked.
|
||||
echo-memory-*-*.plugin
|
||||
# Build artifacts (2.0 policy): the repo tracks ONLY the echo-memory.plugin pointer.
|
||||
# Versioned builds (echo-memory-<ver>.plugin) are published as Gitea release assets
|
||||
# on git.alwisp.com/jason/echo, one release per tag — not committed. Per-user BAKED
|
||||
# artifacts (echo-memory-<ver>-<label>.plugin) carry a live vault bearer token and
|
||||
# must NEVER be committed anywhere (they belong in dist/, also ignored above).
|
||||
*.plugin
|
||||
!echo-memory.plugin
|
||||
|
||||
+127
@@ -1,5 +1,132 @@
|
||||
# Changelog
|
||||
|
||||
## 2.1.0
|
||||
|
||||
The "quick-wins train" from the 2026-07-28 review (`docs/IMPROVEMENT-PLANS.md` #1,
|
||||
#3, #7). Three independently useful changes; no schema or layout changes.
|
||||
|
||||
### Added — `load --brief`: a token-budgeted cold-start digest
|
||||
|
||||
The SessionStart hook injected the full text of six files every session (measured
|
||||
15.9KB and growing without bound — Observations and Scope History never shrink).
|
||||
`load --brief` renders a digest instead: Fact / Pattern rules in full, the last ~10
|
||||
Observations (with a `+N older` note), scope + `scope_updated` + sessions-since,
|
||||
the heartbeat-pointed session log's key sections (Goal / Decisions Made / Open
|
||||
Threads / Suggested Next Step), today's Agent Log lines, and the inbox as a
|
||||
**count + oldest age** — everything the load-time reconcile needs. Over
|
||||
`ECHO_LOAD_BUDGET` (default ~8000 chars), sections trim lowest-priority-first
|
||||
(observations → agent log → session summary) with explicit `(truncated)` markers.
|
||||
The hook now injects the brief form; `/echo-load` and the manual procedure keep the
|
||||
full dump. All `load` reads (plus the sessions listing, which also feeds the
|
||||
heartbeat-absent fallback) are fetched **in parallel** over the pooled connections.
|
||||
|
||||
### Fixed — capture is now offline-durable (the write path's inversion bug)
|
||||
|
||||
Low-level verbs queued on an outage, but the *default* write (`capture`) died: the
|
||||
update path's POST/PATCH and `ensure_daily_log` bypassed the queue entirely, and an
|
||||
unreachable index aborted the whole op. Now: (1) a fully-offline `capture` queues
|
||||
the **whole operation as one semantic record** (title/kind/body/tags/…, plus the
|
||||
capture-time date); `flush` replays it **through `capture`** so create-vs-update,
|
||||
the duplicate gate, and aliasing re-run against the index as it is at replay time —
|
||||
byte-level replay would freeze a decision made blind. (2) A duplicate-gate stop on
|
||||
replay keeps the record, flags it `needs_attention` (surfaced by `flush` and
|
||||
`load`), and does NOT block later records. (3) Mid-capture outages: the update
|
||||
path and `ensure_daily_log` writes ride `safe_request` (accepted edge: a queued
|
||||
agent-log PATCH 400-drops loudly if the daily note never existed — the trace line,
|
||||
not the memory). Same-args offline captures dedupe by idem-key.
|
||||
|
||||
### Added — `session-end`: the one-call session close
|
||||
|
||||
`echo.py session-end <bundle.json> [--apply]` performs the whole end-of-session
|
||||
sequence under ONE lock acquisition, in order: session log PUT → Agent Log line →
|
||||
reflect proposals (via `capture`, gate-aware) → optional `scope set` → **heartbeat
|
||||
LAST as the commit marker** (a part-way failure leaves the previous pointer intact,
|
||||
never dangling). Dry-run by default previews the full plan including the reflect
|
||||
classification. Filename HHMM from `$ECHO_NOW` (else local clock), validated
|
||||
against the canonical session-log regex **before any write**. The Stop hook's nudge
|
||||
now names the command, and recognizes a `session-end` invocation as
|
||||
"already reflected".
|
||||
|
||||
### Fixed — helper-module errors exited 2 instead of their intended codes
|
||||
|
||||
When `echo.py` runs as `__main__`, helper modules' `import echo` creates a twin
|
||||
module whose `EchoError` is a different class object, so `main()`'s handler missed
|
||||
it and the raw traceback leaked. The handler now catches `RuntimeError` (both
|
||||
classes subclass it) and honors `.code`.
|
||||
|
||||
Tests: +19 end-to-end checks (brief-load digest/budget, offline-capture queue /
|
||||
replay-through-capture / gate-on-replay / idem-key dedup, session-end dry-run /
|
||||
apply / ordering / validation-abort). All suites green.
|
||||
|
||||
## 2.0.0
|
||||
|
||||
**Packaging & structure major — nothing about memory behavior changes.** The major
|
||||
version signals "reinstall, re-clone expectations": how the plugin is distributed
|
||||
and installed is what breaks.
|
||||
|
||||
### Removed — the legacy `commands/` directory (BREAKING for pre-skills clients)
|
||||
|
||||
The plugin is **skills-only**. The eight `/echo-*` entry points live exclusively at
|
||||
`skills/<name>/SKILL.md` (introduced 1.6.0, verified on desktop + CoWork
|
||||
2026-07-28, where they already took precedence). A Claude Code old enough to lack
|
||||
skills support loses the slash commands — install 1.6.0 from the release history
|
||||
instead.
|
||||
|
||||
### Changed — artifact policy (BREAKING for anyone pulling zips from the repo)
|
||||
|
||||
- The repo tracks **only** `echo-memory.plugin` (the current installable). The 15
|
||||
historical `echo-memory-<version>.plugin` zips are deleted from the tree (git
|
||||
history retains them).
|
||||
- Versioned builds are published as **Gitea releases** on
|
||||
`git.alwisp.com/jason/echo` — one release per `v<version>` tag, artifact
|
||||
attached, starting with `v2.0.0`.
|
||||
- `.gitignore` now ignores `*.plugin` except the pointer.
|
||||
|
||||
### Added — "packaging for other agent runtimes" README note
|
||||
|
||||
The retired Codex tree stays retired; the README now documents what a port needs
|
||||
(manifest shape, skill entry point, script invocation + hooks) and the rule that a
|
||||
port must be a **build target** from the canonical source tree, never a second
|
||||
hand-maintained copy.
|
||||
|
||||
## 1.6.0
|
||||
|
||||
### Changed — the eight slash commands migrated to the skills format
|
||||
|
||||
Each `commands/<name>.md` now has a `skills/<name>/SKILL.md` twin with the same
|
||||
invocation name and a byte-identical body — confirmed against the current docs
|
||||
that a same-name skill **takes precedence** over the legacy command, so both
|
||||
formats coexist safely in this release (deleting `commands/` is the 2.0
|
||||
packaging break). Done for the features, not the deprecation notice:
|
||||
|
||||
- **`allowed-tools` per skill** — the resolved `python3/python/py -3` script
|
||||
invocations (plus the CoWork `ls /sessions/*` fallback probe) are
|
||||
pre-authorized, so `/echo-load`, `/echo-health`, `/echo-recall` etc. stop
|
||||
prompting.
|
||||
- **`disable-model-invocation: true`** on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) — only the operator triggers them; the
|
||||
read-side skills stay model-invocable.
|
||||
- `argument-hint` / `$ARGUMENTS` carry over unchanged (same semantics in
|
||||
SKILL.md bodies).
|
||||
- The `$ECHO` path-resolution block stays per-skill: the skills format has no
|
||||
cross-skill snippet sharing (each skill directory is self-contained per the
|
||||
official docs), and the block is already the 2-line minimum.
|
||||
|
||||
### Fixed — lint blind spot: retired trees masked by the leaf-README route
|
||||
|
||||
`vault_lint.py` checked retired patterns only for paths matching **no** route,
|
||||
so the permissive `leaf-readme` route (`^(.+/)?README\.md$`) absolved any seed
|
||||
README inside a retired tree — `archive/` and `_agent/outputs/` survived the
|
||||
1.5.0 live cleanup unflagged. Retired patterns are now checked **first**: a
|
||||
path in a retired tree flags `retired-path` regardless of what else matches.
|
||||
New end-to-end test seeds `archive/notes/README.md` and asserts the flag
|
||||
(fails against the pre-fix linter, passes now).
|
||||
|
||||
### Added — plugin.json completeness
|
||||
|
||||
`homepage` + `repository` → `https://git.alwisp.com/jason/echo`; manifest →
|
||||
1.6.0.
|
||||
|
||||
## 1.5.1
|
||||
|
||||
### Fixed — duplicate gate over-firing on vault-common tokens and cross-kind names
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# echo-memory — v1.5.1
|
||||
# echo-memory — v2.1.0
|
||||
|
||||
Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`.
|
||||
|
||||
@@ -67,7 +67,8 @@ echo-v.05/
|
||||
├── docs/history/ ← historical inputs (e.g. the 1.5.0 frontmatter field report)
|
||||
├── build.py ← deterministic .plugin builder (--bake-key/--strip-key/--label/--outdir)
|
||||
├── echo-memory.plugin ← built, installable plugin (zip artifact, rebuilt on version bump)
|
||||
├── echo-memory-<version>.plugin ← versioned build artifacts (history; moves to Gitea releases in 2.0)
|
||||
│ — the ONLY tracked artifact; versioned builds are Gitea release
|
||||
│ assets (one release per v<version> tag), not committed (2.0 policy)
|
||||
├── dist/ ← per-user baked artifacts (secret-bearing) — GITIGNORED, never committed
|
||||
├── eval/ ← credential-free eval + test harness; not bundled
|
||||
│ ├── mock_olrapi.py ← deterministic mock of the REST API + fault injection
|
||||
@@ -80,8 +81,9 @@ echo-v.05/
|
||||
├── .claude-plugin/plugin.json ← manifest (name, version, description)
|
||||
├── README.md ← plugin-level README
|
||||
├── hooks/hooks.json ← session hooks: SessionStart auto-load · Stop reflection nudge
|
||||
├── commands/ ← slash commands (legacy format; skills-format migration = TODO-1.6):
|
||||
│ echo-load|save|recall|triage|health|sweep|reflect|doctor
|
||||
├── skills/echo-{load,save,recall,triage,health,sweep,reflect,doctor}/
|
||||
│ ← the eight slash commands as skills (skills-only since 2.0):
|
||||
│ per-skill allowed-tools; sweep/triage are operator-only
|
||||
└── skills/echo-memory/
|
||||
├── SKILL.md ← operating procedure (authoritative)
|
||||
├── references/
|
||||
@@ -120,6 +122,8 @@ echo-v.05/
|
||||
└── templates/ ← 8 canonical note templates
|
||||
```
|
||||
|
||||
**Packaging for other agent runtimes** (from the retired Codex tree, deleted 2026-07-03): if a port to another runtime (Codex, etc.) is ever wanted, implement it as a **build target** generated from the canonical `echo-memory.plugin.src/` tree (`build.py --target <runtime>` or similar) — never as a second hand-maintained source tree; that's how the original Codex copy drifted. A port needs three adaptations: the manifest shape (`.claude-plugin/plugin.json` → the runtime's equivalent), the skill entry point (`skills/*/SKILL.md` frontmatter + body conventions), and the script-invocation convention (`${CLAUDE_PLUGIN_ROOT}` resolution + the session-hook wiring in `hooks/hooks.json`). The Python toolchain itself is runtime-agnostic (pure stdlib, env/config-driven).
|
||||
|
||||
**Division of responsibility:** `SKILL.md` owns day-to-day *procedure* (loading order, search-first, triage, scope switching, PATCH rules) and points at the bundled tooling. `references/operating-contract.md` owns the durable, client-independent *principles, safety rules, and concurrency model*. `scripts/routing.json` is the machine-readable source of truth for routing; `references/routing-map.md` is its human-readable authority. The other references are the canonical layout, API, and bootstrap specs.
|
||||
|
||||
---
|
||||
@@ -417,6 +421,9 @@ From the credential-free harness (`eval/run_eval.py` against the deterministic m
|
||||
|
||||
| Version | Highlights |
|
||||
|---------|-----------|
|
||||
| **2.1.0** | **Quick-wins train: cheaper sessions, durable capture, one-call session end.** (1) **`load --brief`** — a token-budgeted cold-start digest (Fact/Pattern in full, last ~10 Observations, scope+freshness, last session's key sections, Agent-Log lines, inbox *count*; `ECHO_LOAD_BUDGET` default ~8000 chars) now injected by the SessionStart hook, replacing the full six-file dump that grew unboundedly; all `load` reads are fetched in parallel. (2) **Offline capture durability** — `capture` on an unreachable vault queues the *whole operation* as one semantic record; `flush` replays it **through capture** so routing/gate/aliasing re-run against the current index; a gate stop on replay is kept + flagged, never landed blind; `ensure_daily_log`/update-path writes ride the queue too. (3) **`session-end`** — one call (one lock) writes session log → Agent-Log line → reflect proposals → optional scope switch → **heartbeat last as the commit marker**; dry-run by default; `ECHO_NOW` pins the HHMM. Also fixes the `__main__` twin-module trap so helper-module `EchoError`s exit with their intended codes. +19 end-to-end checks. |
|
||||
| **2.0.0** | **Packaging & structure major — memory behavior unchanged.** Skills-only: the legacy `commands/` directory is deleted (breaking for pre-skills clients; the 1.6.0-verified skills are the sole entry points). Artifact policy: the repo tracks only the `echo-memory.plugin` pointer; the 15 historical versioned zips leave the tree and versioned builds ship as **Gitea releases** (one per `v<version>` tag) from `v2.0.0` on; `.gitignore` blocks `*.plugin` except the pointer. README gains the "packaging for other agent runtimes" port note (build-target rule, never a second source tree). |
|
||||
| **1.6.0** | **Skills-format migration + lint blind-spot fix.** The eight slash commands gain `skills/<name>/SKILL.md` twins (same names, byte-identical bodies; a same-name skill takes precedence, so `commands/` coexists until its 2.0 deletion): per-skill `allowed-tools` end the permission prompts, `disable-model-invocation: true` makes `/echo-sweep` + `/echo-triage` operator-only. `vault_lint` checks retired patterns **before** routes, so a seed README inside a retired tree (`archive/…`) is flagged instead of being absolved by the leaf-README route (+ regression test). `plugin.json` gains `homepage`/`repository`. |
|
||||
| **1.5.1** | **Duplicate-gate precision.** Live use caught the gate blocking a decision note because it shared the single vault-common token "echo" with an archived project (min-normalized scoring gives any single-token entity a 1.0 match). New `gate_candidates()` blocks only on **same-kind** candidates (cross-kind name collisions — a decision titled after its project — warn instead), and a **lone shared token blocks only when unique to that entity** (vault df == 1); multi-token overlaps still block. Advisory warnings (`fuzzy_candidates`) unchanged; exit-76/`--merge-into`/`--force` unchanged. +5 tests; verified against the live vault both ways (false positive creates cleanly, true lookalike still gates). |
|
||||
| **1.5.0** | **Six-improvement pass: retention, routing, and self-firing memory.** (1) **Capture-update keeps the whole body** — the update path previously appended only the first line of a multi-line body (silent data loss); now the full body rides under the dated bullet. (2) **Frontmatter completeness** — capture stamps a kind-default `status` and kind-seeded `tags` (`--tags` enriches); `fm` is create-or-replace (missing keys are surgically inserted instead of 400-failing); `vault_lint` gains a kind-scoped `incomplete-frontmatter` check; `sweep --apply` backfills (fixes the 18/71-notes field-audit drift, one data source: `KIND_STATUS`/`KIND_REQUIRED_FM`). (3) **Pre-write duplicate gate** — a strong fuzzy candidate stops `capture` (exit 76 + candidates) before the duplicate exists; `--merge-into <slug>` updates the canonical entity, `--force` overrides. (4) **Recall corpus + ranking** — sessions and journal notes join the BM25 corpus (down-weighted); scores fuse freshness (half-life on `updated:`) and status (`active` boosted, `archived` demoted); hits show `updated:`/`status:`; `recall --json`; recall-index schema 2 (auto-rebuilds). (5) **Session hooks** — SessionStart auto-loads memory into context, Stop nudges reflection once per substantive session; fail-safe, CoWork-fallback-aware. (6) **One-tap triage** — `echo.py triage` lists the inbox structured, then classifies → previews → routes accepted items via `capture` with automatic processing-log audit lines; `--json` also lands on `link`/`scope show`. +22 mock end-to-end tests; all suites green. |
|
||||
| **1.4.2** | **CoWork sandbox path resilience.** In a remote CoWork sandbox `${CLAUDE_PLUGIN_ROOT}` can point at a host path the sandbox can't reach (`/var/folders/…`) while the plugin is mounted under `…/mnt/.remote-plugins/…`, so script invocations failed until the agent found the real path by hand. SKILL.md and all eight slash commands now resolve the scripts dir with a fallback — prefer `${CLAUDE_PLUGIN_ROOT}`, else locate the mounted copy under `/sessions/*/mnt/.remote-plugins/*/…` — reused via `$ECHO`/`$LINT`/`$SWEEP`/`$SDIR`. On a normal host the primary path always wins (no behaviour change). `echo-health`/`echo-sweep` `allowed-tools` broadened to match the resolved invocation. (The scripts never hardcoded a path — root cause is the harness env var — but the plugin now self-heals.) |
|
||||
|
||||
+16
-14
@@ -20,12 +20,12 @@ The repo root tracks every historical build (`echo-memory-0.6.0.plugin` …
|
||||
`echo-memory-1.5.1.plugin`) plus the `echo-memory.plugin` pointer. 2.0 changes the
|
||||
policy — **breaking for anyone who pulls artifacts straight from the repo**:
|
||||
|
||||
- [ ] Track only `echo-memory.plugin` (the current installable); delete the versioned
|
||||
zips from the tree.
|
||||
- [ ] Publish versioned builds as **Gitea releases** on `git.alwisp.com/jason/echo`
|
||||
instead (one release per tag, artifact attached).
|
||||
- [ ] `.gitignore`: `*.plugin` except the pointer.
|
||||
- [ ] Tag releases going forward (`v2.0.0`, …) so the release page is the version
|
||||
- [x] Track only `echo-memory.plugin` (the current installable); delete the versioned
|
||||
zips from the tree — **done 2.0.0** (15 zips removed; history keeps them).
|
||||
- [x] Publish versioned builds as **Gitea releases** on `git.alwisp.com/jason/echo`
|
||||
instead (one release per tag, artifact attached) — **v2.0.0 onward**.
|
||||
- [x] `.gitignore`: `*.plugin` except the pointer — **done 2.0.0**.
|
||||
- [x] Tag releases going forward (`v2.0.0`, …) so the release page is the version
|
||||
history for artifacts, and the README table stays the narrative history.
|
||||
|
||||
## 2. Complete the skills-format migration (finishes the 1.6 work)
|
||||
@@ -33,10 +33,12 @@ policy — **breaking for anyone who pulls artifacts straight from the repo**:
|
||||
1.6 migrates the eight slash commands to `skills/*/SKILL.md` with both formats
|
||||
coexisting (see `TODO-1.6.md`). 2.0 finishes it:
|
||||
|
||||
- [ ] **Delete the legacy `commands/` directory** — the packaging break that partly
|
||||
motivates the major bump.
|
||||
- [ ] Re-verify desktop + CoWork installs of the skills-only artifact.
|
||||
- [ ] Update SKILL.md / README / command docs that reference `commands/`.
|
||||
- [x] **Delete the legacy `commands/` directory** — **done 2.0.0** (1.6.0's install
|
||||
verification on both surfaces cleared the gate; skills had precedence anyway).
|
||||
- [x] Re-verify desktop + CoWork installs of the skills-only artifact — **verified
|
||||
2026-07-28** on the baked 2.0.0 build: installs cleanly, all eight skills
|
||||
register and work with no legacy `commands/` present.
|
||||
- [x] Update SKILL.md / README / command docs that reference `commands/` — **done 2.0.0**.
|
||||
|
||||
## 3. Codex packaging (from MAINTENANCE › Canonical source tree)
|
||||
|
||||
@@ -47,9 +49,9 @@ regenerated after 2.0 if needed). What remains for 2.0:
|
||||
- [ ] If Codex support returns: implement it as a **build target** generated from the
|
||||
canonical `echo-memory.plugin.src/` tree (`build.py --target codex` or similar) —
|
||||
never as a second hand-maintained source tree (that's how the drift happened).
|
||||
- [ ] Otherwise: a short "packaging for other agent runtimes" note in the README
|
||||
- [x] Otherwise: a short "packaging for other agent runtimes" note in the README
|
||||
documenting what a Codex/other-runtime port needs (manifest shape, skill entry
|
||||
point, script invocation).
|
||||
point, script invocation) — **done 2.0.0** (README › after Repository layout).
|
||||
|
||||
## 4. Eval refresh — publish current-version metrics (from MAINTENANCE › Docs freshness; ROADMAP-1.0 H4 leftover) — ✅ DONE
|
||||
|
||||
@@ -67,8 +69,8 @@ the README. Re-run it per release and refresh the README table.
|
||||
## 5. Repo hygiene odds & ends
|
||||
|
||||
- [x] Retire `ROADMAP-1.0.md` (fully shipped; in git history) — **done 2026-07-03**.
|
||||
- [ ] Root README "Repository layout" section updated for the post-2.0 tree
|
||||
(no versioned zips, no codex tree, skills-only plugin).
|
||||
- [x] Root README "Repository layout" section updated for the post-2.0 tree
|
||||
(no versioned zips, no codex tree, skills-only plugin) — **done 2.0.0**.
|
||||
- [x] `echo-improvements-prompt.md` (the 1.5.0 field report) moved to `docs/history/`
|
||||
— **done 2026-07-03**.
|
||||
|
||||
|
||||
+30
-14
@@ -14,22 +14,36 @@ the official docs (code.claude.com/docs/en/skills.md, plugins.md):
|
||||
invocation name: `/echo-save` works identically. No deprecation date on `commands/`;
|
||||
this is future-proofing, not a fire.
|
||||
- **Do it for the features, not the notice:**
|
||||
- [ ] `allowed-tools` per skill — pre-authorize the resolved `python3 "$ECHO" …`
|
||||
invocations so `/echo-load`, `/echo-health`, `/echo-recall` stop prompting
|
||||
(1.4.2 hand-broadened two commands; the skill format does this properly).
|
||||
- [ ] `disable-model-invocation: true` on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) so only the operator triggers them; leave the
|
||||
read-side ones model-invocable.
|
||||
- [ ] De-duplicate the CoWork `$ECHO` path-resolution block currently copy-pasted
|
||||
into all eight command bodies — each skill folder can carry a shared snippet.
|
||||
- [ ] `argument-hint` carries over as-is; `$ARGUMENTS` substitution unchanged.
|
||||
- [ ] Verify a migrated build installs cleanly on desktop **and** in a CoWork session
|
||||
before deleting `commands/` (deleting the legacy dir is a 2.0 item —
|
||||
see `ROADMAP-2.0.md`; in 1.6 both may coexist).
|
||||
- [x] `allowed-tools` per skill — **done 1.6.0** (all eight skills pre-authorize
|
||||
the resolved `python3`/`python`/`py -3` invocations + the CoWork `ls` probe).
|
||||
- [x] `disable-model-invocation: true` on the write-heavy entry points
|
||||
(`/echo-sweep`, `/echo-triage`) — **done 1.6.0**; read-side stays
|
||||
model-invocable.
|
||||
- [x] De-duplicate the CoWork `$ECHO` path-resolution block — **resolved 1.6.0 as
|
||||
not-supported**: per the official skills docs, skill directories are
|
||||
self-contained (no cross-skill snippet sharing), so the 2-line block stays
|
||||
per-skill. Docs confirmed same-name skill takes precedence over the legacy
|
||||
command, so coexistence is safe.
|
||||
- [x] `argument-hint` carries over as-is; `$ARGUMENTS` substitution unchanged —
|
||||
**done 1.6.0** (save/recall/reflect).
|
||||
- [x] Verify a migrated build installs cleanly on desktop **and** in a CoWork session
|
||||
before deleting `commands/` — **verified 2026-07-28** on the baked 1.6.0-jason
|
||||
artifact, all four checks pass: (1) all eight skills register and take
|
||||
precedence over the coexisting legacy commands; (2) `allowed-tools` present on
|
||||
load/health/recall; (3) `disable-model-invocation` confirmed on exactly
|
||||
sweep+triage; (4) CoWork `$ECHO` fallback resolves via
|
||||
`/sessions/*/mnt/.remote-plugins/*` and executes. **2.0's `commands/` deletion
|
||||
is unblocked.** *Known caveat, accepted (CLI is not a normal operating surface
|
||||
here): the `allowed-tools` globs match the literal script names
|
||||
(`*echo.py*`/`*vault_lint.py*`) while the bodies invoke via the resolved
|
||||
`"$ECHO"` variable — if the Claude Code CLI matches permissions before
|
||||
variable expansion, those skills could still prompt there. Re-confirm on the
|
||||
CLI if that surface ever matters.*
|
||||
|
||||
## 2. plugin.json completeness (from the old MAINTENANCE checklist)
|
||||
|
||||
- [ ] Add `homepage` / repository URL — now decided: `https://git.alwisp.com/jason/echo`.
|
||||
- [x] Add `homepage` / repository URL — **done 1.6.0**: both `homepage` and
|
||||
`repository` set to `https://git.alwisp.com/jason/echo`.
|
||||
|
||||
## 3. Vault follow-ups (noticed during the 1.5.0 dead-link cleanup)
|
||||
|
||||
@@ -46,7 +60,9 @@ they were unwrapped per instruction but repointing them would restore real graph
|
||||
|
||||
*(add items here as the new plugin gets real use)*
|
||||
|
||||
- [ ] **Lint blind spot: retired dirs masked by the leaf-README route** (found 2026-07-03).
|
||||
- [x] **Lint blind spot: retired dirs masked by the leaf-README route** (found
|
||||
2026-07-03; **fixed 1.6.0** — retired patterns now checked before routes, with
|
||||
the `archive/notes/README.md` regression test).
|
||||
`archive/` and `_agent/outputs/` are retired/unrouted pre-0.6 leftovers that survived
|
||||
in the live vault holding only their seed READMEs — and the permissive `leaf-readme`
|
||||
route (`^(.+/)?README\.md$`) matches first, so `vault_lint` never flagged them.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"""build.py — package the echo-memory plugin source into a .plugin artifact.
|
||||
|
||||
Zips the CONTENTS of echo-memory.plugin.src/ at the archive root (the layout the plugin
|
||||
loader expects: .claude-plugin/plugin.json, commands/, skills/ all at top level), excluding
|
||||
loader expects: .claude-plugin/plugin.json, hooks/, skills/ all at top level), excluding
|
||||
dev cruft. The version is read from the manifest, so the output is named automatically.
|
||||
|
||||
Usage:
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
# echo-memory — Improvement Plans (post-1.5.1 review)
|
||||
|
||||
> Status: **planned, not started.** Source: full-code review 2026-07-28.
|
||||
> Nine improvements, each scoped for an independent build session. The tenth item
|
||||
> from that review — the **MCP server** — has its own detailed spec:
|
||||
> `docs/MCP-SERVER-SPEC.md`.
|
||||
>
|
||||
> Sequencing is at the bottom. Nothing here conflicts with `TODO-1.6.md` or
|
||||
> `ROADMAP-2.0.md`; release targets assume 1.6 (skills migration) ships first.
|
||||
|
||||
---
|
||||
|
||||
## 1. Token-budgeted load (`load --brief`) + parallel orientation reads
|
||||
|
||||
**Problem.** The SessionStart hook injects the *full text* of six files into every
|
||||
session. Live measurement: 15.9KB of hook context on a single cold start, and two of
|
||||
the six files grow without bound (operator-preferences `## Observations`,
|
||||
current-context `## Scope History`). Every session pays this tax before any work
|
||||
happens. Separately, `cmd_load` (echo.py) issues its six GETs serially even though
|
||||
`read_many()` exists.
|
||||
|
||||
**Design.**
|
||||
- New `echo.py load --brief` (and make the SessionStart hook use it):
|
||||
- marker → one line (`schema_version`, bootstrapped yes/no).
|
||||
- operator-preferences → `## Fact / Pattern` in full; `## Observations` capped at
|
||||
the last 10 lines with a `(+N older — /echo-load for full)` note.
|
||||
- current-context → `## Scope` + `scope_updated` + sessions-since (i.e. the
|
||||
`scope show` output); omit `## Scope History`.
|
||||
- heartbeat → the pointer line + only the `## Summary`/`## Outcomes` sections of
|
||||
the pointed-at session log (fetch it, extract those headings, cap ~30 lines).
|
||||
- today's daily note → `## Agent Log` lines only (or "absent").
|
||||
- inbox → count + age of oldest item, not the contents (that's all the reconcile
|
||||
needs; `/echo-triage --list` has the details).
|
||||
- Budget guard: after assembly, if the brief output still exceeds
|
||||
`ECHO_LOAD_BUDGET` chars (default ~8000), truncate lowest-priority sections first
|
||||
(observations → agent log → session summary) with explicit `(truncated)` markers.
|
||||
- Full `load` unchanged; `/echo-load` keeps using it.
|
||||
- Fetch all reads via `read_many()` (the 6 targets + the heartbeat-pointed session
|
||||
log + the sessions listing fallback) instead of the serial loop at
|
||||
`echo.py:640`. Cache-put still applies per path.
|
||||
|
||||
**Files.** `echo.py` (cmd_load), `echo_hook_session_start.py`, SKILL.md (document
|
||||
brief-vs-full), `eval/test_features.py`.
|
||||
|
||||
**Tests.** Mock vault with oversized preferences/history files → assert brief output
|
||||
under budget, sections present, truncation markers correct; hook emits brief.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** — highest leverage per line.
|
||||
|
||||
---
|
||||
|
||||
## 2. Local-first recall index
|
||||
|
||||
**Problem.** `echo_recall.update_note()` does GET-whole-index → add → PUT-whole-index
|
||||
against the vault, under the global advisory lock, on every capture and every corpus
|
||||
`put`. The index carries full BM25 postings for the whole corpus, so this round-trip
|
||||
is O(vault) network per write and is the emerging bottleneck + lock hot-spot as the
|
||||
vault grows past a few hundred notes.
|
||||
|
||||
**Design.**
|
||||
- The **live** recall index moves to the local state dir
|
||||
(`~/.echo-memory/recall-index.json`, honoring `ECHO_STATE_DIR`) keyed by endpoint
|
||||
(hash of `BASE`) so multiple vaults don't collide.
|
||||
- `update_note()` becomes a local read-modify-write (file lock via `os.O_EXCL`
|
||||
sidecar or atomic `os.replace`), **no vault round-trip, no advisory lock**.
|
||||
- The vault copy (`_agent/index/recall-index.json`) becomes a **snapshot**, written
|
||||
by `sweep.py` and at session end (piggyback on the heartbeat write / future
|
||||
`session-end` verb). It exists so a *fresh machine* can seed its local index
|
||||
without a full rebuild.
|
||||
- Freshness rule on recall: local index used when present; if the vault snapshot's
|
||||
embedded `built` stamp is newer than the local one (another client swept), pull it.
|
||||
Staleness across clients is tolerable — recall degrades gracefully and improvement
|
||||
#4 (incremental sweep) trues it up cheaply.
|
||||
- Schema bump: recall-index schema 3 adds `built` (ISO timestamp) + `endpoint_hash`.
|
||||
Old schema-2 vault copies are still readable as seeds.
|
||||
- The **entity index stays in the vault** (it is small, and it is the shared routing
|
||||
authority for concurrent clients) — only the BM25 index moves.
|
||||
|
||||
**Files.** `echo_recall.py` (load/save/update_note/rebuild), `echo_queue.py`
|
||||
(state-dir helpers shared), `sweep.py`, `echo.py` (cmd_put upkeep path), routing.json
|
||||
(the vault snapshot path is unchanged), README performance section.
|
||||
|
||||
**Tests.** Capture with vault mock → assert zero recall-index PUTs; sweep → snapshot
|
||||
written; fresh state dir + existing snapshot → seeded without rebuild; two-vault
|
||||
(endpoint) separation.
|
||||
|
||||
**Size/target.** Medium. **Pair with #4 in one index-focused minor** (they touch the
|
||||
same maintenance loop). Fine to land before or after 2.0.
|
||||
|
||||
---
|
||||
|
||||
## 3. Offline durability for the high-level ops (capture path)
|
||||
|
||||
**Problem.** The low-level verbs queue on outage via `safe_request`, but
|
||||
`_append_to_existing()` and `ensure_daily_log()` (echo_ops.py) call `echo.request`
|
||||
directly — so during an outage a raw `append` survives, while a `capture` **update**
|
||||
and its Agent Log line silently vanish. Inverted from the documented contract
|
||||
("capture is the default write").
|
||||
|
||||
**Design.** Queue *semantic* records, not byte-level requests, for the high-level
|
||||
path:
|
||||
- New outbox record type `{"op": "capture", "args": {...}}` alongside the existing
|
||||
raw records. On an unreachable vault, `capture` short-circuits: enqueue the full
|
||||
argument set (title/kind/body/tags/aliases/sources/date/domain/merge_into) and
|
||||
report `queued (offline): capture …`.
|
||||
- `flush()` replays `op:capture` records by calling `echo_ops.capture(...)` against
|
||||
the *current* index — so routing, the duplicate gate, aliasing, and linking all
|
||||
re-run with fresh state instead of replaying stale byte writes. A gate stop on
|
||||
replay (exit 76) keeps the record queued with a `needs_attention` flag and is
|
||||
surfaced by `load`/`flush` output ("1 queued capture needs --merge-into/--force").
|
||||
- Rationale for semantic-over-byte: a byte replay of `_append_to_existing`'s PATCH
|
||||
could target a heading that moved, and the create-vs-update decision made offline
|
||||
may be wrong by replay time. Re-running the op is the idempotent, correct unit
|
||||
(the dated-bullet idempotency key already prevents double-append).
|
||||
- `ensure_daily_log` stays best-effort but routes its POST/PATCH through
|
||||
`safe_request` so a standalone agent-log line survives an outage too.
|
||||
- reflect/triage `--apply` inherit this for free (they call capture).
|
||||
|
||||
**Files.** `echo_ops.py`, `echo_queue.py` (record schema v2 + replay dispatch),
|
||||
`eval/test_offline_queue.py`.
|
||||
|
||||
**Tests.** Fault-injected mock: capture-update offline → queued; flush replays via
|
||||
capture; gate-on-replay keeps record + flags; agent-log line queued; no duplicate
|
||||
bullets on double flush.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** — it's a correctness gap.
|
||||
|
||||
---
|
||||
|
||||
## 4. Incremental sweep via content hashes
|
||||
|
||||
**Problem.** Human edits in Obsidian and writes by other clients never touch the
|
||||
entity/recall indexes; drift is only corrected by a full sweep someone must remember
|
||||
to run. Full sweep re-reads the whole vault (fast now, but O(vault)).
|
||||
|
||||
**Design.**
|
||||
- Entity index entries and the recall-index snapshot gain a per-note
|
||||
`h` — SHA-1 of the note body (entity index schema 2; tolerated-absent for old
|
||||
entries).
|
||||
- `sweep.py --fast`: walk the listing (cheap), `read_many` only paths that are
|
||||
(a) new, (b) missing a stored hash, or (c) whose hash mismatches after fetch — to
|
||||
avoid fetching everything just to hash it, fast mode fetches only paths whose
|
||||
**listing is new/gone** plus a rotating shard (e.g. `hash(path) % 7 == weekday`)
|
||||
of existing notes, so the whole vault is verified over a week of fast sweeps while
|
||||
each run stays tiny. `--fast --all-shards` forces full verification.
|
||||
- Deletions: listing walk catches removed files → drop index + recall entries.
|
||||
- Auto-run: `load` (brief or full) runs `sweep --fast` opportunistically when the
|
||||
last fast-sweep stamp (state dir) is older than `ECHO_FAST_SWEEP_DAYS` (default 7),
|
||||
in the background-tolerant sense: bounded by the same read_many concurrency, and
|
||||
skipped entirely when offline.
|
||||
- `/echo-health` reports last fast/full sweep ages.
|
||||
|
||||
**Files.** `sweep.py`, `echo_index.py` (schema 2 + hash field), `echo_recall.py`
|
||||
(snapshot hash reuse), `echo.py` (load hook-in), `vault_lint.py` (report), migrate
|
||||
note (index schema is machine-owned; no vault migration needed).
|
||||
|
||||
**Tests.** Mock: edit a note out-of-band → fast sweep in that shard re-indexes it;
|
||||
delete → entry dropped; stamp gating; full sweep unchanged.
|
||||
|
||||
**Size/target.** Medium. **Ship with #2** (same schema-bump train).
|
||||
|
||||
---
|
||||
|
||||
## 5. Memory lifecycle — enforce forgetting
|
||||
|
||||
**Problem.** The vault only accretes. Working memory is "time-boxed" by convention
|
||||
only; Observations trimming is manual prose; stale-active detection only reports.
|
||||
Recall precision degrades as dead weight accumulates.
|
||||
|
||||
**Design.**
|
||||
- `capture --kind working` stamps `expires: <today + ECHO_WORKING_TTL_DAYS>`
|
||||
(default 14; `--ttl <days>` overrides). Template updated to carry the field.
|
||||
- `vault_lint.py` new checks: `expired-working` (past `expires`),
|
||||
`stale-observations` (Observations > 30 bullets) — advisory, like the rest.
|
||||
- `sweep.py --decay` (dry-run by default, `--apply` to write; **always
|
||||
preview-first**, honoring the operating contract):
|
||||
- expired working notes → `status: archived` + move body under a dated
|
||||
`## Archived` marker (never delete);
|
||||
- `projects/active/` untouched > 30 days → *propose* `on-hold` (folder move +
|
||||
status, the usual paired change) — apply only the ones confirmed;
|
||||
- Observations > 30 → trim oldest into a dated `_agent/memory/episodic/`
|
||||
overflow note (`observations-archive-YYYY.md` append), keeping the last 30
|
||||
in place. Nothing is lost, the hot file stays small (compounds with #1).
|
||||
- `/echo-health` output nudges: "run `sweep --decay` to review N aging items".
|
||||
|
||||
**Files.** `echo_ops.py` (`--ttl`/expires stamp), `echo_index.py`
|
||||
(KIND_REQUIRED_FM unchanged; working gains optional expires), `vault_lint.py`,
|
||||
`sweep.py`, scaffold working-memory template, SKILL.md (decay section),
|
||||
routing-map note.
|
||||
|
||||
**Tests.** Mock: expired note proposed+archived on apply; observations trim keeps
|
||||
last 30 and appends overflow; stale-active proposal list; dry-run writes nothing.
|
||||
|
||||
**Size/target.** Medium. Feature minor (1.7.x).
|
||||
|
||||
---
|
||||
|
||||
## 6. Contradiction / supersession handling
|
||||
|
||||
**Problem.** A new fact that contradicts a stored one lands beside it; both surface
|
||||
in recall with no signal about which is current. Biggest remaining trust gap in the
|
||||
write path.
|
||||
|
||||
**Design.**
|
||||
- **Write side:** `capture --supersedes "<slug-or-quoted-line>"`:
|
||||
- if it names an entity slug → the old entity gets `status: superseded` +
|
||||
a `superseded_by:` frontmatter path; new note carries `supersedes:` back-link
|
||||
in `## Related`. Recall's STATUS_FACTOR gains `superseded: 0.5`.
|
||||
- if it quotes a Fact/Pattern or Observations line in operator-preferences →
|
||||
the line is struck (`~~old~~ (superseded YYYY-MM-DD)`) and the new line
|
||||
appended, one PATCH each, in the same invocation.
|
||||
- **Detect side (deterministic assist, model decides):** `echo_reflect.classify`
|
||||
gains a `conflict scan` — for each proposal, run its distinctive tokens
|
||||
(`_sig_tokens`) against the Fact/Pattern + recent Observations lines; overlapping
|
||||
lines are attached as `_conflicts: [...]` and the preview row renders
|
||||
`update-or-conflict?` with the matched line(s). The model (which has the
|
||||
conversation) decides supersede-vs-append; the script only guarantees the
|
||||
question gets asked before the write.
|
||||
- Lint: `superseded_by` pointing at a missing note → violation.
|
||||
|
||||
**Files.** `echo_ops.py`, `echo_reflect.py`, `echo_recall.py` (STATUS_FACTOR),
|
||||
`echo.py` (flag plumbing), `vault_lint.py`, SKILL.md + operating-contract
|
||||
(supersession is additive: strike, never delete).
|
||||
|
||||
**Tests.** Mock: supersede entity → status/backlinks both sides; supersede line →
|
||||
strike + append idempotent; reflect preview flags a planted conflict; recall ranks
|
||||
superseded below active twin.
|
||||
|
||||
**Size/target.** Medium. Feature minor (1.7.x/1.8) — after #5 (shares the
|
||||
status-vocabulary touchpoints).
|
||||
|
||||
---
|
||||
|
||||
## 7. `session-end` bundle verb
|
||||
|
||||
**Problem.** Ending a session correctly is 4+ invocations (session-log PUT, Agent
|
||||
Log append, heartbeat PUT, optional scope set, optional reflect apply). Cost per
|
||||
session, and a partial failure leaves broken orientation state (log written,
|
||||
heartbeat stale).
|
||||
|
||||
**Design.**
|
||||
- `echo.py session-end <bundle.json> [--apply]`, bundle:
|
||||
```json
|
||||
{
|
||||
"slug": "echo-mcp-spec",
|
||||
"log_body": "<full session-log markdown, frontmatter included>",
|
||||
"agent_log_line": "- 2026-07-28: wrote MCP spec [[...]]",
|
||||
"scope": "optional new scope text",
|
||||
"reflect": [ { ...PROPOSAL_SCHEMA... } ]
|
||||
}
|
||||
```
|
||||
- Dry-run previews the whole plan (paths + reflect preview). `--apply`, in order,
|
||||
under **one** lock acquisition: PUT `_agent/sessions/<date>-<HHMM>-<slug>.md`
|
||||
(HHMM from `ECHO_NOW` env or wall clock, validated against the canonical regex) →
|
||||
agent-log line (ensure_daily_log) → reflect proposals via capture → scope set if
|
||||
given → heartbeat PUT **last** (it's the commit marker: a crash before it leaves
|
||||
the previous pointer intact, never a dangling one).
|
||||
- Reports per-step ok/queued/gated; offline → every step queues (rides on #3).
|
||||
- Stop hook reason text updated to name the one command; `/echo-reflect` docs point
|
||||
session logging at it. **Prerequisite for the MCP `echo_log_session` tool** —
|
||||
build this verb first so CLI and MCP share one implementation.
|
||||
|
||||
**Files.** new `echo_session.py`, `echo.py` (subcommand), `echo_hook_stop.py`
|
||||
(REASON text), SKILL.md (Session Logging section), commands/echo-reflect.md.
|
||||
|
||||
**Tests.** Mock: full bundle lands all five artifacts; heartbeat-last ordering
|
||||
(inject failure mid-way → heartbeat untouched); dry-run writes nothing; offline
|
||||
queues all steps.
|
||||
|
||||
**Size/target.** Small-medium. **1.6.x point release** (and before the MCP build).
|
||||
|
||||
---
|
||||
|
||||
## 8. Recall robustness — stemming + alias query expansion
|
||||
|
||||
**Problem.** BM25 is purely lexical: "deploy"/"deployment"/"deployed" are distinct
|
||||
terms; a paraphrased query misses. The 8-query gold set can't see this class of miss.
|
||||
|
||||
**Design.**
|
||||
- **Stemming:** a light suffix-stripper (Porter-lite: s/es/ed/ing/ly/tion/ment/ness
|
||||
families, ~40 lines, pure stdlib) applied in `tokenize()` at **both** index and
|
||||
query time. Index schema bump (stemmed postings) → auto-rebuild on next
|
||||
recall/sweep, exactly like the 1→2 transition.
|
||||
- **Alias query expansion:** at recall time, run the query against
|
||||
`fuzzy_candidates()` (top 2, score ≥ 0.5); fold the matched entities' title +
|
||||
alias tokens into the BM25 query as **down-weighted expansion terms** (×0.5 idf
|
||||
contribution) so "the ECHO plugin" also scores `echo-memory`-alias vocabulary.
|
||||
Exact-match seeding stays as-is; expansion never *creates* hits ranked above
|
||||
genuine lexical matches without corpus support.
|
||||
- **Eval:** grow the gold set with ≥6 paraphrase queries (morphological variants +
|
||||
alias phrasings); publish before/after recall@5/MRR in the README table.
|
||||
|
||||
**Files.** `echo_recall.py` (tokenize, INDEX_SCHEMA, score, recall), new
|
||||
`echo_stem.py` (pure function, unit-testable), `eval/run_eval.py` + gold set.
|
||||
|
||||
**Tests.** Stemmer unit table (word → stem, incl. non-cases); paraphrase gold
|
||||
queries pass; schema-2 index discarded and rebuilt cleanly.
|
||||
|
||||
**Size/target.** Small-medium. Ride the **same index-schema train as #2/#4**.
|
||||
|
||||
---
|
||||
|
||||
## 9. Resolve-miss telemetry → alias suggestions
|
||||
|
||||
**Problem.** Every resolve that misses but ends in a confirmed candidate
|
||||
(`--merge-into`, or the operator picking a candidate) is a free mention→entity
|
||||
training pair, currently discarded.
|
||||
|
||||
**Design.**
|
||||
- `echo_index.resolve()` miss + `fuzzy_candidates` non-empty → append one NDJSON
|
||||
record to the local state dir (`~/.echo-memory/resolve-misses.ndjson`):
|
||||
`{mention, candidates:[slugs], ts?}` (no wall clock needed — session date via
|
||||
ECHO_TODAY). `capture --merge-into <slug>` appends the *confirmation* record
|
||||
`{mention, resolved: slug}` (this already learns the alias immediately — that
|
||||
path stays; the log covers the cases that never reach --merge-into).
|
||||
- `sweep.py` (and `/echo-health`) read the log: any `{mention → slug}` pair
|
||||
confirmed ≥2 times, or any mention that repeatedly fuzzy-matched a single
|
||||
candidate ≥3 times without confirmation, is **proposed** as an alias
|
||||
(`sweep --apply` writes it into the note's `aliases:` frontmatter + index, the
|
||||
existing fold-back path). Log entries consumed on apply; file capped (rotate at
|
||||
~500 lines).
|
||||
- Privacy note: mentions can contain operator phrasing — the log lives in the
|
||||
state dir (never the vault), same trust level as the offline queue.
|
||||
|
||||
**Files.** `echo_index.py` (log hooks), `echo_ops.py` (confirmation hook),
|
||||
`sweep.py` (suggest/apply), `vault_lint.py` or sweep report (surface count),
|
||||
SKILL.md one-liner.
|
||||
|
||||
**Tests.** Miss → record written; confirm ×2 → sweep proposes; apply folds alias +
|
||||
consumes log; cap/rotation.
|
||||
|
||||
**Size/target.** Small. Feature minor, pairs naturally with #8.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing (unified with TODO-1.6 and ROADMAP-2.0)
|
||||
|
||||
TODO-1.6 folds in as the opening train — the skills migration restructures the
|
||||
files everything below touches, so it goes first. ROADMAP-2.0 stays a **pure
|
||||
packaging major** (its own stated thesis) and is *not* folded into any feature
|
||||
train; it slots between the quick wins and the MCP server so the MCP work lands
|
||||
on the final packaging baseline (single command format, Gitea releases, one
|
||||
manifest-verification pass).
|
||||
|
||||
| Release | Items | Rationale |
|
||||
|---|---|---|
|
||||
| **1.6.0** | TODO-1.6: skills-format migration (+ per-skill `allowed-tools`, `disable-model-invocation` on write-heavy entries, `$ECHO` block dedupe) · lint blind-spot fix (retired-dir/leaf-README) · plugin.json `homepage` | Restructures the command/SKILL files later work edits; allowed-tools removes prompt friction for every verb added below. Vault link repointing (TODO-1.6 §3) is vault data, not plugin code — any session. |
|
||||
| **1.6.0 — SHIPPED 2026-07-28** | Skills migration + lint fix + homepage | Verified on desktop + CoWork. |
|
||||
| **2.0.0 — SHIPPED 2026-07-28** | ROADMAP-2.0: commands/ deleted · release-based artifacts · gitignore · Codex note | Packaging-only major; skills-only install re-verified. |
|
||||
| **2.1.0 quick wins — SHIPPED 2026-07-28** | #1 brief load · #3 offline capture · #7 session-end | All three landed with +19 e2e checks; see CHANGELOG 2.1.0. The MCP spec's Phase 0 (return-not-print) did NOT ride along — it's the first phase of the MCP build session. |
|
||||
| **2.2** | MCP server (containerized, `echomcp.alwisp.com`) — `docs/MCP-SERVER-SPEC.md` | Needs Phase 0 (return-not-print refactor, first step of the build session). #7 session-end shipped, so `echo_log_session` wraps a real verb. Deployed via CI + PORT, not the plugin manifest. |
|
||||
| **2.3 index train** | #2 local-first recall (CLI-path half) · #4 incremental sweep · #8 stemming+expansion | One schema-bump train; all touch the same index/sweep loop. The container already keeps its index warm in memory (spec §7), so #2 here covers the CLI fallback path; #4's fast sweep also becomes the container's background job. |
|
||||
| **2.4 memory train** | #5 lifecycle/decay · #6 supersession · #9 resolve-miss learning | Capability tier; #6 after #5 (status vocabulary). |
|
||||
@@ -0,0 +1,391 @@
|
||||
# echo-mcp — MCP Server Build Spec (containerized)
|
||||
|
||||
> Status: **spec for a future build session** (written 2026-07-28 against v1.5.1;
|
||||
> revised same day: **remote container architecture**, operator decision — heavy
|
||||
> lifting belongs in a deployed container, not on any one machine).
|
||||
> Companion plan for the other nine review items: `docs/IMPROVEMENT-PLANS.md`.
|
||||
>
|
||||
> **Prerequisites before starting this build:**
|
||||
> 1. The `session-end` verb (IMPROVEMENT-PLANS #7) — the MCP tool wraps it.
|
||||
> 2. Phase 0 below (the return-not-print refactor) — lands in the plugin tree
|
||||
> first and is independently shippable.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why an MCP server
|
||||
|
||||
Every ECHO operation today rides through Bash: resolve `$ECHO` (with the CoWork
|
||||
path-fallback snippet copy-pasted everywhere), quote a Python invocation, run
|
||||
it, re-parse stdout. Costs:
|
||||
|
||||
- **Tokens** — each memory op carries bash scaffolding + output re-parsing; a
|
||||
capture is ~3× the tokens of a typed tool call.
|
||||
- **Fragility** — quoting hazards, the `${CLAUDE_PLUGIN_ROOT}` sandbox mismatch
|
||||
(the whole 1.4.2 release), Windows `python3`-vs-`python` dispatch.
|
||||
- **Reach** — surfaces without a Python-capable shell can't use ECHO at all.
|
||||
- **Efficiency** — every CLI invocation is a cold Python start that re-reads the
|
||||
entity index from the vault over HTTPS.
|
||||
|
||||
What the server does **not** replace: SKILL.md remains the procedure authority
|
||||
(when to load, reconcile, search-first, third-person, etc.). The server replaces
|
||||
the *mechanics* — tools are the verbs, the skill is the discipline.
|
||||
|
||||
## 2. Architecture: remote container (decided)
|
||||
|
||||
A standalone Docker container on the Unraid box (ALPHA), published behind
|
||||
Cloudflare + NPM as **`echomcp.alwisp.com`**, speaking **MCP streamable HTTP
|
||||
(stateless JSON)** with bearer-token auth. Chosen over the earlier local-stdio
|
||||
draft for four reasons:
|
||||
|
||||
1. **The backend is already remote.** Clients round-trip to
|
||||
`echoapi.alwisp.com` today; a server co-located with the vault host turns
|
||||
every vault op into a LAN hop and collapses N client→vault calls into one
|
||||
client→server call.
|
||||
2. **Any device, no per-machine anything.** Desktop, CoWork sandbox, claude.ai
|
||||
(custom connector), mobile — same URL + token. The `${CLAUDE_PLUGIN_ROOT}`
|
||||
CoWork registration risk from the stdio draft disappears entirely.
|
||||
3. **Credentials consolidate server-side.** The vault key lives in the
|
||||
container env (PORT secret store); the image is credential-free; clients
|
||||
hold only an MCP bearer token. The per-user baked-key builds (1.4.x) become
|
||||
unnecessary for any surface that has MCP. Rotation = redeploy.
|
||||
4. **Room to grow.** The plugin's pure-stdlib constraint exists because its
|
||||
scripts run in arbitrary sandboxes. The container is ours: real SDK,
|
||||
SQLite/in-memory indexes that stay warm, background jobs, embeddings later —
|
||||
all without touching the plugin.
|
||||
|
||||
**What stays client-side (unchanged and shipped):** the skill + CLI path — it
|
||||
is the fallback when the server or network is down, the offline queue + read
|
||||
cache keep their per-machine role there, and the SessionStart/Stop **hooks stay
|
||||
CLI-based** (hooks run shell commands regardless of MCP).
|
||||
|
||||
**Cost acknowledged:** a second service to run and an exposed auth surface.
|
||||
Mitigations are existing infra: CF + NPM cert, long random bearer token, Kuma
|
||||
monitor, PORT deploy/rollback. Server down ⇒ exactly today's behavior.
|
||||
|
||||
## 3. Decisions (made — don't relitigate in the build session)
|
||||
|
||||
| Decision | Choice | Why |
|
||||
|---|---|---|
|
||||
| Language | **Python** | Imports the existing `echo_*.py` modules in-process — the entire ops layer is reused, not wrapped. |
|
||||
| SDK | **Official `mcp` Python SDK (FastMCP)** | The stdlib-only rule doesn't bind inside our own image. Hand-rolling the protocol (stdio draft) is no longer justified. |
|
||||
| Transport | **Streamable HTTP, stateless JSON** | Remote, multi-client, simplest to scale; SSE/sessions explicitly avoided. |
|
||||
| Auth | **Static bearer token** (`ECHO_MCP_TOKEN`), constant-time compare, 401 otherwise | Single-operator service; OAuth is overkill. Token from the PORT secret store. |
|
||||
| Vault credentials | Container env: `ECHO_BASE`, `ECHO_KEY`, `ECHO_OWNER` (SECRET: refs in the PORT template) | `echo_config.py` already resolves env-first — zero code change. Image ships no secrets. |
|
||||
| Server name / repo home | `echo-mcp`; lives in this repo under `mcp-server/` (own Dockerfile), deployed from `git.alwisp.com/jason/echo` CI | One canonical tree — the server vendors `skills/echo-memory/scripts/` into the image at build time; no second hand-maintained copy (the Codex-drift lesson). |
|
||||
| Tool prefix | `echo_` | Namespace safety next to other servers. |
|
||||
| Result shape | `structuredContent` (the `echo_output.envelope` dict) + a 1–3 line human text block | Envelope shape already exists. |
|
||||
| Statefulness | Stateless protocol; warm in-process caches | See §7. |
|
||||
| Version target | **2.2** (1.6.0, 2.0.0, and the 2.1.0 quick-wins train all shipped 2026-07-28; `session-end` exists) | Remaining prerequisite: Phase 0 (return-not-print), the build session's first step. |
|
||||
| Local stdio variant | **Dropped for v1** (documented, not built) | The CLI/skill fallback covers the no-server case; two transports = two test matrices for little gain. Phase 0 keeps the door open — the `*_op` core is transport-agnostic. |
|
||||
| Result budgets | Every read tool takes an explicit size budget (`budget_chars` / `max_chars`) and truncates with a marker + "call X for more" | The single biggest token lever: one right-sized answer instead of follow-up fetches. |
|
||||
| Tool profiles | `ECHO_MCP_TOOLS=core\|full` env: `core` exposes only load/recall/capture/triage_inbox/log_session/health | Tool schemas cost context on every surface that lists them; the claude.ai connector doesn't need the escape hatches. Desktop registers `full`. |
|
||||
|
||||
## 4. Phase 0 — the return-not-print refactor (prerequisite, lands in the plugin)
|
||||
|
||||
Unchanged from the stdio draft, and still first: the ops layer prints results
|
||||
(even `--json` mode prints from inside `echo_ops.capture`) and returns exit
|
||||
codes; the server needs return values.
|
||||
|
||||
1. Each high-level op gets a **core function returning an envelope dict**,
|
||||
raising `echo.EchoError` on failure; CLI entry points become thin printing
|
||||
wrappers:
|
||||
|
||||
| Module | New core fn | CLI wrapper keeps |
|
||||
|---|---|---|
|
||||
| `echo_ops` | `capture_op`, `resolve_op`, `link_op` | `capture/resolve/link` |
|
||||
| `echo_recall` | `recall_op(query, limit)` | `recall` |
|
||||
| `echo_triage` | `list_op()` · `route_op(proposals, apply)` | `list_inbox/apply` |
|
||||
| `echo_reflect` | `apply_op(proposals, apply)` | `apply` |
|
||||
| `echo.py` | `load_op(brief)` · `scope_show_op/scope_set_op` · `doctor_op` | `cmd_*` |
|
||||
| `echo_session` (new, #7) | `session_end_op(bundle, apply)` | subcommand |
|
||||
|
||||
2. Special results are **data, not exit codes**: duplicate gate ⇒
|
||||
`{ok: false, action: "duplicate-gate", candidates}` (CLI maps to exit 76);
|
||||
offline queueing ⇒ `{ok: true, queued: true}`.
|
||||
3. Helper chatter (`ok: PUT …`) moves behind a `notify(msg)` callback
|
||||
defaulting to stderr; delete the `redirect_stdout` hack in `capture`.
|
||||
4. Tests: existing suites pass unchanged; new unit tests hit `*_op` directly
|
||||
against the mock and assert envelopes.
|
||||
|
||||
Shippable on its own as a 1.6.x refactor with zero behavior change.
|
||||
|
||||
## 5. Server application (`mcp-server/app.py`)
|
||||
|
||||
- **FastMCP** app, streamable HTTP, stateless. One `@mcp.tool` per tool in §6,
|
||||
each a thin adapter: validate → call the `*_op` core → wrap.
|
||||
- **Auth middleware**: require `Authorization: Bearer <ECHO_MCP_TOKEN>` on every
|
||||
request (constant-time compare); 401 with no detail otherwise. Additionally
|
||||
bind the app to the container interface only; exposure policy lives at
|
||||
NPM/Cloudflare.
|
||||
- **Result wrapper**:
|
||||
|
||||
```python
|
||||
def tool_result(env: dict, summary: str) -> ...:
|
||||
# content: [{type: "text", text: summary}] (1–3 lines, human)
|
||||
# structuredContent: env (echo_output envelope)
|
||||
# isError: not env.get("ok", True) — except the deliberate non-errors below
|
||||
```
|
||||
|
||||
- **Error map** (tool-level results, never protocol errors):
|
||||
|
||||
| Condition | Result |
|
||||
|---|---|
|
||||
| Vault unreachable on a **read** | `isError: true` — "vault unreachable (Obsidian/REST plugin likely down); proceed without memory, writes will queue server-side". |
|
||||
| Vault unreachable on a **write** | **Not an error**: `{ok: true, queued: true}` — the server-side outbox (§7) replays when the vault returns. |
|
||||
| 404 | `isError: true`, `{code: "not-found", path}`. |
|
||||
| Duplicate gate | **Not an error**: `{ok: false, action: "duplicate-gate", candidates}` + text naming the two resolutions (`merge_into` / `force`) — the model must decide, not blind-retry. |
|
||||
| Lock held | `{ok: false, action: "lock-held", holder}`. |
|
||||
| Misconfigured deployment (no `ECHO_BASE`/`ECHO_KEY`) | `isError: true` — "server deployment is missing vault credentials — operator: check the PORT template env". Startup also fails loudly (see §8 healthcheck). |
|
||||
| Anything else | `isError: true`, first line only; traceback to the container log. |
|
||||
|
||||
- **Validation**: FastMCP/pydantic handles types/enums/required; add
|
||||
`model_config = ConfigDict(extra="forbid")` so unknown params (model typos)
|
||||
fail with a field-naming message.
|
||||
|
||||
## 6. Tool surface (13 tools)
|
||||
|
||||
Same surface as the stdio draft **minus `echo_configure`** — with a remote
|
||||
server there is no per-machine config to import; credentials are
|
||||
deployment-side. Descriptions are written for the model: what it does, when to
|
||||
use it, what it returns, ≤3 sentences.
|
||||
|
||||
### 6.1 Orientation & read
|
||||
|
||||
- **`echo_load`** — cold-start orientation. `brief: bool = true`
|
||||
(IMPROVEMENT-PLANS #1 digest; `false` = full six-file dump). Returns
|
||||
`{sections: {marker, preferences, scope, last_session, today, inbox_count,
|
||||
inbox_oldest_days}, queued_flushed, offline}`. Also flushes the server-side
|
||||
outbox. `readOnlyHint: false` (flush), `openWorldHint: true`.
|
||||
- **`echo_recall`** — `query: str`, `limit: int = 6 (1–20)`,
|
||||
`include_linked: bool = true`, `budget_chars: int = 4000 (500–20000)`. Returns
|
||||
the `recall --json` shape (`primary` / `linked` with
|
||||
path/score/type/updated/status/excerpt) **packed to the budget**: the server
|
||||
allocates excerpt space by score, so the model gets the most relevant content
|
||||
in one right-sized answer instead of follow-up fetches. Description: truncated
|
||||
hits are marked — call `echo_get_note` for full content.
|
||||
`readOnlyHint: true`, `idempotentHint: true`.
|
||||
- **`echo_resolve`** — `mention: str` → match `{slug, path, kind, title,
|
||||
aliases}` or `{match: false, suggest_slug, candidates}`. "Call before creating
|
||||
any note by hand; `echo_capture` does this automatically." `readOnlyHint: true`.
|
||||
- **`echo_get_note`** — `path: str` (vault-relative; reject `..`/leading `/`;
|
||||
warn-not-block on paths outside `routing.json`); `section: str = null`
|
||||
(a heading name — return only that section, e.g. `Status`);
|
||||
`max_chars: int = 8000`. Returns `{path, content, frontmatter, truncated?}`.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_get_scope`** — `{scope, scope_updated, sessions_since}`; description
|
||||
tells the model to confirm scope with the operator when `sessions_since ≥ 3`.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_set_scope`** — `scope: str`; atomic switch (history + replace +
|
||||
stamp). `idempotentHint: true`.
|
||||
- **`echo_health`** — `deep: bool = false`. Shallow = doctor checks (vault
|
||||
reachability, auth, marker/schema, outbox depth). Deep = full `vault_lint`
|
||||
→ `{violations: [{check, path, detail}]}`. `readOnlyHint: true`.
|
||||
|
||||
### 6.2 Write
|
||||
|
||||
- **`echo_capture`** — the default write (route + frontmatter + index +
|
||||
auto-link + agent-log in one call). Params: `title` (req); `kind:
|
||||
enum[person, company, concept, reference, meeting, project, area, semantic,
|
||||
episodic, working, skill, decision]` (omit ⇒ inbox line); `body: str = ""`
|
||||
(markdown, inline); `tags/aliases/sources: str[]`; `status: str`;
|
||||
`date: YYYY-MM-DD` (meeting/decision); `domain: enum[business, personal,
|
||||
learning, systems] = business` (area); `merge_into: str` (slug);
|
||||
`force: bool = false`; `dry_run: bool = false`. Returns the capture envelope
|
||||
(`action: created|updated|inbox|duplicate-gate`, `path`, `links_added`,
|
||||
`near_duplicates?`, `candidates?`). Description spells out the gate contract
|
||||
(never blind-retry; `merge_into` or confirmed `force`). `destructiveHint:
|
||||
false`, `idempotentHint: true`.
|
||||
- **`echo_link`** — `a`, `b` (paths **or resolvable names** — server resolves
|
||||
via the index). Returns `{a, b, a_changed, b_changed}`. `idempotentHint: true`.
|
||||
- **`echo_append_note`** — `path`, `line`; whole-line idempotent append.
|
||||
- **`echo_patch_note`** — `path`, `operation: enum[append, prepend, replace]`,
|
||||
`target_type: enum[heading, frontmatter, block]`, `target`, `content`.
|
||||
Description carries the hard-won rules: heading targets are the full
|
||||
`::`-delimited path from the H1; on a 400 invalid-target the server fetches
|
||||
the document map and returns the actual heading list in the error — the worst
|
||||
silent-loss failure becomes self-correcting. `destructiveHint: true`.
|
||||
(Deliberately **no `echo_put_note` / `echo_delete_note`** in v1 — whole-file
|
||||
overwrite and deletion stay behind the CLI + operator explicitness; session
|
||||
logs go through `echo_log_session`.)
|
||||
- **`echo_triage_inbox`** — `proposals: object[] = []` (PROPOSAL_SCHEMA +
|
||||
optional `line`), `apply: bool = false`. Empty ⇒ structured listing; with
|
||||
proposals ⇒ preview, then route + processing-log audit on `apply: true`.
|
||||
"List → propose → preview → apply only after the operator confirms."
|
||||
- **`echo_reflect`** — `proposals: object[]` (req), `apply: bool = false`. Same
|
||||
preview/apply contract as the CLI; description restates: never apply without
|
||||
the operator's go-ahead, never invent memories.
|
||||
- **`echo_log_session`** — the session-end bundle (wraps `session_end_op`, #7):
|
||||
`slug`, `log_body`, `agent_log_line`, `scope?: str`, `reflect?: object[]`,
|
||||
`apply: bool = false`. Per-step results; heartbeat written last as the commit
|
||||
marker.
|
||||
|
||||
### 6.3 v1.1 tools (specced now, built after v1 ships)
|
||||
|
||||
- **`echo_rollup_data`** — `period: enum[week, month]`, `date: YYYY-MM-DD`.
|
||||
Assembles the rollup *digest data* in one call: open threads across
|
||||
`projects/active/` (each note's `## Status` + `updated:`), inbox items aging
|
||||
past 7 days, and the period's `## Scope History` entries. The model writes
|
||||
the prose; the server does the N fetches. Same division of labor as reflect.
|
||||
`readOnlyHint: true`.
|
||||
- **`echo_note_history`** — `path: str` → `{versions: [{id, ts, bytes,
|
||||
summary_line}]}` from the server's shadow write history (§7). `readOnlyHint:
|
||||
true`.
|
||||
- **`echo_restore_note`** — `path: str`, `version_id: str`,
|
||||
`confirm: bool = false` (preview diff unless confirmed). The undo for a bad
|
||||
`replace` PATCH or merge PUT. Description: operator confirmation required
|
||||
before calling with `confirm: true`. `destructiveHint: true`.
|
||||
|
||||
Deliberately absent from v1 (documented in the server README): sweep,
|
||||
bootstrap, migrate, lock/unlock (vault-wide maintenance stays operator-initiated
|
||||
via slash commands — routine sweeps run as background jobs instead), raw
|
||||
search/ls/map (recall/resolve/get_note cover reads; add only if transcripts
|
||||
show the need), and reflect *extraction* (only the model has the conversation —
|
||||
that division of labor is permanent, not a v1 scope cut).
|
||||
|
||||
## 7. Server internals — where the container earns its keep
|
||||
|
||||
### 7.1 v1
|
||||
|
||||
- **Warm entity index**: loaded once, invalidated on any index-writing tool and
|
||||
on a short TTL (`ECHO_MCP_INDEX_TTL`, default 60s) to pick up other clients'
|
||||
writes. Ends the per-invocation index re-read entirely.
|
||||
- **Recall index in memory (+ SQLite file at `/data`)**: the BM25 index lives
|
||||
in process memory, persisted to the container volume — recall answers in
|
||||
milliseconds with **zero** vault round-trips. This *supersedes the
|
||||
server-side half of IMPROVEMENT-PLANS #2*; #2's local-state-dir design still
|
||||
applies to the CLI fallback path.
|
||||
- **Note read cache**: short-TTL (30–60s) body cache for `get_note` and
|
||||
recall's neighbourhood expansion — repeated same-session reads of the same
|
||||
files stop hitting the vault at all.
|
||||
- **Per-path write serialization**: an internal per-path mutex serializes
|
||||
conflicting MCP-path writes — since all server-mediated writes flow through
|
||||
one process, the advisory-lock race window disappears for them. The server
|
||||
still takes the vault advisory lock around index updates to coordinate with
|
||||
CLI clients; idempotent appends stay as the last line of defense.
|
||||
- **Server-side outbox**: the existing `echo_queue` pointed at `/data`
|
||||
(`ECHO_STATE_DIR=/data`) — writes queue when the vault is down and flush on
|
||||
recovery/load. One queue at the always-on host instead of per-laptop.
|
||||
- **Concurrency**: FastMCP serves requests concurrently; guard the in-process
|
||||
caches with a plain `threading.Lock`; write integrity per the bullet above.
|
||||
- **Logging**: one line per call to stdout (container log): tool, ms,
|
||||
ok/queued/gated/error. Never log bodies or keys.
|
||||
|
||||
### 7.2 v1.1 container dividends (backlog, in priority order)
|
||||
|
||||
- **Nightly vault backup** — the vault currently has **no disaster-recovery
|
||||
story**; if the Obsidian host dies, memory is gone. A timer job walks the
|
||||
vault via `read_many`, writes a dated zip to `/data/backups/` (rotate 30),
|
||||
and reports the last-backup age in `/health`. Zero tokens; arguably the
|
||||
highest-value item in this spec.
|
||||
- **Shadow write history + undo** — before any PUT / PATCH-replace the server
|
||||
passes through, snapshot the prior body to `/data/history/` (content-
|
||||
addressed, per-path ring of ~20 versions). Backs the `echo_note_history` /
|
||||
`echo_restore_note` tools (§6.3). The vault's additive philosophy finally
|
||||
gets an undo for its residual risk: a bad `replace` or merge PUT.
|
||||
- **Background maintenance jobs** — a timer thread running `sweep --fast`
|
||||
(IMPROVEMENT-PLANS #4) hourly and the decay pass (#5) proposals weekly; both
|
||||
publish findings to `/health` and the vault-health note rather than
|
||||
auto-fixing.
|
||||
- **Ops alerting** — outbox stuck > N hours, vault unreachable > 1h, new lint
|
||||
violations after a background sweep ⇒ notify via the existing Unraid
|
||||
notification plumbing (Kuma already watches `/health` for liveness).
|
||||
Problems surface without anyone polling.
|
||||
- **Embeddings (explicitly deferred)**: the container is where an embedding
|
||||
recall tier would land later (model API or local), as an image upgrade —
|
||||
no plugin change. Noted so nobody bolts it into the plugin.
|
||||
|
||||
## 8. Container & deployment
|
||||
|
||||
- **Image**: `python:3.12-slim` (not alpine — no musl surprises), `pip install
|
||||
mcp` pinned, copy `skills/echo-memory/scripts/` + `mcp-server/` in.
|
||||
**Dockerfile must be legacy-format** — the git.alwisp.com CI runner has no
|
||||
BuildKit: no `# syntax=` line, no `RUN --mount`.
|
||||
- **Healthcheck**: probe `http://127.0.0.1:<port>/health` — **127.0.0.1, not
|
||||
localhost** (the ::1-vs-IPv4 lesson from cpas/memer/breedr). `/health`
|
||||
(unauthenticated, no vault data): `{ok, vault_reachable, outbox_depth,
|
||||
index_age_s}` — also the Kuma target.
|
||||
- **Env** (via the PORT template, secrets as `SECRET:` refs): `ECHO_BASE`,
|
||||
`ECHO_KEY`, `ECHO_OWNER`, `ECHO_MCP_TOKEN`, `ECHO_STATE_DIR=/data`,
|
||||
`ECHO_MCP_TOOLS=full` (or `core` — see §3 tool profiles), optional
|
||||
`ECHO_WORKERS/ECHO_TIMEOUT/ECHO_MCP_INDEX_TTL`. Volume: `/data`
|
||||
(outbox + recall index + backups + shadow history).
|
||||
- **Startup validation**: fail fast (non-zero exit) if `ECHO_BASE`/`ECHO_KEY`/
|
||||
`ECHO_MCP_TOKEN` are missing — a misdeployed container should crash-loop
|
||||
visibly, not serve errors.
|
||||
- **CI/deploy**: Gitea workflow on push-to-main (tags/releases do **not**
|
||||
autobuild on this host) builds the image and the PORT autodeploy webhook
|
||||
rolls it out; `deploy.unraid.yml` manifest in `mcp-server/`. Publish
|
||||
`echomcp.alwisp.com` via the usual CF + NPM flow; add the Kuma monitor.
|
||||
- **Vault adjacency (confirmed: Obsidian runs on ALPHA)**: the Local REST
|
||||
API's non-encrypted HTTP binding is enabled (2026-07-28) —
|
||||
**`ECHO_BASE=http://10.2.0.35:27123`** — so the container talks to Obsidian
|
||||
directly, never through the public `echoapi.alwisp.com` hairpin. All vault
|
||||
chatter behind a tool call stays host/LAN-local; vault ops stop depending on
|
||||
Cloudflare/DNS/NPM health entirely (the public chain is only needed for the
|
||||
`echomcp` ingress). Why HTTP not HTTPS here: the REST API's HTTPS port
|
||||
(27124) uses a self-signed cert, which `echo.py`'s default-verifying
|
||||
`HTTPSConnection` rejects — cleartext on the internal network beats adding an
|
||||
insecure-TLS flag to the client. **Exposure check (do once):** 27123 carries
|
||||
the vault bearer token in cleartext, so it must not be reachable from
|
||||
outside — confirm no router/firewall forward for 27123; the only public
|
||||
doors stay HTTPS 443 via NPM (`echoapi` for the CLI fallback path, which
|
||||
remains unchanged, and `echomcp` for this server).
|
||||
|
||||
## 9. Client registration & fallback
|
||||
|
||||
- **Claude Code / CoWork**: register as a remote MCP server —
|
||||
`https://echomcp.alwisp.com/mcp` with the `Authorization: Bearer` header
|
||||
(project or user scope). No plugin-manifest coupling; the plugin does **not**
|
||||
register the server.
|
||||
- **claude.ai**: custom connector with the same URL + token — ECHO memory from
|
||||
the browser/phone, a surface the plugin could never reach.
|
||||
- **SKILL.md** gains: *"When the `echo_*` MCP tools are available, prefer them
|
||||
for every operation they cover; the `$ECHO` CLI recipes are the fallback for
|
||||
hosts without the connector or when the server is unreachable."* Procedures
|
||||
unchanged.
|
||||
- **Fallback matrix**: server up ⇒ tools. Server down, machine has the plugin ⇒
|
||||
CLI path exactly as today (including its own offline queue). Both down ⇒
|
||||
today's "vault unreachable, proceed without memory".
|
||||
|
||||
## 10. Multi-user note (Andy / Bryan / Gretchen / Arsalan)
|
||||
|
||||
One container serves **one vault**. The image is credential-free, so additional
|
||||
users are additional deployments of the same image with their own env (their
|
||||
vault endpoint/key + their own `ECHO_MCP_TOKEN`), e.g. `echomcp-bryan` on
|
||||
another port/subdomain. Cheap, isolated, no code change. A single multi-tenant
|
||||
server (token → vault map) is deliberately out of scope — that's CHORUS's
|
||||
territory.
|
||||
|
||||
## 11. Testing & eval
|
||||
|
||||
1. **Protocol/integration tests** (`eval/test_mcp_server.py`): run the FastMCP
|
||||
app in-process (or via `httpx` test client) against `mock_olrapi`; assert
|
||||
initialize/tools-list schemas, auth (no token ⇒ 401; bad token ⇒ 401;
|
||||
`/health` open), envelope purity on every tool.
|
||||
2. **Tool behavior**: per tool — happy path, validation failure (unknown param
|
||||
rejected with field name), vault-down (writes ⇒ `queued: true`, reads ⇒
|
||||
actionable error), duplicate-gate flow (gate → `merge_into` retry succeeds).
|
||||
3. **Parity**: `capture` via MCP and via CLI against twin mock vaults produce
|
||||
byte-identical notes/index entries (guards Phase 0).
|
||||
4. **Container smoke** (CI): build image, start with mock env, `/health` green,
|
||||
one authed `tools/call` round-trip; healthcheck probes 127.0.0.1.
|
||||
5. **MCP Inspector** manual pass against the deployed container.
|
||||
6. **Eval set** (mcp-builder Phase 4): 10 read-only Q&A pairs against a seeded
|
||||
mock vault exercising recall→get_note→resolve chains; `eval/mcp_eval.xml`.
|
||||
|
||||
## 12. Build-session plan of record
|
||||
|
||||
| Phase | Deliverable | Est. size |
|
||||
|---|---|---|
|
||||
| 0 | Return-not-print refactor in the plugin tree (`*_op` cores); suites green | ~⅓ the session |
|
||||
| 1 | `mcp-server/`: FastMCP app, auth middleware, result wrapper, error map, `/health` | ~200 lines |
|
||||
| 2 | Read tools (load, recall, resolve, get_note, scope×2, health) | wiring |
|
||||
| 3 | Write tools (capture, link, append/patch, triage, reflect, log_session) | wiring + descriptions |
|
||||
| 4 | Warm caches + `/data` outbox + startup validation | small |
|
||||
| 5 | Dockerfile (legacy syntax) + CI workflow + PORT manifest + deploy + CF/NPM publish + Kuma | infra pass |
|
||||
| 6 | Tests (§11) + Inspector pass + eval XML; SKILL.md/README/CHANGELOG | ~⅓ the session |
|
||||
|
||||
Definition of done: Inspector lists 13 tools against `echomcp.alwisp.com`; a
|
||||
live session (desktop **and** CoWork) performs a full memory day — load →
|
||||
recall → capture → gate → merge_into → triage → log_session — without one Bash
|
||||
call; parity test green; container smoke test in CI; Kuma monitor green.
|
||||
@@ -0,0 +1,99 @@
|
||||
# ECHO Memory — Usage Guide
|
||||
|
||||
*Written for Bryan (and anyone else coming back to ECHO after a break). Current as of v1.5.1, July 2026.*
|
||||
|
||||
ECHO gives Claude persistent memory across sessions. Everything durable — people, companies, projects, decisions, preferences, session history — lives as markdown notes in an Obsidian vault, read and written over the Obsidian Local REST API. The plugin ships the whole toolchain (pure Python, stdlib only — works the same on Windows/macOS/Linux) plus the operating procedure Claude follows.
|
||||
|
||||
The short version of how to use it: **you mostly don't have to do anything.** Since v1.5 the system loads itself at session start and nudges itself to save at session end. The slash commands below are for when you want to drive it explicitly.
|
||||
|
||||
---
|
||||
|
||||
## One-time setup
|
||||
|
||||
If Jason handed you a **per-user baked plugin** (`echo-memory-1.x.x-bryan.plugin`), just install it. Your vault credentials are baked into the artifact — no config file, no key pasting, works on desktop and in every CoWork session. This is the normal path.
|
||||
|
||||
If you have the **generic plugin** instead, it will report `NOT CONFIGURED` on first use and ask for your config file. Install it once with:
|
||||
|
||||
```bash
|
||||
python3 echo.py config import <your-config.json>
|
||||
```
|
||||
|
||||
That writes `~/.claude/echo-memory/config.json` (owner, endpoint, API key). It survives plugin updates — one time per machine.
|
||||
|
||||
**Verify either way:** run `/echo-doctor`. Green across the board (Python, vault reachable, auth, bootstrapped, key source) means you're live.
|
||||
|
||||
---
|
||||
|
||||
## Day-to-day usage
|
||||
|
||||
### What happens automatically (v1.5+)
|
||||
|
||||
- **Session start** — a hook runs the cold-start load and injects your memory into context: your profile/preferences, current scope, the last session log, today's journal note, and inbox depth. You don't have to ask Claude to "load memory" anymore.
|
||||
- **Session end** — if the session was substantive and nothing was saved, a hook nudges (once) to run `/echo-reflect` and write the session log. Reflection always previews before writing — nothing lands without your OK.
|
||||
- **If the vault is unreachable**, writes queue to a local outbox and replay automatically next time it's up; reads fall back to a cached last-known-good. An outage degrades gracefully instead of erroring.
|
||||
|
||||
### The slash commands
|
||||
|
||||
| Command | When to use it |
|
||||
|---|---|
|
||||
| `/echo-load` | Manually load memory context. Rarely needed now — the session-start hook does this — but useful mid-session after a lot of vault writes, or if the hook didn't fire. |
|
||||
| `/echo-save <thing>` | Save something right now: "remember the Henagar tower password is in the vault", "log that we decided X". Routes it to the right note automatically. |
|
||||
| `/echo-recall <topic>` | Ask what memory knows about a topic. Returns ranked matches **plus their linked neighbourhood** — a project comes back with its people, decisions, and recent session mentions attached. |
|
||||
| `/echo-reflect` | End-of-session sweep: extracts durable facts, decisions, and new entities from the conversation, shows you the proposed writes, applies on confirm. The main way memory grows. |
|
||||
| `/echo-triage` | Empty the inbox: classifies each quick capture, previews destinations, routes accepted items with an audit trail. Run when load reports the inbox is getting deep. |
|
||||
| `/echo-doctor` | Readiness check — Python, vault reachability, auth, bootstrap/schema version, where the key came from. First stop when anything seems off. |
|
||||
| `/echo-health` | Full vault lint: routing violations, broken wikilinks, orphan notes, stale scope, incomplete frontmatter, index drift. Run occasionally or when things feel inconsistent. |
|
||||
| `/echo-sweep` | Bring the vault up to the current plugin spec after an upgrade — rebuilds the entity index, backfills frontmatter, fixes one-way links. Dry-run by default; **run this once right after updating from an old version.** |
|
||||
|
||||
### A typical session
|
||||
|
||||
1. Start a conversation — memory loads itself. Claude knows who you are, what's active, and what happened last session.
|
||||
2. Work normally. Say "save that" / "remember this" whenever something durable comes up (or `/echo-save` explicitly). Not sure where something belongs? Save it anyway — it lands in the inbox and `/echo-triage` sorts it later.
|
||||
3. At the end, accept the reflection nudge (or run `/echo-reflect` yourself). Confirm the preview. Done — next session picks up from here.
|
||||
|
||||
---
|
||||
|
||||
## What the memory system can do (current capabilities)
|
||||
|
||||
**Smart, deduplicating capture.** One `capture` call routes content to its canonical home via an entity index, stamps complete frontmatter (status, tags, timestamps), cross-links every entity it mentions, and logs the write. Names resolve through aliases and fuzzy matching, so "echo memory" finds the `echo` project instead of spawning a duplicate note. If a new title strongly resembles an existing **same-kind** entity, a pre-write duplicate gate *stops* the write and shows you the candidates — you choose merge or create. Duplicates get blocked before they exist, not cleaned up after.
|
||||
|
||||
**Real retrieval, not keyword grep.** Recall fuses BM25 full-text search with graph expansion along note links, over entities *and* session logs *and* journal entries. Ranking is freshness- and status-aware: recently updated and `active` notes float up, `archived` sinks. You get a topic's connected neighbourhood, not one isolated file.
|
||||
|
||||
**Structured memory model.** Working (transient) / episodic (what happened, when) / semantic (durable facts and preferences) memory layers, plus a current-context scope, per-session logs, an append-only journal with rollups, and PARA-style projects/areas/resources/decisions. A machine-readable routing manifest defines what may be written where, and the linter enforces it.
|
||||
|
||||
**Reflection.** `/echo-reflect` turns a conversation into memory: extract → classify → dedup against the index → confidence-filter → preview → apply. Low-confidence items go to the inbox instead of polluting the graph.
|
||||
|
||||
**Durability and safety.** Every write is HTTP-status-checked and read-back-verified (a failed write fails loudly, never silently). Appends are idempotent — retries can't double-write. Offline writes queue and replay. A cooperative advisory lock keeps Claude Code and CoWork from trampling each other on the shared vault.
|
||||
|
||||
**Self-maintaining.** The vault bootstraps itself from an empty Obsidian vault, migrates its own schema on version bumps, and `/echo-sweep` + `/echo-health` keep the graph honest (index rebuilds, link symmetry, frontmatter backfill, orphan/broken-link detection).
|
||||
|
||||
**Fast.** Connection pooling + concurrent reads mean full-vault operations that used to time out in the sandbox now finish in under a second.
|
||||
|
||||
---
|
||||
|
||||
## What's new since 0.7 (your last consistent version)
|
||||
|
||||
You left off right after the toolchain got its executable spine (`echo.py`, routing manifest, linter). Since then, in rough order of what you'll actually notice:
|
||||
|
||||
- **You don't route anything by hand anymore.** 0.7 was "pick the path, write with the right verb." Now `capture`/`recall`/`resolve`/`link` do routing, frontmatter, indexing, and cross-linking in one call — and in practice you just talk to Claude and it uses them.
|
||||
- **Memory loads and saves itself** (v1.5 hooks). No more remembering to run the loading procedure or write the session log.
|
||||
- **Recall is actually good now** (v0.9–1.5): entity index + hybrid BM25/graph search + freshness/status ranking, spanning sessions and journal too.
|
||||
- **Duplicates get blocked at write time** (v1.5 gate) instead of accumulating.
|
||||
- **Offline resilience** (v1.0): vault down ≠ data lost; writes queue and replay.
|
||||
- **Everything is pure Python** (v0.8): no bash, works identically on Windows.
|
||||
- **Credentials moved out of the plugin** (v1.3–1.4): generic builds prompt once per machine; your baked build carries them invisibly. Treat a baked `.plugin` file like a password — it contains your vault key. Don't share or commit it.
|
||||
|
||||
**After installing the new version, run `/echo-doctor`, then `/echo-sweep` once** to bring your vault up to the current schema (it dry-runs first and shows the plan).
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Fix |
|
||||
|---|---|
|
||||
| `NOT CONFIGURED` banner / doctor shows red config | Generic build with no config on this machine — `echo.py config import <file>`, or ask Jason for your baked build. |
|
||||
| Vault unreachable | Check that Obsidian + the Local REST API plugin are running on the backend and the endpoint is reachable. Meanwhile writes queue safely and replay on reconnect. |
|
||||
| Save blocked with "duplicate gate" (exit 76) | Not an error — it found a likely-existing entity. Review the candidates: merge into the existing note, or force-create if it's genuinely distinct. |
|
||||
| Memory feels stale or inconsistent | `/echo-health` to see what's off, `/echo-sweep` to repair index/links/frontmatter. |
|
||||
| Inbox keeps getting mentioned at load | `/echo-triage` — one pass empties it. |
|
||||
| Anything else weird | `/echo-doctor` first, then ask Jason. |
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,6 +1,8 @@
|
||||
{
|
||||
"name": "echo-memory",
|
||||
"version": "1.5.1",
|
||||
"version": "2.1.0",
|
||||
"homepage": "https://git.alwisp.com/jason/echo",
|
||||
"repository": "https://git.alwisp.com/jason/echo",
|
||||
"description": "Persistent memory via the ECHO Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /echo-* commands.",
|
||||
"author": {
|
||||
"name": "Jason Stedwell"
|
||||
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
---
|
||||
name: echo-doctor
|
||||
description: Check ECHO readiness — Python, vault reachability, auth, bootstrap/schema, and key source
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run a one-call readiness check before relying on memory.
|
||||
+2
-1
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: echo-health
|
||||
description: Run the ECHO vault-health linter and summarize any invariant violations
|
||||
allowed-tools: Bash(python3 *vault_lint.py*), Bash(python *vault_lint.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
allowed-tools: Bash(python3 *vault_lint.py*), Bash(python *vault_lint.py*), Bash(py -3 *vault_lint.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
---
|
||||
|
||||
Run the bundled, read-only vault linter and report findings. Pass the conversation's current date (`ECHO_TODAY`) so stale/aging math uses the same clock the agent writes with:
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
---
|
||||
name: echo-load
|
||||
description: Load ECHO memory — cold-start context read (profile, scope, latest session, today, inbox)
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to load memory now. Run the cold-start **Loading procedure** from `SKILL.md`: the six orientation reads (marker, operator-preferences, current-context, heartbeat, today's daily note, inbox) in **one call**, then do the load-time **reconcile** (inbox-depth + scope-drift) and surface it in a single line.
|
||||
@@ -140,7 +140,7 @@ Load at the start of any substantive conversation — anything beyond a single q
|
||||
|
||||
### Loading procedure
|
||||
|
||||
**Run `python3 "$ECHO" load`** — one call that performs the six orientation reads below and prints each labeled section (404s on today's note / inbox show as absent, not errors; an absent marker is flagged). This replaces issuing six separate GETs by hand. The table documents what `load` reads and why:
|
||||
**Run `python3 "$ECHO" load`** — one call that performs the six orientation reads below (fetched in parallel) and prints each labeled section (404s on today's note / inbox show as absent, not errors; an absent marker is flagged). This replaces issuing six separate GETs by hand. **`load --brief`** renders a token-budgeted digest instead — Fact/Pattern rules in full, the last ~10 Observations, scope + freshness, the last session's key sections, today's Agent Log lines, and the inbox as a *count* (`ECHO_LOAD_BUDGET`, default ~8000 chars, trims lowest-priority sections first). The SessionStart hook injects the brief form; `/echo-load` and this procedure use the full form. The table documents what `load` reads and why:
|
||||
|
||||
| # | GET | Notes |
|
||||
|---|-----|-------|
|
||||
@@ -405,25 +405,40 @@ At the end of substantive conversations (ones that produced decisions, artifacts
|
||||
|
||||
**Filename format is canonical: `_agent/sessions/YYYY-MM-DD-HHMM-<slug>.md`.** Always include the four-digit local-time HHMM component — it makes filenames lexically sort in true chronological order, which is what Step 3 of loading relies on. Older session logs without the HHMM part exist; leave them alone, but every new one must use the full form.
|
||||
|
||||
Write the body (see `references/session-log-template.md`) to a file with the Write tool, then:
|
||||
**Preferred — one call (`session-end`).** Write a bundle JSON with the Write tool and run it; it performs the whole session-end sequence under one lock, in order, with the **heartbeat written last as the commit marker** (a failure part-way leaves the previous pointer intact):
|
||||
|
||||
```json
|
||||
{ "slug": "<kebab-slug>",
|
||||
"log_body": "<full session-log markdown, frontmatter included — see references/session-log-template.md>",
|
||||
"agent_log_line": "- <currentDate>: <one-line summary>",
|
||||
"scope": "<new scope text — omit to leave unchanged>",
|
||||
"reflect": [ { "title": "…", "kind": "…", "body": "…", "confidence": 0.9 } ] }
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 "$ECHO" session-end bundle.json # dry-run: previews the full plan
|
||||
ECHO_NOW=<HHMM> python3 "$ECHO" session-end bundle.json --apply
|
||||
```
|
||||
|
||||
`--apply` writes: session log → Agent Log line → reflect proposals (via `capture`, gate-aware) → optional scope switch → heartbeat. Set `ECHO_NOW` to the conversation's local HHMM so the filename sorts truthfully. Reflect proposals still follow the reflect contract — preview with the dry-run and apply only with the operator's go-ahead. Offline, every step queues durably.
|
||||
|
||||
**Manual fallback** (only if `session-end` is unavailable) — the same sequence by hand:
|
||||
|
||||
```bash
|
||||
python3 "$ECHO" put _agent/sessions/<currentDate>-HHMM-<slug>.md <bodyfile>
|
||||
```
|
||||
|
||||
Then add a one-line entry to today's daily note via the **Daily Note — Agent Log** procedure above.
|
||||
|
||||
Finally, update the heartbeat pointer so the next session can orient in one read (this closes the loop with loading-procedure Step 4 — a pointer nobody writes is a pointer nobody can read). It is a single line, `<session-log-path> @ <ISO-timestamp>`; write it to a file and PUT it:
|
||||
Then add a one-line entry to today's daily note via the **Daily Note — Agent Log** procedure above, and finally update the heartbeat pointer so the next session can orient in one read (a pointer nobody writes is a pointer nobody can read) — a single line, `<session-log-path> @ <ISO-timestamp>`:
|
||||
|
||||
```bash
|
||||
python3 "$ECHO" put _agent/heartbeat/last-session.md <bodyfile>
|
||||
```
|
||||
|
||||
`last-session.md` is overwritten (PUT) each session end — never appended, so it can't grow or duplicate.
|
||||
`last-session.md` is overwritten (PUT) each session end — never appended, so it can't grow or duplicate. Keep the heartbeat write **last** in the manual sequence too.
|
||||
|
||||
## Vault Unreachable
|
||||
|
||||
If the API returns a connection error, timeout, or `502`, tell the operator once that the memory vault is unreachable (a `502` usually means Obsidian/the REST plugin is not running on the backend), then proceed without memory. Do not retry repeatedly.
|
||||
If the API returns a connection error, timeout, or `502`, tell the operator once that the memory vault is unreachable (a `502` usually means Obsidian/the REST plugin is not running on the backend), then proceed — reads degrade to the last-known-good cache at load, and **writes queue durably**: the low-level verbs queue the request, and `capture` queues the whole operation as one semantic record that replays *through capture* (routing + duplicate gate re-run against the index at replay time) on the next reachable session. A queued capture that stops at the duplicate gate on replay is kept and flagged (`flush` lists it) rather than landed blind. Do not retry repeatedly.
|
||||
|
||||
## Style Rules
|
||||
|
||||
|
||||
@@ -605,9 +605,136 @@ def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) -
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_load() -> int:
|
||||
"""Cold-start orientation: the canonical 6 reads in one call. 404s on today's
|
||||
daily note and the inbox are normal (printed as absent, not errors)."""
|
||||
def _fetch_statuses(paths: list[str]) -> dict[str, tuple[int, bytes]]:
|
||||
"""Concurrently GET vault paths keeping (status, body) per path — unlike read_many,
|
||||
the 404-vs-offline distinction survives, which cmd_load's cache logic needs."""
|
||||
if not paths:
|
||||
return {}
|
||||
workers = min(MAX_WORKERS, len(paths))
|
||||
with ThreadPoolExecutor(max_workers=workers) as pool:
|
||||
return dict(zip(paths, pool.map(lambda p: request("GET", vault_url(p)), paths)))
|
||||
|
||||
|
||||
def _bullet_lines(text: str) -> list[str]:
|
||||
return [ln for ln in text.splitlines() if ln.lstrip().startswith("- ")]
|
||||
|
||||
|
||||
def _fm_line(text: str, field: str) -> str:
|
||||
for ln in text.splitlines():
|
||||
if ln.startswith(f"{field}:"):
|
||||
return ln.split(":", 1)[1].strip().strip('"').strip("'")
|
||||
return ""
|
||||
|
||||
|
||||
def _render_brief(texts: dict[str, str | None], listing_files: list[str],
|
||||
offline: bool) -> str:
|
||||
"""The token-budgeted cold-start digest (2.1.0). Selection over compression: Fact /
|
||||
Pattern rules in full, recent Observations only, scope + freshness (not the whole
|
||||
history), the last session's key sections, today's Agent Log lines, inbox COUNT.
|
||||
Sections are trimmed lowest-priority-first when over ECHO_LOAD_BUDGET."""
|
||||
budget = int(os.environ.get("ECHO_LOAD_BUDGET", "8000"))
|
||||
S: list[tuple[str, str]] = [] # (section-name, text) in display order
|
||||
|
||||
marker = texts.get("marker")
|
||||
if marker is None:
|
||||
S.append(("marker", "marker: _agent/echo-vault.md ABSENT — vault not bootstrapped "
|
||||
"(run bootstrap.py before relying on memory)"))
|
||||
else:
|
||||
ver = _fm_line(marker, "schema_version") or "?"
|
||||
S.append(("marker", f"marker: _agent/echo-vault.md — bootstrapped, schema_version {ver}"))
|
||||
|
||||
ctx = texts.get("context")
|
||||
if ctx:
|
||||
scope = extract_heading(ctx, "Scope") or "(empty)"
|
||||
upd = _fm_line(ctx, "scope_updated") or "unknown"
|
||||
since = sum(1 for f in listing_files if f.endswith(".md") and f[:10] > upd) \
|
||||
if upd != "unknown" else None
|
||||
head = f"scope (updated {upd}"
|
||||
if since is not None:
|
||||
head += f", {since} session(s) logged since"
|
||||
S.append(("scope", head + "):\n" + scope))
|
||||
else:
|
||||
S.append(("scope", "scope: current-context.md absent"))
|
||||
|
||||
prefs = texts.get("preferences")
|
||||
if prefs:
|
||||
fact = extract_heading(prefs, "Fact / Pattern")
|
||||
if fact:
|
||||
S.append(("facts", "preferences — Fact / Pattern:\n" + fact))
|
||||
obs = _bullet_lines(extract_heading(prefs, "Observations"))
|
||||
if obs:
|
||||
shown = obs[-10:]
|
||||
head = f"preferences — Observations (last {len(shown)} of {len(obs)}):"
|
||||
S.append(("observations", head + "\n" + "\n".join(shown)))
|
||||
|
||||
hb = texts.get("heartbeat")
|
||||
if hb and hb.strip():
|
||||
pointer = hb.strip().splitlines()[0]
|
||||
sess_path = pointer.split(" @ ", 1)[0].strip()
|
||||
part = [f"last session: {pointer}"]
|
||||
log_text = texts.get("_pointed_session")
|
||||
if log_text:
|
||||
for h in ("Goal", "Decisions Made", "Open Threads", "Suggested Next Step"):
|
||||
sec = extract_heading(log_text, h)
|
||||
if sec and sec.strip("() \n"):
|
||||
part.append(f" {h}: " + " / ".join(ln.strip() for ln in sec.splitlines()
|
||||
if ln.strip())[:400])
|
||||
elif sess_path:
|
||||
part.append(" (session log not readable — see the path above)")
|
||||
S.append(("session", "\n".join(part)))
|
||||
elif listing_files:
|
||||
recent = sorted((f for f in listing_files if f.endswith(".md")), reverse=True)[:3]
|
||||
S.append(("session", "last session: heartbeat absent — recent logs:\n"
|
||||
+ "\n".join(f" _agent/sessions/{f}" for f in recent)))
|
||||
|
||||
today_note = texts.get("today")
|
||||
if today_note:
|
||||
log_lines = _bullet_lines(extract_heading(today_note, "Agent Log"))
|
||||
S.append(("agent-log", "today's Agent Log:\n" + ("\n".join(log_lines) or " (empty)")))
|
||||
else:
|
||||
S.append(("agent-log", "today's Agent Log: (no daily note yet)"))
|
||||
|
||||
inbox = texts.get("inbox")
|
||||
if inbox:
|
||||
items = re.findall(r"(?m)^\s*-\s*(\d{4}-\d{2}-\d{2})", inbox)
|
||||
if items:
|
||||
try:
|
||||
oldest = (dt.date.fromisoformat(today()) - dt.date.fromisoformat(min(items))).days
|
||||
S.append(("inbox", f"inbox: {len(items)} capture(s), oldest {oldest}d — "
|
||||
"offer triage if any are older than ~7 days"))
|
||||
except ValueError:
|
||||
S.append(("inbox", f"inbox: {len(items)} capture(s)"))
|
||||
else:
|
||||
S.append(("inbox", "inbox: empty"))
|
||||
else:
|
||||
S.append(("inbox", "inbox: empty"))
|
||||
|
||||
def total() -> int:
|
||||
return sum(len(t) + 2 for _, t in S)
|
||||
|
||||
# Over budget -> trim lowest-priority sections first, with explicit markers.
|
||||
for name, keep in (("observations", 4), ("agent-log", 4), ("session", 9)):
|
||||
if total() <= budget:
|
||||
break
|
||||
for i, (n, t) in enumerate(S):
|
||||
if n == name:
|
||||
lines = t.splitlines()
|
||||
if len(lines) > keep:
|
||||
S[i] = (n, "\n".join(lines[:keep]) + "\n (truncated — /echo-load for full)")
|
||||
out = f"ECHO load (brief) — {today()}\n\n" + "\n\n".join(t for _, t in S)
|
||||
if len(out) > budget:
|
||||
out = out[:budget] + "\n(truncated — /echo-load for full)"
|
||||
if offline:
|
||||
out += ("\n\nNOTE: vault unreachable — context above is last-known-good cache; "
|
||||
"writes this session will be queued and synced when the vault returns.")
|
||||
return out
|
||||
|
||||
|
||||
def cmd_load(brief: bool = False) -> int:
|
||||
"""Cold-start orientation: the canonical 6 reads in one call (fetched in parallel).
|
||||
404s on today's daily note and the inbox are normal (printed as absent, not errors).
|
||||
`brief` renders the token-budgeted digest the SessionStart hook injects; the default
|
||||
full mode (and /echo-load) prints the raw sections unchanged."""
|
||||
if not echo_config.is_configured(_CFG):
|
||||
print("echo: NOT CONFIGURED — this machine has no usable ECHO key file yet.")
|
||||
print(f"Expected at: {echo_config.config_path()}")
|
||||
@@ -631,35 +758,77 @@ def cmd_load() -> int:
|
||||
synced = echo_queue.flush()
|
||||
if synced:
|
||||
print(f"(synced {synced} write(s) queued during a prior offline session)\n")
|
||||
flagged = echo_queue.needs_attention()
|
||||
if flagged:
|
||||
print(f"NOTE: {len(flagged)} queued write(s) need attention (e.g. a capture "
|
||||
"stopped at the duplicate gate on replay) — resolve with capture "
|
||||
"--merge-into/--force; `echo.py flush` re-lists them.\n")
|
||||
except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load
|
||||
print(f"(queue flush skipped: {exc})\n", file=sys.stderr)
|
||||
|
||||
# All orientation reads (+ the sessions listing) fetched in parallel over the
|
||||
# warm connection pool — the listing feeds both the brief digest's scope-freshness
|
||||
# count and the heartbeat-absent fallback, so it's no longer a second round-trip.
|
||||
listing_path = "_agent/sessions/"
|
||||
results = _fetch_statuses([p for _, p in targets] + [listing_path])
|
||||
|
||||
marker_missing = False
|
||||
heartbeat_absent = False
|
||||
offline = False
|
||||
texts: dict[str, str | None] = {}
|
||||
for label, path in targets:
|
||||
status, body = request("GET", vault_url(path))
|
||||
status, body = results[path]
|
||||
if status == 200:
|
||||
echo_queue.cache_put(path, body) # refresh last-known-good for offline fallback
|
||||
texts[label] = body.decode(errors="replace")
|
||||
elif status == 0: # vault unreachable -> degrade to last-known-good cache
|
||||
offline = True
|
||||
cached = echo_queue.cache_get(path)
|
||||
texts[label] = cached.decode(errors="replace") if cached is not None else None
|
||||
else:
|
||||
texts[label] = None
|
||||
if status == 404:
|
||||
if label == "marker":
|
||||
marker_missing = True
|
||||
if label == "heartbeat":
|
||||
heartbeat_absent = True
|
||||
|
||||
lst_status, lst_body = results[listing_path]
|
||||
listing_files: list[str] = []
|
||||
if lst_status == 200:
|
||||
try:
|
||||
listing_files = list(json.loads(lst_body).get("files", []))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
if brief:
|
||||
hb = texts.get("heartbeat")
|
||||
if hb and hb.strip(): # follow-up: the pointed session log's key sections
|
||||
sess_path = hb.strip().splitlines()[0].split(" @ ", 1)[0].strip()
|
||||
if sess_path:
|
||||
st2, b2 = request("GET", vault_url(sess_path))
|
||||
if st2 == 200:
|
||||
texts["_pointed_session"] = b2.decode(errors="replace")
|
||||
print(_render_brief(texts, listing_files, offline))
|
||||
return 0
|
||||
|
||||
# ---- full mode: raw sections, output format unchanged ----
|
||||
for label, path in targets:
|
||||
status, body = results[path]
|
||||
if status == 200:
|
||||
print(f"===== {label}: {path} (HTTP 200) =====")
|
||||
text = body.decode(errors="replace")
|
||||
text = texts[label] or ""
|
||||
sys.stdout.write(text)
|
||||
if not text.endswith("\n"):
|
||||
sys.stdout.write("\n")
|
||||
elif status == 404:
|
||||
print(f"===== {label}: {path} (HTTP 404) =====")
|
||||
print("(absent — fine)")
|
||||
if label == "marker":
|
||||
marker_missing = True
|
||||
if label == "heartbeat":
|
||||
heartbeat_absent = True
|
||||
elif status == 0: # vault unreachable -> degrade to last-known-good cache
|
||||
offline = True
|
||||
cached = echo_queue.cache_get(path)
|
||||
if cached is not None:
|
||||
elif status == 0:
|
||||
if texts[label] is not None:
|
||||
print(f"===== {label}: {path} (OFFLINE — serving stale cache) =====")
|
||||
sys.stdout.write(cached.decode(errors="replace"))
|
||||
if not cached.endswith(b"\n"):
|
||||
sys.stdout.write(texts[label])
|
||||
if not texts[label].endswith("\n"):
|
||||
sys.stdout.write("\n")
|
||||
else:
|
||||
print(f"===== {label}: {path} (OFFLINE — no cache) =====")
|
||||
@@ -669,15 +838,9 @@ def cmd_load() -> int:
|
||||
print(f"(error HTTP {status}: {body.decode(errors='replace')[:200]})")
|
||||
print()
|
||||
# M3: heartbeat pointer missing/stale -> fall back to the recent sessions listing
|
||||
# (matches the documented loading procedure) so orientation works without the pointer.
|
||||
if heartbeat_absent and not offline:
|
||||
st, body = request("GET", vault_url("_agent/sessions/"))
|
||||
if st == 200:
|
||||
try:
|
||||
files = sorted((f for f in json.loads(body).get("files", []) if f.endswith(".md")),
|
||||
reverse=True)
|
||||
except json.JSONDecodeError:
|
||||
files = []
|
||||
# (already fetched in the batch) so orientation works without the pointer.
|
||||
if heartbeat_absent and not offline and listing_files:
|
||||
files = sorted((f for f in listing_files if f.endswith(".md")), reverse=True)
|
||||
if files:
|
||||
print("===== recent sessions (heartbeat absent — fallback) =====")
|
||||
for f in files[:5]:
|
||||
@@ -732,7 +895,8 @@ def cmd_config(args) -> int:
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description="Validated cross-platform ECHO vault client")
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
sub.add_parser("load")
|
||||
p = sub.add_parser("load")
|
||||
p.add_argument("--brief", action="store_true") # token-budgeted digest (hook default)
|
||||
sub.add_parser("flush") # H2: replay writes queued during a prior outage
|
||||
sub.add_parser("doctor") # M3: one-call readiness check
|
||||
sub.add_parser("get").add_argument("path")
|
||||
@@ -761,6 +925,9 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
p = sub.add_parser("reflect") # H5: apply a JSON proposal set (dry-run unless --apply)
|
||||
p.add_argument("file", nargs="?")
|
||||
p.add_argument("--apply", action="store_true")
|
||||
p = sub.add_parser("session-end") # 2.1.0: one-call session end (log+agent-log+reflect+scope+heartbeat)
|
||||
p.add_argument("file", nargs="?") # bundle JSON; - or stdin
|
||||
p.add_argument("--apply", action="store_true")
|
||||
p = sub.add_parser("triage") # one-tap inbox triage (reflect pipeline + audit log)
|
||||
p.add_argument("file", nargs="?") # proposals JSON; omit to list
|
||||
p.add_argument("--list", action="store_true", dest="list_inbox")
|
||||
@@ -796,21 +963,27 @@ def main(argv: list[str] | None = None) -> int:
|
||||
args = build_parser().parse_args(argv)
|
||||
try:
|
||||
if args.cmd == "load":
|
||||
return cmd_load()
|
||||
return cmd_load(brief=args.brief)
|
||||
if args.cmd == "doctor":
|
||||
import echo_doctor
|
||||
return echo_doctor.run()
|
||||
if args.cmd == "flush":
|
||||
import echo_queue
|
||||
n = echo_queue.flush()
|
||||
remaining = len(echo_queue.pending())
|
||||
remaining = echo_queue.pending()
|
||||
flagged = [r for r in remaining if r.get("needs_attention")]
|
||||
if n:
|
||||
print(f"ok: flushed {n} queued write(s)"
|
||||
+ (f"; {remaining} still queued (vault unreachable)" if remaining else ""))
|
||||
+ (f"; {len(remaining)} still queued" if remaining else ""))
|
||||
elif remaining:
|
||||
print(f"ok: {remaining} write(s) still queued (vault unreachable)")
|
||||
print(f"ok: {len(remaining)} write(s) still queued")
|
||||
else:
|
||||
print("ok: queue empty")
|
||||
for r in flagged:
|
||||
what = (f"capture '{(r.get('args') or {}).get('title')}'" if r.get("op") == "capture"
|
||||
else f"{r.get('method')} {r.get('url')}")
|
||||
print(f" needs attention ({r['needs_attention']}): {what} — resolve "
|
||||
"(e.g. capture --merge-into <slug> / --force), it will not auto-land")
|
||||
return 0
|
||||
if args.cmd == "config":
|
||||
return cmd_config(args)
|
||||
@@ -852,6 +1025,14 @@ def main(argv: list[str] | None = None) -> int:
|
||||
if not isinstance(proposals, list):
|
||||
raise EchoError("reflect: proposals must be a JSON array of objects", 2)
|
||||
return echo_reflect.apply(proposals, confirm=args.apply)
|
||||
if args.cmd == "session-end":
|
||||
import echo_session
|
||||
raw = read_body(args.file).decode("utf-8", errors="replace")
|
||||
try:
|
||||
bundle = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise EchoError(f"session-end: bundle must be a JSON object ({exc})", 2)
|
||||
return echo_session.session_end(bundle, apply=args.apply)
|
||||
if args.cmd == "triage":
|
||||
import echo_triage
|
||||
if args.list_inbox or not args.file:
|
||||
@@ -880,9 +1061,12 @@ def main(argv: list[str] | None = None) -> int:
|
||||
date=args.date, domain=args.domain, inbox=args.inbox, no_log=args.no_log,
|
||||
as_json=args.json, dry_run=args.dry_run,
|
||||
force=args.force, merge_into=args.merge_into)
|
||||
except EchoError as exc:
|
||||
except RuntimeError as exc:
|
||||
# Catches EchoError from THIS module and from helper modules' own `import echo`
|
||||
# twin (when echo.py runs as __main__ the two class objects differ, but both
|
||||
# subclass RuntimeError and carry .code).
|
||||
print(f"echo.py: {exc}", file=sys.stderr)
|
||||
return exc.code
|
||||
return getattr(exc, "code", 1)
|
||||
return 2
|
||||
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ def main() -> int:
|
||||
try:
|
||||
import echo
|
||||
with contextlib.redirect_stdout(buf):
|
||||
rc = echo.cmd_load()
|
||||
rc = echo.cmd_load(brief=True) # token-budgeted digest; /echo-load stays full
|
||||
text = buf.getvalue()
|
||||
if rc == 78:
|
||||
context = ("[echo-memory] ECHO is NOT CONFIGURED on this machine. Before "
|
||||
|
||||
@@ -30,9 +30,11 @@ MIN_TURNS = int(os.environ.get("ECHO_REFLECT_MIN_TURNS", "5"))
|
||||
REASON = (
|
||||
"[echo-memory] Session-end reflection has not run yet. If this session produced "
|
||||
"durable facts, decisions, commitments, or artifacts worth remembering: run the "
|
||||
"/echo-reflect flow (extract -> preview -> apply only with the operator's confirm) "
|
||||
"and write the session log + heartbeat per the echo-memory skill. If nothing "
|
||||
"durable emerged, simply finish — do not invent memories. "
|
||||
"/echo-reflect flow (extract -> preview -> apply only with the operator's confirm), "
|
||||
"then finish with ONE call — `echo.py session-end <bundle.json> --apply` — which "
|
||||
"writes the session log, Agent Log line, reflect proposals, optional scope switch, "
|
||||
"and the heartbeat (last, as the commit marker) together. If nothing durable "
|
||||
"emerged, simply finish — do not invent memories. "
|
||||
"(This reminder fires once per session.)"
|
||||
)
|
||||
|
||||
@@ -64,6 +66,7 @@ def already_reflected(raw: str) -> bool:
|
||||
"""Transcript evidence that reflection or session logging already happened."""
|
||||
return ("/echo-reflect" in raw
|
||||
or re.search(r'echo\.py"?\s+reflect\b', raw) is not None
|
||||
or re.search(r'echo\.py"?\s+session-end\b', raw) is not None
|
||||
or "put _agent/sessions/" in raw
|
||||
or "put _agent/heartbeat/last-session.md" in raw)
|
||||
|
||||
|
||||
@@ -83,8 +83,13 @@ def recall(query, limit: int = 8, as_json: bool = False) -> int:
|
||||
# ----------------------------------------------------------- agent log -------
|
||||
def ensure_daily_log(line: str) -> None:
|
||||
"""Resilient: ensure today's daily note + its `## Agent Log` heading exist, then
|
||||
idempotently append the line. Best-effort — never raises into the caller."""
|
||||
idempotently append the line. Best-effort — never raises into the caller. Writes go
|
||||
through the offline queue (2.1.0), so a mid-session outage queues the line instead
|
||||
of silently dropping it. Offline edge accepted: if the daily note still doesn't
|
||||
exist at replay time, the queued PATCH 400-drops with a loud warning — the agent-log
|
||||
line is a convenience trace, not the memory itself."""
|
||||
try:
|
||||
import echo_queue
|
||||
date = echo.today()
|
||||
path = f"journal/daily/{date}.md"
|
||||
status, body = echo.request("GET", echo.vault_url(path))
|
||||
@@ -92,18 +97,19 @@ def ensure_daily_log(line: str) -> None:
|
||||
ts, tb = echo.request("GET", echo.vault_url("journal/templates/daily-note-template.md"))
|
||||
tmpl = tb.decode(errors="replace") if ts == 200 else f"# {date}\n\n## Agent Log\n\n## Related\n"
|
||||
tmpl = tmpl.replace("{{date:YYYY-MM-DD}}", date).replace("{{DATE}}", date)
|
||||
echo.request("PUT", echo.vault_url(path), data=tmpl.encode(),
|
||||
echo_queue.safe_request("PUT", echo.vault_url(path), data=tmpl.encode(),
|
||||
headers={"Content-Type": "text/markdown"})
|
||||
text = tmpl
|
||||
else:
|
||||
text = body.decode(errors="replace")
|
||||
if not re.search(r"(?m)^## Agent Log\s*$", text):
|
||||
echo.request("POST", echo.vault_url(path), data=b"\n\n## Agent Log\n",
|
||||
text = body.decode(errors="replace") if status == 200 else ""
|
||||
if status == 200 and not re.search(r"(?m)^## Agent Log\s*$", text):
|
||||
echo_queue.safe_request("POST", echo.vault_url(path), data=b"\n\n## Agent Log\n",
|
||||
headers={"Content-Type": "text/markdown"})
|
||||
status, body = echo.request("GET", echo.vault_url(path))
|
||||
if status == 200 and line in body.decode(errors="replace"):
|
||||
st2, body2 = echo.request("GET", echo.vault_url(path))
|
||||
if st2 == 200 and line in body2.decode(errors="replace"):
|
||||
return
|
||||
echo.request("PATCH", echo.vault_url(path),
|
||||
echo_queue.safe_request(
|
||||
"PATCH", echo.vault_url(path),
|
||||
data=echo.normalize_patch_body((line + "\n").encode(), "append", "heading"),
|
||||
headers={"Operation": "append", "Target-Type": "heading",
|
||||
"Target": f"{date}::Agent Log", "Content-Type": "text/markdown"})
|
||||
@@ -148,17 +154,22 @@ def _dated_block(today_s: str, body_text: str) -> tuple[str, str]:
|
||||
|
||||
|
||||
def _append_to_existing(path: str, kind: str, today_s: str, body_text: str) -> None:
|
||||
# Writes route through the offline queue (2.1.0) so a mid-capture outage queues the
|
||||
# update instead of silently losing it. (A fully-offline capture never reaches here —
|
||||
# the short-circuit in capture() queues the whole op as one semantic record.)
|
||||
import echo_queue
|
||||
text = links.get_text(path) or ""
|
||||
h1 = links.first_h1(text) or path.rsplit("/", 1)[-1][:-3]
|
||||
heading = LOG_HEADING.get(kind, "Notes")
|
||||
if not re.search(rf"(?m)^##\s+{re.escape(heading)}\s*$", text):
|
||||
echo.request("POST", echo.vault_url(path), data=f"\n\n## {heading}\n".encode(),
|
||||
echo_queue.safe_request("POST", echo.vault_url(path), data=f"\n\n## {heading}\n".encode(),
|
||||
headers={"Content-Type": "text/markdown"})
|
||||
bullet, block = _dated_block(today_s, body_text)
|
||||
status, body = echo.request("GET", echo.vault_url(path))
|
||||
if status == 200 and bullet in body.decode(errors="replace"):
|
||||
return
|
||||
echo.request("PATCH", echo.vault_url(path),
|
||||
echo_queue.safe_request(
|
||||
"PATCH", echo.vault_url(path),
|
||||
data=echo.normalize_patch_body((block + "\n").encode(), "append", "heading"),
|
||||
headers={"Operation": "append", "Target-Type": "heading",
|
||||
"Target": f"{h1}::{heading}", "Content-Type": "text/markdown"})
|
||||
@@ -213,7 +224,35 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
|
||||
return done("inbox", "inbox/captures/inbox.md", ok=rc == 0)
|
||||
|
||||
slug = idx_mod.slugify(title)
|
||||
# Offline short-circuit (2.1.0). The routed path needs the entity index and several
|
||||
# round-trips; when the vault is unreachable, queue the WHOLE capture as one semantic
|
||||
# record — flush replays it back through this function against the index as it is at
|
||||
# replay time, so routing, the duplicate gate, and aliasing re-run with fresh state.
|
||||
# (Byte-level request replay would freeze a create-vs-update decision made blind.)
|
||||
try:
|
||||
index = idx_mod.load()
|
||||
except echo.EchoError as exc:
|
||||
if "unreachable" not in str(exc):
|
||||
raise
|
||||
if dry_run:
|
||||
print("offline: vault unreachable — a real run would queue this capture "
|
||||
"for replay on the next reachable session.", file=real_stdout)
|
||||
return 0
|
||||
import echo_queue
|
||||
echo_queue.enqueue_capture({
|
||||
"kind": kind, "title": title, "body_text": body_text, "status_v": status_v,
|
||||
"aliases": aliases, "sources": sources, "tags": tags, "date": date,
|
||||
"domain": domain, "inbox": False, "no_log": no_log,
|
||||
"force": force, "merge_into": merge_into, "today": today_s,
|
||||
})
|
||||
if as_json:
|
||||
env = echo_output.envelope("queued:capture",
|
||||
{"kind": kind, "title": title, "queued": True})
|
||||
print(json.dumps(env, ensure_ascii=False), file=real_stdout)
|
||||
else:
|
||||
print(f"queued (offline): capture {kind} '{title}' — will replay through "
|
||||
"capture on the next reachable session", file=real_stdout)
|
||||
return 0
|
||||
match_slug, existing = idx_mod.resolve(index, title)
|
||||
# --merge-into: the operator has already identified the canonical entity (e.g. after
|
||||
# a duplicate-gate stop) — route this capture as an UPDATE to it, whatever the title.
|
||||
|
||||
@@ -81,6 +81,23 @@ def enqueue(method: str, url: str, data: bytes | None, headers: dict, idem_key:
|
||||
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def enqueue_capture(args: dict) -> None:
|
||||
"""Queue a whole capture as ONE semantic record (2.1.0). Byte-level replay is wrong
|
||||
for the routed capture path: the create-vs-update decision, duplicate gate, and
|
||||
aliasing must re-run against the index AS IT IS AT REPLAY TIME, so flush re-invokes
|
||||
echo_ops.capture with the recorded arguments instead of replaying stale requests.
|
||||
`args` carries body_text inline (never a temp-file path) plus `today` (the capture-
|
||||
time ECHO_TODAY) so replayed frontmatter/log dates reflect when it was said."""
|
||||
key = "capture:" + hashlib.sha1(
|
||||
json.dumps(args, sort_keys=True, ensure_ascii=False).encode("utf-8")).hexdigest()
|
||||
if any(rec.get("idem_key") == key for rec in pending()):
|
||||
return
|
||||
_ensure_dirs()
|
||||
rec = {"op": "capture", "args": args, "idem_key": key}
|
||||
with outbox_path().open("a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def pending() -> list[dict]:
|
||||
p = outbox_path()
|
||||
if not p.exists():
|
||||
@@ -88,6 +105,12 @@ def pending() -> list[dict]:
|
||||
return [json.loads(ln) for ln in p.read_text(encoding="utf-8").splitlines() if ln.strip()]
|
||||
|
||||
|
||||
def needs_attention() -> list[dict]:
|
||||
"""Queued records that replay could not land automatically (e.g. a duplicate-gate
|
||||
stop) — surfaced by `flush` and `load` so the operator resolves them deliberately."""
|
||||
return [rec for rec in pending() if rec.get("needs_attention")]
|
||||
|
||||
|
||||
def _rewrite(records: list[dict]) -> None:
|
||||
p = outbox_path()
|
||||
if not records:
|
||||
@@ -108,9 +131,57 @@ def _rebase(url: str) -> str:
|
||||
return echo.BASE + parts.path + (("?" + parts.query) if parts.query else "")
|
||||
|
||||
|
||||
def _replay_capture(rec: dict) -> str:
|
||||
"""Replay a queued semantic capture through echo_ops.capture against the CURRENT
|
||||
index. Returns 'ok', 'retry' (still offline), or 'gated' (duplicate gate / needs
|
||||
the operator — the record is kept and flagged, later records still replay)."""
|
||||
# Cheap reachability probe FIRST: if the vault is still down, echo_ops.capture's
|
||||
# own offline short-circuit would try to re-enqueue this very record — probe and
|
||||
# bail as 'retry' instead of recursing into the queue.
|
||||
st, _ = echo.request("GET", echo.vault_url("_agent/echo-vault.md"))
|
||||
if st == 0:
|
||||
return "retry"
|
||||
import echo_ops
|
||||
a = dict(rec.get("args") or {})
|
||||
body_text = a.pop("body_text", "") or ""
|
||||
day = a.pop("today", None)
|
||||
bodyfile = echo.temp_file(body_text.encode("utf-8")) if body_text else None
|
||||
prev = os.environ.get("ECHO_TODAY")
|
||||
try:
|
||||
if day:
|
||||
os.environ["ECHO_TODAY"] = day # replayed dates reflect when it was said
|
||||
rc = echo_ops.capture(
|
||||
a.get("kind"), a.get("title") or "(untitled)", bodyfile,
|
||||
status_v=a.get("status_v", ""), aliases=a.get("aliases") or [],
|
||||
sources=a.get("sources") or [], tags=a.get("tags") or [],
|
||||
date=a.get("date"), domain=a.get("domain", "business"),
|
||||
inbox=bool(a.get("inbox")), no_log=bool(a.get("no_log")),
|
||||
force=bool(a.get("force")), merge_into=a.get("merge_into"))
|
||||
except Exception as exc: # noqa: BLE001 — a broken record must not wedge the queue
|
||||
print(f"echo_queue: queued capture '{a.get('title')}' failed on replay ({exc}) "
|
||||
"— kept and flagged", file=sys.stderr)
|
||||
return "gated"
|
||||
finally:
|
||||
if day:
|
||||
if prev is None:
|
||||
os.environ.pop("ECHO_TODAY", None)
|
||||
else:
|
||||
os.environ["ECHO_TODAY"] = prev
|
||||
if rc == 0:
|
||||
return "ok"
|
||||
if rc == 76:
|
||||
print(f"echo_queue: queued capture '{a.get('title')}' stopped at the duplicate "
|
||||
"gate on replay — resolve with capture --merge-into <slug> or --force, "
|
||||
"then remove it from the outbox", file=sys.stderr)
|
||||
return "gated"
|
||||
|
||||
|
||||
def _replay(rec: dict) -> str:
|
||||
"""Re-issue one queued write idempotently. Returns 'ok' (landed/already-present),
|
||||
'drop' (permanent 4xx — can never succeed), or 'retry' (still offline / 5xx)."""
|
||||
'drop' (permanent 4xx — can never succeed), 'gated' (kept + flagged for the
|
||||
operator), or 'retry' (still offline / 5xx)."""
|
||||
if rec.get("op") == "capture":
|
||||
return _replay_capture(rec)
|
||||
method = rec["method"]
|
||||
url = _rebase(rec["url"])
|
||||
data = base64.b64decode(rec["body_b64"]) if rec.get("body_b64") else None
|
||||
@@ -137,21 +208,30 @@ def _replay(rec: dict) -> str:
|
||||
|
||||
def flush() -> int:
|
||||
"""Replay queued writes in order; stop at the first still-unreachable one (keeping it
|
||||
and the rest). Returns the number actually replayed. Safe to call when empty."""
|
||||
and everything after). A 'gated' record (duplicate gate / replay error) is kept and
|
||||
flagged but does NOT block later records — they are independent writes. Returns the
|
||||
number actually replayed. Safe to call when empty."""
|
||||
items = pending()
|
||||
if not items:
|
||||
return 0
|
||||
replayed = 0
|
||||
remaining: list[dict] = []
|
||||
for i, rec in enumerate(items):
|
||||
stopped = False
|
||||
for rec in items:
|
||||
if stopped:
|
||||
remaining.append(rec)
|
||||
continue
|
||||
result = _replay(rec)
|
||||
if result == "ok":
|
||||
replayed += 1
|
||||
elif result == "drop":
|
||||
continue # warned in _replay; remove from queue
|
||||
elif result == "gated":
|
||||
rec["needs_attention"] = rec.get("needs_attention") or "replay-gated"
|
||||
remaining.append(rec)
|
||||
else: # retry -> still offline; keep this and everything after, in order
|
||||
remaining = items[i:]
|
||||
break
|
||||
remaining.append(rec)
|
||||
stopped = True
|
||||
_rewrite(remaining)
|
||||
return replayed
|
||||
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env python3
|
||||
"""echo_session.py — the session-end bundle: one call ends a session correctly. [2.1.0]
|
||||
|
||||
Ending a substantive session used to take four-plus separate invocations (session-log
|
||||
PUT, Agent Log append, heartbeat PUT, optional scope set, optional reflect apply) —
|
||||
a cost per session, and a partial failure left broken orientation state (log written
|
||||
but heartbeat stale). This verb does all of it in one call, under ONE advisory-lock
|
||||
acquisition, in a fixed order with the heartbeat written LAST as the commit marker:
|
||||
a failure part-way leaves the previous pointer intact, never a dangling one.
|
||||
|
||||
Bundle shape (JSON object):
|
||||
{
|
||||
"slug": "echo-mcp-spec", # required, kebab-case
|
||||
"log_body": "<full session-log markdown, frontmatter included>", # required
|
||||
"agent_log_line": "- 2026-07-28: ...", # optional; derived when omitted
|
||||
"scope": "new scope text", # optional -> scope set
|
||||
"reflect": [ { ...PROPOSAL_SCHEMA... } ] # optional -> routed via capture
|
||||
}
|
||||
|
||||
Dry-run by default (previews the whole plan, reflect included); --apply writes.
|
||||
Times: the filename HHMM comes from $ECHO_NOW (HHMM) else the local clock; the date
|
||||
from ECHO_TODAY via echo.today(). Offline: every step rides the offline queue, so a
|
||||
session end during an outage queues durably instead of failing.
|
||||
|
||||
CLI: echo.py session-end <bundle.json> [--apply]
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import echo # noqa: E402
|
||||
|
||||
SESSION_RE = re.compile(r"^\d{4}-\d{2}-\d{2}-\d{4}-[a-z0-9-]+\.md$")
|
||||
SLUG_RE = re.compile(r"^[a-z0-9][a-z0-9-]*$")
|
||||
|
||||
|
||||
def _hhmm() -> str:
|
||||
v = os.environ.get("ECHO_NOW", "").strip()
|
||||
if v:
|
||||
if not re.match(r"^\d{4}$", v):
|
||||
raise echo.EchoError(f"session-end: $ECHO_NOW must be HHMM, got '{v}'", 2)
|
||||
return v
|
||||
return dt.datetime.now().strftime("%H%M")
|
||||
|
||||
|
||||
def validate(bundle: dict) -> tuple[str, str]:
|
||||
"""Return (session_path, agent_log_line) or raise EchoError(2). Validation runs
|
||||
BEFORE any write — a bad bundle writes nothing at all."""
|
||||
if not isinstance(bundle, dict):
|
||||
raise echo.EchoError("session-end: bundle must be a JSON object", 2)
|
||||
slug = str(bundle.get("slug", "")).strip()
|
||||
if not SLUG_RE.match(slug):
|
||||
raise echo.EchoError(f"session-end: slug must be kebab-case, got '{slug}'", 2)
|
||||
if not str(bundle.get("log_body", "")).strip():
|
||||
raise echo.EchoError("session-end: log_body is required (the full session-log markdown)", 2)
|
||||
reflect = bundle.get("reflect")
|
||||
if reflect is not None and not isinstance(reflect, list):
|
||||
raise echo.EchoError("session-end: reflect must be a JSON array of proposals", 2)
|
||||
fname = f"{echo.today()}-{_hhmm()}-{slug}.md"
|
||||
if not SESSION_RE.match(fname):
|
||||
raise echo.EchoError(f"session-end: derived filename '{fname}' violates the "
|
||||
"canonical YYYY-MM-DD-HHMM-<slug>.md form", 2)
|
||||
path = f"_agent/sessions/{fname}"
|
||||
line = str(bundle.get("agent_log_line") or "").strip() \
|
||||
or f"- {echo.today()}: session logged [[{path[:-3]}]]"
|
||||
return path, line
|
||||
|
||||
|
||||
def session_end(bundle: dict, apply: bool = False) -> int:
|
||||
import echo_reflect
|
||||
path, line = validate(bundle)
|
||||
scope = str(bundle.get("scope") or "").strip()
|
||||
proposals = bundle.get("reflect") or []
|
||||
|
||||
valid, errors = echo_reflect.validate(proposals)
|
||||
for e in errors:
|
||||
print(f"skip: {e}", file=sys.stderr)
|
||||
|
||||
print(f"session-end plan ({'APPLY' if apply else 'dry-run'}):")
|
||||
print(f" 1. session log -> {path}")
|
||||
print(f" 2. agent-log line: {line}")
|
||||
if valid:
|
||||
echo_reflect.classify(valid)
|
||||
print(f" 3. reflect: {len(valid)} proposal(s)")
|
||||
print(echo_reflect.preview(valid))
|
||||
else:
|
||||
print(" 3. reflect: (none)")
|
||||
print(f" 4. scope set: {scope!r}" if scope else " 4. scope: (unchanged)")
|
||||
print(f" 5. heartbeat -> {path} @ <now> (written LAST — the commit marker)")
|
||||
|
||||
if not apply:
|
||||
print("\nsession-end: dry-run — re-run with --apply to write.")
|
||||
return 0
|
||||
|
||||
import echo_concurrency
|
||||
import echo_ops
|
||||
steps: dict[str, str] = {}
|
||||
with echo_concurrency.vault_lock():
|
||||
# 1. The session log itself. A hard failure here aborts the whole bundle —
|
||||
# nothing after it (including the heartbeat) runs, so orientation state
|
||||
# can never point at a log that was never written.
|
||||
echo.cmd_put(path, echo.temp_file(str(bundle["log_body"]).encode("utf-8")))
|
||||
steps["session_log"] = "ok"
|
||||
# 2. Agent-log line (best-effort by contract; queues itself when offline).
|
||||
echo_ops.ensure_daily_log(line)
|
||||
steps["agent_log"] = "ok"
|
||||
# 3. Reflect proposals through the normal capture path (gate-aware).
|
||||
if valid:
|
||||
gated = 0
|
||||
for p in valid:
|
||||
if p.get("_action") == "error":
|
||||
continue
|
||||
bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None
|
||||
rc = echo_ops.capture(
|
||||
p.get("kind"), p["title"], bodyfile,
|
||||
aliases=p.get("aliases") or [], sources=p.get("sources") or [],
|
||||
tags=p.get("tags") or [], date=p.get("date"),
|
||||
domain=p.get("domain", "business"),
|
||||
inbox=bool(p.get("inbox")) or not p.get("kind"))
|
||||
gated += 1 if rc == 76 else 0
|
||||
steps["reflect"] = f"{len(valid) - gated}/{len(valid)} applied" \
|
||||
+ (f", {gated} gated (re-propose with --merge-into)" if gated else "")
|
||||
else:
|
||||
steps["reflect"] = "skipped (none)"
|
||||
# 4. Scope switch (optional).
|
||||
if scope:
|
||||
echo.cmd_scope("set", scope)
|
||||
steps["scope"] = "ok"
|
||||
else:
|
||||
steps["scope"] = "unchanged"
|
||||
# 5. Heartbeat LAST — the commit marker for the whole bundle.
|
||||
echo.cmd_put("_agent/heartbeat/last-session.md",
|
||||
echo.temp_file(f"{path} @ {echo.now_iso()}\n".encode("utf-8")))
|
||||
steps["heartbeat"] = "ok"
|
||||
|
||||
print("\nsession-end: done — "
|
||||
+ "; ".join(f"{k}: {v}" for k, v in steps.items()))
|
||||
return 0
|
||||
@@ -165,13 +165,15 @@ def main() -> int:
|
||||
# this shared cache instead of issuing a fresh GET per file per section.
|
||||
texts = echo.read_many(md_files)
|
||||
|
||||
# Path membership + retired-path detection
|
||||
# Path membership + retired-path detection. Retired patterns are checked FIRST:
|
||||
# a file inside a retired tree is flagged even when a permissive route (the
|
||||
# leaf-README rule) would otherwise match it — previously `archive/x/README.md`
|
||||
# was silently absolved by `leaf-readme` and retired dirs survived unflagged.
|
||||
for path in all_files:
|
||||
if routes and not any(rx.match(path) for _, rx in routes):
|
||||
replacement = next((repl for rx, repl in retired if rx.match(path)), None)
|
||||
if replacement is not None:
|
||||
flag("retired-path", f"{path}: retired location — should be {replacement}")
|
||||
else:
|
||||
elif routes and not any(rx.match(path) for _, rx in routes):
|
||||
flag("unknown-path", f"{path}: matches no route in routing.json")
|
||||
|
||||
# Per-note frontmatter checks
|
||||
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-recall
|
||||
description: Recall a topic from ECHO memory — matching notes plus their linked neighbourhood
|
||||
argument-hint: "[topic, person, or project]"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to recall context for:
|
||||
+8
-1
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-reflect
|
||||
description: Reflect on this session — extract durable memories and propose them for one-tap capture
|
||||
argument-hint: "[optional focus, e.g. 'just decisions']"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run **session reflection** (H5). Scan this conversation for
|
||||
@@ -28,4 +30,9 @@ python3 "$ECHO" reflect proposals.json --apply # write — routes each via ca
|
||||
`reflect` dedups every proposal against the entity index (so it shows create-vs-update),
|
||||
skips below-confidence items, and routes `inbox` proposals to the capture inbox. Each
|
||||
applied item goes through the normal `capture` path — canonical frontmatter, auto-linking,
|
||||
and the recall index — and the writes are lock-guarded. $ARGUMENTS
|
||||
and the recall index — and the writes are lock-guarded.
|
||||
|
||||
**Finishing the session?** Prefer the one-call bundle — `python3 "$ECHO" session-end
|
||||
bundle.json --apply` — which applies the reflect proposals AND writes the session log,
|
||||
Agent Log line, optional scope switch, and the heartbeat (last, as the commit marker)
|
||||
together. See the echo-memory skill's **Session Logging** section for the bundle shape. $ARGUMENTS
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-save
|
||||
description: Save to ECHO memory — route content to its canonical home (search-first, idempotent)
|
||||
argument-hint: "[what to remember]"
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to persist this to the ECHO vault:
|
||||
+3
-1
@@ -1,6 +1,8 @@
|
||||
---
|
||||
name: echo-sweep
|
||||
description: Bring the ECHO vault up to the current feature spec (entity index + cross-links)
|
||||
allowed-tools: Bash(python3 *sweep.py*), Bash(python *sweep.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
allowed-tools: Bash(python3 *sweep.py*), Bash(python *sweep.py*), Bash(py -3 *sweep.py*), Bash(ls /sessions/*), Bash(dirname *)
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to bring the vault up to the current spec. Run the sweep **dry-run first**, show the operator the plan, and only `--apply` on their go-ahead:
|
||||
+3
@@ -1,5 +1,8 @@
|
||||
---
|
||||
name: echo-triage
|
||||
description: Triage the ECHO inbox — route aging captures to their canonical homes and log the moves
|
||||
allowed-tools: Bash(python3 *echo.py*), Bash(python *echo.py*), Bash(py -3 *echo.py*), Bash(ls /sessions/*)
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Use the **echo-memory** skill to run **one-tap Inbox Triage** via the `triage` verb (it reuses the reflect pipeline: classify against the entity index → preview → apply via `capture`, and writes the audit log for you).
|
||||
@@ -298,6 +298,14 @@ def main():
|
||||
check("v1.5 sweep backfills the kind tag",
|
||||
"tags: [company]" in drift or '<fm:tags=["company"]>' in drift, drift)
|
||||
|
||||
# 16b. lint blind spot (1.6.0): a seed README inside a retired tree must be
|
||||
# flagged retired-path — previously the permissive leaf-readme route matched
|
||||
# first and absolved it, so retired dirs holding only a README went unflagged.
|
||||
h.seed("archive/notes/README.md", "This folder holds archived notes.\n")
|
||||
r = h.echo(LINT)
|
||||
check("v1.6 lint flags a README inside a retired tree",
|
||||
"archive/notes/README.md: retired location" in r.stdout, r.stdout[-800:])
|
||||
|
||||
# 17. hooks: stop-hook nudges once on a substantive session, then stays quiet.
|
||||
import tempfile
|
||||
tdir = Path(tempfile.mkdtemp())
|
||||
@@ -337,6 +345,63 @@ def main():
|
||||
check("v1.5 session-start hook stays quiet on resume",
|
||||
r.returncode == 0 and not r.stdout.strip(), r.stdout)
|
||||
|
||||
# 18. (2.1.0) brief load: digest form, budget respected, key facts present.
|
||||
h.seed("_agent/memory/semantic/operator-preferences.md",
|
||||
"---\ntype: semantic-memory\ncreated: 2026-06-01\n---\n# Operator Preferences\n\n"
|
||||
"## Fact / Pattern\n- prefers concise output\n\n## Observations\n"
|
||||
+ "\n".join(f"- 2026-06-{i:02d}: observation {i}" for i in range(1, 15)) + "\n")
|
||||
h.seed("inbox/captures/inbox.md", "- 2026-06-01: old capture\n- 2026-06-20: new capture\n")
|
||||
r = h.echo(ECHO, "load", "--brief")
|
||||
check("v2.1 brief load renders the digest header",
|
||||
"ECHO load (brief)" in r.stdout, r.stdout[:200])
|
||||
check("v2.1 brief load keeps Fact / Pattern",
|
||||
"prefers concise output" in r.stdout, r.stdout[:800])
|
||||
check("v2.1 brief load trims observations to the last 10",
|
||||
"last 10 of 14" in r.stdout and "observation 14" in r.stdout
|
||||
and "observation 2" not in r.stdout.replace("observation 2026", ""), r.stdout[-1200:])
|
||||
check("v2.1 brief load reports inbox as a count",
|
||||
"inbox: 2 capture(s), oldest 20d" in r.stdout, r.stdout[-400:])
|
||||
check("v2.1 brief load includes the marker line",
|
||||
"echo-vault.md" in r.stdout and "marker" in r.stdout, r.stdout[:300])
|
||||
|
||||
# 19. (2.1.0) session-end: dry-run writes nothing; apply lands all steps with
|
||||
# the heartbeat LAST; a bad bundle aborts before any write.
|
||||
se_env = dict(os.environ, ECHO_BASE=base, ECHO_KEY=KEY, ECHO_VERIFY="1",
|
||||
ECHO_TODAY="2026-06-21", ECHO_NOW="2359")
|
||||
bundle = {"slug": "quickwins-ship",
|
||||
"log_body": "---\ntype: session-log\nstatus: complete\ncreated: 2026-06-21\n---\n"
|
||||
"# Session Log\n\n## Goal\nShip the quick wins.\n",
|
||||
"agent_log_line": "- 2026-06-21: shipped the quick wins",
|
||||
"reflect": [{"title": "Quark Preference", "kind": "semantic",
|
||||
"body": "The operator prefers quark.", "confidence": 0.9}]}
|
||||
bfile = HERE / "_session_bundle.json"
|
||||
bfile.write_text(json.dumps(bundle), encoding="utf-8")
|
||||
sess_path = "_agent/sessions/2026-06-21-2359-quickwins-ship.md"
|
||||
r = subprocess.run([sys.executable, str(ECHO), "session-end", str(bfile)],
|
||||
capture_output=True, text=True, env=se_env)
|
||||
check("v2.1 session-end dry-run previews the plan",
|
||||
"dry-run" in r.stdout and sess_path in r.stdout, r.stdout)
|
||||
check("v2.1 session-end dry-run writes nothing", h.ground(sess_path) is None)
|
||||
r = subprocess.run([sys.executable, str(ECHO), "session-end", str(bfile), "--apply"],
|
||||
capture_output=True, text=True, env=se_env)
|
||||
check("v2.1 session-end apply lands the session log",
|
||||
"Ship the quick wins" in (h.ground(sess_path) or ""), r.stdout + r.stderr)
|
||||
check("v2.1 session-end apply routes the reflect proposal",
|
||||
h.ground("_agent/memory/semantic/quark-preference.md") is not None)
|
||||
check("v2.1 session-end apply writes the heartbeat last (commit marker)",
|
||||
sess_path in (h.ground("_agent/heartbeat/last-session.md") or ""))
|
||||
daily = h.ground("journal/daily/2026-06-21.md") or ""
|
||||
check("v2.1 session-end apply appends the agent-log line",
|
||||
"shipped the quick wins" in daily, daily[-300:])
|
||||
bad = dict(bundle, slug="Bad_Slug!")
|
||||
bfile.write_text(json.dumps(bad), encoding="utf-8")
|
||||
r = subprocess.run([sys.executable, str(ECHO), "session-end", str(bfile), "--apply"],
|
||||
capture_output=True, text=True, env=se_env)
|
||||
bfile.unlink()
|
||||
check("v2.1 session-end rejects a bad slug before any write",
|
||||
r.returncode == 2 and sess_path in (h.ground("_agent/heartbeat/last-session.md") or ""),
|
||||
r.stderr)
|
||||
|
||||
print(f"\n{len(failures)} failure(s)" if failures else "\nall feature tests passed")
|
||||
return 1 if failures else 0
|
||||
finally:
|
||||
|
||||
@@ -115,6 +115,40 @@ def main():
|
||||
check("offline load flags OFFLINE", "OFFLINE" in r.stdout, r.stdout)
|
||||
check("offline load serves cached marker", "EVALMARK-marker" in r.stdout, r.stdout)
|
||||
|
||||
# --- Phase 5 (2.1.0): vault DOWN -> capture queues a SEMANTIC record ------
|
||||
r = echo(DEAD, "capture", "Offline Person", "--kind", "person")
|
||||
check("offline capture exits 0 (queued)", r.returncode == 0, r.stdout + r.stderr)
|
||||
check("offline capture reports queued", "queued (offline): capture" in r.stdout, r.stdout)
|
||||
outbox = Path(state) / "outbox.ndjson"
|
||||
check("capture queued as an op record",
|
||||
outbox.exists() and '"op": "capture"' in outbox.read_text(encoding="utf-8"))
|
||||
# same capture again while offline -> deduped by idem_key, not double-queued
|
||||
echo(DEAD, "capture", "Offline Person", "--kind", "person")
|
||||
recs = [ln for ln in outbox.read_text(encoding="utf-8").splitlines() if '"op": "capture"' in ln]
|
||||
check("offline capture is idempotent in the queue", len(recs) == 1, str(len(recs)))
|
||||
|
||||
# --- Phase 6 (2.1.0): vault UP -> flush replays capture THROUGH capture ---
|
||||
srv = start_mock()
|
||||
r = echo(mock_base, "flush")
|
||||
check("flush replays the queued capture", "flushed" in r.stdout, r.stdout + r.stderr)
|
||||
note = ground("resources/people/offline-person.md")
|
||||
check("replayed capture created the routed note",
|
||||
note is not None and "type: person" in note, str(note)[:200])
|
||||
check("replayed capture used the capture-time date",
|
||||
note is not None and "created: 2026-06-22" in note, str(note)[:200])
|
||||
|
||||
# --- Phase 7 (2.1.0): duplicate gate ON REPLAY keeps + flags the record ---
|
||||
echo(mock_base, "capture", "Zebulon Quargle", "--kind", "person") # the existing entity
|
||||
r = echo(DEAD, "capture", "Zebulon Quargle Junior", "--kind", "person")
|
||||
check("lookalike capture queues offline", "queued (offline)" in r.stdout, r.stdout)
|
||||
r = echo(mock_base, "flush")
|
||||
check("gated replay is flagged, not landed",
|
||||
"needs attention" in r.stdout and "replay-gated" in r.stdout, r.stdout + r.stderr)
|
||||
check("gated replay did NOT create the duplicate",
|
||||
ground("resources/people/zebulon-quargle-junior.md") is None)
|
||||
check("gated record survives in the outbox",
|
||||
'"needs_attention"' in (outbox.read_text(encoding="utf-8") if outbox.exists() else ""))
|
||||
|
||||
print(f"\n{len(failures)} failure(s)" if failures else "\nall offline-queue tests passed")
|
||||
return 1 if failures else 0
|
||||
finally:
|
||||
|
||||
Reference in New Issue
Block a user