echo/CHANGELOG.md

# Changelog

## 2.3.0

**The index train** (IMPROVEMENT-PLANS #2/#4/#8) — one schema-bump release:
recall-index schema 3, entity-index schema 2. Old recall indexes are discarded
and rebuilt automatically (sub-second); no vault migration.

### Changed — the recall index is LOCAL-FIRST

The live BM25 index moves out of the vault into the machine state dir
(`recall-index-<endpoint-hash>.json` under `ECHO_STATE_DIR`). Capture's index
upkeep — previously GET-whole-index → PUT-whole-index against the vault under
the advisory lock, an O(vault) network round trip per write — is now a
zero-network atomic file write with no lock at all. The vault copy at
`_agent/index/recall-index.json` becomes a **snapshot**: written by
`sweep --apply` and best-effort at `session-end`, read only to seed a fresh
machine or to catch up after another client swept (checked at most every
`ECHO_RECALL_SYNC_HOURS`, default 24). The read path never PUTs the vault.

### Added — incremental sweep (`sweep --fast`) + content hashes

Entity-index entries and recall-index doc meta now carry a 16-char content
hash. `sweep --fast` walks the listing (cheap) and fetches only what is new,
gone, missing a hash, or in **today's rotating verification shard**
(`hash(path) % 7 == weekday` — the whole vault gets verified across a week of
fast sweeps while each run stays tiny; `--all-shards` forces everything).
Index-only and machine-owned, so no `--apply` gate; deletions are dropped from
both indexes. **`load` auto-runs it** when this machine's last sweep is older
than `ECHO_FAST_SWEEP_DAYS` (default 7) — edits made directly in Obsidian or by
other clients now reach the indexes without anyone remembering maintenance.
`doctor` reports index freshness.

### Added — stemming + alias query expansion

`echo_stem.py`: a conservative Porter-lite stemmer applied by `tokenize()` at
BOTH index and query time, so "deployed"/"deployment"/"deploys" meet at
"deploy" and "penalties" finds "penalty" (guards against over-stemming: "sing"
is not "s"; the -al/d~s irregulars are deliberately not conflated). At recall
time, a query that fuzzy-matches an entity (score ≥ 0.5) folds that entity's
title/alias vocabulary into the BM25 query at **half weight** — "turbokappa
tuning" finds the Kappa Engine note even though the alias appears in no body.
Expansion can only boost documents that actually contain the terms.

### Eval

Gold set grows 8 → **14 queries** (6 paraphrases the pre-2.3 lexical index
missed). v2.3.0: recall@5 / MRR **1.00 / 1.00**, 3/3 session-journal queries;
keyword baseline 0.75 / 0.79 and 0/3. Tests: +3 unit (stemmer families,
over-stemming guards, hash), +10 end-to-end (local-first no-snapshot-write,
local recallability, stemmed recall, alias expansion, fast-sweep pickup/
deletion/hash backfill). All suites — including the offline-queue, ops-api,
and MCP server suites — green; test harnesses now isolate `ECHO_STATE_DIR`.

## 2.2.0

### Added — echo-mcp: the containerized MCP server

ECHO's operations are now reachable as **typed MCP tools** from any surface with
an MCP connector (Claude Code, CoWork, claude.ai) — no Python-capable shell, no
`$ECHO` path resolution, no output re-parsing. New `mcp-server/` + `Dockerfile`
in this repo; deployed on ALPHA as the `echo-mcp` container behind
`https://echomcp.alwisp.com` (streamable HTTP, stateless JSON, bearer auth,
open `/health` for Docker/Kuma).

- **14 tools** wrapping the 2.1.1 `*_op` cores: `echo_load`, `echo_recall`
  (score-packed `budget_chars`), `echo_resolve`, `echo_get_note`
  (`section`/`max_chars`, traversal-guarded), `echo_get_scope`/`echo_set_scope`,
  `echo_health` (`deep` runs the linter), `echo_capture` (gate as data —
  `merge_into`/`force` are parameters), `echo_link` (paths OR resolvable names),
  `echo_append_note`/`echo_patch_note` (invalid-target errors include the note's
  actual headings), `echo_triage_inbox` (list/preview/apply in one tool),
  `echo_reflect`, `echo_log_session` (the session-end bundle, heartbeat-last).
- **Tool profiles**: `ECHO_MCP_TOOLS=core` exposes only the six daily drivers.
- **Vault adjacency**: the container talks to the Obsidian REST API's HTTP
  binding on the same box (`ECHO_BASE=http://10.2.0.35:27123`) — vault ops stop
  depending on Cloudflare/NPM/DNS; the public chain only fronts the `echomcp`
  ingress. Server-side offline queue at `/data`.
- **All MCP writes serialize** through the server process; the vault advisory
  lock still coordinates with CLI clients.
- SKILL.md: prefer the `echo_*` tools when present; CLI recipes are the fallback.
- New suite `eval/test_mcp_server.py` (skips without the SDK): health/auth/
  initialize/tools-list + the capture→gate→merge→recall→log_session flow over
  real streamable HTTP.

The plugin itself is unchanged apart from the SKILL.md note — the container
vendors `skills/echo-memory/scripts/` at build time (one canonical tree).

## 2.1.1

### Changed — Phase 0 of the MCP build: every high-level op returns an envelope

Internal refactor (MCP-SERVER-SPEC §4), no behavior change intended: each
high-level operation now has a core `*_op` function that **returns an envelope
dict and never prints to stdout** — the transport-agnostic seam the upcoming MCP
server (2.2) wraps. The CLI verbs are thin wrappers that print exactly what they
printed before and map envelope outcomes to the same exit codes.

- `echo_ops`: `capture_op` / `resolve_op` / `link_op` — the duplicate gate and
  offline queueing are now **data** (`action: duplicate-gate` with candidates;
  `queued: true`), mapped to exit 76 / "queued" text only in the wrapper. The
  `--json` stdout-swallowing hack is gone: helper chatter from the low-level
  verbs routes to **stderr** wholesale inside the cores. `capture_op` accepts
  `body_text=` directly (no temp file needed — the MCP path).
- `echo_recall.recall_op` — one structured result backs both the prose and
  `--json` renderings (the duplicated prose path was deleted).
- `echo_triage.list_op` / `route_op`, `echo_reflect.apply_op`,
  `echo_session.session_end_op` — same pattern; reflect/triage/session now call
  `capture_op` internally and count gate outcomes from envelopes.
- `echo.scope_show_op` / `scope_set_op` / `load_op` (sections + brief digest as
  data), `echo_doctor.run_op` (checks as a list).

New suite `eval/test_ops_api.py` asserts the contract for every core: envelope
shape AND stdout purity (a stray print would corrupt an MCP response stream).
All existing suites pass unchanged.

## 2.1.0

The "quick-wins train" from the 2026-07-28 review (`docs/IMPROVEMENT-PLANS.md` #1,
#3, #7). Three independently useful changes; no schema or layout changes.

### Added — `load --brief`: a token-budgeted cold-start digest

The SessionStart hook injected the full text of six files every session (measured
15.9KB and growing without bound — Observations and Scope History never shrink).
`load --brief` renders a digest instead: Fact / Pattern rules in full, the last ~10
Observations (with a `+N older` note), scope + `scope_updated` + sessions-since,
the heartbeat-pointed session log's key sections (Goal / Decisions Made / Open
Threads / Suggested Next Step), today's Agent Log lines, and the inbox as a
**count + oldest age** — everything the load-time reconcile needs. Over
`ECHO_LOAD_BUDGET` (default ~8000 chars), sections trim lowest-priority-first
(observations → agent log → session summary) with explicit `(truncated)` markers.
The hook now injects the brief form; `/echo-load` and the manual procedure keep the
full dump. All `load` reads (plus the sessions listing, which also feeds the
heartbeat-absent fallback) are fetched **in parallel** over the pooled connections.

### Fixed — capture is now offline-durable (the write path's inversion bug)

Low-level verbs queued on an outage, but the *default* write (`capture`) died: the
update path's POST/PATCH and `ensure_daily_log` bypassed the queue entirely, and an
unreachable index aborted the whole op. Now: (1) a fully-offline `capture` queues
the **whole operation as one semantic record** (title/kind/body/tags/…, plus the
capture-time date); `flush` replays it **through `capture`** so create-vs-update,
the duplicate gate, and aliasing re-run against the index as it is at replay time —
byte-level replay would freeze a decision made blind. (2) A duplicate-gate stop on
replay keeps the record, flags it `needs_attention` (surfaced by `flush` and
`load`), and does NOT block later records. (3) Mid-capture outages: the update
path and `ensure_daily_log` writes ride `safe_request` (accepted edge: a queued
agent-log PATCH 400-drops loudly if the daily note never existed — the trace line,
not the memory). Same-args offline captures dedupe by idem-key.

### Added — `session-end`: the one-call session close

`echo.py session-end <bundle.json> [--apply]` performs the whole end-of-session
sequence under ONE lock acquisition, in order: session log PUT → Agent Log line →
reflect proposals (via `capture`, gate-aware) → optional `scope set` → **heartbeat
LAST as the commit marker** (a part-way failure leaves the previous pointer intact,
never dangling). Dry-run by default previews the full plan including the reflect
classification. Filename HHMM from `$ECHO_NOW` (else local clock), validated
against the canonical session-log regex **before any write**. The Stop hook's nudge
now names the command, and recognizes a `session-end` invocation as
"already reflected".

### Fixed — helper-module errors exited 2 instead of their intended codes

When `echo.py` runs as `__main__`, helper modules' `import echo` creates a twin
module whose `EchoError` is a different class object, so `main()`'s handler missed
it and the raw traceback leaked. The handler now catches `RuntimeError` (both
classes subclass it) and honors `.code`.

Tests: +19 end-to-end checks (brief-load digest/budget, offline-capture queue /
replay-through-capture / gate-on-replay / idem-key dedup, session-end dry-run /
apply / ordering / validation-abort). All suites green.

## 2.0.0

**Packaging & structure major — nothing about memory behavior changes.** The major
version signals "reinstall, re-clone expectations": how the plugin is distributed
and installed is what breaks.

### Removed — the legacy `commands/` directory (BREAKING for pre-skills clients)

The plugin is **skills-only**. The eight `/echo-*` entry points live exclusively at
`skills/<name>/SKILL.md` (introduced 1.6.0, verified on desktop + CoWork
2026-07-28, where they already took precedence). A Claude Code old enough to lack
skills support loses the slash commands — install 1.6.0 from the release history
instead.

### Changed — artifact policy (BREAKING for anyone pulling zips from the repo)

- The repo tracks **only** `echo-memory.plugin` (the current installable). The 15
  historical `echo-memory-<version>.plugin` zips are deleted from the tree (git
  history retains them).
- Versioned builds are published as **Gitea releases** on
  `git.alwisp.com/jason/echo` — one release per `v<version>` tag, artifact
  attached, starting with `v2.0.0`.
- `.gitignore` now ignores `*.plugin` except the pointer.

### Added — "packaging for other agent runtimes" README note

The retired Codex tree stays retired; the README now documents what a port needs
(manifest shape, skill entry point, script invocation + hooks) and the rule that a
port must be a **build target** from the canonical source tree, never a second
hand-maintained copy.

## 1.6.0

### Changed — the eight slash commands migrated to the skills format

Each `commands/<name>.md` now has a `skills/<name>/SKILL.md` twin with the same
invocation name and a byte-identical body — confirmed against the current docs
that a same-name skill **takes precedence** over the legacy command, so both
formats coexist safely in this release (deleting `commands/` is the 2.0
packaging break). Done for the features, not the deprecation notice:

- **`allowed-tools` per skill** — the resolved `python3/python/py -3` script
  invocations (plus the CoWork `ls /sessions/*` fallback probe) are
  pre-authorized, so `/echo-load`, `/echo-health`, `/echo-recall` etc. stop
  prompting.
- **`disable-model-invocation: true`** on the write-heavy entry points
  (`/echo-sweep`, `/echo-triage`) — only the operator triggers them; the
  read-side skills stay model-invocable.
- `argument-hint` / `$ARGUMENTS` carry over unchanged (same semantics in
  SKILL.md bodies).
- The `$ECHO` path-resolution block stays per-skill: the skills format has no
  cross-skill snippet sharing (each skill directory is self-contained per the
  official docs), and the block is already the 2-line minimum.

### Fixed — lint blind spot: retired trees masked by the leaf-README route

`vault_lint.py` checked retired patterns only for paths matching **no** route,
so the permissive `leaf-readme` route (`^(.+/)?README\.md$`) absolved any seed
README inside a retired tree — `archive/` and `_agent/outputs/` survived the
1.5.0 live cleanup unflagged. Retired patterns are now checked **first**: a
path in a retired tree flags `retired-path` regardless of what else matches.
New end-to-end test seeds `archive/notes/README.md` and asserts the flag
(fails against the pre-fix linter, passes now).

### Added — plugin.json completeness

`homepage` + `repository` → `https://git.alwisp.com/jason/echo`; manifest →
1.6.0.

## 1.5.1

### Fixed — duplicate gate over-firing on vault-common tokens and cross-kind names

Live 1.5.0 use surfaced a false positive: creating the decision "echo-memory 1.5.0
improvement set" was blocked by the archived project `echo-v-05` (score 1.0), because
`score = overlap / min(|tokens|)` lets any entity whose single distinctive token
appears in the new title score 1.0 — and "echo" appears across many entities in an
echo-centric vault.

New `echo_index.gate_candidates()` (backed by `token_df()`) applies two precision
rules before blocking; `capture` gates through it. `fuzzy_candidates()` — the advisory
warning list — is unchanged:

- **Same-kind only.** A cross-kind name collision (a decision titled after its
  project, a meeting named for a company) is intentional naming, not a duplicate.
  Cross-kind lookalikes still warn, never block.
- **No single-common-token blocks.** A lone shared token gates only when it is unique
  to that entity (vault df == 1). Multi-token overlaps still gate even when the
  individual tokens are common.

Exit-76 / `--merge-into` / `--force` semantics unchanged. Tests: 3 new offline unit
tests (`gate_candidates`), end-to-end gate cases restructured + 2 new (cross-kind
no-gate, common-token no-gate); verified against the live vault (the original false
positive now creates cleanly; a same-kind multi-token lookalike still gates).

## 1.5.0

Six improvements from the July 2026 review + the frontmatter-drift field report.

### Fixed — capture-on-update kept only the first body line (silent data loss)

`capture` on an existing entity appended `- <date>: <first line>` and **discarded the
rest of the body**. It now appends the whole body as a dated block — first line on the
bullet, continuation lines indented under it. Idempotency (bullet-per-day) unchanged.

### Added — frontmatter completeness (prevent → tool → detect → remediate)

A field audit found 18/71 entity notes missing `status`/`tags`; the drift was created
at write time and invisible to the tooling. Four coordinated changes, driven by one
data source (`KIND_STATUS` / `KIND_REQUIRED_FM` in `echo_index.py`):

- **`capture`** stamps a kind-appropriate default `status` (all kinds, not just
  projects) and seeds `tags` with the note's kind; new `--tags a,b` enriches it.
  Reflect/triage proposals accept a `"tags"` field.
- **`echo.py fm`** is now **create-or-replace**: on `400 invalid-target` (missing key)
  it surgically inserts the one line into the frontmatter block and re-PUTs, verified —
  it previously failed on exactly the notes that needed repair.
- **`vault_lint.py`** gains a kind-scoped `incomplete-frontmatter` check
  (missing/empty `status`/`tags` on entity notes; block-style lists handled).
- **`sweep.py --apply`** backfills flagged notes (kind-default status + baseline tag).

### Added — pre-write duplicate gate in capture

A strong fuzzy candidate (score ≥ 0.6, env `ECHO_DUP_GATE`) now **stops** `capture`
before it creates a near-duplicate (exit `76` + candidate list) instead of warning
after the file exists. `--merge-into <slug>` routes the capture as an update to the
existing entity (mention learned as an alias); `--force` creates anyway. Weak
resemblances still create + warn as before.

### Changed — recall corpus + ranking (recall-index schema 2)

- The BM25 corpus now includes **session logs and journal notes** (down-weighted 0.5 /
  0.4 so entity notes win ties) — past discussions are findable without having been
  promoted into an entity note. `echo.py put` keeps the index current for corpus paths.
- Ranking fuses BM25 with **freshness** (half-life decay on `updated:`, floored at 0.6,
  env `ECHO_FRESH_HALF_LIFE`) and **status** (`active` ×1.1, `on-hold` ×0.9,
  `archived` ×0.75) — a stale archived note no longer outranks the live one.
- Hits print `updated:`/`status:`; `recall --json` emits structured results.
- The recall index carries per-doc meta (schema 2); an old index is discarded and
  rebuilt automatically on the next recall/sweep.

### Added — session hooks (self-firing load & reflect)

New `hooks/hooks.json` + two hook scripts. **SessionStart** runs `echo.py load` and
injects the output as context (skips resume/compact; NOT-CONFIGURED degrades to an
instruction, never an error). **Stop** nudges once per substantive session (≥5 user
turns, env `ECHO_REFLECT_MIN_TURNS`) when nothing was reflected or session-logged —
reflection itself still previews and asks before writing. Both use the 1.4.2 CoWork
path fallback and always exit 0.

### Added — one-tap inbox triage + `--json` on read verbs

New `echo.py triage`: `--list --json` parses the inbox into `{line, date, text,
age_days}`; a proposals file (reflect schema + `"line"`) is classified → previewed →
applied via `capture`, with the `inbox/processing-log/` audit line written per move
(originals never deleted). `/echo-triage` rewritten around it. Also: `--json` for
`recall`, `link`, and `scope show` (`resolve`/`capture` already had it).

Tests: +22 feature cases (mock end-to-end incl. hooks), patch-semantics updated for
the new `fm` contract; all suites green. Docs: SKILL.md, README, command docs updated.

## 1.4.2

### Fixed — CoWork sandbox plugin-path resolution

In a remote CoWork sandbox, `${CLAUDE_PLUGIN_ROOT}` can resolve to a host path the
sandbox can't reach (e.g. `/var/folders/…`), while the plugin is actually mounted under
`…/mnt/.remote-plugins/…`. Every `python3 "${CLAUDE_PLUGIN_ROOT}/…"` invocation then
failed until the agent manually located the real path. (Root cause is the harness env
var, not the plugin — the scripts always used the variable correctly, never a hardcoded
path — but the plugin now self-heals.)

- SKILL.md and all eight slash commands resolve the scripts path with a fallback:
  prefer `${CLAUDE_PLUGIN_ROOT}`, else locate the copy under
  `/sessions/*/mnt/.remote-plugins/*/skills/echo-memory/scripts/`. Reused via
  `$ECHO`/`$LINT`/`$SWEEP`/`$SDIR`; on a normal host the primary path always wins, so
  behaviour is unchanged.
- `echo-health` / `echo-sweep` `allowed-tools` broadened to match the resolved
  (variable-dir) invocation and the `ls`/`dirname` resolver steps.

## 1.4.1

### Fixed — baked credentials now win unconditionally

A per-user baked artifact could still be prompted for credentials on install. The
baked `DEFAULT_*` tier was lowest priority, so a stale `ECHO_*` env var or a
leftover/placeholder `~/.claude/echo-memory/config.json` shadowed the baked key —
even an unedited `"<placeholder>"` value won over it, flipping `is_configured()` to
false and triggering the first-run "ask the operator" flow despite valid baked creds.

- `echo_config.load()` now treats a **complete** baked set (endpoint + key both
  present) as authoritative: it wins over env vars and the config file and cannot be
  shadowed. The generic (unbaked) plugin keeps its env → file → first-run flow.
- Added `baked_complete()`; `source()` reports `baked-in (plugin)` for that tier.

### Security — untracked leaked baked artifacts

- Two labeled baked artifacts (`echo-memory-1.4.0-jason.plugin`,
  `echo-memory-1.4.0-gretchen.plugin`), each carrying a live vault bearer token, had
  been committed at the repo root. They are now `git rm --cached`-removed and
  `.gitignore` gains `echo-memory-*-*.plugin` so labeled builds can't be re-committed
  (the version-only pointer `echo-memory.plugin` and `echo-memory-<ver>.plugin` stay
  tracked). Tokens remain in prior history — rotate if the repo was ever shared.

## 1.4.0

### Added — per-user baked-key builds (CoWork zero-touch)

The plugin is the only thing reliably mounted into a CoWork session (the sandbox's
`~/.claude` is synthetic, not your real one), so the config now *can* travel inside
the artifact. `echo_config` gains a lowest-priority fallback tier — the
`DEFAULT_OWNER` / `DEFAULT_BASE` / `DEFAULT_KEY` constants — **empty in source**, so
the committed tree still ships zero credentials.

- `build.py --bake-key --from <config.json> [--label <name>]` substitutes a single
  user's owner/endpoint/key into those constants, producing a per-user artifact that
  needs **no config file and no per-session paste**. Values may also come from
  `--owner/--endpoint/--key` or `ECHO_OWNER/ECHO_BASE/ECHO_KEY`.
- Baked builds default to `dist/` (gitignored) and never touch the shared
  `echo-memory.plugin` pointer.
- `build.py --strip-key` now force-blanks the same constants (guaranteed token-free).
- `config show` / `doctor` report `[baked-in default]` when a field resolves from the
  baked tier.

### Changed — artifact hygiene

- `.gitignore` excludes `dist/` and the filled-in `echo-memory.config.json`, where
  baked builds and the key file are meant to live. A baked artifact carries a vault
  key and must be **delivered directly to that one user, never committed, pushed, or
  published.** (Note: labeled baked artifacts built at the repo *root* were not yet
  guarded — see 1.4.1.)

### Resolution order (unchanged for hosts)

`env` → `$ECHO_CONFIG` file → canonical `~/.claude/echo-memory/config.json` →
**baked `DEFAULT_*`**. On a normal desktop the empty defaults mean behaviour is
identical to 1.3.x; the baked tier only matters in per-user artifacts.