diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cb9f21..01c909e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,33 @@ # Changelog +## 2.1.1 + +### Changed — Phase 0 of the MCP build: every high-level op returns an envelope + +Internal refactor (MCP-SERVER-SPEC §4), no behavior change intended: each +high-level operation now has a core `*_op` function that **returns an envelope +dict and never prints to stdout** — the transport-agnostic seam the upcoming MCP +server (2.2) wraps. The CLI verbs are thin wrappers that print exactly what they +printed before and map envelope outcomes to the same exit codes. + +- `echo_ops`: `capture_op` / `resolve_op` / `link_op` — the duplicate gate and + offline queueing are now **data** (`action: duplicate-gate` with candidates; + `queued: true`), mapped to exit 76 / "queued" text only in the wrapper. The + `--json` stdout-swallowing hack is gone: helper chatter from the low-level + verbs routes to **stderr** wholesale inside the cores. `capture_op` accepts + `body_text=` directly (no temp file needed — the MCP path). +- `echo_recall.recall_op` — one structured result backs both the prose and + `--json` renderings (the duplicated prose path was deleted). +- `echo_triage.list_op` / `route_op`, `echo_reflect.apply_op`, + `echo_session.session_end_op` — same pattern; reflect/triage/session now call + `capture_op` internally and count gate outcomes from envelopes. +- `echo.scope_show_op` / `scope_set_op` / `load_op` (sections + brief digest as + data), `echo_doctor.run_op` (checks as a list). + +New suite `eval/test_ops_api.py` asserts the contract for every core: envelope +shape AND stdout purity (a stray print would corrupt an MCP response stream). +All existing suites pass unchanged. + ## 2.1.0 The "quick-wins train" from the 2026-07-28 review (`docs/IMPROVEMENT-PLANS.md` #1, diff --git a/README.md b/README.md index e17007f..bf9045b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# echo-memory — v2.1.0 +# echo-memory — v2.1.1 Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`. @@ -421,6 +421,7 @@ From the credential-free harness (`eval/run_eval.py` against the deterministic m | Version | Highlights | |---------|-----------| +| **2.1.1** | **MCP Phase 0 — return-not-print.** Every high-level op gains a core `*_op` function returning an envelope dict with a stdout-purity guarantee (helper chatter → stderr); CLI verbs become thin wrappers with identical output and exit codes. Duplicate gate and offline queueing become data (`action: duplicate-gate` / `queued: true`). New `eval/test_ops_api.py` contract suite. This is the seam the 2.2 containerized MCP server wraps. | | **2.1.0** | **Quick-wins train: cheaper sessions, durable capture, one-call session end.** (1) **`load --brief`** — a token-budgeted cold-start digest (Fact/Pattern in full, last ~10 Observations, scope+freshness, last session's key sections, Agent-Log lines, inbox *count*; `ECHO_LOAD_BUDGET` default ~8000 chars) now injected by the SessionStart hook, replacing the full six-file dump that grew unboundedly; all `load` reads are fetched in parallel. (2) **Offline capture durability** — `capture` on an unreachable vault queues the *whole operation* as one semantic record; `flush` replays it **through capture** so routing/gate/aliasing re-run against the current index; a gate stop on replay is kept + flagged, never landed blind; `ensure_daily_log`/update-path writes ride the queue too. (3) **`session-end`** — one call (one lock) writes session log → Agent-Log line → reflect proposals → optional scope switch → **heartbeat last as the commit marker**; dry-run by default; `ECHO_NOW` pins the HHMM. Also fixes the `__main__` twin-module trap so helper-module `EchoError`s exit with their intended codes. +19 end-to-end checks. | | **2.0.0** | **Packaging & structure major — memory behavior unchanged.** Skills-only: the legacy `commands/` directory is deleted (breaking for pre-skills clients; the 1.6.0-verified skills are the sole entry points). Artifact policy: the repo tracks only the `echo-memory.plugin` pointer; the 15 historical versioned zips leave the tree and versioned builds ship as **Gitea releases** (one per `v` tag) from `v2.0.0` on; `.gitignore` blocks `*.plugin` except the pointer. README gains the "packaging for other agent runtimes" port note (build-target rule, never a second source tree). | | **1.6.0** | **Skills-format migration + lint blind-spot fix.** The eight slash commands gain `skills//SKILL.md` twins (same names, byte-identical bodies; a same-name skill takes precedence, so `commands/` coexists until its 2.0 deletion): per-skill `allowed-tools` end the permission prompts, `disable-model-invocation: true` makes `/echo-sweep` + `/echo-triage` operator-only. `vault_lint` checks retired patterns **before** routes, so a seed README inside a retired tree (`archive/…`) is flagged instead of being absolved by the leaf-README route (+ regression test). `plugin.json` gains `homepage`/`repository`. | diff --git a/docs/MCP-SERVER-SPEC.md b/docs/MCP-SERVER-SPEC.md index 5bc1f8d..bc84b14 100644 --- a/docs/MCP-SERVER-SPEC.md +++ b/docs/MCP-SERVER-SPEC.md @@ -75,7 +75,7 @@ monitor, PORT deploy/rollback. Server down ⇒ exactly today's behavior. | Tool prefix | `echo_` | Namespace safety next to other servers. | | Result shape | `structuredContent` (the `echo_output.envelope` dict) + a 1–3 line human text block | Envelope shape already exists. | | Statefulness | Stateless protocol; warm in-process caches | See §7. | -| Version target | **2.2** (1.6.0, 2.0.0, and the 2.1.0 quick-wins train all shipped 2026-07-28; `session-end` exists) | Remaining prerequisite: Phase 0 (return-not-print), the build session's first step. | +| Version target | **2.2** — ALL prerequisites shipped 2026-07-28 (`session-end` in 2.1.0; **Phase 0 in 2.1.1**, contract-tested by `eval/test_ops_api.py`) | The build session starts directly at §5 (the server app). | | Local stdio variant | **Dropped for v1** (documented, not built) | The CLI/skill fallback covers the no-server case; two transports = two test matrices for little gain. Phase 0 keeps the door open — the `*_op` core is transport-agnostic. | | Result budgets | Every read tool takes an explicit size budget (`budget_chars` / `max_chars`) and truncates with a marker + "call X for more" | The single biggest token lever: one right-sized answer instead of follow-up fetches. | | Tool profiles | `ECHO_MCP_TOOLS=core\|full` env: `core` exposes only load/recall/capture/triage_inbox/log_session/health | Tool schemas cost context on every surface that lists them; the claude.ai connector doesn't need the escape hatches. Desktop registers `full`. | diff --git a/echo-memory.plugin b/echo-memory.plugin index 8659cd4..6ed6d87 100644 Binary files a/echo-memory.plugin and b/echo-memory.plugin differ diff --git a/echo-memory.plugin.src/.claude-plugin/plugin.json b/echo-memory.plugin.src/.claude-plugin/plugin.json index 5f57621..840f542 100644 --- a/echo-memory.plugin.src/.claude-plugin/plugin.json +++ b/echo-memory.plugin.src/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "echo-memory", - "version": "2.1.0", + "version": "2.1.1", "homepage": "https://git.alwisp.com/jason/echo", "repository": "https://git.alwisp.com/jason/echo", "description": "Persistent memory via the ECHO Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /echo-* commands.", diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo.py index 30eed46..1898dd0 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo.py @@ -549,59 +549,76 @@ def extract_heading(markdown: str, heading: str) -> str: return "\n".join(out).strip() -def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) -> int: +def scope_show_op() -> dict: + """Core scope-show (Phase 0): active scope + freshness + sessions-since, as data.""" path = "_agent/context/current-context.md" status, body = request("GET", vault_url(path)) - check(status, body, f"scope {subcommand}") + check(status, body, "scope show") current = body.decode(errors="replace") + scope_text = extract_heading(current, "Scope") + scope_updated = "" + for line in current.splitlines(): + if line.startswith("scope_updated:"): + scope_updated = line.split(":", 1)[1].strip().strip('"').strip("'") + break + sessions_since = None + # Count session logs dated after scope_updated (the drift signal). + if scope_updated: + status, body = request("GET", vault_url("_agent/sessions/")) + if status == 200: + try: + files = json.loads(body).get("files", []) + sessions_since = sum(1 for f in files if f.endswith(".md") and f[:10] > scope_updated) + except json.JSONDecodeError: + pass + return {"ok": True, "action": "scope-show", "scope": scope_text, + "scope_updated": scope_updated or None, "sessions_since": sessions_since} - if subcommand == "show": - scope_text = extract_heading(current, "Scope") - scope_updated = "" - for line in current.splitlines(): - if line.startswith("scope_updated:"): - scope_updated = line.split(":", 1)[1].strip().strip('"').strip("'") - break - sessions_since = None - # Count session logs dated after scope_updated (the drift signal). - if scope_updated: - status, body = request("GET", vault_url("_agent/sessions/")) - if status == 200: - try: - files = json.loads(body).get("files", []) - sessions_since = sum(1 for f in files if f.endswith(".md") and f[:10] > scope_updated) - except json.JSONDecodeError: - pass - if as_json: - print(json.dumps({"ok": True, "action": "scope-show", "scope": scope_text, - "scope_updated": scope_updated or None, - "sessions_since": sessions_since}, ensure_ascii=False)) - return 0 - print("-- Active scope --") - print(scope_text) - print(f"scope_updated: {scope_updated or ''}") - if sessions_since is not None: - print(f"sessions logged since: {sessions_since}") - return 0 - if subcommand != "set": - raise EchoError("scope: use 'show' or 'set \"\"'", 2) +def scope_set_op(text: str) -> dict: + """Core scope-set (Phase 0): atomic switch (history + replace + stamp). Chatter + from the underlying PATCHes routes to stderr; returns the switch envelope.""" + import contextlib if not text: raise EchoError("scope set needs the new scope text", 2) + path = "_agent/context/current-context.md" + status, body = request("GET", vault_url(path)) + check(status, body, "scope set") + current = body.decode(errors="replace") prior = extract_heading(current, "Scope").replace("\n", " ").strip()[:140] or "(prior scope)" - cmd_patch(path, "prepend", "heading", "Current Context::Scope History", - temp_file(f"- {today()}: {prior}\n".encode())) - cmd_patch(path, "replace", "heading", "Current Context::Scope", - temp_file(f"{text}\n".encode())) - try: - cmd_patch(path, "replace", "frontmatter", "scope_updated", - temp_file(json.dumps(today()).encode())) - except EchoError as exc: - raise EchoError( - "scope set: body switched, but scope_updated frontmatter is missing " - f"(run bootstrap.py to add it) [{exc}]" - ) - print(f"ok: scope switched (prior archived to Scope History; scope_updated={today()})") + with contextlib.redirect_stdout(sys.stderr): + cmd_patch(path, "prepend", "heading", "Current Context::Scope History", + temp_file(f"- {today()}: {prior}\n".encode())) + cmd_patch(path, "replace", "heading", "Current Context::Scope", + temp_file(f"{text}\n".encode())) + try: + cmd_patch(path, "replace", "frontmatter", "scope_updated", + temp_file(json.dumps(today()).encode())) + except EchoError as exc: + raise EchoError( + "scope set: body switched, but scope_updated frontmatter is missing " + f"(run bootstrap.py to add it) [{exc}]" + ) + return {"ok": True, "action": "scope-set", "scope": text, "prior": prior, + "scope_updated": today()} + + +def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) -> int: + if subcommand == "show": + env = scope_show_op() + if as_json: + print(json.dumps(env, ensure_ascii=False)) + return 0 + print("-- Active scope --") + print(env["scope"]) + print(f"scope_updated: {env['scope_updated'] or ''}") + if env["sessions_since"] is not None: + print(f"sessions logged since: {env['sessions_since']}") + return 0 + if subcommand != "set": + raise EchoError("scope: use 'show' or 'set \"\"'", 2) + env = scope_set_op(text or "") + print(f"ok: scope switched (prior archived to Scope History; scope_updated={env['scope_updated']})") return 0 @@ -730,51 +747,22 @@ def _render_brief(texts: dict[str, str | None], listing_files: list[str], return out -def cmd_load(brief: bool = False) -> int: - """Cold-start orientation: the canonical 6 reads in one call (fetched in parallel). - 404s on today's daily note and the inbox are normal (printed as absent, not errors). - `brief` renders the token-budgeted digest the SessionStart hook injects; the default - full mode (and /echo-load) prints the raw sections unchanged.""" - if not echo_config.is_configured(_CFG): - print("echo: NOT CONFIGURED — this machine has no usable ECHO key file yet.") - print(f"Expected at: {echo_config.config_path()}") - print("ACTION: ask the operator for their echo-memory config file (it holds the") - print(" vault owner, endpoint, and API key), then install it with ONE of:") - print(" python3 echo.py config import ") - print(" python3 echo.py config set --owner \"…\" --endpoint \"https://…\" --key \"…\"") - print("Then re-run load. (Memory is unavailable until configured.)") - return 78 # distinct: configuration required - targets = [ - ("marker", "_agent/echo-vault.md"), - ("preferences", "_agent/memory/semantic/operator-preferences.md"), - ("context", "_agent/context/current-context.md"), - ("heartbeat", "_agent/heartbeat/last-session.md"), - ("today", f"journal/daily/{today()}.md"), - ("inbox", "inbox/captures/inbox.md"), - ] - import echo_queue - # H2: sync any writes queued during a prior outage, best-effort and quiet on empty. - try: - synced = echo_queue.flush() - if synced: - print(f"(synced {synced} write(s) queued during a prior offline session)\n") - flagged = echo_queue.needs_attention() - if flagged: - print(f"NOTE: {len(flagged)} queued write(s) need attention (e.g. a capture " - "stopped at the duplicate gate on replay) — resolve with capture " - "--merge-into/--force; `echo.py flush` re-lists them.\n") - except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load - print(f"(queue flush skipped: {exc})\n", file=sys.stderr) +LOAD_TARGETS = [ + ("marker", "_agent/echo-vault.md"), + ("preferences", "_agent/memory/semantic/operator-preferences.md"), + ("context", "_agent/context/current-context.md"), + ("heartbeat", "_agent/heartbeat/last-session.md"), +] - # All orientation reads (+ the sessions listing) fetched in parallel over the - # warm connection pool — the listing feeds both the brief digest's scope-freshness - # count and the heartbeat-absent fallback, so it's no longer a second round-trip. + +def _load_gather(targets): + """Fetch the orientation reads (+ the sessions listing) in parallel and resolve + each to text via status/cache. Shared by cmd_load and load_op. Returns + (results, texts, listing_files, offline, marker_missing, heartbeat_absent).""" + import echo_queue listing_path = "_agent/sessions/" results = _fetch_statuses([p for _, p in targets] + [listing_path]) - - marker_missing = False - heartbeat_absent = False - offline = False + marker_missing = heartbeat_absent = offline = False texts: dict[str, str | None] = {} for label, path in targets: status, body = results[path] @@ -792,7 +780,6 @@ def cmd_load(brief: bool = False) -> int: marker_missing = True if label == "heartbeat": heartbeat_absent = True - lst_status, lst_body = results[listing_path] listing_files: list[str] = [] if lst_status == 200: @@ -800,15 +787,84 @@ def cmd_load(brief: bool = False) -> int: listing_files = list(json.loads(lst_body).get("files", [])) except json.JSONDecodeError: pass + return results, texts, listing_files, offline, marker_missing, heartbeat_absent + + +def _fetch_pointed_session(texts: dict) -> None: + """Follow the heartbeat pointer and stash the pointed session log for the digest.""" + hb = texts.get("heartbeat") + if hb and hb.strip(): + sess_path = hb.strip().splitlines()[0].split(" @ ", 1)[0].strip() + if sess_path: + st2, b2 = request("GET", vault_url(sess_path)) + if st2 == 200: + texts["_pointed_session"] = b2.decode(errors="replace") + + +def load_op(brief: bool = True) -> dict: + """Core load (Phase 0): the orientation reads as data — sections keyed by label, + plus offline/bootstrap flags, queue counts, recent sessions, and (with brief) the + rendered digest. Raises EchoError(78) when the machine is not configured.""" + if not echo_config.is_configured(_CFG): + raise EchoError("NOT CONFIGURED — no usable ECHO key file on this machine " + f"(expected at {echo_config.config_path()})", 78) + import echo_queue + synced = flagged = 0 + try: + synced = echo_queue.flush() + flagged = len(echo_queue.needs_attention()) + except Exception: # noqa: BLE001 — queue upkeep must never block a load + pass + targets = LOAD_TARGETS + [("today", f"journal/daily/{today()}.md"), + ("inbox", "inbox/captures/inbox.md")] + _, texts, listing_files, offline, marker_missing, _ = _load_gather(targets) + data = {"ok": True, "action": "load", "offline": offline, + "marker_missing": marker_missing, "synced": synced, + "needs_attention": flagged, + "sections": {k: v for k, v in texts.items() if not k.startswith("_")}, + "recent_sessions": sorted((f for f in listing_files if f.endswith(".md")), + reverse=True)[:5]} + if brief: + _fetch_pointed_session(texts) + data["brief"] = _render_brief(texts, listing_files, offline) + return data + + +def cmd_load(brief: bool = False) -> int: + """Cold-start orientation: the canonical 6 reads in one call (fetched in parallel). + 404s on today's daily note and the inbox are normal (printed as absent, not errors). + `brief` renders the token-budgeted digest the SessionStart hook injects; the default + full mode (and /echo-load) prints the raw sections unchanged.""" + if not echo_config.is_configured(_CFG): + print("echo: NOT CONFIGURED — this machine has no usable ECHO key file yet.") + print(f"Expected at: {echo_config.config_path()}") + print("ACTION: ask the operator for their echo-memory config file (it holds the") + print(" vault owner, endpoint, and API key), then install it with ONE of:") + print(" python3 echo.py config import ") + print(" python3 echo.py config set --owner \"…\" --endpoint \"https://…\" --key \"…\"") + print("Then re-run load. (Memory is unavailable until configured.)") + return 78 # distinct: configuration required + targets = LOAD_TARGETS + [("today", f"journal/daily/{today()}.md"), + ("inbox", "inbox/captures/inbox.md")] + import echo_queue + # H2: sync any writes queued during a prior outage, best-effort and quiet on empty. + try: + synced = echo_queue.flush() + if synced: + print(f"(synced {synced} write(s) queued during a prior offline session)\n") + flagged = echo_queue.needs_attention() + if flagged: + print(f"NOTE: {len(flagged)} queued write(s) need attention (e.g. a capture " + "stopped at the duplicate gate on replay) — resolve with capture " + "--merge-into/--force; `echo.py flush` re-lists them.\n") + except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load + print(f"(queue flush skipped: {exc})\n", file=sys.stderr) + + results, texts, listing_files, offline, marker_missing, heartbeat_absent = \ + _load_gather(targets) if brief: - hb = texts.get("heartbeat") - if hb and hb.strip(): # follow-up: the pointed session log's key sections - sess_path = hb.strip().splitlines()[0].split(" @ ", 1)[0].strip() - if sess_path: - st2, b2 = request("GET", vault_url(sess_path)) - if st2 == 200: - texts["_pointed_session"] = b2.decode(errors="replace") + _fetch_pointed_session(texts) print(_render_brief(texts, listing_files, offline)) return 0 diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_doctor.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_doctor.py index 521b1a8..e308768 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_doctor.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_doctor.py @@ -22,17 +22,18 @@ import echo # noqa: E402 MIN_PY = (3, 9) -def run() -> int: - reds = 0 +def run_op() -> dict: + """Core doctor (Phase 0): the readiness checks as data — {checks: [{ok, label, + detail}], endpoint, fatal, ok}. `fatal` names an early-exit condition (not + configured / unreachable) after which later checks were skipped.""" + checks: list[dict] = [] def line(ok: bool, label: str, detail: str = "") -> None: - nonlocal reds - reds += 0 if ok else 1 - print(f" [{'OK ' if ok else 'RED'}] {label}" + (f" — {detail}" if detail else "")) + checks.append({"ok": ok, "label": label, "detail": detail}) import echo_config cfg = echo_config.load() - print(f"echo doctor — endpoint {cfg['endpoint'] or '(not configured)'}") + fatal = None # 1. interpreter line(sys.version_info >= MIN_PY, f"python >= {MIN_PY[0]}.{MIN_PY[1]}", @@ -53,33 +54,50 @@ def run() -> int: line(bool(cfg["owner"]), "vault owner set", cfg["owner"] if cfg["owner"] else "optional, but recommended for third-person writes") if not echo_config.is_configured(cfg): + fatal = "not-configured" + else: + # 3. reachability + auth + marker, in one GET of the bootstrap marker + status, body = echo.request("GET", echo.vault_url("_agent/echo-vault.md")) + if status == 0: + line(False, "vault reachable", body.decode(errors="replace")[:120]) + fatal = "unreachable" + else: + line(status not in (401, 403), "auth accepted", + f"HTTP {status}" if status in (401, 403) else "") + if status == 404: + line(False, "vault bootstrapped", "marker absent — run bootstrap.py") + elif status == 200: + text = body.decode(errors="replace") + ver = next((ln.split(":", 1)[1].strip() for ln in text.splitlines() + if ln.startswith("schema_version:")), "?") + line(True, "vault bootstrapped", f"schema_version={ver}") + else: + line(status < 400, "marker fetch", f"HTTP {status}") + + reds = sum(1 for c in checks if not c["ok"]) + return {"ok": reds == 0 and fatal is None, "action": "doctor", + "endpoint": cfg["endpoint"], "fatal": fatal, "reds": reds, "checks": checks} + + +def run() -> int: + env = run_op() + print(f"echo doctor — endpoint {env['endpoint'] or '(not configured)'}") + for c in env["checks"]: + print(f" [{'OK ' if c['ok'] else 'RED'}] {c['label']}" + + (f" — {c['detail']}" if c["detail"] else "")) + if env["fatal"] == "not-configured": print("\ndoctor: not configured — ask the operator for their key file and install it " "(`echo.py config import `, or `config set --owner … --endpoint … --key …`), " "then re-run.") return 1 - - # 3. reachability + auth + marker, in one GET of the bootstrap marker - status, body = echo.request("GET", echo.vault_url("_agent/echo-vault.md")) - if status == 0: - line(False, "vault reachable", body.decode(errors="replace")[:120]) + if env["fatal"] == "unreachable": print("\ndoctor: endpoint unreachable — is Obsidian + the Local REST API running?") return 1 - line(status not in (401, 403), "auth accepted", f"HTTP {status}" if status in (401, 403) else "") - if status == 404: - line(False, "vault bootstrapped", "marker absent — run bootstrap.py") - elif status == 200: - text = body.decode(errors="replace") - ver = next((ln.split(":", 1)[1].strip() for ln in text.splitlines() - if ln.startswith("schema_version:")), "?") - line(True, "vault bootstrapped", f"schema_version={ver}") - else: - line(status < 400, "marker fetch", f"HTTP {status}") - - # 4. invariants pointer. + # invariants pointer. print(" [i] invariants — run `/echo-health` (vault_lint.py) for the full check") - - print(f"\ndoctor: {'all green' if reds == 0 else f'{reds} issue(s) — see RED above'}") - return 1 if reds else 0 + summary = "all green" if env["reds"] == 0 else f"{env['reds']} issue(s) — see RED above" + print(f"\ndoctor: {summary}") + return 1 if env["reds"] else 0 if __name__ == "__main__": diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_ops.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_ops.py index b6abf6d..d63e7bc 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_ops.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_ops.py @@ -37,12 +37,13 @@ LOG_HEADING = { # ---------------------------------------------------------------- resolve ----- -def resolve(mention: str) -> int: +def resolve_op(mention: str) -> dict: + """Core resolve: mention -> match/candidates dict. No printing (Phase 0 — the same + return value backs the CLI wrapper and the MCP `echo_resolve` tool).""" index = idx_mod.load() slug, e = idx_mod.resolve(index, mention) if e: - print(json.dumps({"match": True, "slug": slug, **e}, ensure_ascii=False, indent=2)) - return 0 + return {"match": True, "slug": slug, **e} # No exact match — surface fuzzy candidates so a shortened/expanded name (e.g. "echo # memory" for the project "echo") reveals the existing note instead of looking absent. cands = idx_mod.fuzzy_candidates(index, mention) @@ -54,21 +55,30 @@ def resolve(mention: str) -> int: "these before creating a new note; reuse the right path or `echo.py link`.") else: out["note"] = "no index entry — derive a path with the right --kind and create it" - print(json.dumps(out, ensure_ascii=False, indent=2)) + return out + + +def resolve(mention: str) -> int: + print(json.dumps(resolve_op(mention), ensure_ascii=False, indent=2)) return 0 # ------------------------------------------------------------------- link ----- -def link(a_path: str, b_path: str, as_json: bool = False) -> int: +def link_op(a_path: str, b_path: str) -> dict: + """Core link: reciprocal `## Related` links, returns the change envelope.""" + import echo_output a_changed, b_changed = links.link_bidirectional(a_path, b_path) + return echo_output.envelope("link", {"a": a_path, "b": b_path, + "a_changed": a_changed, "b_changed": b_changed}) + + +def link(a_path: str, b_path: str, as_json: bool = False) -> int: + env = link_op(a_path, b_path) if as_json: - import echo_output - env = echo_output.envelope("link", {"a": a_path, "b": b_path, - "a_changed": a_changed, "b_changed": b_changed}) print(json.dumps(env, ensure_ascii=False)) return 0 - print(f"ok: linked {a_path} <-> {b_path} " - f"(added: {'A' if a_changed else '-'}{'B' if b_changed else '-'})") + print(f"ok: linked {env['a']} <-> {env['b']} " + f"(added: {'A' if env['a_changed'] else '-'}{'B' if env['b_changed'] else '-'})") return 0 @@ -176,52 +186,52 @@ def _append_to_existing(path: str, kind: str, today_s: str, body_text: str) -> N echo.cmd_fm(path, "updated", json.dumps(today_s)) -def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "", - aliases=None, sources=None, tags=None, date: str | None = None, - domain: str = "business", inbox: bool = False, no_log: bool = False, - as_json: bool = False, dry_run: bool = False, force: bool = False, - merge_into: str | None = None) -> int: +def capture_op(kind: str | None, title: str, file_arg: str | None = None, status_v: str = "", + aliases=None, sources=None, tags=None, date: str | None = None, + domain: str = "business", inbox: bool = False, no_log: bool = False, + dry_run: bool = False, force: bool = False, + merge_into: str | None = None, body_text: str | None = None) -> dict: + """Core capture (Phase 0): route + frontmatter + index + auto-link + agent-log, + returning an envelope dict — never printing to stdout (helper chatter from the + low-level verbs is redirected to stderr). Special outcomes are DATA, not exit + codes: `duplicate-gate` (ok=false, candidates), `queued:capture` (queued=true), + `dry-run:*`. Raises echo.EchoError on hard failure. `body_text` may be passed + directly (MCP path); otherwise it is read from file_arg/stdin.""" import contextlib - import io import echo_output import echo_quality - real_stdout = sys.stdout + def chatter(): + # Low-level verbs (cmd_put/cmd_append/cmd_fm) print progress lines; keep + # stdout clean for the envelope by routing them to stderr wholesale. + return contextlib.redirect_stdout(sys.stderr) - def quiet(): - # M4: in --json mode, swallow the helper "ok:" chatter so stdout is clean JSON. - return contextlib.redirect_stdout(io.StringIO()) if as_json else contextlib.nullcontext() - - def done(action: str, path: str, links: int = 0, ok: bool = True, dry: bool = False, - near=None) -> int: - if as_json: - act = f"dry-run:{action}" if dry else action - data = {"kind": kind, "path": path, "title": title, "links_added": links} - if near: - data["near_duplicates"] = near - env = echo_output.envelope(act, data, ok=ok) - print(json.dumps(env, ensure_ascii=False), file=real_stdout) - elif dry: - print(f"would {action} {kind or '-'} -> {path}") - # non-dry human output is the helper "ok:" lines + the summary printed by the caller. - return 0 if ok else 1 - - body_text = echo.read_body(file_arg).decode("utf-8", errors="replace") + if body_text is None: + body_text = echo.read_body(file_arg).decode("utf-8", errors="replace") today_s = echo.today() aliases = [a.strip() for a in (aliases or []) if a.strip()] sources = [s.strip() for s in (sources or []) if s.strip()] tags = [t.strip() for t in (tags or []) if t.strip()] + def env_for(action: str, path: str, links: int = 0, ok: bool = True, dry: bool = False, + near=None, **extra) -> dict: + act = f"dry-run:{action}" if dry else action + data = {"kind": kind, "path": path, "title": title, "links_added": links} + if near: + data["near_duplicates"] = near + data.update(extra) + return echo_output.envelope(act, data, ok=ok) + # Unknown home -> defer to the inbox (a single idempotent capture line). if inbox or not kind: if dry_run: - return done("inbox", "inbox/captures/inbox.md", dry=True) + return env_for("inbox", "inbox/captures/inbox.md", dry=True) line = f"- {today_s}: {title}" if body_text.strip(): line += f" — {body_text.strip().splitlines()[0]}" - with quiet(): + with chatter(): rc = echo.cmd_append("inbox/captures/inbox.md", line) - return done("inbox", "inbox/captures/inbox.md", ok=rc == 0) + return env_for("inbox", "inbox/captures/inbox.md", ok=rc == 0) slug = idx_mod.slugify(title) # Offline short-circuit (2.1.0). The routed path needs the entity index and several @@ -235,9 +245,8 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = if "unreachable" not in str(exc): raise if dry_run: - print("offline: vault unreachable — a real run would queue this capture " - "for replay on the next reachable session.", file=real_stdout) - return 0 + return echo_output.envelope("dry-run:queued:capture", + {"kind": kind, "title": title, "queued": True}) import echo_queue echo_queue.enqueue_capture({ "kind": kind, "title": title, "body_text": body_text, "status_v": status_v, @@ -245,23 +254,16 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "domain": domain, "inbox": False, "no_log": no_log, "force": force, "merge_into": merge_into, "today": today_s, }) - if as_json: - env = echo_output.envelope("queued:capture", - {"kind": kind, "title": title, "queued": True}) - print(json.dumps(env, ensure_ascii=False), file=real_stdout) - else: - print(f"queued (offline): capture {kind} '{title}' — will replay through " - "capture on the next reachable session", file=real_stdout) - return 0 + return echo_output.envelope("queued:capture", + {"kind": kind, "title": title, "queued": True}) match_slug, existing = idx_mod.resolve(index, title) # --merge-into: the operator has already identified the canonical entity (e.g. after # a duplicate-gate stop) — route this capture as an UPDATE to it, whatever the title. if merge_into and not existing: match_slug, existing = idx_mod.resolve(index, merge_into) if not existing: - print(f"echo_ops: --merge-into '{merge_into}' matches no entity in the index " - f"(try `resolve` first)", file=sys.stderr) - return 2 + raise echo.EchoError(f"--merge-into '{merge_into}' matches no entity in the " + "index (try `resolve` first)", 2) existing_reachable = bool(existing and echo.request("GET", echo.vault_url(existing["path"]))[0] == 200) # Pre-write duplicate gate: a strong fuzzy candidate means this title is very likely @@ -276,40 +278,26 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = for s, c, sc in idx_mod.gate_candidates(index, title, kind=kind, threshold=DUP_GATE)] if gate_hits: - if as_json: - env = echo_output.envelope("duplicate-gate", { - "kind": kind, "title": title, "candidates": gate_hits, - "note": "likely duplicate — re-run with --merge-into to update the " - "existing entity, or --force to create anyway"}, ok=False) - print(json.dumps(env, ensure_ascii=False), file=real_stdout) - else: - print(f"STOP: '{title}' likely already exists — not creating a duplicate.", - file=real_stdout) - for c in gate_hits: - print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']})", - file=real_stdout) - print(" re-run with --merge-into to update the existing entity, " - "or --force to create anyway.", file=real_stdout) - return 76 # distinct exit: duplicate gate (cf. 75 lock, 78 config) + return echo_output.envelope("duplicate-gate", { + "kind": kind, "title": title, "candidates": gate_hits, + "note": "likely duplicate — re-run with --merge-into to update the " + "existing entity, or --force to create anyway"}, ok=False) if dry_run: if existing_reachable: - return done("update", existing["path"], dry=True) + return env_for("update", existing["path"], dry=True) s2 = echo_quality.safe_slug(set(index.get("entities", {}).keys()), slug) near = [c.get("path") for _, c, _ in idx_mod.fuzzy_candidates(index, title)][:3] - plan = done("create", idx_mod.derive_path(kind, s2, date=date, domain=domain), - dry=True, near=near) - if (not as_json and not force - and idx_mod.gate_candidates(index, title, kind=kind, threshold=DUP_GATE)): - # (in --json mode the near_duplicates field carries this; keep stdout clean) - print("note: a real run would STOP at the duplicate gate — use --merge-into " - "or --force.", file=real_stdout) - return plan + would_gate = (not force + and bool(idx_mod.gate_candidates(index, title, kind=kind, + threshold=DUP_GATE))) + return env_for("create", idx_mod.derive_path(kind, s2, date=date, domain=domain), + dry=True, near=near, would_gate=would_gate) near_dupes: list[str] = [] index_title = title # title to record in the index entry index_aliases = list(aliases) - with quiet(): + with chatter(): if existing_reachable: # Reuse the matched entity's CANONICAL slug — never re-slug from the mention, or # two index slugs end up pointing at one note. Keep its title; learn the mention @@ -374,13 +362,61 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = if not no_log: ensure_daily_log(f"- {today_s}: {action} {kind} [[{links.link_token(path)}]]") + return env_for(action, path, links=linked, near=near_dupes) + + +def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "", + aliases=None, sources=None, tags=None, date: str | None = None, + domain: str = "business", inbox: bool = False, no_log: bool = False, + as_json: bool = False, dry_run: bool = False, force: bool = False, + merge_into: str | None = None) -> int: + """CLI wrapper around capture_op: prints exactly what pre-Phase-0 capture printed + (human prose or the --json envelope) and maps envelope outcomes to exit codes + (76 duplicate-gate, 2 usage-class errors, 0 otherwise).""" + try: + env = capture_op(kind, title, file_arg, status_v=status_v, aliases=aliases, + sources=sources, tags=tags, date=date, domain=domain, + inbox=inbox, no_log=no_log, dry_run=dry_run, force=force, + merge_into=merge_into) + except echo.EchoError as exc: + print(f"echo_ops: {exc}", file=sys.stderr) + return getattr(exc, "code", 1) + action = env.get("action", "") + if as_json: - done(action, path, links=linked, near=near_dupes) - else: - print(f"ok: {action} {kind} -> {path}" + (f"; auto-linked {linked}" if linked else "")) - if near_dupes: - kind_word = "entity" if len(near_dupes) == 1 else "entities" - print(f"WARNING: similar existing {kind_word} ({', '.join(near_dupes)}) — if this is " - f"the same thing, merge or `echo.py link` instead of keeping a duplicate.", - file=real_stdout) - return 0 + print(json.dumps(env, ensure_ascii=False)) + if action == "duplicate-gate": + return 76 + return 0 if env.get("ok") else 1 + + if action == "duplicate-gate": + print(f"STOP: '{title}' likely already exists — not creating a duplicate.") + for c in env.get("candidates", []): + print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']})") + print(" re-run with --merge-into to update the existing entity, " + "or --force to create anyway.") + return 76 # distinct exit: duplicate gate (cf. 75 lock, 78 config) + if action == "queued:capture": + print(f"queued (offline): capture {kind} '{title}' — will replay through " + "capture on the next reachable session") + return 0 + if action == "dry-run:queued:capture": + print("offline: vault unreachable — a real run would queue this capture " + "for replay on the next reachable session.") + return 0 + if action.startswith("dry-run:"): + print(f"would {action.split(':', 1)[1]} {kind or '-'} -> {env.get('path')}") + if env.get("would_gate"): + print("note: a real run would STOP at the duplicate gate — use --merge-into " + "or --force.") + return 0 + if action == "inbox": + return 0 if env.get("ok") else 1 + print(f"ok: {action} {kind} -> {env.get('path')}" + + (f"; auto-linked {env['links_added']}" if env.get("links_added") else "")) + near = env.get("near_duplicates") + if near: + kind_word = "entity" if len(near) == 1 else "entities" + print(f"WARNING: similar existing {kind_word} ({', '.join(near)}) — if this is " + f"the same thing, merge or `echo.py link` instead of keeping a duplicate.") + return 0 if env.get("ok") else 1 diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_recall.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_recall.py index 13ecfba..bb172c3 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_recall.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_recall.py @@ -396,32 +396,10 @@ def _doc_summary(path: str, text: str) -> dict: return out -def _brief(path: str, score: float | None = None, via: str | None = None) -> None: - text = links.get_text(path) - if text is None: - return - info = _doc_summary(path, text) - head = f"\n### {path}" - if score is not None: - head += f" (score {score:.2f})" - if via: - head += f" (via {via})" - print(head) - stamps = [] - if info.get("type"): - stamps.append(f"type: {info['type']}") - if info.get("updated"): - stamps.append(f"updated: {info['updated']}") - if info.get("status"): - stamps.append(f"status: {info['status']}") - if stamps: - print("_" + " · ".join(stamps) + "_") # staleness is visible, not guessed - if info.get("excerpt"): - print(info["excerpt"]) - - # ------------------------------------------------------------------- entrypoint -def recall(query, limit: int = 8, as_json: bool = False) -> int: +def recall_op(query, limit: int = 8) -> dict: + """Core recall (Phase 0): hybrid BM25 + graph, returning the structured envelope + ({query, primary, linked}) that backs both the CLI renderings and the MCP tool.""" q = " ".join(query) if isinstance(query, list) else query today_s = echo.today() index = idx_mod.load() @@ -464,31 +442,49 @@ def recall(query, limit: int = 8, as_json: bool = False) -> int: # --- graph layer ---------------------------------------------------------- neighbours = expand_graph(hits, nmap, base, max_hops=MAX_HOPS) + import echo_output + texts = echo.read_many(hits + [p for p, _ in neighbours[: 2 * limit]]) + primary = [] + for p in hits: + if texts.get(p) is None: + continue + primary.append({**_doc_summary(p, texts[p]), + "score": round(base.get(p, 0.0), 3)}) + linked = [] + for p, (sc, via) in neighbours[: 2 * limit]: + if texts.get(p) is None: + continue + linked.append({**_doc_summary(p, texts[p]), + "score": round(sc, 3), "via": via}) + return echo_output.envelope("recall", {"query": q, "primary": primary, + "linked": linked}) + + +def _print_hit(info: dict) -> None: + """Prose rendering of one recall hit — same fields the envelope carries.""" + head = f"\n### {info['path']}" + if info.get("score") is not None: + head += f" (score {info['score']:.2f})" + if info.get("via"): + head += f" (via {info['via']})" + print(head) + stamps = [f"{k}: {info[k]}" for k in ("type", "updated", "status") if info.get(k)] + if stamps: + print("_" + " · ".join(stamps) + "_") # staleness is visible, not guessed + if info.get("excerpt"): + print(info["excerpt"]) + + +def recall(query, limit: int = 8, as_json: bool = False) -> int: + env = recall_op(query, limit=limit) if as_json: - import echo_output - texts = echo.read_many(hits + [p for p, _ in neighbours[: 2 * limit]]) - primary = [] - for p in hits: - if texts.get(p) is None: - continue - primary.append({**_doc_summary(p, texts[p]), - "score": round(base.get(p, 0.0), 3)}) - linked = [] - for p, (sc, via) in neighbours[: 2 * limit]: - if texts.get(p) is None: - continue - linked.append({**_doc_summary(p, texts[p]), - "score": round(sc, 3), "via": via}) - env = echo_output.envelope("recall", {"query": q, "primary": primary, - "linked": linked}) print(json.dumps(env, ensure_ascii=False)) return 0 - - print(f"== recall: {q} ==") - print(f"\n# Primary hits ({len(hits)})") - for p in hits: - _brief(p, score=base.get(p)) - print(f"\n# Linked context ({len(neighbours)})") - for p, (sc, via) in neighbours[: 2 * limit]: - _brief(p, score=sc, via=via) + print(f"== recall: {env['query']} ==") + print(f"\n# Primary hits ({len(env['primary'])})") + for info in env["primary"]: + _print_hit(info) + print(f"\n# Linked context ({len(env['linked'])})") + for info in env["linked"]: + _print_hit(info) return 0 diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_reflect.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_reflect.py index f7ffbc0..29e7f1a 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_reflect.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_reflect.py @@ -99,43 +99,69 @@ def preview(proposals: list[dict]) -> str: return "\n".join(rows) -def apply(proposals: list[dict], confirm: bool = False) -> int: - """Validate -> classify -> preview; with confirm=True, apply each via echo_ops.capture - (which routes/indexes/links/logs, each self-lock-guarded). Without confirm it is a - dry-run that writes nothing — the preview IS the confirmation step.""" +def apply_op(proposals: list[dict], confirm: bool = False) -> dict: + """Core reflect (Phase 0): validate -> classify -> (apply via capture_op). Returns + an envelope — rows carry the classified preview; with confirm, `results` carries + each proposal's capture outcome. Never prints to stdout.""" + import echo_output valid, errors = validate(proposals) - for e in errors: - print(f"skip: {e}", file=sys.stderr) - if not valid: - print("reflect: no valid proposals to apply.") - return 0 - - classify(valid) - counts = {a: sum(1 for p in valid if p.get("_action") == a) for a in ("create", "update", "inbox", "error")} - print(f"reflect: {len(valid)} proposal(s) — " - f"{counts['create']} new, {counts['update']} update, {counts['inbox']} inbox" - + (f", {counts['error']} error" if counts["error"] else "")) - print(preview(valid)) - - if not confirm: - print("\nreflect: dry-run — re-run with --apply to write these to memory.") - return 0 + rows: list[dict] = [] + counts: dict[str, int] = {} + if valid: + classify(valid) + counts = {a: sum(1 for p in valid if p.get("_action") == a) + for a in ("create", "update", "inbox", "error")} + rows = [{"action": p.get("_action"), "kind": p.get("kind"), + "title": p["title"], "path": p.get("_path")} for p in valid] + data = {"proposals": len(proposals or []), "valid": len(valid), "errors": errors, + "counts": counts, "rows": rows, "dry_run": not confirm, + "applied": 0, "gated": 0, "results": []} + if not valid or not confirm: + return echo_output.envelope("reflect", data) import echo_ops # lazy: the apply path pulls in the capture/index/link stack applied = gated = 0 + results = [] for p in valid: if p.get("_action") == "error": continue - bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None - rc = echo_ops.capture( - p.get("kind"), p["title"], bodyfile, + env = echo_ops.capture_op( + p.get("kind"), p["title"], body_text=p.get("body") or "", aliases=p.get("aliases") or [], sources=p.get("sources") or [], tags=p.get("tags") or [], date=p.get("date"), domain=p.get("domain", "business"), inbox=bool(p.get("inbox")) or not p.get("kind")) - applied += 1 if rc == 0 else 0 - gated += 1 if rc == 76 else 0 - print(f"reflect: applied {applied}/{len(valid)} proposal(s)." + act = env.get("action", "") + if act == "duplicate-gate": + gated += 1 + elif env.get("ok"): + applied += 1 + results.append({"title": p["title"], "action": act, + "path": env.get("path"), "ok": bool(env.get("ok"))}) + data.update(applied=applied, gated=gated, results=results) + return echo_output.envelope("reflect", data) + + +def apply(proposals: list[dict], confirm: bool = False) -> int: + """CLI wrapper: same preview/summary text as pre-Phase-0. Without confirm it is a + dry-run that writes nothing — the preview IS the confirmation step.""" + env = apply_op(proposals, confirm=confirm) + for e in env["errors"]: + print(f"skip: {e}", file=sys.stderr) + if not env["valid"]: + print("reflect: no valid proposals to apply.") + return 0 + counts = env["counts"] + print(f"reflect: {env['valid']} proposal(s) — " + f"{counts['create']} new, {counts['update']} update, {counts['inbox']} inbox" + + (f", {counts['error']} error" if counts["error"] else "")) + print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} " + f"| {r['title']} -> {r['path'] or '?'}" for r in env["rows"])) + if env["dry_run"]: + print("\nreflect: dry-run — re-run with --apply to write these to memory.") + return 0 + gated = env["gated"] + print(f"reflect: applied {env['applied']}/{env['valid']} proposal(s)." + (f" {gated} stopped at the duplicate gate — re-propose with the existing " f"entity's title (or capture --merge-into )." if gated else "")) return 0 diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_session.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_session.py index 3bc614b..c650b90 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_session.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_session.py @@ -72,36 +72,34 @@ def validate(bundle: dict) -> tuple[str, str]: return path, line -def session_end(bundle: dict, apply: bool = False) -> int: +def session_end_op(bundle: dict, apply: bool = False) -> dict: + """Core session-end (Phase 0): validate -> plan -> (apply). Returns an envelope + (plan rows + per-step results); never prints to stdout — helper chatter from the + underlying verbs routes to stderr. Raises echo.EchoError(2) on a bad bundle, + BEFORE any write.""" + import contextlib + import echo_output import echo_reflect path, line = validate(bundle) scope = str(bundle.get("scope") or "").strip() proposals = bundle.get("reflect") or [] valid, errors = echo_reflect.validate(proposals) - for e in errors: - print(f"skip: {e}", file=sys.stderr) - - print(f"session-end plan ({'APPLY' if apply else 'dry-run'}):") - print(f" 1. session log -> {path}") - print(f" 2. agent-log line: {line}") + rows: list[dict] = [] if valid: echo_reflect.classify(valid) - print(f" 3. reflect: {len(valid)} proposal(s)") - print(echo_reflect.preview(valid)) - else: - print(" 3. reflect: (none)") - print(f" 4. scope set: {scope!r}" if scope else " 4. scope: (unchanged)") - print(f" 5. heartbeat -> {path} @ (written LAST — the commit marker)") - + rows = [{"action": p.get("_action"), "kind": p.get("kind"), + "title": p["title"], "path": p.get("_path")} for p in valid] + data = {"path": path, "agent_log_line": line, "scope": scope or None, + "reflect_rows": rows, "reflect_errors": errors, + "dry_run": not apply, "steps": {}} if not apply: - print("\nsession-end: dry-run — re-run with --apply to write.") - return 0 + return echo_output.envelope("session-end", data) import echo_concurrency import echo_ops steps: dict[str, str] = {} - with echo_concurrency.vault_lock(): + with echo_concurrency.vault_lock(), contextlib.redirect_stdout(sys.stderr): # 1. The session log itself. A hard failure here aborts the whole bundle — # nothing after it (including the heartbeat) runs, so orientation state # can never point at a log that was never written. @@ -116,14 +114,13 @@ def session_end(bundle: dict, apply: bool = False) -> int: for p in valid: if p.get("_action") == "error": continue - bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None - rc = echo_ops.capture( - p.get("kind"), p["title"], bodyfile, + env = echo_ops.capture_op( + p.get("kind"), p["title"], body_text=p.get("body") or "", aliases=p.get("aliases") or [], sources=p.get("sources") or [], tags=p.get("tags") or [], date=p.get("date"), domain=p.get("domain", "business"), inbox=bool(p.get("inbox")) or not p.get("kind")) - gated += 1 if rc == 76 else 0 + gated += 1 if env.get("action") == "duplicate-gate" else 0 steps["reflect"] = f"{len(valid) - gated}/{len(valid)} applied" \ + (f", {gated} gated (re-propose with --merge-into)" if gated else "") else: @@ -139,6 +136,30 @@ def session_end(bundle: dict, apply: bool = False) -> int: echo.temp_file(f"{path} @ {echo.now_iso()}\n".encode("utf-8"))) steps["heartbeat"] = "ok" + data["steps"] = steps + return echo_output.envelope("session-end", data) + + +def session_end(bundle: dict, apply: bool = False) -> int: + """CLI wrapper: same plan/summary text as before; envelope logic in session_end_op.""" + env = session_end_op(bundle, apply=apply) + for e in env["reflect_errors"]: + print(f"skip: {e}", file=sys.stderr) + print(f"session-end plan ({'APPLY' if apply else 'dry-run'}):") + print(f" 1. session log -> {env['path']}") + print(f" 2. agent-log line: {env['agent_log_line']}") + rows = env["reflect_rows"] + if rows: + print(f" 3. reflect: {len(rows)} proposal(s)") + print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} " + f"| {r['title']} -> {r['path'] or '?'}" for r in rows)) + else: + print(" 3. reflect: (none)") + print(f" 4. scope set: {env['scope']!r}" if env["scope"] else " 4. scope: (unchanged)") + print(f" 5. heartbeat -> {env['path']} @ (written LAST — the commit marker)") + if env["dry_run"]: + print("\nsession-end: dry-run — re-run with --apply to write.") + return 0 print("\nsession-end: done — " - + "; ".join(f"{k}: {v}" for k, v in steps.items())) + + "; ".join(f"{k}: {v}" for k, v in env["steps"].items())) return 0 diff --git a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_triage.py b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_triage.py index 3f5d652..bb59fe1 100644 --- a/echo-memory.plugin.src/skills/echo-memory/scripts/echo_triage.py +++ b/echo-memory.plugin.src/skills/echo-memory/scripts/echo_triage.py @@ -55,19 +55,25 @@ def parse_inbox(text: str) -> list[dict]: return items -def list_inbox(as_json: bool = False) -> int: +def list_op() -> dict: + """Core inbox listing (Phase 0): structured captures envelope, no printing.""" + import echo_output status, body = echo.request("GET", echo.vault_url(INBOX_PATH)) if status == 404: items = [] else: echo.check(status, body, f"triage list {INBOX_PATH}") items = parse_inbox(body.decode(errors="replace")) + return echo_output.envelope("triage-list", {"path": INBOX_PATH, "items": items, + "count": len(items)}) + + +def list_inbox(as_json: bool = False) -> int: + env = list_op() if as_json: - import echo_output - env = echo_output.envelope("triage-list", {"path": INBOX_PATH, "items": items, - "count": len(items)}) print(json.dumps(env, ensure_ascii=False)) return 0 + items = env["items"] if not items: print("triage: inbox is empty — nothing to route.") return 0 @@ -80,53 +86,76 @@ def list_inbox(as_json: bool = False) -> int: return 0 -def apply(proposals: list[dict], confirm: bool = False) -> int: - """Validate -> classify -> preview; with confirm, route each via capture AND write - the processing-log audit line. Mirrors echo_reflect.apply's contract exactly.""" +def route_op(proposals: list[dict], confirm: bool = False) -> dict: + """Core triage routing (Phase 0): reflect pipeline + processing-log audit lines. + Returns an envelope (rows = classified preview; results = per-item outcomes); + never prints to stdout (audit-append chatter routes to stderr).""" + import contextlib + import echo_output import echo_reflect valid, errors = echo_reflect.validate(proposals) - for e in errors: - print(f"skip: {e}", file=sys.stderr) - if not valid: - print("triage: no valid proposals to route.") - return 0 - - echo_reflect.classify(valid) - counts = {a: sum(1 for p in valid if p.get("_action") == a) - for a in ("create", "update", "inbox", "error")} - print(f"triage: {len(valid)} proposal(s) — " - f"{counts['create']} new, {counts['update']} update, {counts['inbox']} stay-in-inbox" - + (f", {counts['error']} error" if counts["error"] else "")) - print(echo_reflect.preview(valid)) - - if not confirm: - print("\ntriage: dry-run — re-run with --apply to route these and log the moves.") - return 0 + rows: list[dict] = [] + counts: dict[str, int] = {} + if valid: + echo_reflect.classify(valid) + counts = {a: sum(1 for p in valid if p.get("_action") == a) + for a in ("create", "update", "inbox", "error")} + rows = [{"action": p.get("_action"), "kind": p.get("kind"), + "title": p["title"], "path": p.get("_path")} for p in valid] + data = {"proposals": len(proposals or []), "valid": len(valid), "errors": errors, + "counts": counts, "rows": rows, "dry_run": not confirm, + "routed": 0, "gated": 0, "log": f"{LOG_DIR}/{echo.today()}.md", "results": []} + if not valid or not confirm: + return echo_output.envelope("triage", data) import echo_ops today_s = echo.today() - applied = gated = 0 + routed = gated = 0 + results = [] for p in valid: - if p.get("_action") == "error": - continue - if p.get("_action") == "inbox": + if p.get("_action") in ("error", "inbox"): continue # routing an inbox line back to the inbox is a no-op, not a move - bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None - rc = echo_ops.capture( - p.get("kind"), p["title"], bodyfile, + env = echo_ops.capture_op( + p.get("kind"), p["title"], body_text=p.get("body") or "", aliases=p.get("aliases") or [], sources=p.get("sources") or [], tags=p.get("tags") or [], date=p.get("date"), domain=p.get("domain", "business")) - if rc == 76: + act = env.get("action", "") + results.append({"title": p["title"], "action": act, + "path": env.get("path"), "ok": bool(env.get("ok"))}) + if act == "duplicate-gate": gated += 1 continue - if rc != 0: + if not env.get("ok"): continue - applied += 1 + routed += 1 original = (p.get("line") or p["title"]).strip() - echo.cmd_append(f"{LOG_DIR}/{today_s}.md", - f"- {original} → {p.get('_path', '?')}") - print(f"triage: routed {applied}/{len(valid)} item(s); audit in {LOG_DIR}/{today_s}.md. " + with contextlib.redirect_stdout(sys.stderr): + echo.cmd_append(f"{LOG_DIR}/{today_s}.md", + f"- {original} → {p.get('_path', '?')}") + data.update(routed=routed, gated=gated, results=results) + return echo_output.envelope("triage", data) + + +def apply(proposals: list[dict], confirm: bool = False) -> int: + """CLI wrapper: same preview/summary text as pre-Phase-0. Mirrors reflect's contract.""" + env = route_op(proposals, confirm=confirm) + for e in env["errors"]: + print(f"skip: {e}", file=sys.stderr) + if not env["valid"]: + print("triage: no valid proposals to route.") + return 0 + counts = env["counts"] + print(f"triage: {env['valid']} proposal(s) — " + f"{counts['create']} new, {counts['update']} update, {counts['inbox']} stay-in-inbox" + + (f", {counts['error']} error" if counts["error"] else "")) + print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} " + f"| {r['title']} -> {r['path'] or '?'}" for r in env["rows"])) + if env["dry_run"]: + print("\ntriage: dry-run — re-run with --apply to route these and log the moves.") + return 0 + gated = env["gated"] + print(f"triage: routed {env['routed']}/{env['valid']} item(s); audit in {env['log']}. " "Originals kept in the inbox (deletion is explicit-only)." + (f" {gated} stopped at the duplicate gate — re-propose with the existing " f"entity's title or use capture --merge-into." if gated else "")) diff --git a/eval/test_ops_api.py b/eval/test_ops_api.py new file mode 100644 index 0000000..1a4114f --- /dev/null +++ b/eval/test_ops_api.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +"""test_ops_api.py — Phase 0 (return-not-print) contract: every high-level op has a +core `*_op` function that RETURNS an envelope dict and prints nothing to stdout. + +This is the seam the MCP server wraps (docs/MCP-SERVER-SPEC.md §4): the CLI wrappers +are tested by the other suites; here we call the cores in-process against the mock and +assert (a) the envelope shapes and (b) stdout purity — a stray print() would corrupt +an MCP stdio/HTTP response stream. + +Run: python test_ops_api.py [--port 8850] +""" +from __future__ import annotations + +import argparse +import contextlib +import io +import json +import os +import subprocess +import sys +import tempfile +import time +import urllib.request +from pathlib import Path + +HERE = Path(__file__).resolve().parent +SCRIPTS = HERE.parent / "echo-memory.plugin.src" / "skills" / "echo-memory" / "scripts" +KEY = "test-key-not-a-real-secret" + +ap = argparse.ArgumentParser() +ap.add_argument("--port", type=int, default=8850) +a = ap.parse_args() +BASE = f"http://127.0.0.1:{a.port}" + +# env must be set BEFORE importing echo (it resolves config at import time) +os.environ.update(ECHO_BASE=BASE, ECHO_KEY=KEY, ECHO_TODAY="2026-06-21", + ECHO_NOW="2300", ECHO_STATE_DIR=tempfile.mkdtemp(), ECHO_VERIFY="0") +sys.path.insert(0, str(SCRIPTS)) + +failures = [] + + +def check(name, cond, detail=""): + print(f"{'ok ' if cond else 'FAIL'} {name}" + (f" -- {detail}" if not cond else "")) + if not cond: + failures.append(name) + + +def http(method, url, body=None): + data = body.encode() if isinstance(body, str) else body + req = urllib.request.Request(url, data=data, method=method, + headers={"Authorization": f"Bearer {KEY}"}) + try: + with urllib.request.urlopen(req, timeout=10) as r: + return r.status, r.read().decode("utf-8", "replace") + except Exception as e: # noqa: BLE001 + return getattr(e, "code", 0), "" + + +def pure(fn, *args, **kw): + """Call fn capturing stdout; return (result, captured_stdout).""" + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + out = fn(*args, **kw) + return out, buf.getvalue() + + +def main(): + srv = subprocess.Popen([sys.executable, str(HERE / "mock_olrapi.py"), "--port", str(a.port)], + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + try: + for _ in range(50): + try: + urllib.request.urlopen(f"{BASE}/__debug__reset", data=b"", timeout=1) + break + except Exception: + time.sleep(0.1) + http("PUT", f"{BASE}/vault/_agent/echo-vault.md", "---\nschema_version: 4\n---\n# marker\n") + http("PUT", f"{BASE}/vault/_agent/context/current-context.md", + "---\ntype: context-bundle\nscope_updated: \"2026-06-20\"\ncreated: 2026-06-01\n---\n" + "# Current Context\n\n## Scope\ntesting phase 0\n\n## Scope History\n") + + import echo + import echo_doctor + import echo_ops + import echo_recall + import echo_reflect + import echo_session + import echo_triage + + # capture_op: create / duplicate-gate / dry-run — envelopes, stdout pure + env, out = pure(echo_ops.capture_op, "person", "Zara Quix", body_text="Met at the expo.") + check("capture_op create envelope", env.get("ok") is True and env.get("action") == "created" + and env.get("path") == "resources/people/zara-quix.md", json.dumps(env)) + check("capture_op prints nothing to stdout", out == "", out[:200]) + env, out = pure(echo_ops.capture_op, "person", "Zara Quix Junior", body_text="") + check("capture_op duplicate-gate is data, not an exit code", + env.get("ok") is False and env.get("action") == "duplicate-gate" + and env.get("candidates"), json.dumps(env)) + check("capture_op gate prints nothing", out == "", out[:200]) + env, out = pure(echo_ops.capture_op, "company", "Plan Co", body_text="", dry_run=True) + check("capture_op dry-run envelope", env.get("action") == "dry-run:create" + and env.get("path") == "resources/companies/plan-co.md", json.dumps(env)) + + # resolve_op / link_op + env, out = pure(echo_ops.resolve_op, "zara quix") + check("resolve_op returns the match dict", env.get("match") is True + and env.get("path") == "resources/people/zara-quix.md", json.dumps(env)) + pure(echo_ops.capture_op, "concept", "Gizmo", body_text="") + env, out = pure(echo_ops.link_op, "resources/people/zara-quix.md", "resources/concepts/gizmo.md") + check("link_op envelope", env.get("ok") is True and env.get("a_changed") in (True, False), + json.dumps(env)) + check("link_op prints nothing", out == "", out[:200]) + + # recall_op + env, out = pure(echo_recall.recall_op, "expo") + check("recall_op envelope shape", env.get("action") == "recall" + and isinstance(env.get("primary"), list) and isinstance(env.get("linked"), list), + json.dumps(env)[:200]) + check("recall_op prints nothing", out == "", out[:200]) + + # triage list_op / route_op (dry-run) + http("PUT", f"{BASE}/vault/inbox/captures/inbox.md", "- 2026-06-10: try the quorlab tool\n") + env, out = pure(echo_triage.list_op) + check("triage list_op envelope", env.get("count") == 1 + and env["items"][0]["age_days"] == 11, json.dumps(env)) + props = [{"title": "Quorlab Tool", "kind": "reference", "confidence": 0.9, + "line": "- 2026-06-10: try the quorlab tool"}] + env, out = pure(echo_triage.route_op, props, False) + check("triage route_op dry-run rows", env.get("dry_run") is True + and env["rows"][0]["action"] == "create", json.dumps(env)) + check("triage route_op prints nothing", out == "", out[:200]) + + # reflect apply_op (applied path, using capture_op internally) + env, out = pure(echo_reflect.apply_op, + [{"title": "Blorp Pattern", "kind": "semantic", + "body": "The operator prefers blorp.", "confidence": 0.9}], True) + check("reflect apply_op applies via capture_op", env.get("applied") == 1 + and env["results"][0]["action"] == "created", json.dumps(env)) + check("reflect apply_op prints nothing", out == "", out[:200]) + + # scope ops + env, out = pure(echo.scope_show_op) + check("scope_show_op returns scope + freshness", env.get("scope") == "testing phase 0" + and env.get("scope_updated") == "2026-06-20", json.dumps(env)) + env, out = pure(echo.scope_set_op, "phase zero refactor") + check("scope_set_op envelope", env.get("action") == "scope-set" + and env.get("scope_updated") == "2026-06-21", json.dumps(env)) + check("scope_set_op prints nothing", out == "", out[:200]) + + # load_op (brief) + env, out = pure(echo.load_op, True) + check("load_op returns sections + brief", "marker" in env.get("sections", {}) + and "ECHO load (brief)" in env.get("brief", ""), json.dumps(env)[:200]) + check("load_op prints nothing", out == "", out[:200]) + + # doctor run_op + env, out = pure(echo_doctor.run_op) + check("doctor run_op checks list", isinstance(env.get("checks"), list) + and env.get("fatal") is None and env.get("ok") is True, json.dumps(env)) + check("doctor run_op prints nothing", out == "", out[:200]) + + # session_end_op: dry-run envelope; bad bundle raises EchoError(2) pre-write + bundle = {"slug": "phase-zero", "log_body": "---\ntype: session-log\n---\n# S\n\n## Goal\nx\n"} + env, out = pure(echo_session.session_end_op, bundle, False) + check("session_end_op dry-run envelope", env.get("dry_run") is True + and env.get("path") == "_agent/sessions/2026-06-21-2300-phase-zero.md", json.dumps(env)) + check("session_end_op prints nothing", out == "", out[:200]) + try: + echo_session.session_end_op({"slug": "Bad Slug!", "log_body": "x"}, True) + check("session_end_op rejects a bad slug", False, "no exception raised") + except RuntimeError as exc: + check("session_end_op rejects a bad slug", getattr(exc, "code", 0) == 2, str(exc)) + + print(f"\n{len(failures)} failure(s)" if failures else "\nall ops-api tests passed") + return 1 if failures else 0 + finally: + srv.terminate() + + +if __name__ == "__main__": + raise SystemExit(main())