1 Commits

Author SHA1 Message Date
Jason Stedwell e7792003a7 2.1.1 — MCP Phase 0: every high-level op returns an envelope (return-not-print)
Internal refactor per docs/MCP-SERVER-SPEC.md §4; CLI output and exit codes
unchanged (wrappers reproduce them from the envelopes):

- echo_ops: capture_op/resolve_op/link_op — duplicate gate and offline queueing
  are data (action: duplicate-gate + candidates; queued: true); the --json
  stdout-swallow hack is deleted (helper chatter -> stderr inside the cores);
  capture_op takes body_text= directly (the MCP path, no temp file).
- echo_recall.recall_op — one structured result backs prose and --json.
- echo_triage.list_op/route_op, echo_reflect.apply_op,
  echo_session.session_end_op — call capture_op internally, count gate outcomes
  from envelopes.
- echo.scope_show_op/scope_set_op/load_op (+ shared _load_gather), and
  echo_doctor.run_op (checks as data).

New eval/test_ops_api.py asserts the contract for every core: envelope shape
AND stdout purity (a stray print would corrupt an MCP response stream). All six
suites green. The 2.2 server build now starts directly at the app layer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 22:51:33 -05:00
13 changed files with 738 additions and 345 deletions
+28
View File
@@ -1,5 +1,33 @@
# Changelog
## 2.1.1
### Changed — Phase 0 of the MCP build: every high-level op returns an envelope
Internal refactor (MCP-SERVER-SPEC §4), no behavior change intended: each
high-level operation now has a core `*_op` function that **returns an envelope
dict and never prints to stdout** — the transport-agnostic seam the upcoming MCP
server (2.2) wraps. The CLI verbs are thin wrappers that print exactly what they
printed before and map envelope outcomes to the same exit codes.
- `echo_ops`: `capture_op` / `resolve_op` / `link_op` — the duplicate gate and
offline queueing are now **data** (`action: duplicate-gate` with candidates;
`queued: true`), mapped to exit 76 / "queued" text only in the wrapper. The
`--json` stdout-swallowing hack is gone: helper chatter from the low-level
verbs routes to **stderr** wholesale inside the cores. `capture_op` accepts
`body_text=` directly (no temp file needed — the MCP path).
- `echo_recall.recall_op` — one structured result backs both the prose and
`--json` renderings (the duplicated prose path was deleted).
- `echo_triage.list_op` / `route_op`, `echo_reflect.apply_op`,
`echo_session.session_end_op` — same pattern; reflect/triage/session now call
`capture_op` internally and count gate outcomes from envelopes.
- `echo.scope_show_op` / `scope_set_op` / `load_op` (sections + brief digest as
data), `echo_doctor.run_op` (checks as a list).
New suite `eval/test_ops_api.py` asserts the contract for every core: envelope
shape AND stdout purity (a stray print would corrupt an MCP response stream).
All existing suites pass unchanged.
## 2.1.0
The "quick-wins train" from the 2026-07-28 review (`docs/IMPROVEMENT-PLANS.md` #1,
+2 -1
View File
@@ -1,4 +1,4 @@
# echo-memory — v2.1.0
# echo-memory — v2.1.1
Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`.
@@ -421,6 +421,7 @@ From the credential-free harness (`eval/run_eval.py` against the deterministic m
| Version | Highlights |
|---------|-----------|
| **2.1.1** | **MCP Phase 0 — return-not-print.** Every high-level op gains a core `*_op` function returning an envelope dict with a stdout-purity guarantee (helper chatter → stderr); CLI verbs become thin wrappers with identical output and exit codes. Duplicate gate and offline queueing become data (`action: duplicate-gate` / `queued: true`). New `eval/test_ops_api.py` contract suite. This is the seam the 2.2 containerized MCP server wraps. |
| **2.1.0** | **Quick-wins train: cheaper sessions, durable capture, one-call session end.** (1) **`load --brief`** — a token-budgeted cold-start digest (Fact/Pattern in full, last ~10 Observations, scope+freshness, last session's key sections, Agent-Log lines, inbox *count*; `ECHO_LOAD_BUDGET` default ~8000 chars) now injected by the SessionStart hook, replacing the full six-file dump that grew unboundedly; all `load` reads are fetched in parallel. (2) **Offline capture durability**`capture` on an unreachable vault queues the *whole operation* as one semantic record; `flush` replays it **through capture** so routing/gate/aliasing re-run against the current index; a gate stop on replay is kept + flagged, never landed blind; `ensure_daily_log`/update-path writes ride the queue too. (3) **`session-end`** — one call (one lock) writes session log → Agent-Log line → reflect proposals → optional scope switch → **heartbeat last as the commit marker**; dry-run by default; `ECHO_NOW` pins the HHMM. Also fixes the `__main__` twin-module trap so helper-module `EchoError`s exit with their intended codes. +19 end-to-end checks. |
| **2.0.0** | **Packaging & structure major — memory behavior unchanged.** Skills-only: the legacy `commands/` directory is deleted (breaking for pre-skills clients; the 1.6.0-verified skills are the sole entry points). Artifact policy: the repo tracks only the `echo-memory.plugin` pointer; the 15 historical versioned zips leave the tree and versioned builds ship as **Gitea releases** (one per `v<version>` tag) from `v2.0.0` on; `.gitignore` blocks `*.plugin` except the pointer. README gains the "packaging for other agent runtimes" port note (build-target rule, never a second source tree). |
| **1.6.0** | **Skills-format migration + lint blind-spot fix.** The eight slash commands gain `skills/<name>/SKILL.md` twins (same names, byte-identical bodies; a same-name skill takes precedence, so `commands/` coexists until its 2.0 deletion): per-skill `allowed-tools` end the permission prompts, `disable-model-invocation: true` makes `/echo-sweep` + `/echo-triage` operator-only. `vault_lint` checks retired patterns **before** routes, so a seed README inside a retired tree (`archive/…`) is flagged instead of being absolved by the leaf-README route (+ regression test). `plugin.json` gains `homepage`/`repository`. |
+1 -1
View File
@@ -75,7 +75,7 @@ monitor, PORT deploy/rollback. Server down ⇒ exactly today's behavior.
| Tool prefix | `echo_` | Namespace safety next to other servers. |
| Result shape | `structuredContent` (the `echo_output.envelope` dict) + a 13 line human text block | Envelope shape already exists. |
| Statefulness | Stateless protocol; warm in-process caches | See §7. |
| Version target | **2.2** (1.6.0, 2.0.0, and the 2.1.0 quick-wins train all shipped 2026-07-28; `session-end` exists) | Remaining prerequisite: Phase 0 (return-not-print), the build session's first step. |
| Version target | **2.2** — ALL prerequisites shipped 2026-07-28 (`session-end` in 2.1.0; **Phase 0 in 2.1.1**, contract-tested by `eval/test_ops_api.py`) | The build session starts directly at §5 (the server app). |
| Local stdio variant | **Dropped for v1** (documented, not built) | The CLI/skill fallback covers the no-server case; two transports = two test matrices for little gain. Phase 0 keeps the door open — the `*_op` core is transport-agnostic. |
| Result budgets | Every read tool takes an explicit size budget (`budget_chars` / `max_chars`) and truncates with a marker + "call X for more" | The single biggest token lever: one right-sized answer instead of follow-up fetches. |
| Tool profiles | `ECHO_MCP_TOOLS=core\|full` env: `core` exposes only load/recall/capture/triage_inbox/log_session/health | Tool schemas cost context on every surface that lists them; the claude.ai connector doesn't need the escape hatches. Desktop registers `full`. |
Binary file not shown.
@@ -1,6 +1,6 @@
{
"name": "echo-memory",
"version": "2.1.0",
"version": "2.1.1",
"homepage": "https://git.alwisp.com/jason/echo",
"repository": "https://git.alwisp.com/jason/echo",
"description": "Persistent memory via the ECHO Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /echo-* commands.",
@@ -549,59 +549,76 @@ def extract_heading(markdown: str, heading: str) -> str:
return "\n".join(out).strip()
def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) -> int:
def scope_show_op() -> dict:
"""Core scope-show (Phase 0): active scope + freshness + sessions-since, as data."""
path = "_agent/context/current-context.md"
status, body = request("GET", vault_url(path))
check(status, body, f"scope {subcommand}")
check(status, body, "scope show")
current = body.decode(errors="replace")
scope_text = extract_heading(current, "Scope")
scope_updated = ""
for line in current.splitlines():
if line.startswith("scope_updated:"):
scope_updated = line.split(":", 1)[1].strip().strip('"').strip("'")
break
sessions_since = None
# Count session logs dated after scope_updated (the drift signal).
if scope_updated:
status, body = request("GET", vault_url("_agent/sessions/"))
if status == 200:
try:
files = json.loads(body).get("files", [])
sessions_since = sum(1 for f in files if f.endswith(".md") and f[:10] > scope_updated)
except json.JSONDecodeError:
pass
return {"ok": True, "action": "scope-show", "scope": scope_text,
"scope_updated": scope_updated or None, "sessions_since": sessions_since}
if subcommand == "show":
scope_text = extract_heading(current, "Scope")
scope_updated = ""
for line in current.splitlines():
if line.startswith("scope_updated:"):
scope_updated = line.split(":", 1)[1].strip().strip('"').strip("'")
break
sessions_since = None
# Count session logs dated after scope_updated (the drift signal).
if scope_updated:
status, body = request("GET", vault_url("_agent/sessions/"))
if status == 200:
try:
files = json.loads(body).get("files", [])
sessions_since = sum(1 for f in files if f.endswith(".md") and f[:10] > scope_updated)
except json.JSONDecodeError:
pass
if as_json:
print(json.dumps({"ok": True, "action": "scope-show", "scope": scope_text,
"scope_updated": scope_updated or None,
"sessions_since": sessions_since}, ensure_ascii=False))
return 0
print("-- Active scope --")
print(scope_text)
print(f"scope_updated: {scope_updated or '<missing — drift cannot be detected; run scope set or repair>'}")
if sessions_since is not None:
print(f"sessions logged since: {sessions_since}")
return 0
if subcommand != "set":
raise EchoError("scope: use 'show' or 'set \"<text>\"'", 2)
def scope_set_op(text: str) -> dict:
"""Core scope-set (Phase 0): atomic switch (history + replace + stamp). Chatter
from the underlying PATCHes routes to stderr; returns the switch envelope."""
import contextlib
if not text:
raise EchoError("scope set needs the new scope text", 2)
path = "_agent/context/current-context.md"
status, body = request("GET", vault_url(path))
check(status, body, "scope set")
current = body.decode(errors="replace")
prior = extract_heading(current, "Scope").replace("\n", " ").strip()[:140] or "(prior scope)"
cmd_patch(path, "prepend", "heading", "Current Context::Scope History",
temp_file(f"- {today()}: {prior}\n".encode()))
cmd_patch(path, "replace", "heading", "Current Context::Scope",
temp_file(f"{text}\n".encode()))
try:
cmd_patch(path, "replace", "frontmatter", "scope_updated",
temp_file(json.dumps(today()).encode()))
except EchoError as exc:
raise EchoError(
"scope set: body switched, but scope_updated frontmatter is missing "
f"(run bootstrap.py to add it) [{exc}]"
)
print(f"ok: scope switched (prior archived to Scope History; scope_updated={today()})")
with contextlib.redirect_stdout(sys.stderr):
cmd_patch(path, "prepend", "heading", "Current Context::Scope History",
temp_file(f"- {today()}: {prior}\n".encode()))
cmd_patch(path, "replace", "heading", "Current Context::Scope",
temp_file(f"{text}\n".encode()))
try:
cmd_patch(path, "replace", "frontmatter", "scope_updated",
temp_file(json.dumps(today()).encode()))
except EchoError as exc:
raise EchoError(
"scope set: body switched, but scope_updated frontmatter is missing "
f"(run bootstrap.py to add it) [{exc}]"
)
return {"ok": True, "action": "scope-set", "scope": text, "prior": prior,
"scope_updated": today()}
def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) -> int:
if subcommand == "show":
env = scope_show_op()
if as_json:
print(json.dumps(env, ensure_ascii=False))
return 0
print("-- Active scope --")
print(env["scope"])
print(f"scope_updated: {env['scope_updated'] or '<missing — drift cannot be detected; run scope set or repair>'}")
if env["sessions_since"] is not None:
print(f"sessions logged since: {env['sessions_since']}")
return 0
if subcommand != "set":
raise EchoError("scope: use 'show' or 'set \"<text>\"'", 2)
env = scope_set_op(text or "")
print(f"ok: scope switched (prior archived to Scope History; scope_updated={env['scope_updated']})")
return 0
@@ -730,51 +747,22 @@ def _render_brief(texts: dict[str, str | None], listing_files: list[str],
return out
def cmd_load(brief: bool = False) -> int:
"""Cold-start orientation: the canonical 6 reads in one call (fetched in parallel).
404s on today's daily note and the inbox are normal (printed as absent, not errors).
`brief` renders the token-budgeted digest the SessionStart hook injects; the default
full mode (and /echo-load) prints the raw sections unchanged."""
if not echo_config.is_configured(_CFG):
print("echo: NOT CONFIGURED — this machine has no usable ECHO key file yet.")
print(f"Expected at: {echo_config.config_path()}")
print("ACTION: ask the operator for their echo-memory config file (it holds the")
print(" vault owner, endpoint, and API key), then install it with ONE of:")
print(" python3 echo.py config import <path-to-their-file>")
print(" python3 echo.py config set --owner \"\" --endpoint \"https://…\" --key \"\"")
print("Then re-run load. (Memory is unavailable until configured.)")
return 78 # distinct: configuration required
targets = [
("marker", "_agent/echo-vault.md"),
("preferences", "_agent/memory/semantic/operator-preferences.md"),
("context", "_agent/context/current-context.md"),
("heartbeat", "_agent/heartbeat/last-session.md"),
("today", f"journal/daily/{today()}.md"),
("inbox", "inbox/captures/inbox.md"),
]
import echo_queue
# H2: sync any writes queued during a prior outage, best-effort and quiet on empty.
try:
synced = echo_queue.flush()
if synced:
print(f"(synced {synced} write(s) queued during a prior offline session)\n")
flagged = echo_queue.needs_attention()
if flagged:
print(f"NOTE: {len(flagged)} queued write(s) need attention (e.g. a capture "
"stopped at the duplicate gate on replay) — resolve with capture "
"--merge-into/--force; `echo.py flush` re-lists them.\n")
except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load
print(f"(queue flush skipped: {exc})\n", file=sys.stderr)
LOAD_TARGETS = [
("marker", "_agent/echo-vault.md"),
("preferences", "_agent/memory/semantic/operator-preferences.md"),
("context", "_agent/context/current-context.md"),
("heartbeat", "_agent/heartbeat/last-session.md"),
]
# All orientation reads (+ the sessions listing) fetched in parallel over the
# warm connection pool — the listing feeds both the brief digest's scope-freshness
# count and the heartbeat-absent fallback, so it's no longer a second round-trip.
def _load_gather(targets):
"""Fetch the orientation reads (+ the sessions listing) in parallel and resolve
each to text via status/cache. Shared by cmd_load and load_op. Returns
(results, texts, listing_files, offline, marker_missing, heartbeat_absent)."""
import echo_queue
listing_path = "_agent/sessions/"
results = _fetch_statuses([p for _, p in targets] + [listing_path])
marker_missing = False
heartbeat_absent = False
offline = False
marker_missing = heartbeat_absent = offline = False
texts: dict[str, str | None] = {}
for label, path in targets:
status, body = results[path]
@@ -792,7 +780,6 @@ def cmd_load(brief: bool = False) -> int:
marker_missing = True
if label == "heartbeat":
heartbeat_absent = True
lst_status, lst_body = results[listing_path]
listing_files: list[str] = []
if lst_status == 200:
@@ -800,15 +787,84 @@ def cmd_load(brief: bool = False) -> int:
listing_files = list(json.loads(lst_body).get("files", []))
except json.JSONDecodeError:
pass
return results, texts, listing_files, offline, marker_missing, heartbeat_absent
def _fetch_pointed_session(texts: dict) -> None:
"""Follow the heartbeat pointer and stash the pointed session log for the digest."""
hb = texts.get("heartbeat")
if hb and hb.strip():
sess_path = hb.strip().splitlines()[0].split(" @ ", 1)[0].strip()
if sess_path:
st2, b2 = request("GET", vault_url(sess_path))
if st2 == 200:
texts["_pointed_session"] = b2.decode(errors="replace")
def load_op(brief: bool = True) -> dict:
"""Core load (Phase 0): the orientation reads as data — sections keyed by label,
plus offline/bootstrap flags, queue counts, recent sessions, and (with brief) the
rendered digest. Raises EchoError(78) when the machine is not configured."""
if not echo_config.is_configured(_CFG):
raise EchoError("NOT CONFIGURED — no usable ECHO key file on this machine "
f"(expected at {echo_config.config_path()})", 78)
import echo_queue
synced = flagged = 0
try:
synced = echo_queue.flush()
flagged = len(echo_queue.needs_attention())
except Exception: # noqa: BLE001 — queue upkeep must never block a load
pass
targets = LOAD_TARGETS + [("today", f"journal/daily/{today()}.md"),
("inbox", "inbox/captures/inbox.md")]
_, texts, listing_files, offline, marker_missing, _ = _load_gather(targets)
data = {"ok": True, "action": "load", "offline": offline,
"marker_missing": marker_missing, "synced": synced,
"needs_attention": flagged,
"sections": {k: v for k, v in texts.items() if not k.startswith("_")},
"recent_sessions": sorted((f for f in listing_files if f.endswith(".md")),
reverse=True)[:5]}
if brief:
_fetch_pointed_session(texts)
data["brief"] = _render_brief(texts, listing_files, offline)
return data
def cmd_load(brief: bool = False) -> int:
"""Cold-start orientation: the canonical 6 reads in one call (fetched in parallel).
404s on today's daily note and the inbox are normal (printed as absent, not errors).
`brief` renders the token-budgeted digest the SessionStart hook injects; the default
full mode (and /echo-load) prints the raw sections unchanged."""
if not echo_config.is_configured(_CFG):
print("echo: NOT CONFIGURED — this machine has no usable ECHO key file yet.")
print(f"Expected at: {echo_config.config_path()}")
print("ACTION: ask the operator for their echo-memory config file (it holds the")
print(" vault owner, endpoint, and API key), then install it with ONE of:")
print(" python3 echo.py config import <path-to-their-file>")
print(" python3 echo.py config set --owner \"\" --endpoint \"https://…\" --key \"\"")
print("Then re-run load. (Memory is unavailable until configured.)")
return 78 # distinct: configuration required
targets = LOAD_TARGETS + [("today", f"journal/daily/{today()}.md"),
("inbox", "inbox/captures/inbox.md")]
import echo_queue
# H2: sync any writes queued during a prior outage, best-effort and quiet on empty.
try:
synced = echo_queue.flush()
if synced:
print(f"(synced {synced} write(s) queued during a prior offline session)\n")
flagged = echo_queue.needs_attention()
if flagged:
print(f"NOTE: {len(flagged)} queued write(s) need attention (e.g. a capture "
"stopped at the duplicate gate on replay) — resolve with capture "
"--merge-into/--force; `echo.py flush` re-lists them.\n")
except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load
print(f"(queue flush skipped: {exc})\n", file=sys.stderr)
results, texts, listing_files, offline, marker_missing, heartbeat_absent = \
_load_gather(targets)
if brief:
hb = texts.get("heartbeat")
if hb and hb.strip(): # follow-up: the pointed session log's key sections
sess_path = hb.strip().splitlines()[0].split(" @ ", 1)[0].strip()
if sess_path:
st2, b2 = request("GET", vault_url(sess_path))
if st2 == 200:
texts["_pointed_session"] = b2.decode(errors="replace")
_fetch_pointed_session(texts)
print(_render_brief(texts, listing_files, offline))
return 0
@@ -22,17 +22,18 @@ import echo # noqa: E402
MIN_PY = (3, 9)
def run() -> int:
reds = 0
def run_op() -> dict:
"""Core doctor (Phase 0): the readiness checks as data — {checks: [{ok, label,
detail}], endpoint, fatal, ok}. `fatal` names an early-exit condition (not
configured / unreachable) after which later checks were skipped."""
checks: list[dict] = []
def line(ok: bool, label: str, detail: str = "") -> None:
nonlocal reds
reds += 0 if ok else 1
print(f" [{'OK ' if ok else 'RED'}] {label}" + (f"{detail}" if detail else ""))
checks.append({"ok": ok, "label": label, "detail": detail})
import echo_config
cfg = echo_config.load()
print(f"echo doctor — endpoint {cfg['endpoint'] or '(not configured)'}")
fatal = None
# 1. interpreter
line(sys.version_info >= MIN_PY, f"python >= {MIN_PY[0]}.{MIN_PY[1]}",
@@ -53,33 +54,50 @@ def run() -> int:
line(bool(cfg["owner"]), "vault owner set",
cfg["owner"] if cfg["owner"] else "optional, but recommended for third-person writes")
if not echo_config.is_configured(cfg):
fatal = "not-configured"
else:
# 3. reachability + auth + marker, in one GET of the bootstrap marker
status, body = echo.request("GET", echo.vault_url("_agent/echo-vault.md"))
if status == 0:
line(False, "vault reachable", body.decode(errors="replace")[:120])
fatal = "unreachable"
else:
line(status not in (401, 403), "auth accepted",
f"HTTP {status}" if status in (401, 403) else "")
if status == 404:
line(False, "vault bootstrapped", "marker absent — run bootstrap.py")
elif status == 200:
text = body.decode(errors="replace")
ver = next((ln.split(":", 1)[1].strip() for ln in text.splitlines()
if ln.startswith("schema_version:")), "?")
line(True, "vault bootstrapped", f"schema_version={ver}")
else:
line(status < 400, "marker fetch", f"HTTP {status}")
reds = sum(1 for c in checks if not c["ok"])
return {"ok": reds == 0 and fatal is None, "action": "doctor",
"endpoint": cfg["endpoint"], "fatal": fatal, "reds": reds, "checks": checks}
def run() -> int:
env = run_op()
print(f"echo doctor — endpoint {env['endpoint'] or '(not configured)'}")
for c in env["checks"]:
print(f" [{'OK ' if c['ok'] else 'RED'}] {c['label']}"
+ (f"{c['detail']}" if c["detail"] else ""))
if env["fatal"] == "not-configured":
print("\ndoctor: not configured — ask the operator for their key file and install it "
"(`echo.py config import <file>`, or `config set --owner … --endpoint … --key …`), "
"then re-run.")
return 1
# 3. reachability + auth + marker, in one GET of the bootstrap marker
status, body = echo.request("GET", echo.vault_url("_agent/echo-vault.md"))
if status == 0:
line(False, "vault reachable", body.decode(errors="replace")[:120])
if env["fatal"] == "unreachable":
print("\ndoctor: endpoint unreachable — is Obsidian + the Local REST API running?")
return 1
line(status not in (401, 403), "auth accepted", f"HTTP {status}" if status in (401, 403) else "")
if status == 404:
line(False, "vault bootstrapped", "marker absent — run bootstrap.py")
elif status == 200:
text = body.decode(errors="replace")
ver = next((ln.split(":", 1)[1].strip() for ln in text.splitlines()
if ln.startswith("schema_version:")), "?")
line(True, "vault bootstrapped", f"schema_version={ver}")
else:
line(status < 400, "marker fetch", f"HTTP {status}")
# 4. invariants pointer.
# invariants pointer.
print(" [i] invariants — run `/echo-health` (vault_lint.py) for the full check")
print(f"\ndoctor: {'all green' if reds == 0 else f'{reds} issue(s) — see RED above'}")
return 1 if reds else 0
summary = "all green" if env["reds"] == 0 else f"{env['reds']} issue(s) — see RED above"
print(f"\ndoctor: {summary}")
return 1 if env["reds"] else 0
if __name__ == "__main__":
@@ -37,12 +37,13 @@ LOG_HEADING = {
# ---------------------------------------------------------------- resolve -----
def resolve(mention: str) -> int:
def resolve_op(mention: str) -> dict:
"""Core resolve: mention -> match/candidates dict. No printing (Phase 0 — the same
return value backs the CLI wrapper and the MCP `echo_resolve` tool)."""
index = idx_mod.load()
slug, e = idx_mod.resolve(index, mention)
if e:
print(json.dumps({"match": True, "slug": slug, **e}, ensure_ascii=False, indent=2))
return 0
return {"match": True, "slug": slug, **e}
# No exact match — surface fuzzy candidates so a shortened/expanded name (e.g. "echo
# memory" for the project "echo") reveals the existing note instead of looking absent.
cands = idx_mod.fuzzy_candidates(index, mention)
@@ -54,21 +55,30 @@ def resolve(mention: str) -> int:
"these before creating a new note; reuse the right path or `echo.py link`.")
else:
out["note"] = "no index entry — derive a path with the right --kind and create it"
print(json.dumps(out, ensure_ascii=False, indent=2))
return out
def resolve(mention: str) -> int:
print(json.dumps(resolve_op(mention), ensure_ascii=False, indent=2))
return 0
# ------------------------------------------------------------------- link -----
def link(a_path: str, b_path: str, as_json: bool = False) -> int:
def link_op(a_path: str, b_path: str) -> dict:
"""Core link: reciprocal `## Related` links, returns the change envelope."""
import echo_output
a_changed, b_changed = links.link_bidirectional(a_path, b_path)
return echo_output.envelope("link", {"a": a_path, "b": b_path,
"a_changed": a_changed, "b_changed": b_changed})
def link(a_path: str, b_path: str, as_json: bool = False) -> int:
env = link_op(a_path, b_path)
if as_json:
import echo_output
env = echo_output.envelope("link", {"a": a_path, "b": b_path,
"a_changed": a_changed, "b_changed": b_changed})
print(json.dumps(env, ensure_ascii=False))
return 0
print(f"ok: linked {a_path} <-> {b_path} "
f"(added: {'A' if a_changed else '-'}{'B' if b_changed else '-'})")
print(f"ok: linked {env['a']} <-> {env['b']} "
f"(added: {'A' if env['a_changed'] else '-'}{'B' if env['b_changed'] else '-'})")
return 0
@@ -176,52 +186,52 @@ def _append_to_existing(path: str, kind: str, today_s: str, body_text: str) -> N
echo.cmd_fm(path, "updated", json.dumps(today_s))
def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "",
aliases=None, sources=None, tags=None, date: str | None = None,
domain: str = "business", inbox: bool = False, no_log: bool = False,
as_json: bool = False, dry_run: bool = False, force: bool = False,
merge_into: str | None = None) -> int:
def capture_op(kind: str | None, title: str, file_arg: str | None = None, status_v: str = "",
aliases=None, sources=None, tags=None, date: str | None = None,
domain: str = "business", inbox: bool = False, no_log: bool = False,
dry_run: bool = False, force: bool = False,
merge_into: str | None = None, body_text: str | None = None) -> dict:
"""Core capture (Phase 0): route + frontmatter + index + auto-link + agent-log,
returning an envelope dict — never printing to stdout (helper chatter from the
low-level verbs is redirected to stderr). Special outcomes are DATA, not exit
codes: `duplicate-gate` (ok=false, candidates), `queued:capture` (queued=true),
`dry-run:*`. Raises echo.EchoError on hard failure. `body_text` may be passed
directly (MCP path); otherwise it is read from file_arg/stdin."""
import contextlib
import io
import echo_output
import echo_quality
real_stdout = sys.stdout
def chatter():
# Low-level verbs (cmd_put/cmd_append/cmd_fm) print progress lines; keep
# stdout clean for the envelope by routing them to stderr wholesale.
return contextlib.redirect_stdout(sys.stderr)
def quiet():
# M4: in --json mode, swallow the helper "ok:" chatter so stdout is clean JSON.
return contextlib.redirect_stdout(io.StringIO()) if as_json else contextlib.nullcontext()
def done(action: str, path: str, links: int = 0, ok: bool = True, dry: bool = False,
near=None) -> int:
if as_json:
act = f"dry-run:{action}" if dry else action
data = {"kind": kind, "path": path, "title": title, "links_added": links}
if near:
data["near_duplicates"] = near
env = echo_output.envelope(act, data, ok=ok)
print(json.dumps(env, ensure_ascii=False), file=real_stdout)
elif dry:
print(f"would {action} {kind or '-'} -> {path}")
# non-dry human output is the helper "ok:" lines + the summary printed by the caller.
return 0 if ok else 1
body_text = echo.read_body(file_arg).decode("utf-8", errors="replace")
if body_text is None:
body_text = echo.read_body(file_arg).decode("utf-8", errors="replace")
today_s = echo.today()
aliases = [a.strip() for a in (aliases or []) if a.strip()]
sources = [s.strip() for s in (sources or []) if s.strip()]
tags = [t.strip() for t in (tags or []) if t.strip()]
def env_for(action: str, path: str, links: int = 0, ok: bool = True, dry: bool = False,
near=None, **extra) -> dict:
act = f"dry-run:{action}" if dry else action
data = {"kind": kind, "path": path, "title": title, "links_added": links}
if near:
data["near_duplicates"] = near
data.update(extra)
return echo_output.envelope(act, data, ok=ok)
# Unknown home -> defer to the inbox (a single idempotent capture line).
if inbox or not kind:
if dry_run:
return done("inbox", "inbox/captures/inbox.md", dry=True)
return env_for("inbox", "inbox/captures/inbox.md", dry=True)
line = f"- {today_s}: {title}"
if body_text.strip():
line += f"{body_text.strip().splitlines()[0]}"
with quiet():
with chatter():
rc = echo.cmd_append("inbox/captures/inbox.md", line)
return done("inbox", "inbox/captures/inbox.md", ok=rc == 0)
return env_for("inbox", "inbox/captures/inbox.md", ok=rc == 0)
slug = idx_mod.slugify(title)
# Offline short-circuit (2.1.0). The routed path needs the entity index and several
@@ -235,9 +245,8 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
if "unreachable" not in str(exc):
raise
if dry_run:
print("offline: vault unreachable — a real run would queue this capture "
"for replay on the next reachable session.", file=real_stdout)
return 0
return echo_output.envelope("dry-run:queued:capture",
{"kind": kind, "title": title, "queued": True})
import echo_queue
echo_queue.enqueue_capture({
"kind": kind, "title": title, "body_text": body_text, "status_v": status_v,
@@ -245,23 +254,16 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
"domain": domain, "inbox": False, "no_log": no_log,
"force": force, "merge_into": merge_into, "today": today_s,
})
if as_json:
env = echo_output.envelope("queued:capture",
{"kind": kind, "title": title, "queued": True})
print(json.dumps(env, ensure_ascii=False), file=real_stdout)
else:
print(f"queued (offline): capture {kind} '{title}' — will replay through "
"capture on the next reachable session", file=real_stdout)
return 0
return echo_output.envelope("queued:capture",
{"kind": kind, "title": title, "queued": True})
match_slug, existing = idx_mod.resolve(index, title)
# --merge-into: the operator has already identified the canonical entity (e.g. after
# a duplicate-gate stop) — route this capture as an UPDATE to it, whatever the title.
if merge_into and not existing:
match_slug, existing = idx_mod.resolve(index, merge_into)
if not existing:
print(f"echo_ops: --merge-into '{merge_into}' matches no entity in the index "
f"(try `resolve` first)", file=sys.stderr)
return 2
raise echo.EchoError(f"--merge-into '{merge_into}' matches no entity in the "
"index (try `resolve` first)", 2)
existing_reachable = bool(existing and echo.request("GET", echo.vault_url(existing["path"]))[0] == 200)
# Pre-write duplicate gate: a strong fuzzy candidate means this title is very likely
@@ -276,40 +278,26 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
for s, c, sc in idx_mod.gate_candidates(index, title, kind=kind,
threshold=DUP_GATE)]
if gate_hits:
if as_json:
env = echo_output.envelope("duplicate-gate", {
"kind": kind, "title": title, "candidates": gate_hits,
"note": "likely duplicate — re-run with --merge-into <slug> to update the "
"existing entity, or --force to create anyway"}, ok=False)
print(json.dumps(env, ensure_ascii=False), file=real_stdout)
else:
print(f"STOP: '{title}' likely already exists — not creating a duplicate.",
file=real_stdout)
for c in gate_hits:
print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']})",
file=real_stdout)
print(" re-run with --merge-into <slug> to update the existing entity, "
"or --force to create anyway.", file=real_stdout)
return 76 # distinct exit: duplicate gate (cf. 75 lock, 78 config)
return echo_output.envelope("duplicate-gate", {
"kind": kind, "title": title, "candidates": gate_hits,
"note": "likely duplicate — re-run with --merge-into <slug> to update the "
"existing entity, or --force to create anyway"}, ok=False)
if dry_run:
if existing_reachable:
return done("update", existing["path"], dry=True)
return env_for("update", existing["path"], dry=True)
s2 = echo_quality.safe_slug(set(index.get("entities", {}).keys()), slug)
near = [c.get("path") for _, c, _ in idx_mod.fuzzy_candidates(index, title)][:3]
plan = done("create", idx_mod.derive_path(kind, s2, date=date, domain=domain),
dry=True, near=near)
if (not as_json and not force
and idx_mod.gate_candidates(index, title, kind=kind, threshold=DUP_GATE)):
# (in --json mode the near_duplicates field carries this; keep stdout clean)
print("note: a real run would STOP at the duplicate gate — use --merge-into "
"or --force.", file=real_stdout)
return plan
would_gate = (not force
and bool(idx_mod.gate_candidates(index, title, kind=kind,
threshold=DUP_GATE)))
return env_for("create", idx_mod.derive_path(kind, s2, date=date, domain=domain),
dry=True, near=near, would_gate=would_gate)
near_dupes: list[str] = []
index_title = title # title to record in the index entry
index_aliases = list(aliases)
with quiet():
with chatter():
if existing_reachable:
# Reuse the matched entity's CANONICAL slug — never re-slug from the mention, or
# two index slugs end up pointing at one note. Keep its title; learn the mention
@@ -374,13 +362,61 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
if not no_log:
ensure_daily_log(f"- {today_s}: {action} {kind} [[{links.link_token(path)}]]")
return env_for(action, path, links=linked, near=near_dupes)
def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "",
aliases=None, sources=None, tags=None, date: str | None = None,
domain: str = "business", inbox: bool = False, no_log: bool = False,
as_json: bool = False, dry_run: bool = False, force: bool = False,
merge_into: str | None = None) -> int:
"""CLI wrapper around capture_op: prints exactly what pre-Phase-0 capture printed
(human prose or the --json envelope) and maps envelope outcomes to exit codes
(76 duplicate-gate, 2 usage-class errors, 0 otherwise)."""
try:
env = capture_op(kind, title, file_arg, status_v=status_v, aliases=aliases,
sources=sources, tags=tags, date=date, domain=domain,
inbox=inbox, no_log=no_log, dry_run=dry_run, force=force,
merge_into=merge_into)
except echo.EchoError as exc:
print(f"echo_ops: {exc}", file=sys.stderr)
return getattr(exc, "code", 1)
action = env.get("action", "")
if as_json:
done(action, path, links=linked, near=near_dupes)
else:
print(f"ok: {action} {kind} -> {path}" + (f"; auto-linked {linked}" if linked else ""))
if near_dupes:
kind_word = "entity" if len(near_dupes) == 1 else "entities"
print(f"WARNING: similar existing {kind_word} ({', '.join(near_dupes)}) — if this is "
f"the same thing, merge or `echo.py link` instead of keeping a duplicate.",
file=real_stdout)
return 0
print(json.dumps(env, ensure_ascii=False))
if action == "duplicate-gate":
return 76
return 0 if env.get("ok") else 1
if action == "duplicate-gate":
print(f"STOP: '{title}' likely already exists — not creating a duplicate.")
for c in env.get("candidates", []):
print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']})")
print(" re-run with --merge-into <slug> to update the existing entity, "
"or --force to create anyway.")
return 76 # distinct exit: duplicate gate (cf. 75 lock, 78 config)
if action == "queued:capture":
print(f"queued (offline): capture {kind} '{title}' — will replay through "
"capture on the next reachable session")
return 0
if action == "dry-run:queued:capture":
print("offline: vault unreachable — a real run would queue this capture "
"for replay on the next reachable session.")
return 0
if action.startswith("dry-run:"):
print(f"would {action.split(':', 1)[1]} {kind or '-'} -> {env.get('path')}")
if env.get("would_gate"):
print("note: a real run would STOP at the duplicate gate — use --merge-into "
"or --force.")
return 0
if action == "inbox":
return 0 if env.get("ok") else 1
print(f"ok: {action} {kind} -> {env.get('path')}"
+ (f"; auto-linked {env['links_added']}" if env.get("links_added") else ""))
near = env.get("near_duplicates")
if near:
kind_word = "entity" if len(near) == 1 else "entities"
print(f"WARNING: similar existing {kind_word} ({', '.join(near)}) — if this is "
f"the same thing, merge or `echo.py link` instead of keeping a duplicate.")
return 0 if env.get("ok") else 1
@@ -396,32 +396,10 @@ def _doc_summary(path: str, text: str) -> dict:
return out
def _brief(path: str, score: float | None = None, via: str | None = None) -> None:
text = links.get_text(path)
if text is None:
return
info = _doc_summary(path, text)
head = f"\n### {path}"
if score is not None:
head += f" (score {score:.2f})"
if via:
head += f" (via {via})"
print(head)
stamps = []
if info.get("type"):
stamps.append(f"type: {info['type']}")
if info.get("updated"):
stamps.append(f"updated: {info['updated']}")
if info.get("status"):
stamps.append(f"status: {info['status']}")
if stamps:
print("_" + " · ".join(stamps) + "_") # staleness is visible, not guessed
if info.get("excerpt"):
print(info["excerpt"])
# ------------------------------------------------------------------- entrypoint
def recall(query, limit: int = 8, as_json: bool = False) -> int:
def recall_op(query, limit: int = 8) -> dict:
"""Core recall (Phase 0): hybrid BM25 + graph, returning the structured envelope
({query, primary, linked}) that backs both the CLI renderings and the MCP tool."""
q = " ".join(query) if isinstance(query, list) else query
today_s = echo.today()
index = idx_mod.load()
@@ -464,31 +442,49 @@ def recall(query, limit: int = 8, as_json: bool = False) -> int:
# --- graph layer ----------------------------------------------------------
neighbours = expand_graph(hits, nmap, base, max_hops=MAX_HOPS)
import echo_output
texts = echo.read_many(hits + [p for p, _ in neighbours[: 2 * limit]])
primary = []
for p in hits:
if texts.get(p) is None:
continue
primary.append({**_doc_summary(p, texts[p]),
"score": round(base.get(p, 0.0), 3)})
linked = []
for p, (sc, via) in neighbours[: 2 * limit]:
if texts.get(p) is None:
continue
linked.append({**_doc_summary(p, texts[p]),
"score": round(sc, 3), "via": via})
return echo_output.envelope("recall", {"query": q, "primary": primary,
"linked": linked})
def _print_hit(info: dict) -> None:
"""Prose rendering of one recall hit — same fields the envelope carries."""
head = f"\n### {info['path']}"
if info.get("score") is not None:
head += f" (score {info['score']:.2f})"
if info.get("via"):
head += f" (via {info['via']})"
print(head)
stamps = [f"{k}: {info[k]}" for k in ("type", "updated", "status") if info.get(k)]
if stamps:
print("_" + " · ".join(stamps) + "_") # staleness is visible, not guessed
if info.get("excerpt"):
print(info["excerpt"])
def recall(query, limit: int = 8, as_json: bool = False) -> int:
env = recall_op(query, limit=limit)
if as_json:
import echo_output
texts = echo.read_many(hits + [p for p, _ in neighbours[: 2 * limit]])
primary = []
for p in hits:
if texts.get(p) is None:
continue
primary.append({**_doc_summary(p, texts[p]),
"score": round(base.get(p, 0.0), 3)})
linked = []
for p, (sc, via) in neighbours[: 2 * limit]:
if texts.get(p) is None:
continue
linked.append({**_doc_summary(p, texts[p]),
"score": round(sc, 3), "via": via})
env = echo_output.envelope("recall", {"query": q, "primary": primary,
"linked": linked})
print(json.dumps(env, ensure_ascii=False))
return 0
print(f"== recall: {q} ==")
print(f"\n# Primary hits ({len(hits)})")
for p in hits:
_brief(p, score=base.get(p))
print(f"\n# Linked context ({len(neighbours)})")
for p, (sc, via) in neighbours[: 2 * limit]:
_brief(p, score=sc, via=via)
print(f"== recall: {env['query']} ==")
print(f"\n# Primary hits ({len(env['primary'])})")
for info in env["primary"]:
_print_hit(info)
print(f"\n# Linked context ({len(env['linked'])})")
for info in env["linked"]:
_print_hit(info)
return 0
@@ -99,43 +99,69 @@ def preview(proposals: list[dict]) -> str:
return "\n".join(rows)
def apply(proposals: list[dict], confirm: bool = False) -> int:
"""Validate -> classify -> preview; with confirm=True, apply each via echo_ops.capture
(which routes/indexes/links/logs, each self-lock-guarded). Without confirm it is a
dry-run that writes nothing — the preview IS the confirmation step."""
def apply_op(proposals: list[dict], confirm: bool = False) -> dict:
"""Core reflect (Phase 0): validate -> classify -> (apply via capture_op). Returns
an envelope — rows carry the classified preview; with confirm, `results` carries
each proposal's capture outcome. Never prints to stdout."""
import echo_output
valid, errors = validate(proposals)
for e in errors:
print(f"skip: {e}", file=sys.stderr)
if not valid:
print("reflect: no valid proposals to apply.")
return 0
classify(valid)
counts = {a: sum(1 for p in valid if p.get("_action") == a) for a in ("create", "update", "inbox", "error")}
print(f"reflect: {len(valid)} proposal(s) — "
f"{counts['create']} new, {counts['update']} update, {counts['inbox']} inbox"
+ (f", {counts['error']} error" if counts["error"] else ""))
print(preview(valid))
if not confirm:
print("\nreflect: dry-run — re-run with --apply to write these to memory.")
return 0
rows: list[dict] = []
counts: dict[str, int] = {}
if valid:
classify(valid)
counts = {a: sum(1 for p in valid if p.get("_action") == a)
for a in ("create", "update", "inbox", "error")}
rows = [{"action": p.get("_action"), "kind": p.get("kind"),
"title": p["title"], "path": p.get("_path")} for p in valid]
data = {"proposals": len(proposals or []), "valid": len(valid), "errors": errors,
"counts": counts, "rows": rows, "dry_run": not confirm,
"applied": 0, "gated": 0, "results": []}
if not valid or not confirm:
return echo_output.envelope("reflect", data)
import echo_ops # lazy: the apply path pulls in the capture/index/link stack
applied = gated = 0
results = []
for p in valid:
if p.get("_action") == "error":
continue
bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None
rc = echo_ops.capture(
p.get("kind"), p["title"], bodyfile,
env = echo_ops.capture_op(
p.get("kind"), p["title"], body_text=p.get("body") or "",
aliases=p.get("aliases") or [], sources=p.get("sources") or [],
tags=p.get("tags") or [],
date=p.get("date"), domain=p.get("domain", "business"),
inbox=bool(p.get("inbox")) or not p.get("kind"))
applied += 1 if rc == 0 else 0
gated += 1 if rc == 76 else 0
print(f"reflect: applied {applied}/{len(valid)} proposal(s)."
act = env.get("action", "")
if act == "duplicate-gate":
gated += 1
elif env.get("ok"):
applied += 1
results.append({"title": p["title"], "action": act,
"path": env.get("path"), "ok": bool(env.get("ok"))})
data.update(applied=applied, gated=gated, results=results)
return echo_output.envelope("reflect", data)
def apply(proposals: list[dict], confirm: bool = False) -> int:
"""CLI wrapper: same preview/summary text as pre-Phase-0. Without confirm it is a
dry-run that writes nothing — the preview IS the confirmation step."""
env = apply_op(proposals, confirm=confirm)
for e in env["errors"]:
print(f"skip: {e}", file=sys.stderr)
if not env["valid"]:
print("reflect: no valid proposals to apply.")
return 0
counts = env["counts"]
print(f"reflect: {env['valid']} proposal(s) — "
f"{counts['create']} new, {counts['update']} update, {counts['inbox']} inbox"
+ (f", {counts['error']} error" if counts["error"] else ""))
print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} "
f"| {r['title']} -> {r['path'] or '?'}" for r in env["rows"]))
if env["dry_run"]:
print("\nreflect: dry-run — re-run with --apply to write these to memory.")
return 0
gated = env["gated"]
print(f"reflect: applied {env['applied']}/{env['valid']} proposal(s)."
+ (f" {gated} stopped at the duplicate gate — re-propose with the existing "
f"entity's title (or capture --merge-into <slug>)." if gated else ""))
return 0
@@ -72,36 +72,34 @@ def validate(bundle: dict) -> tuple[str, str]:
return path, line
def session_end(bundle: dict, apply: bool = False) -> int:
def session_end_op(bundle: dict, apply: bool = False) -> dict:
"""Core session-end (Phase 0): validate -> plan -> (apply). Returns an envelope
(plan rows + per-step results); never prints to stdout — helper chatter from the
underlying verbs routes to stderr. Raises echo.EchoError(2) on a bad bundle,
BEFORE any write."""
import contextlib
import echo_output
import echo_reflect
path, line = validate(bundle)
scope = str(bundle.get("scope") or "").strip()
proposals = bundle.get("reflect") or []
valid, errors = echo_reflect.validate(proposals)
for e in errors:
print(f"skip: {e}", file=sys.stderr)
print(f"session-end plan ({'APPLY' if apply else 'dry-run'}):")
print(f" 1. session log -> {path}")
print(f" 2. agent-log line: {line}")
rows: list[dict] = []
if valid:
echo_reflect.classify(valid)
print(f" 3. reflect: {len(valid)} proposal(s)")
print(echo_reflect.preview(valid))
else:
print(" 3. reflect: (none)")
print(f" 4. scope set: {scope!r}" if scope else " 4. scope: (unchanged)")
print(f" 5. heartbeat -> {path} @ <now> (written LAST — the commit marker)")
rows = [{"action": p.get("_action"), "kind": p.get("kind"),
"title": p["title"], "path": p.get("_path")} for p in valid]
data = {"path": path, "agent_log_line": line, "scope": scope or None,
"reflect_rows": rows, "reflect_errors": errors,
"dry_run": not apply, "steps": {}}
if not apply:
print("\nsession-end: dry-run — re-run with --apply to write.")
return 0
return echo_output.envelope("session-end", data)
import echo_concurrency
import echo_ops
steps: dict[str, str] = {}
with echo_concurrency.vault_lock():
with echo_concurrency.vault_lock(), contextlib.redirect_stdout(sys.stderr):
# 1. The session log itself. A hard failure here aborts the whole bundle —
# nothing after it (including the heartbeat) runs, so orientation state
# can never point at a log that was never written.
@@ -116,14 +114,13 @@ def session_end(bundle: dict, apply: bool = False) -> int:
for p in valid:
if p.get("_action") == "error":
continue
bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None
rc = echo_ops.capture(
p.get("kind"), p["title"], bodyfile,
env = echo_ops.capture_op(
p.get("kind"), p["title"], body_text=p.get("body") or "",
aliases=p.get("aliases") or [], sources=p.get("sources") or [],
tags=p.get("tags") or [], date=p.get("date"),
domain=p.get("domain", "business"),
inbox=bool(p.get("inbox")) or not p.get("kind"))
gated += 1 if rc == 76 else 0
gated += 1 if env.get("action") == "duplicate-gate" else 0
steps["reflect"] = f"{len(valid) - gated}/{len(valid)} applied" \
+ (f", {gated} gated (re-propose with --merge-into)" if gated else "")
else:
@@ -139,6 +136,30 @@ def session_end(bundle: dict, apply: bool = False) -> int:
echo.temp_file(f"{path} @ {echo.now_iso()}\n".encode("utf-8")))
steps["heartbeat"] = "ok"
data["steps"] = steps
return echo_output.envelope("session-end", data)
def session_end(bundle: dict, apply: bool = False) -> int:
"""CLI wrapper: same plan/summary text as before; envelope logic in session_end_op."""
env = session_end_op(bundle, apply=apply)
for e in env["reflect_errors"]:
print(f"skip: {e}", file=sys.stderr)
print(f"session-end plan ({'APPLY' if apply else 'dry-run'}):")
print(f" 1. session log -> {env['path']}")
print(f" 2. agent-log line: {env['agent_log_line']}")
rows = env["reflect_rows"]
if rows:
print(f" 3. reflect: {len(rows)} proposal(s)")
print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} "
f"| {r['title']} -> {r['path'] or '?'}" for r in rows))
else:
print(" 3. reflect: (none)")
print(f" 4. scope set: {env['scope']!r}" if env["scope"] else " 4. scope: (unchanged)")
print(f" 5. heartbeat -> {env['path']} @ <now> (written LAST — the commit marker)")
if env["dry_run"]:
print("\nsession-end: dry-run — re-run with --apply to write.")
return 0
print("\nsession-end: done — "
+ "; ".join(f"{k}: {v}" for k, v in steps.items()))
+ "; ".join(f"{k}: {v}" for k, v in env["steps"].items()))
return 0
@@ -55,19 +55,25 @@ def parse_inbox(text: str) -> list[dict]:
return items
def list_inbox(as_json: bool = False) -> int:
def list_op() -> dict:
"""Core inbox listing (Phase 0): structured captures envelope, no printing."""
import echo_output
status, body = echo.request("GET", echo.vault_url(INBOX_PATH))
if status == 404:
items = []
else:
echo.check(status, body, f"triage list {INBOX_PATH}")
items = parse_inbox(body.decode(errors="replace"))
return echo_output.envelope("triage-list", {"path": INBOX_PATH, "items": items,
"count": len(items)})
def list_inbox(as_json: bool = False) -> int:
env = list_op()
if as_json:
import echo_output
env = echo_output.envelope("triage-list", {"path": INBOX_PATH, "items": items,
"count": len(items)})
print(json.dumps(env, ensure_ascii=False))
return 0
items = env["items"]
if not items:
print("triage: inbox is empty — nothing to route.")
return 0
@@ -80,53 +86,76 @@ def list_inbox(as_json: bool = False) -> int:
return 0
def apply(proposals: list[dict], confirm: bool = False) -> int:
"""Validate -> classify -> preview; with confirm, route each via capture AND write
the processing-log audit line. Mirrors echo_reflect.apply's contract exactly."""
def route_op(proposals: list[dict], confirm: bool = False) -> dict:
"""Core triage routing (Phase 0): reflect pipeline + processing-log audit lines.
Returns an envelope (rows = classified preview; results = per-item outcomes);
never prints to stdout (audit-append chatter routes to stderr)."""
import contextlib
import echo_output
import echo_reflect
valid, errors = echo_reflect.validate(proposals)
for e in errors:
print(f"skip: {e}", file=sys.stderr)
if not valid:
print("triage: no valid proposals to route.")
return 0
echo_reflect.classify(valid)
counts = {a: sum(1 for p in valid if p.get("_action") == a)
for a in ("create", "update", "inbox", "error")}
print(f"triage: {len(valid)} proposal(s) — "
f"{counts['create']} new, {counts['update']} update, {counts['inbox']} stay-in-inbox"
+ (f", {counts['error']} error" if counts["error"] else ""))
print(echo_reflect.preview(valid))
if not confirm:
print("\ntriage: dry-run — re-run with --apply to route these and log the moves.")
return 0
rows: list[dict] = []
counts: dict[str, int] = {}
if valid:
echo_reflect.classify(valid)
counts = {a: sum(1 for p in valid if p.get("_action") == a)
for a in ("create", "update", "inbox", "error")}
rows = [{"action": p.get("_action"), "kind": p.get("kind"),
"title": p["title"], "path": p.get("_path")} for p in valid]
data = {"proposals": len(proposals or []), "valid": len(valid), "errors": errors,
"counts": counts, "rows": rows, "dry_run": not confirm,
"routed": 0, "gated": 0, "log": f"{LOG_DIR}/{echo.today()}.md", "results": []}
if not valid or not confirm:
return echo_output.envelope("triage", data)
import echo_ops
today_s = echo.today()
applied = gated = 0
routed = gated = 0
results = []
for p in valid:
if p.get("_action") == "error":
continue
if p.get("_action") == "inbox":
if p.get("_action") in ("error", "inbox"):
continue # routing an inbox line back to the inbox is a no-op, not a move
bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None
rc = echo_ops.capture(
p.get("kind"), p["title"], bodyfile,
env = echo_ops.capture_op(
p.get("kind"), p["title"], body_text=p.get("body") or "",
aliases=p.get("aliases") or [], sources=p.get("sources") or [],
tags=p.get("tags") or [],
date=p.get("date"), domain=p.get("domain", "business"))
if rc == 76:
act = env.get("action", "")
results.append({"title": p["title"], "action": act,
"path": env.get("path"), "ok": bool(env.get("ok"))})
if act == "duplicate-gate":
gated += 1
continue
if rc != 0:
if not env.get("ok"):
continue
applied += 1
routed += 1
original = (p.get("line") or p["title"]).strip()
echo.cmd_append(f"{LOG_DIR}/{today_s}.md",
f"- {original}{p.get('_path', '?')}")
print(f"triage: routed {applied}/{len(valid)} item(s); audit in {LOG_DIR}/{today_s}.md. "
with contextlib.redirect_stdout(sys.stderr):
echo.cmd_append(f"{LOG_DIR}/{today_s}.md",
f"- {original}{p.get('_path', '?')}")
data.update(routed=routed, gated=gated, results=results)
return echo_output.envelope("triage", data)
def apply(proposals: list[dict], confirm: bool = False) -> int:
"""CLI wrapper: same preview/summary text as pre-Phase-0. Mirrors reflect's contract."""
env = route_op(proposals, confirm=confirm)
for e in env["errors"]:
print(f"skip: {e}", file=sys.stderr)
if not env["valid"]:
print("triage: no valid proposals to route.")
return 0
counts = env["counts"]
print(f"triage: {env['valid']} proposal(s) — "
f"{counts['create']} new, {counts['update']} update, {counts['inbox']} stay-in-inbox"
+ (f", {counts['error']} error" if counts["error"] else ""))
print("\n".join(f" {r['action'] or '?':7} | {('-' if r['action'] == 'inbox' else r['kind'] or '?'):9} "
f"| {r['title']} -> {r['path'] or '?'}" for r in env["rows"]))
if env["dry_run"]:
print("\ntriage: dry-run — re-run with --apply to route these and log the moves.")
return 0
gated = env["gated"]
print(f"triage: routed {env['routed']}/{env['valid']} item(s); audit in {env['log']}. "
"Originals kept in the inbox (deletion is explicit-only)."
+ (f" {gated} stopped at the duplicate gate — re-propose with the existing "
f"entity's title or use capture --merge-into." if gated else ""))
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""test_ops_api.py — Phase 0 (return-not-print) contract: every high-level op has a
core `*_op` function that RETURNS an envelope dict and prints nothing to stdout.
This is the seam the MCP server wraps (docs/MCP-SERVER-SPEC.md §4): the CLI wrappers
are tested by the other suites; here we call the cores in-process against the mock and
assert (a) the envelope shapes and (b) stdout purity — a stray print() would corrupt
an MCP stdio/HTTP response stream.
Run: python test_ops_api.py [--port 8850]
"""
from __future__ import annotations
import argparse
import contextlib
import io
import json
import os
import subprocess
import sys
import tempfile
import time
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
SCRIPTS = HERE.parent / "echo-memory.plugin.src" / "skills" / "echo-memory" / "scripts"
KEY = "test-key-not-a-real-secret"
ap = argparse.ArgumentParser()
ap.add_argument("--port", type=int, default=8850)
a = ap.parse_args()
BASE = f"http://127.0.0.1:{a.port}"
# env must be set BEFORE importing echo (it resolves config at import time)
os.environ.update(ECHO_BASE=BASE, ECHO_KEY=KEY, ECHO_TODAY="2026-06-21",
ECHO_NOW="2300", ECHO_STATE_DIR=tempfile.mkdtemp(), ECHO_VERIFY="0")
sys.path.insert(0, str(SCRIPTS))
failures = []
def check(name, cond, detail=""):
print(f"{'ok ' if cond else 'FAIL'} {name}" + (f" -- {detail}" if not cond else ""))
if not cond:
failures.append(name)
def http(method, url, body=None):
data = body.encode() if isinstance(body, str) else body
req = urllib.request.Request(url, data=data, method=method,
headers={"Authorization": f"Bearer {KEY}"})
try:
with urllib.request.urlopen(req, timeout=10) as r:
return r.status, r.read().decode("utf-8", "replace")
except Exception as e: # noqa: BLE001
return getattr(e, "code", 0), ""
def pure(fn, *args, **kw):
"""Call fn capturing stdout; return (result, captured_stdout)."""
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
out = fn(*args, **kw)
return out, buf.getvalue()
def main():
srv = subprocess.Popen([sys.executable, str(HERE / "mock_olrapi.py"), "--port", str(a.port)],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
try:
for _ in range(50):
try:
urllib.request.urlopen(f"{BASE}/__debug__reset", data=b"", timeout=1)
break
except Exception:
time.sleep(0.1)
http("PUT", f"{BASE}/vault/_agent/echo-vault.md", "---\nschema_version: 4\n---\n# marker\n")
http("PUT", f"{BASE}/vault/_agent/context/current-context.md",
"---\ntype: context-bundle\nscope_updated: \"2026-06-20\"\ncreated: 2026-06-01\n---\n"
"# Current Context\n\n## Scope\ntesting phase 0\n\n## Scope History\n")
import echo
import echo_doctor
import echo_ops
import echo_recall
import echo_reflect
import echo_session
import echo_triage
# capture_op: create / duplicate-gate / dry-run — envelopes, stdout pure
env, out = pure(echo_ops.capture_op, "person", "Zara Quix", body_text="Met at the expo.")
check("capture_op create envelope", env.get("ok") is True and env.get("action") == "created"
and env.get("path") == "resources/people/zara-quix.md", json.dumps(env))
check("capture_op prints nothing to stdout", out == "", out[:200])
env, out = pure(echo_ops.capture_op, "person", "Zara Quix Junior", body_text="")
check("capture_op duplicate-gate is data, not an exit code",
env.get("ok") is False and env.get("action") == "duplicate-gate"
and env.get("candidates"), json.dumps(env))
check("capture_op gate prints nothing", out == "", out[:200])
env, out = pure(echo_ops.capture_op, "company", "Plan Co", body_text="", dry_run=True)
check("capture_op dry-run envelope", env.get("action") == "dry-run:create"
and env.get("path") == "resources/companies/plan-co.md", json.dumps(env))
# resolve_op / link_op
env, out = pure(echo_ops.resolve_op, "zara quix")
check("resolve_op returns the match dict", env.get("match") is True
and env.get("path") == "resources/people/zara-quix.md", json.dumps(env))
pure(echo_ops.capture_op, "concept", "Gizmo", body_text="")
env, out = pure(echo_ops.link_op, "resources/people/zara-quix.md", "resources/concepts/gizmo.md")
check("link_op envelope", env.get("ok") is True and env.get("a_changed") in (True, False),
json.dumps(env))
check("link_op prints nothing", out == "", out[:200])
# recall_op
env, out = pure(echo_recall.recall_op, "expo")
check("recall_op envelope shape", env.get("action") == "recall"
and isinstance(env.get("primary"), list) and isinstance(env.get("linked"), list),
json.dumps(env)[:200])
check("recall_op prints nothing", out == "", out[:200])
# triage list_op / route_op (dry-run)
http("PUT", f"{BASE}/vault/inbox/captures/inbox.md", "- 2026-06-10: try the quorlab tool\n")
env, out = pure(echo_triage.list_op)
check("triage list_op envelope", env.get("count") == 1
and env["items"][0]["age_days"] == 11, json.dumps(env))
props = [{"title": "Quorlab Tool", "kind": "reference", "confidence": 0.9,
"line": "- 2026-06-10: try the quorlab tool"}]
env, out = pure(echo_triage.route_op, props, False)
check("triage route_op dry-run rows", env.get("dry_run") is True
and env["rows"][0]["action"] == "create", json.dumps(env))
check("triage route_op prints nothing", out == "", out[:200])
# reflect apply_op (applied path, using capture_op internally)
env, out = pure(echo_reflect.apply_op,
[{"title": "Blorp Pattern", "kind": "semantic",
"body": "The operator prefers blorp.", "confidence": 0.9}], True)
check("reflect apply_op applies via capture_op", env.get("applied") == 1
and env["results"][0]["action"] == "created", json.dumps(env))
check("reflect apply_op prints nothing", out == "", out[:200])
# scope ops
env, out = pure(echo.scope_show_op)
check("scope_show_op returns scope + freshness", env.get("scope") == "testing phase 0"
and env.get("scope_updated") == "2026-06-20", json.dumps(env))
env, out = pure(echo.scope_set_op, "phase zero refactor")
check("scope_set_op envelope", env.get("action") == "scope-set"
and env.get("scope_updated") == "2026-06-21", json.dumps(env))
check("scope_set_op prints nothing", out == "", out[:200])
# load_op (brief)
env, out = pure(echo.load_op, True)
check("load_op returns sections + brief", "marker" in env.get("sections", {})
and "ECHO load (brief)" in env.get("brief", ""), json.dumps(env)[:200])
check("load_op prints nothing", out == "", out[:200])
# doctor run_op
env, out = pure(echo_doctor.run_op)
check("doctor run_op checks list", isinstance(env.get("checks"), list)
and env.get("fatal") is None and env.get("ok") is True, json.dumps(env))
check("doctor run_op prints nothing", out == "", out[:200])
# session_end_op: dry-run envelope; bad bundle raises EchoError(2) pre-write
bundle = {"slug": "phase-zero", "log_body": "---\ntype: session-log\n---\n# S\n\n## Goal\nx\n"}
env, out = pure(echo_session.session_end_op, bundle, False)
check("session_end_op dry-run envelope", env.get("dry_run") is True
and env.get("path") == "_agent/sessions/2026-06-21-2300-phase-zero.md", json.dumps(env))
check("session_end_op prints nothing", out == "", out[:200])
try:
echo_session.session_end_op({"slug": "Bad Slug!", "log_body": "x"}, True)
check("session_end_op rejects a bad slug", False, "no exception raised")
except RuntimeError as exc:
check("session_end_op rejects a bad slug", getattr(exc, "code", 0) == 2, str(exc))
print(f"\n{len(failures)} failure(s)" if failures else "\nall ops-api tests passed")
return 1 if failures else 0
finally:
srv.terminate()
if __name__ == "__main__":
raise SystemExit(main())