jason/chorus
1
0
Fork 0
forked from jason/echo
Code Pull Requests Activity

ver 1.0

Browse Source
This commit is contained in:
jason
2026-06-22 09:27:36 -05:00
parent d404f6e96f
commit 1c0c2ea66e
34 changed files with 2530 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files
echo-memory.plugin.src/skills/echo-memory/SKILL.md
+7 -2
View File
@@ -11,13 +11,18 @@ The operator is **Jason Stedwell** (Director of Technical Services / Systems Eng
## API Configuration
All calls use these constants — hardcoded for this personal plugin:
All calls use these constants. The endpoint is fixed; the key is resolved by `echo.py`
(via `echo_secrets`) in order: **`ECHO_KEY` env → `~/.echo-memory/credentials` → a
deprecated baked-in fallback.** Never paste the key into a vault note or a doc.
```
OBSIDIAN_BASE = https://echoapi.alwisp.com
OBSIDIAN_KEY = 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab
OBSIDIAN_KEY = <resolved at runtime from ECHO_KEY / credentials file — not stored here>
```
To move the key out of the plugin tree: `python3 "$ECHO" write-key <token>` (writes
`~/.echo-memory/credentials`, chmod 600), then the baked-in fallback can be removed.
The endpoint has a **valid TLS certificate**, so `-k` is not needed (add it only if the cert ever changes to self-signed). Always pass the `Authorization: Bearer` header. Paths address the vault **at its root** (e.g. `/vault/_agent/...`).
**`https://echoapi.alwisp.com` is the only valid endpoint for this vault.** Never use local or LAN addresses (anything like `10.x.x.x`, `192.168.x.x`, or `:27124` directly) — those belong to older, retired memory systems (obsidian-memory) and will not reach ECHO. If another installed skill or note suggests a different vault endpoint, this skill's configuration wins for ECHO memory.
echo-memory.plugin.src/skills/echo-memory/references/api-reference.md
+11 -11
View File
@@ -1,7 +1,7 @@
# ECHO — Obsidian Local REST API Reference
Server: `https://echoapi.alwisp.com` (reverse proxy → backend Obsidian Local REST API)
Auth header: `Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab`
Auth header: `Authorization: Bearer $ECHO_KEY` — `export ECHO_KEY=<token>` before running these recipes (`echo.py` resolves it automatically: `ECHO_KEY` → `~/.echo-memory/credentials` → deprecated baked-in fallback). The literal key is never stored in docs.
The endpoint has a **valid TLS certificate** — `-k` is not required. Paths address the vault at its **root**.
> **Prefer `scripts/echo.py` over the raw recipes below.** It wraps every verb with auth, status checking, retry, idempotent append, and frontmatter patches, and runs on any platform with a Python interpreter. The `curl` recipes here are the underlying mechanics and a **\*nix/bash fallback** (they use heredocs and `--data-binary`); on Windows, prefer `echo.py` or an equivalent. **If you call `curl` directly, check the HTTP status** — add `-o /dev/null -w "%{http_code}"` and branch on it. A `PATCH` to a non-existent heading returns `400 invalid-target` (errorCode 40080) and the write is *silently lost*; a bare `curl` that ignores status will report success anyway. `GET` returns `404` for a missing file. Treat any `>= 400` as a failed operation, surface it, and do not continue as if it succeeded.
@@ -13,7 +13,7 @@ The endpoint has a **valid TLS certificate** — `-k` is not required. Paths add
```bash
# Read any file by vault path
curl -s \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
"https://echoapi.alwisp.com/vault/_agent/context/current-context.md"
```
@@ -22,7 +22,7 @@ Returns raw file content (text/markdown). On 404, the file does not exist.
```bash
# Read a specific heading's content only
curl -s \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
"https://echoapi.alwisp.com/vault/_agent/memory/semantic/operator-preferences.md/heading/Operator"
```
@@ -38,7 +38,7 @@ Nested headings: separate levels with `::` (URL-encode spaces as `%20`):
```bash
# List contents of a directory (trailing slash required)
curl -s \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
"https://echoapi.alwisp.com/vault/_agent/sessions/"
```
@@ -56,7 +56,7 @@ cat > /tmp/obs_entry.md << 'OBSEOF'
OBSEOF
curl -s -X POST \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
-H "Content-Type: text/markdown" \
--data-binary @/tmp/obs_entry.md \
"https://echoapi.alwisp.com/vault/inbox/captures/inbox.md"
@@ -87,7 +87,7 @@ source_notes: []
OBSEOF
curl -s -X PUT \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
-H "Content-Type: text/markdown" \
--data-binary @/tmp/obs_file.md \
"https://echoapi.alwisp.com/vault/_agent/sessions/2026-06-05-1430-my-session.md"
@@ -109,7 +109,7 @@ Jason prefers concise status updates — lead with the decision.
OBSEOF
curl -s -X PATCH \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
-H "Operation: append" \
-H "Target-Type: heading" \
-H "Target: Operator Preferences::Fact / Pattern" \
@@ -124,7 +124,7 @@ When unsure of the exact heading path, GET the note with the document-map Accept
```bash
curl -s \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
-H "Accept: application/vnd.olrapi.document-map+json" \
"https://echoapi.alwisp.com/vault/_agent/memory/semantic/operator-preferences.md"
```
@@ -143,7 +143,7 @@ Same call with `Operation: prepend`.
```bash
curl -s -X PATCH \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
-H "Operation: replace" \
-H "Target-Type: frontmatter" \
-H "Target: updated" \
@@ -158,7 +158,7 @@ curl -s -X PATCH \
```bash
curl -s -X POST \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
"https://echoapi.alwisp.com/search/simple/?query=weekly+review"
```
@@ -170,7 +170,7 @@ Returns an array of `{ filename, score, matches: [{ context, match }] }`.
```bash
curl -s -X DELETE \
-H "Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab" \
-H "Authorization: Bearer $ECHO_KEY" \
"https://echoapi.alwisp.com/vault/inbox/imports/old-note.md"
```
echo-memory.plugin.src/skills/echo-memory/references/bootstrap.md
+1 -1
View File
@@ -20,7 +20,7 @@ python3 "$SCRIPTS/migrate.py" --apply # perform the migration (moves/delet
`bootstrap.py` writes through `echo.py`, so every step is status-checked and the marker is written last. The scripts are pure Python (only an interpreter required — no bash, no platform-specific `date`), so they run identically on Windows, macOS, and Linux. The manual steps below document what the script does (and serve as a fallback if the script can't run).
```
AUTH="Authorization: Bearer 241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab"
AUTH="Authorization: Bearer $ECHO_KEY" # export ECHO_KEY first; echo.py resolves it automatically
BASE="https://echoapi.alwisp.com"
```
echo-memory.plugin.src/skills/echo-memory/scripts/echo.py
+127 -13
View File
@@ -66,9 +66,14 @@ for _stream in (sys.stdout, sys.stderr):
pass
BASE = os.environ.get("ECHO_BASE", "https://echoapi.alwisp.com").rstrip("/")
# The token stays hardcoded for this personal plugin; ECHO_KEY env still overrides.
# Key resolution is centralized in echo_secrets: ECHO_KEY env -> ~/.echo-memory/credentials
# -> the deprecated baked-in fallback below. Move the key out of the tree with
# `echo.py write-key <token>`; remove DEFAULT_KEY before the 1.0 tag (see ROADMAP-1.0.md > M1).
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo_secrets # noqa: E402
DEFAULT_KEY = "241265fbe6830934a9a4ad3e69335f64a42153b663aa5b0017cb1ea1217b2bab"
KEY = os.environ.get("ECHO_KEY") or DEFAULT_KEY
KEY = echo_secrets.resolve_key(DEFAULT_KEY)
VERIFY = os.environ.get("ECHO_VERIFY", "1") != "0"
LOCK_TTL = int(os.environ.get("ECHO_LOCK_TTL", "900"))
@@ -207,10 +212,20 @@ def cmd_search(query: list[str]) -> int:
return 0
def _qwrite(method: str, url: str, data: bytes | None = None, headers: dict | None = None):
"""A mutating request that survives an outage: routes through echo_queue.safe_request,
which enqueues the write and returns (202, ...) when the vault is unreachable (H2)."""
import echo_queue
return echo_queue.safe_request(method, url, data=data, headers=headers)
def cmd_put(path: str, file_arg: str | None) -> int:
data = read_body(file_arg)
status, body = request("PUT", vault_url(path), data=data,
status, body = _qwrite("PUT", vault_url(path), data=data,
headers={"Content-Type": "text/markdown"})
if status == 202:
print(f"queued (offline): PUT {path} — will sync on the next reachable session")
return 0
check(status, body, f"put {path}")
if VERIFY:
status, _ = request("GET", vault_url(path))
@@ -222,8 +237,11 @@ def cmd_put(path: str, file_arg: str | None) -> int:
def cmd_post(path: str, file_arg: str | None) -> int:
data = read_body(file_arg)
status, body = request("POST", vault_url(path), data=data,
status, body = _qwrite("POST", vault_url(path), data=data,
headers={"Content-Type": "text/markdown"})
if status == 202:
print(f"queued (offline): POST {path} — will sync on the next reachable session")
return 0
check(status, body, f"post {path}")
print(f"ok: POST {path}")
return 0
@@ -238,10 +256,15 @@ def cmd_append(path: str, line: str) -> int:
if line.rstrip("\r") in existing:
print(f"skip: line already present in {path}")
return 0
elif status == 0:
pass # offline: skip the idempotency read; queue the POST (flush dedups by content)
elif status != 404:
check(status, body, f"append(read) {path}")
status, body = request("POST", vault_url(path), data=f"{line}\n".encode(),
status, body = _qwrite("POST", vault_url(path), data=f"{line}\n".encode(),
headers={"Content-Type": "text/markdown"})
if status == 202:
print(f"queued (offline): APPEND {path} — will sync on the next reachable session")
return 0
check(status, body, f"append {path}")
print(f"ok: APPEND {path}")
return 0
@@ -253,7 +276,7 @@ def cmd_patch(path: str, op: str, target_type: str, target: str, file_arg: str |
if target_type not in {"heading", "frontmatter", "block"}:
raise EchoError("patch: target-type must be heading, frontmatter, or block", 2)
data = normalize_patch_body(read_body(file_arg), op, target_type)
status, body = request(
status, body = _qwrite(
"PATCH",
vault_url(path),
data=data,
@@ -264,6 +287,9 @@ def cmd_patch(path: str, op: str, target_type: str, target: str, file_arg: str |
"Content-Type": "application/json" if target_type == "frontmatter" else "text/markdown",
},
)
if status == 202:
print(f"queued (offline): PATCH {op} '{target}' -> {path} — will sync on the next reachable session")
return 0
check(status, body, f"patch {path} ({target})")
print(f"ok: PATCH {op} {target_type} '{target}' -> {path}")
return 0
@@ -274,7 +300,10 @@ def cmd_fm(path: str, field: str, value: str) -> int:
def cmd_delete(path: str) -> int:
status, body = request("DELETE", vault_url(path))
status, body = _qwrite("DELETE", vault_url(path))
if status == 202:
print(f"queued (offline): DELETE {path} — will sync on the next reachable session")
return 0
check(status, body, f"delete {path}")
print(f"ok: DELETE {path}")
return 0
@@ -290,7 +319,7 @@ def parse_lock_time(value: str) -> int:
return int(time.time())
def cmd_lock(owner: str) -> int:
def cmd_lock(owner: str, quiet: bool = False) -> int:
status, body = request("GET", vault_url(LOCK_PATH))
if status == 200 and body.strip():
current = body.decode(errors="replace").strip()
@@ -310,11 +339,12 @@ def cmd_lock(owner: str) -> int:
if held_owner != owner:
print(f"lock race: now held by '{held_owner}', not '{owner}' — backing off", file=sys.stderr)
return 75
print(f"ok: locked by {owner}")
if not quiet:
print(f"ok: locked by {owner}")
return 0
def cmd_unlock(owner: str) -> int:
def cmd_unlock(owner: str, quiet: bool = False) -> int:
status, body = request("GET", vault_url(LOCK_PATH))
if status == 200:
held_owner = body.decode(errors="replace").split(" @ ", 1)[0].strip()
@@ -324,7 +354,8 @@ def cmd_unlock(owner: str) -> int:
status, body = request("DELETE", vault_url(LOCK_PATH))
if status != 404:
check(status, body, "unlock")
print("ok: unlocked")
if not quiet:
print("ok: unlocked")
return 0
@@ -403,22 +434,67 @@ def cmd_load() -> int:
("today", f"journal/daily/{today()}.md"),
("inbox", "inbox/captures/inbox.md"),
]
import echo_queue
# H2: sync any writes queued during a prior outage, best-effort and quiet on empty.
try:
synced = echo_queue.flush()
if synced:
print(f"(synced {synced} write(s) queued during a prior offline session)\n")
except Exception as exc: # noqa: BLE001 — never let queue upkeep block a load
print(f"(queue flush skipped: {exc})\n", file=sys.stderr)
marker_missing = False
heartbeat_absent = False
offline = False
for label, path in targets:
status, body = request("GET", vault_url(path))
print(f"===== {label}: {path} (HTTP {status}) =====")
if status == 200:
echo_queue.cache_put(path, body) # refresh last-known-good for offline fallback
print(f"===== {label}: {path} (HTTP 200) =====")
text = body.decode(errors="replace")
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
elif status == 404:
print(f"===== {label}: {path} (HTTP 404) =====")
print("(absent — fine)")
if label == "marker":
marker_missing = True
if label == "heartbeat":
heartbeat_absent = True
elif status == 0: # vault unreachable -> degrade to last-known-good cache
offline = True
cached = echo_queue.cache_get(path)
if cached is not None:
print(f"===== {label}: {path} (OFFLINE — serving stale cache) =====")
sys.stdout.write(cached.decode(errors="replace"))
if not cached.endswith(b"\n"):
sys.stdout.write("\n")
else:
print(f"===== {label}: {path} (OFFLINE — no cache) =====")
print("(unreachable and not cached)")
else:
print(f"===== {label}: {path} (HTTP {status}) =====")
print(f"(error HTTP {status}: {body.decode(errors='replace')[:200]})")
print()
# M3: heartbeat pointer missing/stale -> fall back to the recent sessions listing
# (matches the documented loading procedure) so orientation works without the pointer.
if heartbeat_absent and not offline:
st, body = request("GET", vault_url("_agent/sessions/"))
if st == 200:
try:
files = sorted((f for f in json.loads(body).get("files", []) if f.endswith(".md")),
reverse=True)
except json.JSONDecodeError:
files = []
if files:
print("===== recent sessions (heartbeat absent — fallback) =====")
for f in files[:5]:
print(f" _agent/sessions/{f}")
print()
if offline:
print("NOTE: vault unreachable — context above is last-known-good cache; writes this "
"session will be queued and synced when the vault returns.")
if marker_missing:
print("NOTE: marker absent — vault not bootstrapped. Run bootstrap.py before relying on memory.")
return 0
@@ -428,6 +504,8 @@ def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Validated cross-platform ECHO vault client")
sub = parser.add_subparsers(dest="cmd", required=True)
sub.add_parser("load")
sub.add_parser("flush") # H2: replay writes queued during a prior outage
sub.add_parser("doctor") # M3: one-call readiness check
sub.add_parser("get").add_argument("path")
sub.add_parser("map").add_argument("path")
sub.add_parser("ls").add_argument("path")
@@ -443,9 +521,13 @@ def build_parser() -> argparse.ArgumentParser:
sub.add_parser("delete").add_argument("path")
sub.add_parser("lock").add_argument("owner")
sub.add_parser("unlock").add_argument("owner")
sub.add_parser("write-key").add_argument("token") # M1: move the key to ~/.echo-memory/credentials
p = sub.add_parser("scope")
p.add_argument("subcommand", nargs="?", default="show", choices=["show", "set"])
p.add_argument("text", nargs="?")
p = sub.add_parser("reflect") # H5: apply a JSON proposal set (dry-run unless --apply)
p.add_argument("file", nargs="?")
p.add_argument("--apply", action="store_true")
# High-level operations (entity index + cross-links + recall). See echo_ops.py.
sub.add_parser("resolve").add_argument("mention", nargs="+")
@@ -462,6 +544,8 @@ def build_parser() -> argparse.ArgumentParser:
p.add_argument("--date")
p.add_argument("--domain", default="business")
p.add_argument("--no-log", action="store_true")
p.add_argument("--json", action="store_true") # M4: emit a JSON result envelope
p.add_argument("--dry-run", action="store_true") # M4: preview the plan, write nothing
return parser
@@ -470,6 +554,25 @@ def main(argv: list[str] | None = None) -> int:
try:
if args.cmd == "load":
return cmd_load()
if args.cmd == "doctor":
import echo_doctor
return echo_doctor.run()
if args.cmd == "flush":
import echo_queue
n = echo_queue.flush()
remaining = len(echo_queue.pending())
if n:
print(f"ok: flushed {n} queued write(s)"
+ (f"; {remaining} still queued (vault unreachable)" if remaining else ""))
elif remaining:
print(f"ok: {remaining} write(s) still queued (vault unreachable)")
else:
print("ok: queue empty")
return 0
if args.cmd == "write-key":
path = echo_secrets.write_key(args.token)
print(f"ok: wrote credentials to {path} (chmod 600 best-effort); ECHO_KEY env still overrides")
return 0
if args.cmd == "get":
return cmd_get(args.path)
if args.cmd == "map":
@@ -498,6 +601,16 @@ def main(argv: list[str] | None = None) -> int:
return cmd_unlock(args.owner)
if args.cmd == "scope":
return cmd_scope(args.subcommand, args.text)
if args.cmd == "reflect":
import echo_reflect
raw = read_body(args.file).decode("utf-8", errors="replace")
try:
proposals = json.loads(raw)
except json.JSONDecodeError as exc:
raise EchoError(f"reflect: proposals must be a JSON array ({exc})", 2)
if not isinstance(proposals, list):
raise EchoError("reflect: proposals must be a JSON array of objects", 2)
return echo_reflect.apply(proposals, confirm=args.apply)
if args.cmd in ("resolve", "recall", "link", "capture"):
import echo_ops # lazy: only the high-level ops pull in the index/link modules
if args.cmd == "resolve":
@@ -510,7 +623,8 @@ def main(argv: list[str] | None = None) -> int:
args.kind, args.title, args.file, status_v=args.status,
aliases=args.aliases.split(",") if args.aliases else [],
sources=args.source.split(",") if args.source else [],
date=args.date, domain=args.domain, inbox=args.inbox, no_log=args.no_log)
date=args.date, domain=args.domain, inbox=args.inbox, no_log=args.no_log,
as_json=args.json, dry_run=args.dry_run)
except EchoError as exc:
print(f"echo.py: {exc}", file=sys.stderr)
return exc.code
echo-memory.plugin.src/skills/echo-memory/scripts/echo_concurrency.py
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""echo_concurrency.py — H3: make the multi-writer story actually safe. [v1.0 SCAFFOLD — not yet wired]
The plugin advertises a shared vault (Claude Code + CoWork), but two real races exist:
1. The entity index is a full-file load -> mutate -> PUT (echo_index.load/save) with
NO lock. Two concurrent `capture` calls both read the old index; last PUT wins;
the other entity is silently dropped from the index (the note file survives, but
resolve/recall miss it until the next sweep).
2. The advisory lock is MANUAL (echo.py cmd_lock) — capture / scope set never take it;
it relies on the model remembering to.
This module provides (a) a context manager that auto-acquires/releases the existing
advisory lock around a write burst, with crash-safe release, and (b) an atomic
index-update that re-reads UNDER the lock before saving, so concurrent captures merge
instead of clobber.
ACCEPTANCE (see ROADMAP-1.0.md > H3):
- two concurrent captures both land in the index (no lost entries)
- lock auto-released on normal AND error exit
"""
from __future__ import annotations
import contextlib
import os
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo # noqa: E402
import echo_index as idx_mod # noqa: E402
def auto_owner() -> str:
"""A stable-per-process owner id: client tag + pid. Avoids Math.random/time-based
ids; pid is unique enough for cooperative locking and is reproducible in tests."""
tag = os.environ.get("ECHO_CLIENT", "cc")
return f"{tag}-{os.getpid()}"
@contextlib.contextmanager
def vault_lock(owner: str | None = None, required: bool = False):
"""Hold the advisory lock for a write burst; release on exit (incl. exceptions).
Yields `held: bool`. The lock is cooperative (read-back-confirmed, TTL-reclaimable),
so if another session holds it we do NOT block: with `required=False` we yield
`held=False` and let the caller proceed (advisory) or warn; with `required=True` we
raise. Release is best-effort and never masks the body's own exception.
"""
owner = owner or auto_owner()
rc = echo.cmd_lock(owner, quiet=True)
held = rc == 0
if not held and required:
raise echo.EchoError(f"vault_lock: could not acquire lock for '{owner}' (held elsewhere)", 75)
try:
yield held
finally:
if held:
try:
echo.cmd_unlock(owner, quiet=True)
except Exception: # noqa: BLE001 — release is best-effort; TTL reclaims a leak
pass
def atomic_index_update(mutate, owner: str | None = None):
"""Apply `mutate(index)` to the entity index without losing a concurrent writer's
entry: take the lock, **re-read the index fresh** (not a stale in-memory copy),
run the mutator, save, release. Returns the freshly-read-and-saved index.
The fresh re-read inside the critical section is the core of the fix — even if the
advisory lock can't be taken (another session active), reading immediately before
the save collapses the read-modify-write window to two calls instead of spanning the
whole capture, so an unlocked update is still far safer than the prior code.
"""
with vault_lock(owner) as held:
index = idx_mod.load()
mutate(index)
idx_mod.save(index)
if not held:
print("echo_concurrency: entity index updated WITHOUT the advisory lock "
"(another session may be active); the fresh re-read minimizes the race.",
file=sys.stderr)
return index
echo-memory.plugin.src/skills/echo-memory/scripts/echo_doctor.py
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env python3
"""echo_doctor.py — M3: one-call readiness check. [v1.0 SCAFFOLD — run() implemented]
There is no single "is everything OK" command. doctor checks, in order, the things
that make ECHO usable this session and prints a green/red report. Intended to back an
`echo.py doctor` subcommand and the start of /echo-load when something looks wrong.
Also tracked here (TODO): complete the `load` fallback — echo.cmd_load reads the
heartbeat pointer but never lists _agent/sessions/ when it's missing/stale, though
SKILL.md:122 documents that fallback. See patch_load_fallback() below.
Exit: 0 all green · 1 a check is red.
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo # noqa: E402
MIN_PY = (3, 9)
def run() -> int:
reds = 0
def line(ok: bool, label: str, detail: str = "") -> None:
nonlocal reds
reds += 0 if ok else 1
print(f" [{'OK ' if ok else 'RED'}] {label}" + (f" — {detail}" if detail else ""))
print(f"echo doctor — endpoint {echo.BASE}")
# 1. interpreter
line(sys.version_info >= MIN_PY, f"python >= {MIN_PY[0]}.{MIN_PY[1]}",
f"running {sys.version_info.major}.{sys.version_info.minor}")
# 2. reachability + auth + marker, in one GET of the bootstrap marker
status, body = echo.request("GET", echo.vault_url("_agent/echo-vault.md"))
if status == 0:
line(False, "vault reachable", body.decode(errors="replace")[:120])
print("\ndoctor: endpoint unreachable — is Obsidian + the Local REST API running?")
return 1
line(status not in (401, 403), "auth accepted", f"HTTP {status}" if status in (401, 403) else "")
if status == 404:
line(False, "vault bootstrapped", "marker absent — run bootstrap.py")
elif status == 200:
text = body.decode(errors="replace")
ver = next((ln.split(":", 1)[1].strip() for ln in text.splitlines()
if ln.startswith("schema_version:")), "?")
line(True, "vault bootstrapped", f"schema_version={ver}")
else:
line(status < 400, "marker fetch", f"HTTP {status}")
# 3. key source (M1) + invariants pointer.
import echo_secrets
key_src = ("ECHO_KEY env" if os.environ.get("ECHO_KEY")
else "credentials file" if (echo_secrets._state_dir() / "credentials").exists()
else "baked-in fallback (deprecated — see API-KEY-SETUP.md)")
line(os.environ.get("ECHO_KEY") is not None or (echo_secrets._state_dir() / "credentials").exists(),
"API key source", key_src)
print(" [i] invariants — run `/echo-health` (vault_lint.py) for the full check")
print(f"\ndoctor: {'all green' if reds == 0 else f'{reds} issue(s) — see RED above'}")
return 1 if reds else 0
if __name__ == "__main__":
raise SystemExit(run())
echo-memory.plugin.src/skills/echo-memory/scripts/echo_ops.py
+90 -106
View File
@@ -14,7 +14,6 @@ from __future__ import annotations
import json
import re
import sys
import urllib.parse
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
@@ -54,79 +53,11 @@ def link(a_path: str, b_path: str) -> int:
# ----------------------------------------------------------------- recall -----
def _resolve_link_to_path(target: str, nmap: dict) -> str | None:
target = target.strip()
if "/" in target:
return target if target.endswith(".md") else target + ".md"
return nmap.get(idx_mod.slugify(target))
def _brief(path: str) -> None:
text = links.get_text(path)
if text is None:
return
print(f"\n### {path}")
fm_type = re.search(r"(?m)^type:\s*(.+)$", text)
if fm_type:
print(f"_type: {fm_type.group(1).strip()}_")
# Prefer a Status paragraph; else the first handful of non-empty body lines.
body = text.split("\n---", 2)[-1] if text.startswith("---") else text
status = re.search(r"(?ms)^##\s+Status\s*$(.*?)(?=^##\s|\Z)", body)
if status and status.group(1).strip():
print(status.group(1).strip()[:400])
return
shown = 0
for line in body.splitlines():
if not line.strip() or line.startswith("# "):
continue
print(line[:200])
shown += 1
if shown >= 8:
break
def recall(query, limit: int = 6) -> int:
q = " ".join(query) if isinstance(query, list) else query
index = idx_mod.load()
nmap = idx_mod.name_map(index)
status, body = echo.request("POST", f"{echo.BASE}/search/simple/?query={urllib.parse.quote_plus(q)}")
echo.check(status, body, "recall search")
try:
results = json.loads(body)
except json.JSONDecodeError:
results = []
hits = []
for r in results[:limit]:
fn = r.get("filename") or r.get("path")
if fn and fn not in hits:
hits.append(fn)
rslug, e = idx_mod.resolve(index, q)
if e and e.get("path") and e["path"] not in hits:
hits.insert(0, e["path"])
seen = set(hits)
neighbors = [] # (via, path)
for p in hits:
text = links.get_text(p)
if text is None:
continue
targets = set(links.all_wikilinks(text)) | set(links.source_notes(text))
for t in targets:
tp = _resolve_link_to_path(t, nmap)
if tp and tp not in seen:
seen.add(tp)
neighbors.append((p, tp))
print(f"== recall: {q} ==")
print(f"\n# Primary hits ({len(hits)})")
for p in hits:
_brief(p)
print(f"\n# Linked context ({len(neighbors)})")
for via, p in neighbors:
print(f"\n(via {via})")
_brief(p)
return 0
def recall(query, limit: int = 8) -> int:
"""Hybrid lexical (BM25) + graph recall — implemented in echo_recall. Kept here as
the stable public entrypoint (echo.py and /echo-recall call echo_ops.recall)."""
import echo_recall # lazy: only pulls the BM25 modules when recall is actually run
return echo_recall.recall(query, limit=limit)
# ----------------------------------------------------------- agent log -------
@@ -197,7 +128,30 @@ def _append_to_existing(path: str, kind: str, today_s: str, body_text: str) -> N
def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = "",
aliases=None, sources=None, date: str | None = None, domain: str = "business",
inbox: bool = False, no_log: bool = False) -> int:
inbox: bool = False, no_log: bool = False, as_json: bool = False,
dry_run: bool = False) -> int:
import contextlib
import io
import echo_output
import echo_quality
real_stdout = sys.stdout
def quiet():
# M4: in --json mode, swallow the helper "ok:" chatter so stdout is clean JSON.
return contextlib.redirect_stdout(io.StringIO()) if as_json else contextlib.nullcontext()
def done(action: str, path: str, links: int = 0, ok: bool = True, dry: bool = False) -> int:
if as_json:
act = f"dry-run:{action}" if dry else action
env = echo_output.envelope(act, {"kind": kind, "path": path, "title": title,
"links_added": links}, ok=ok)
print(json.dumps(env, ensure_ascii=False), file=real_stdout)
elif dry:
print(f"would {action} {kind or '-'} -> {path}")
# non-dry human output is the helper "ok:" lines + the summary printed by the caller.
return 0 if ok else 1
body_text = echo.read_body(file_arg).decode("utf-8", errors="replace")
today_s = echo.today()
aliases = [a.strip() for a in (aliases or []) if a.strip()]
@@ -205,44 +159,74 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str =
# Unknown home -> defer to the inbox (a single idempotent capture line).
if inbox or not kind:
if dry_run:
return done("inbox", "inbox/captures/inbox.md", dry=True)
line = f"- {today_s}: {title}"
if body_text.strip():
line += f" — {body_text.strip().splitlines()[0]}"
return echo.cmd_append("inbox/captures/inbox.md", line)
with quiet():
rc = echo.cmd_append("inbox/captures/inbox.md", line)
return done("inbox", "inbox/captures/inbox.md", ok=rc == 0)
slug = idx_mod.slugify(title)
index = idx_mod.load()
_, existing = idx_mod.resolve(index, title)
existing_reachable = bool(existing and echo.request("GET", echo.vault_url(existing["path"]))[0] == 200)
if existing and echo.request("GET", echo.vault_url(existing["path"]))[0] == 200:
path = existing["path"]
kind = existing.get("kind", kind)
_append_to_existing(path, kind, today_s, body_text)
action = "updated"
if dry_run:
if existing_reachable:
return done("update", existing["path"], dry=True)
s2 = echo_quality.safe_slug(set(index.get("entities", {}).keys()), slug)
return done("create", idx_mod.derive_path(kind, s2, date=date, domain=domain), dry=True)
with quiet():
if existing_reachable:
path = existing["path"]
kind = existing.get("kind", kind)
_append_to_existing(path, kind, today_s, body_text)
action = "updated"
else:
if not status_v and kind == "project":
status_v = "active"
# M2: don't let a 40-char-truncated slug silently collide with a different
# entity already in the index — disambiguate to slug-2, slug-3, ...
slug = echo_quality.safe_slug(set(index.get("entities", {}).keys()), slug)
path = idx_mod.derive_path(kind, slug, date=date, domain=domain)
note = _build_note(idx_mod.KIND_TYPE.get(kind, kind), status_v, today_s, title, body_text, sources)
echo.cmd_put(path, echo.temp_file(note.encode()))
action = "created"
# H3: the entity-index write is the race the review flagged — a bare load->save lets
# two concurrent captures clobber each other's entry. Route it through a lock-guarded,
# fresh-re-read transaction. Returns the fresh index (used by auto-link below).
import echo_concurrency
index = echo_concurrency.atomic_index_update(
lambda idx: idx_mod.upsert(idx, slug, path, kind, title, aliases))
# auto-link: any other known entity CONFIDENTLY named in the body (M2: is_confident_link
# rejects short/common tokens so a 3-char alias or a word like "API" can't link everywhere).
linked = 0
for s2, e2 in index.get("entities", {}).items():
if e2.get("path") in (None, path):
continue
names = [n for n in [e2.get("title", ""), s2, *e2.get("aliases", [])] if n]
if any(echo_quality.is_confident_link(n, body_text) for n in names):
ca, cb = links.link_bidirectional(path, e2["path"])
linked += int(ca or cb)
# Keep the BM25 recall index current for this note (best-effort; lock-guarded inside
# update_note, so it can't clobber a concurrent writer either).
try:
import echo_recall
echo_recall.update_note(path, links.get_text(path) or "")
except Exception as exc: # never let recall-index upkeep fail a capture
print(f"echo_ops: recall-index update skipped ({exc})", file=sys.stderr)
if not no_log:
ensure_daily_log(f"- {today_s}: {action} {kind} [[{links.link_token(path)}]]")
if as_json:
done(action, path, links=linked)
else:
if not status_v and kind == "project":
status_v = "active"
path = idx_mod.derive_path(kind, slug, date=date, domain=domain)
note = _build_note(idx_mod.KIND_TYPE.get(kind, kind), status_v, today_s, title, body_text, sources)
echo.cmd_put(path, echo.temp_file(note.encode()))
action = "created"
idx_mod.upsert(index, slug, path, kind, title, aliases)
idx_mod.save(index)
# auto-link: any other known entity whose name/alias appears in the body, matched
# on word boundaries (and >=3 chars) so a short alias can't match inside a word.
linked = 0
for s2, e2 in index.get("entities", {}).items():
if e2.get("path") in (None, path):
continue
names = [n for n in [e2.get("title", ""), s2, *e2.get("aliases", [])] if len(n) >= 3]
if any(re.search(rf"\b{re.escape(n)}\b", body_text, re.IGNORECASE) for n in names):
ca, cb = links.link_bidirectional(path, e2["path"])
linked += int(ca or cb)
if not no_log:
ensure_daily_log(f"- {today_s}: {action} {kind} [[{links.link_token(path)}]]")
print(f"ok: {action} {kind} -> {path}" + (f"; auto-linked {linked}" if linked else ""))
print(f"ok: {action} {kind} -> {path}" + (f"; auto-linked {linked}" if linked else ""))
return 0
echo-memory.plugin.src/skills/echo-memory/scripts/echo_output.py
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""echo_output.py — M4: machine-readable output + dry-run plumbing. [v1.0 SCAFFOLD — implemented]
The high-level ops (capture/recall/resolve/scope) print prose, forcing the agent to
re-parse free text to act on a result. This provides a single JSON envelope and a
small dry-run helper so every op can emit a structured result and preview writes.
INTEGRATION (merge step): thread a global `--json` / `--dry-run` flag through echo.py's
parser and have each cmd_* return its data dict; emit() it. In --dry-run, the write
verbs print the envelope with action="dry-run" and perform no network mutation.
"""
from __future__ import annotations
import json
import sys
def envelope(action: str, data: dict | None = None, ok: bool = True,
error: str | None = None) -> dict:
"""Canonical result shape for every high-level op."""
env = {"ok": ok, "action": action}
if data:
env.update(data)
if error:
env["error"] = error
return env
def emit(env: dict, as_json: bool) -> None:
"""Print either the JSON envelope (machine mode) or a one-line human summary."""
if as_json:
print(json.dumps(env, ensure_ascii=False))
return
if not env.get("ok"):
print(f"error: {env.get('error', 'failed')}", file=sys.stderr)
return
bits = [env.get("action", "ok")]
if env.get("path"):
bits.append(str(env["path"]))
if env.get("links_added"):
bits.append(f"+{env['links_added']} links")
print("ok: " + " ".join(bits))
def dry_run_envelope(action: str, data: dict | None = None) -> dict:
"""Result for a previewed-but-not-executed write."""
return envelope(f"dry-run:{action}", data, ok=True)
echo-memory.plugin.src/skills/echo-memory/scripts/echo_quality.py
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""echo_quality.py — M2: keep the graph (which recall depends on) clean.
[v1.0 SCAFFOLD — helpers implemented; integration into echo_links/echo_index is the merge step]
Two graph-polluting bugs today:
1. Auto-link fires on any indexed entity whose name/alias is >=3 chars and word-matches
the body (echo_ops.py:236). A concept titled "API" or a 2-3 char alias links to every
note that happens to mention the word -> noisy, wrong edges.
2. slugify truncates to 40 chars (echo_index.py:58) with NO collision check, so two
distinct long titles can resolve to the same slug -> same path -> silent merge.
This module supplies the two guards. Integration:
* echo_ops auto-link loop -> gate each candidate on is_confident_link(name, body)
* echo_index.derive_path -> route the slug through safe_slug(existing_slugs, slug)
"""
from __future__ import annotations
import re
# Tokens too short or too common to be a confident entity reference on their own.
MIN_NAME_LEN = 4
_COMMON = frozenset(
"api app data note task user team work code test main repo file plan goal item "
"the and for with mpm".split()
)
def is_confident_link(name: str, body: str) -> bool:
"""True only if `name` is a strong enough signal to auto-link on its appearance in
`body`. Rejects short names, common words, and non-matches. Multi-word names are
trusted at lower length (a two-word phrase is rarely incidental)."""
n = (name or "").strip()
if not n:
return False
multiword = len(n.split()) > 1
if not multiword:
if len(n) < MIN_NAME_LEN or n.lower() in _COMMON:
return False
return re.search(rf"\b{re.escape(n)}\b", body or "", re.IGNORECASE) is not None
def safe_slug(existing_slugs, slug: str, max_len: int = 40) -> str:
"""Return `slug` if free, else a disambiguated `slug-2`, `slug-3`, ... that still
fits `max_len` and does not collide with anything in `existing_slugs`."""
existing = set(existing_slugs or ())
if slug not in existing:
return slug
n = 2
while True:
suffix = f"-{n}"
base = slug[: max_len - len(suffix)] if len(slug) + len(suffix) > max_len else slug
cand = f"{base}{suffix}"
if cand not in existing:
return cand
n += 1
echo-memory.plugin.src/skills/echo-memory/scripts/echo_queue.py
+191
View File
@@ -0,0 +1,191 @@
#!/usr/bin/env python3
"""echo_queue.py — H2: offline write-ahead queue + read cache. [v1.0 — wired]
Today vault-unreachable => "proceed without memory" (SKILL.md), so a 502 (Obsidian not
running — the most common real failure) loses everything said that session. This module
makes explicit writes durable across an outage and lets cold-start `load` degrade to
last-known context instead of nothing.
DESIGN:
* Outbox : append-only NDJSON of intended writes. Replayed in order on the next
reachable session. Replay is idempotent by construction — PUT/DELETE/PATCH-
replace overwrite, and POST / PATCH-append are content-deduped (skip if the
line/block is already present), the same strategy echo.py append uses. Each
record carries an idem_key so the SAME write is never queued twice.
* Cache : last-known-good GET bodies, so `load` has a fallback. Written by cmd_load.
* Location: a LOCAL state dir — it CANNOT live in the vault, because the vault is the
thing that's down. Default ~/.echo-memory/, override ECHO_STATE_DIR.
Integration seam: `safe_request()` — the write verbs (cmd_put/post/append/patch/delete)
call it instead of echo.request; on an outage it enqueues and returns a synthesized
(202, b"queued ...") so the caller reports "queued, will sync" instead of failing.
"""
from __future__ import annotations
import base64
import hashlib
import json
import os
import sys
import urllib.parse
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo # noqa: E402
MUTATING = {"PUT", "POST", "PATCH", "DELETE"}
def state_dir() -> Path:
return Path(os.environ.get("ECHO_STATE_DIR") or (Path.home() / ".echo-memory"))
def outbox_path() -> Path:
return state_dir() / "outbox.ndjson"
def cache_dir() -> Path:
return state_dir() / "cache"
def _ensure_dirs() -> None:
state_dir().mkdir(parents=True, exist_ok=True)
cache_dir().mkdir(parents=True, exist_ok=True)
def _key(method: str, url: str, data: bytes | None) -> str:
h = hashlib.sha1()
h.update(method.encode())
h.update(b"\0")
h.update(url.encode())
h.update(b"\0")
h.update(data or b"")
return h.hexdigest()
# --------------------------------------------------------------------- outbox
def enqueue(method: str, url: str, data: bytes | None, headers: dict, idem_key: str | None = None) -> None:
"""Append one intended write to the outbox (NDJSON; base64 body for binary safety).
Dedups: if a record with the same idem_key is already pending, do nothing."""
idem_key = idem_key or _key(method, url, data)
if any(rec.get("idem_key") == idem_key for rec in pending()):
return
_ensure_dirs()
rec = {
"method": method.upper(), "url": url,
"body_b64": base64.b64encode(data).decode() if data else None,
"headers": headers or {}, "idem_key": idem_key,
# No wall-clock: replay order is file order, not a timestamp (keeps this testable).
}
with outbox_path().open("a", encoding="utf-8") as fh:
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
def pending() -> list[dict]:
p = outbox_path()
if not p.exists():
return []
return [json.loads(ln) for ln in p.read_text(encoding="utf-8").splitlines() if ln.strip()]
def _rewrite(records: list[dict]) -> None:
p = outbox_path()
if not records:
if p.exists():
p.unlink()
return
_ensure_dirs()
tmp = p.with_suffix(".ndjson.tmp")
tmp.write_text("".join(json.dumps(r, ensure_ascii=False) + "\n" for r in records), encoding="utf-8")
os.replace(tmp, p) # atomic swap so a crash mid-flush can't corrupt the queue
def _rebase(url: str) -> str:
"""Re-point a queued URL at the CURRENT echo.BASE. Writes are queued with the base
that was active when they failed; on replay the vault may be back at a different base
(e.g. an endpoint migration), so we always reconstruct the origin from echo.BASE."""
parts = urllib.parse.urlsplit(url)
return echo.BASE + parts.path + (("?" + parts.query) if parts.query else "")
def _replay(rec: dict) -> str:
"""Re-issue one queued write idempotently. Returns 'ok' (landed/already-present),
'drop' (permanent 4xx — can never succeed), or 'retry' (still offline / 5xx)."""
method = rec["method"]
url = _rebase(rec["url"])
data = base64.b64decode(rec["body_b64"]) if rec.get("body_b64") else None
headers = rec.get("headers") or {}
# Append-style writes: skip if the content is already present (idempotency on replay).
is_append = method == "POST" or (method == "PATCH" and headers.get("Operation") == "append")
if is_append and data:
st, cur = echo.request("GET", url)
if st == 0:
return "retry" # still offline
if st == 200 and data.decode("utf-8", "replace").strip("\n") in cur.decode("utf-8", "replace"):
return "ok" # already landed (or a duplicate we must not re-add)
st, body = echo.request(method, url, data=data, headers=headers)
if st == 0 or 500 <= st < 600:
return "retry"
if 400 <= st < 500:
print(f"echo_queue: dropping a queued {method} {url} — permanent HTTP {st} on replay",
file=sys.stderr)
return "drop"
return "ok"
def flush() -> int:
"""Replay queued writes in order; stop at the first still-unreachable one (keeping it
and the rest). Returns the number actually replayed. Safe to call when empty."""
items = pending()
if not items:
return 0
replayed = 0
remaining: list[dict] = []
for i, rec in enumerate(items):
result = _replay(rec)
if result == "ok":
replayed += 1
elif result == "drop":
continue # warned in _replay; remove from queue
else: # retry -> still offline; keep this and everything after, in order
remaining = items[i:]
break
_rewrite(remaining)
return replayed
# --------------------------------------------------------------------- cache
def cache_key(path: str) -> Path:
return cache_dir() / (path.replace("/", "__") + ".cache")
def cache_put(path: str, body: bytes) -> None:
_ensure_dirs()
cache_key(path).write_bytes(body)
def cache_get(path: str) -> bytes | None:
p = cache_key(path)
return p.read_bytes() if p.exists() else None
# ----------------------------------------------------------- integration seam
def safe_request(method: str, url: str, data=None, headers=None, idem_key: str | None = None):
"""Network-first wrapper around echo.request with offline queueing for writes.
- success or a client error (4xx): return (status, body) as-is — 4xx is a bug, fail loud.
- transport failure (status 0) or 5xx (after echo.request's own retry) on a MUTATING
verb: enqueue the write and return (202, b"queued (offline)") so the caller reports it
as durably queued rather than failed.
- otherwise (e.g. a failing GET): return (status, body); the read-cache fallback lives
in cmd_load, which knows which reads are worth serving stale.
"""
method = method.upper()
status, body = echo.request(method, url, data=data, headers=headers)
offline = status == 0 or 500 <= status < 600
if offline and method in MUTATING:
enqueue(method, url, data, headers or {}, idem_key)
return 202, b"queued (offline)"
return status, body
echo-memory.plugin.src/skills/echo-memory/scripts/echo_recall.py
+349
View File
@@ -0,0 +1,349 @@
#!/usr/bin/env python3
"""echo_recall.py — H1: hybrid lexical (BM25) + graph recall. [v1.0 — wired]
Replaces the keyword-only recall (search/simple + fixed one-hop) with a ranked, local
retriever that finds a note even when the query is phrased differently than the note
was stored, then expands along the graph with a per-hop decay so the *connected*
context is surfaced in relevance order.
DESIGN (bounded by the project's pure-stdlib, no-deps constraint):
* Lexical layer : BM25 over note bodies. No embeddings / external API — BM25 is a
strong, dependency-free baseline that handles vocabulary mismatch
far better than substring search.
* Graph layer : from the top lexical hits, walk body wikilinks + source_notes up
to MAX_HOPS, scoring each node `parent * GRAPH_DECAY**hop` (max over
paths) so near, strongly-cued neighbours rank above distant ones.
* Persistence : the BM25 postings + doc stats are a DERIVED artifact, so they live
in the vault's machine index dir (`_agent/index/recall-index.json`),
maintained on `capture` (update_note) and fully rebuilt by `sweep.py`
— exactly how `entities.json` is maintained. Rebuildable => portable.
* Resilience : if the index can't be built (vault unreachable), recall falls back
to the server's /search/simple so it degrades instead of dying.
Corpus = the indexable ENTITY notes (idx_mod.kind_for_path is not None): people,
companies, concepts, references, meetings, projects, areas, decisions, semantic/
episodic memory, skills — the durable memory graph. (Sessions/journal are a future
add; tracked in ROADMAP-1.0.md.)
"""
from __future__ import annotations
import json
import math
import re
import sys
import urllib.parse
from collections import Counter, deque
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo # noqa: E402
import echo_index as idx_mod # noqa: E402
import echo_links as links # noqa: E402
RECALL_INDEX_PATH = "_agent/index/recall-index.json"
# Tuning knobs. Defaults are standard BM25 (k1/b) + a 0.6 hop decay over <=2 hops.
K1 = 1.5
B = 0.75
MAX_HOPS = 2
GRAPH_DECAY = 0.6
MIN_SCORE = 0.0 # relevance floor; raise to trim weak lexical hits
_WORD = re.compile(r"[a-z0-9]+")
# Minimal English stoplist — keeps BM25 idf from being dominated by glue words.
_STOP = frozenset(
"the a an and or of to in is are was were be been for on at by with as it this that "
"from into over under not no yes do does did has have had will would can could".split()
)
_TEMPLATE_RE = re.compile(r"(^|/)(templates/|.*-template\.md$)")
_SKIP_BASENAMES = {"README.md"}
def tokenize(text: str) -> list[str]:
"""Lowercase word tokens, stopworded. Frontmatter should be stripped by the caller."""
return [t for t in _WORD.findall(text.lower()) if t not in _STOP and len(t) > 1]
def strip_frontmatter(text: str) -> str:
if not text.startswith("---"):
return text
end = text.find("\n---", 3)
return text[end + 4:] if end != -1 else text
# --------------------------------------------------------------------- BM25 core
class Bm25Index:
"""A compact, serializable BM25 index. add()/remove() are idempotent per path."""
def __init__(self) -> None:
self.df: Counter[str] = Counter() # term -> #docs containing it
self.postings: dict[str, dict[str, int]] = {} # term -> {doc_path: tf}
self.length: dict[str, int] = {} # doc_path -> token count
self.n_docs = 0
self.avg_len = 0.0
def _recompute(self) -> None:
self.n_docs = len(self.length)
self.avg_len = (sum(self.length.values()) / self.n_docs) if self.n_docs else 0.0
def add(self, path: str, text: str) -> None:
if path in self.length: # re-index in place (idempotent)
self.remove(path)
toks = tokenize(text)
self.length[path] = len(toks)
for term, tf in Counter(toks).items():
self.postings.setdefault(term, {})[path] = tf
self.df[term] += 1
self._recompute()
def remove(self, path: str) -> None:
if path not in self.length:
return
for term in list(self.postings.keys()):
if path in self.postings[term]:
del self.postings[term][path]
self.df[term] -= 1
if not self.postings[term]:
del self.postings[term]
del self.df[term]
del self.length[path]
self._recompute()
def score(self, query: str, limit: int = 10) -> list[tuple[str, float]]:
scores: Counter[str] = Counter()
for term in set(tokenize(query)):
posting = self.postings.get(term)
if not posting:
continue
idf = math.log(1 + (self.n_docs - self.df[term] + 0.5) / (self.df[term] + 0.5))
for path, tf in posting.items():
dl = self.length.get(path, 0)
denom = tf + K1 * (1 - B + B * (dl / self.avg_len if self.avg_len else 1))
scores[path] += idf * (tf * (K1 + 1) / denom if denom else 0)
return [(p, s) for p, s in scores.most_common(limit) if s > MIN_SCORE]
# ---- serialization (compact; rebuildable, so the format can change freely) ----
def to_json(self) -> dict:
return {"schema": 1, "n_docs": self.n_docs, "avg_len": self.avg_len,
"length": self.length, "postings": self.postings}
@classmethod
def from_json(cls, d: dict) -> "Bm25Index":
ix = cls()
ix.length = d.get("length", {})
ix.postings = d.get("postings", {})
ix.n_docs = d.get("n_docs", len(ix.length))
ix.avg_len = d.get("avg_len", 0.0)
ix.df = Counter({term: len(posting) for term, posting in ix.postings.items()})
return ix
# ------------------------------------------------------------------- vault I/O
def _get(path: str) -> str | None:
status, body = echo.request("GET", echo.vault_url(path))
if status == 404:
return None
echo.check(status, body, f"recall get {path}")
return body.decode(errors="replace")
def _list_dir(path: str):
p = "" if path in ("", "/") else (path if path.endswith("/") else path + "/")
body = _get(p)
if body is None:
return [], []
try:
j = json.loads(body)
except json.JSONDecodeError:
return [], []
entries = list(j.get("files", [])) + list(j.get("folders", []))
files = [e for e in entries if not e.endswith("/")]
folders = [e[:-1] for e in entries if e.endswith("/")]
return files, folders
def _walk(prefix: str = ""):
files, folders = _list_dir(prefix)
for f in files:
yield prefix + f
for d in folders:
yield from _walk(f"{prefix}{d}/")
def _indexable(path: str) -> bool:
base = path.rsplit("/", 1)[-1]
return (path.endswith(".md") and idx_mod.kind_for_path(path) is not None
and base not in _SKIP_BASENAMES and not _TEMPLATE_RE.search(path))
# ------------------------------------------------------------------- persistence
def load_index() -> Bm25Index:
status, body = echo.request("GET", echo.vault_url(RECALL_INDEX_PATH))
if status == 404:
return Bm25Index()
echo.check(status, body, "recall-index load")
try:
return Bm25Index.from_json(json.loads(body))
except (json.JSONDecodeError, KeyError, TypeError):
return Bm25Index()
def save_index(ix: Bm25Index) -> None:
body = json.dumps(ix.to_json(), ensure_ascii=False).encode("utf-8")
status, b = echo.request("PUT", echo.vault_url(RECALL_INDEX_PATH), data=body,
headers={"Content-Type": "application/json"})
echo.check(status, b, "recall-index save")
def rebuild() -> Bm25Index:
"""Full rebuild from every indexable entity note. Called by sweep.py and as the
cold-cache path inside recall(). Saves and returns the index."""
ix = Bm25Index()
for path in _walk():
if not _indexable(path):
continue
text = _get(path)
if text is not None:
ix.add(path, strip_frontmatter(text))
save_index(ix)
return ix
def update_note(path: str, text: str) -> None:
"""Best-effort incremental upkeep for a single note (mirrors how entities.json is
maintained on capture). Never raises into the caller. The load->save is wrapped in the
advisory lock with a fresh re-read (H3), so concurrent captures can't clobber the
recall index either."""
try:
if not _indexable(path):
return # only entity notes are part of the recall corpus
import echo_concurrency
with echo_concurrency.vault_lock():
ix = load_index() # fresh read inside the lock
ix.add(path, strip_frontmatter(text))
save_index(ix)
except Exception as exc: # noqa: BLE001 — upkeep must never fail a capture
print(f"echo_recall: index update skipped ({exc})", file=sys.stderr)
# ------------------------------------------------------------------- graph fusion
def _resolve_target(target: str, nmap: dict) -> str | None:
target = target.strip()
if not target:
return None
if "/" in target:
return target if target.endswith(".md") else target + ".md"
return nmap.get(idx_mod.slugify(target))
def expand_graph(seeds, nmap: dict, base_scores: dict, max_hops: int = MAX_HOPS):
"""BFS from seeds over body wikilinks + source_notes, scoring each neighbour
`parent_score * GRAPH_DECAY**hop` (kept as the max over reaching paths). Returns a
score-sorted list of (path, (score, via)) excluding the seeds themselves."""
score_of = dict(base_scores)
seen = set(seeds)
results: dict[str, tuple[float, str]] = {}
dq = deque((s, 0) for s in seeds)
while dq:
path, hop = dq.popleft()
if hop >= max_hops:
continue
text = links.get_text(path)
if text is None:
continue
body = strip_frontmatter(text)
targets = set(links.all_wikilinks(body)) | set(links.source_notes(text))
parent = score_of.get(path, 1.0)
for t in targets:
tp = _resolve_target(t, nmap)
if not tp or tp in seeds:
continue
decayed = parent * (GRAPH_DECAY ** (hop + 1))
if decayed > results.get(tp, (0.0, ""))[0]:
results[tp] = (decayed, path)
if tp not in seen:
seen.add(tp)
dq.append((tp, hop + 1))
return sorted(results.items(), key=lambda kv: -kv[1][0])
# ------------------------------------------------------------------- presentation
def _brief(path: str, score: float | None = None, via: str | None = None) -> None:
text = links.get_text(path)
if text is None:
return
head = f"\n### {path}"
if score is not None:
head += f" (score {score:.2f})"
if via:
head += f" (via {via})"
print(head)
mtype = re.search(r"(?m)^type:\s*(.+)$", text)
if mtype:
print(f"_type: {mtype.group(1).strip()}_")
body = strip_frontmatter(text)
status = re.search(r"(?ms)^##\s+Status\s*$(.*?)(?=^##\s|\Z)", body)
if status and status.group(1).strip():
print(status.group(1).strip()[:400])
return
shown = 0
for line in body.splitlines():
if not line.strip() or line.startswith("# "):
continue
print(line[:200])
shown += 1
if shown >= 6:
break
# ------------------------------------------------------------------- entrypoint
def recall(query, limit: int = 8) -> int:
q = " ".join(query) if isinstance(query, list) else query
index = idx_mod.load()
nmap = idx_mod.name_map(index)
# --- lexical layer (BM25); rebuild the cache on a cold miss --------------
lexical: list[tuple[str, float]] = []
try:
ix = load_index()
if ix.n_docs == 0:
ix = rebuild()
lexical = ix.score(q, limit=limit)
except echo.EchoError as exc:
print(f"recall: BM25 index unavailable ({exc}); falling back to server search",
file=sys.stderr)
base: dict[str, float] = {p: s for p, s in lexical}
hits: list[str] = [p for p, _ in lexical]
# --- entity-index seed (alias-aware exact match) -------------------------
_, e = idx_mod.resolve(index, q)
if e and e.get("path") and e["path"] not in base:
hits.insert(0, e["path"])
base[e["path"]] = max(base.values(), default=1.0)
# --- resilience fallback: server search when lexical found nothing -------
if not hits:
try:
status, body = echo.request(
"POST", f"{echo.BASE}/search/simple/?query={urllib.parse.quote_plus(q)}")
if status == 200:
for r in (json.loads(body) or [])[:limit]:
fn = r.get("filename") or r.get("path")
if fn and fn not in base:
hits.append(fn)
base[fn] = 0.0
except Exception: # noqa: BLE001 — fallback only; stay quiet
pass
# --- graph layer ----------------------------------------------------------
neighbours = expand_graph(hits, nmap, base, max_hops=MAX_HOPS)
print(f"== recall: {q} ==")
print(f"\n# Primary hits ({len(hits)})")
for p in hits:
_brief(p, score=base.get(p))
print(f"\n# Linked context ({len(neighbours)})")
for p, (sc, via) in neighbours[: 2 * limit]:
_brief(p, score=sc, via=via)
return 0
echo-memory.plugin.src/skills/echo-memory/scripts/echo_reflect.py
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""echo_reflect.py — H5: automatic session-reflection capture. [v1.0 — wired]
Today memory only accrues when the agent decides to write. This makes accrual the
default: at session end the model extracts durable items from the conversation and
emits a PROPOSAL set; this module dedups them against the entity index, previews them,
and applies the accepted ones via the normal `capture` path — so nothing is written
without a go-ahead (honors the "show before large/sensitive writes" safety rule).
DIVISION OF LABOR:
* Extraction is MODEL-side (only the LLM has the conversation). The agent produces a
JSON array of proposals matching PROPOSAL_SCHEMA.
* This script is the deterministic spine: validate -> classify (new/update/inbox vs the
entity index) -> preview -> apply on confirm. No NL understanding lives here, so it's
testable offline.
PROPOSAL_SCHEMA (one per durable item the agent wants to remember):
{
"title": "Bob Smith", # required
"kind": "person", # required unless "inbox": true
"body": "Principal at MPM; ...", # markdown body (optional)
"aliases": ["bob", "rs"], # optional
"sources": ["_agent/sessions/..."], # optional backward links
"date": "YYYY-MM-DD", # optional (meeting/decision kinds)
"domain": "business", # optional (area kind)
"inbox": false, # route to inbox when the home is unknown
"confidence": 0.0-1.0 # agent's own confidence; gates auto-suggest
}
CLI (echo.py reflect): dry-run previews; --apply writes — matching bootstrap/migrate/sweep.
"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import echo # noqa: E402
import echo_index as idx_mod # noqa: E402
MIN_CONFIDENCE = 0.6
def validate(proposals: list[dict]) -> tuple[list[dict], list[str]]:
"""Return (valid, errors). title always required; kind required unless inbox; a
confidence below the floor is routed away (the agent should send it to the inbox)."""
valid, errors = [], []
for i, p in enumerate(proposals or []):
if not str(p.get("title", "")).strip():
errors.append(f"proposal[{i}]: missing title")
continue
is_inbox = bool(p.get("inbox"))
if not is_inbox:
kind = str(p.get("kind", "")).strip()
if not kind:
errors.append(f"proposal[{i}] '{p['title']}': missing kind (or set inbox:true)")
continue
if kind not in idx_mod.KIND_FOLDER:
errors.append(f"proposal[{i}] '{p['title']}': unknown kind '{kind}'")
continue
if float(p.get("confidence", 1.0)) < MIN_CONFIDENCE:
errors.append(f"proposal[{i}] '{p['title']}': confidence < {MIN_CONFIDENCE} — send to inbox instead")
continue
valid.append(p)
return valid, errors
def classify(proposals: list[dict]) -> list[dict]:
"""Annotate each proposal with `_action` (create / update / inbox) and `_path`, by
resolving its title against the entity index (alias-aware), so the preview shows
create-vs-append and dedups against what's already in memory."""
index = idx_mod.load()
for p in proposals:
if p.get("inbox") or not p.get("kind"):
p["_action"], p["_path"] = "inbox", "inbox/captures/inbox.md"
continue
_, e = idx_mod.resolve(index, p["title"])
if e and e.get("path"):
p["_action"], p["_path"] = "update", e["path"]
else:
try:
p["_path"] = idx_mod.derive_path(
p["kind"], idx_mod.slugify(p["title"]),
date=p.get("date"), domain=p.get("domain", "business"))
p["_action"] = "create"
except echo.EchoError as exc:
p["_action"], p["_path"] = "error", str(exc)
return proposals
def preview(proposals: list[dict]) -> str:
"""One confirmable line per proposal: action | kind | title -> target path."""
rows = []
for p in proposals:
act = p.get("_action", "?")
kind = "-" if act == "inbox" else p.get("kind", "?")
rows.append(f" {act:7} | {kind:9} | {p['title']} -> {p.get('_path', '?')}")
return "\n".join(rows)
def apply(proposals: list[dict], confirm: bool = False) -> int:
"""Validate -> classify -> preview; with confirm=True, apply each via echo_ops.capture
(which routes/indexes/links/logs, each self-lock-guarded). Without confirm it is a
dry-run that writes nothing — the preview IS the confirmation step."""
valid, errors = validate(proposals)
for e in errors:
print(f"skip: {e}", file=sys.stderr)
if not valid:
print("reflect: no valid proposals to apply.")
return 0
classify(valid)
counts = {a: sum(1 for p in valid if p.get("_action") == a) for a in ("create", "update", "inbox", "error")}
print(f"reflect: {len(valid)} proposal(s) — "
f"{counts['create']} new, {counts['update']} update, {counts['inbox']} inbox"
+ (f", {counts['error']} error" if counts["error"] else ""))
print(preview(valid))
if not confirm:
print("\nreflect: dry-run — re-run with --apply to write these to memory.")
return 0
import echo_ops # lazy: the apply path pulls in the capture/index/link stack
applied = 0
for p in valid:
if p.get("_action") == "error":
continue
bodyfile = echo.temp_file((p.get("body") or "").encode()) if p.get("body") else None
rc = echo_ops.capture(
p.get("kind"), p["title"], bodyfile,
aliases=p.get("aliases") or [], sources=p.get("sources") or [],
date=p.get("date"), domain=p.get("domain", "business"),
inbox=bool(p.get("inbox")) or not p.get("kind"))
applied += 1 if rc == 0 else 0
print(f"reflect: applied {applied}/{len(valid)} proposal(s).")
return 0
echo-memory.plugin.src/skills/echo-memory/scripts/echo_secrets.py
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""echo_secrets.py — M1: resolve the API bearer token without shipping it in source.
[v1.0 SCAFFOLD — resolve_key() implemented; wiring into echo.py is the integration step]
Problem: the token is hardcoded at echo.py:70 and repeated through api-reference.md /
README — committed to git and shipped inside every .plugin zip, rotatable only by a
rebuild. The plugin's own safety rule is "never store secrets". This resolves the key
from progressively less-trusted sources, with a loud warning if the legacy baked
default is ever reached.
RESOLUTION ORDER (first hit wins):
1. env ECHO_KEY — preferred; CI / per-session override.
2. ECHO_STATE_DIR/credentials — local file, one `key=...` line, chmod 0600.
3. legacy baked default (deprecated) — warns to stderr; remove before 1.0 ship.
Pure stdlib only (no `keyring`); the local file is the cross-platform stand-in for an
OS keychain and keeps the secret OUT of the tracked tree.
INTEGRATION (do at merge time):
echo.py: KEY = echo_secrets.resolve_key()
docs: replace the literal token with `<ECHO_KEY>` placeholders
rotate: document `echo write-key` (writes the credentials file) — see TODO below
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
# Kept ONLY so a fresh install still works during the migration window; delete for 1.0.
_LEGACY_DEFAULT = "" # intentionally blank in the scaffold — do not re-introduce the literal
def _state_dir() -> Path:
return Path(os.environ.get("ECHO_STATE_DIR") or (Path.home() / ".echo-memory"))
def _from_file() -> str | None:
p = _state_dir() / "credentials"
if not p.exists():
return None
for line in p.read_text(encoding="utf-8").splitlines():
line = line.strip()
if line.startswith("key=") or line.startswith("ECHO_KEY="):
return line.split("=", 1)[1].strip()
return None
def resolve_key(legacy: str | None = None) -> str:
"""Return the bearer token from the most trusted available source:
ECHO_KEY env -> credentials file -> `legacy` baked-in fallback (with a loud, once-per-
process warning that can be silenced via ECHO_KEY_LEGACY_OK=1). Raises if none exist."""
env = os.environ.get("ECHO_KEY")
if env:
return env
from_file = _from_file()
if from_file:
return from_file
legacy = legacy or _LEGACY_DEFAULT
if legacy:
if os.environ.get("ECHO_KEY_LEGACY_OK") != "1":
print("echo_secrets: WARNING — using the deprecated baked-in API key. Run "
"`echo.py write-key <token>` (stores ~/.echo-memory/credentials) or set "
"ECHO_KEY; set ECHO_KEY_LEGACY_OK=1 to silence.", file=sys.stderr)
return legacy
raise RuntimeError(
"echo_secrets: no API key found. Set ECHO_KEY or create "
f"{_state_dir() / 'credentials'} with a 'key=<token>' line."
)
def write_key(token: str) -> Path:
"""Persist the token to the local credentials file (one `key=...` line, chmod 0600).
Best-effort perms — Windows has no POSIX mode bits, so the chmod is advisory there."""
d = _state_dir()
d.mkdir(parents=True, exist_ok=True)
path = d / "credentials"
path.write_text(f"key={token.strip()}\n", encoding="utf-8")
try:
os.chmod(path, 0o600)
except OSError:
pass # platform without POSIX perms (e.g. some Windows filesystems)
return path
echo-memory.plugin.src/skills/echo-memory/scripts/migrate.py
+6 -1
View File
@@ -25,7 +25,7 @@ sys.path.insert(0, str(SCRIPT_DIR))
import echo # noqa: E402
CURRENT_SCHEMA = 3
CURRENT_SCHEMA = 4
def get_text(path: str) -> str | None:
@@ -141,6 +141,11 @@ def main(argv: list[str] | None = None) -> int:
"# index\n\nMachine-maintained entity index. See the echo-memory plugin.\n"))
print("migrate: [2->3] then run `sweep.py --apply` to build entities.json and symmetrize links.")
if start < 4:
print("migrate: [3->4] add the recall (BM25) index — hybrid lexical+graph recall")
print("migrate: [3->4] _agent/index/ already exists (schema 3); no moves needed.")
print("migrate: [3->4] then run `sweep.py --apply` to build _agent/index/recall-index.json.")
do_or_show(args.apply, f"set _agent/echo-vault.md schema_version -> {CURRENT_SCHEMA}",
lambda: echo.cmd_fm("_agent/echo-vault.md", "schema_version", str(CURRENT_SCHEMA)))
if args.apply:
echo-memory.plugin.src/skills/echo-memory/scripts/sweep.py
+10 -1
View File
@@ -33,8 +33,9 @@ sys.path.insert(0, str(SCRIPT_DIR))
import echo # noqa: E402
import echo_index as idx_mod # noqa: E402
import echo_links as links # noqa: E402
import echo_recall # noqa: E402
CURRENT_SCHEMA = 3
CURRENT_SCHEMA = 4
TEMPLATE_RE = re.compile(r"(^|/)(templates/|.*-template\.md$)")
SKIP_BASENAMES = {"README.md"}
@@ -123,6 +124,14 @@ def main(argv: list[str] | None = None) -> int:
if apply:
idx_mod.save(rebuilt)
# ---- 2b. (re)build the recall (BM25) index -------------------------------
if apply:
rix = echo_recall.rebuild()
print(f"sweep: {tag} recall index — {rix.n_docs} docs (BM25)")
else:
n_idx = sum(1 for p in all_files if echo_recall._indexable(p))
print(f"sweep: {tag} recall index — would rebuild BM25 over {n_idx} entity notes")
# ---- 3. symmetrize existing Related links --------------------------------
fileset = set(all_files)
basemap: dict[str, str] = {}
echo-memory.plugin.src/skills/echo-memory/scripts/test_v1_scaffold.py
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""test_v1_scaffold.py — H4 interface guard for the v1.0 scaffolds. [offline, no creds]
Asserts the scaffold contract holds so the skeleton can't silently rot while 1.0 is
built: every planned module imports, the parts that ARE implemented work (BM25 ranking,
queue/cache round-trip, slug helpers, secret resolution order), and the parts that are
TODO raise NotImplementedError (so a stub can't masquerade as done).
Run: python test_v1_scaffold.py
"""
from __future__ import annotations
import os
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
failures: list[str] = []
def check(name: str, cond: bool, detail: str = "") -> None:
print(f"{'ok ' if cond else 'FAIL'} {name}" + (f" -- {detail}" if not cond else ""))
if not cond:
failures.append(name)
def main() -> int:
# H1 — BM25 core is implemented; assert it actually ranks a relevant doc first.
import echo_recall as r
ix = r.Bm25Index()
ix.add("notes/mqtt.md", "the mqtt broker config and tls certificate rotation")
ix.add("notes/payroll.md", "quarterly payroll run and employee deductions")
top = ix.score("certificate rotation broker", limit=2)
check("H1 BM25 ranks relevant doc first", bool(top) and top[0][0] == "notes/mqtt.md", str(top))
check("H1 BM25 survives json round-trip",
r.Bm25Index.from_json(ix.to_json()).score("payroll")[0][0] == "notes/payroll.md")
ix.remove("notes/payroll.md")
check("H1 BM25 remove() drops a doc's postings", ix.score("payroll") == [])
check("H1 recall path is wired (not a stub)",
all(callable(getattr(r, n, None))
for n in ("recall", "rebuild", "update_note", "load_index", "save_index")))
# H2 — cache round-trips, enqueue dedups, flush is a no-op on an empty queue (offline-safe).
import echo_queue as q
with tempfile.TemporaryDirectory() as d:
os.environ["ECHO_STATE_DIR"] = d
q.cache_put("a/b.md", b"hello")
check("H2 cache round-trip", q.cache_get("a/b.md") == b"hello")
q.enqueue("PUT", "http://x/vault/a.md", b"body", {}, idem_key="k1")
check("H2 enqueue persists a record", len(q.pending()) == 1)
q.enqueue("PUT", "http://x/vault/a.md", b"body", {}, idem_key="k1")
check("H2 enqueue dedups by idem_key", len(q.pending()) == 1)
with tempfile.TemporaryDirectory() as d2:
os.environ["ECHO_STATE_DIR"] = d2
check("H2 flush() returns 0 on an empty queue (no network)", q.flush() == 0)
# H3 — lock CM + atomic index update are stubs.
import echo_concurrency as c
check("H3 auto_owner is stable", c.auto_owner() == c.auto_owner())
check("H3 vault_lock returns a context manager (no network until entered)",
hasattr(c.vault_lock(), "__enter__") and hasattr(c.vault_lock(), "__exit__"))
check("H3 atomic_index_update is wired", callable(c.atomic_index_update))
# H5 — proposal validation is pure (offline): keeps valid (incl. inbox), drops the rest.
import echo_reflect as rf
v, errs = rf.validate([
{"title": "Ok", "kind": "person", "confidence": 0.9},
{"title": "", "kind": "person"}, # missing title
{"title": "Bad", "kind": "bogus"}, # unknown kind
{"title": "Weak", "kind": "person", "confidence": 0.2}, # below confidence floor
{"title": "Note", "inbox": True}, # inbox needs no kind
])
check("H5 validate keeps valid (incl. inbox), drops invalid", len(v) == 2 and len(errs) == 3, (len(v), errs))
check("H5 classify/preview/apply wired",
all(callable(getattr(rf, n, None)) for n in ("classify", "preview", "apply")))
# M1 — secret resolution order is implemented; env wins, then the credentials file.
import echo_secrets as s
os.environ["ECHO_KEY"] = "env-key-123"
check("M1 env ECHO_KEY wins", s.resolve_key("legacy-fallback") == "env-key-123")
with tempfile.TemporaryDirectory() as d:
os.environ["ECHO_STATE_DIR"] = d
os.environ.pop("ECHO_KEY", None)
s.write_key("file-key-xyz")
check("M1 write_key -> credentials -> resolve_key reads it", s.resolve_key() == "file-key-xyz")
os.environ["ECHO_KEY_LEGACY_OK"] = "1" # silence the warning for the legacy-path check
os.environ["ECHO_STATE_DIR"] = os.path.join(d, "empty") # an empty dir: no creds file
check("M1 legacy fallback when env+file absent", s.resolve_key("legacy-fallback") == "legacy-fallback")
os.environ["ECHO_KEY"] = "env-key-123" # restore for any later checks
# M2 — slug collision + link-confidence helpers are implemented.
import echo_quality as qual
check("M2 safe_slug disambiguates a collision",
qual.safe_slug({"foo"}, "foo") != "foo")
check("M2 short/common tokens are not confident links",
qual.is_confident_link("API", "we built an api") is False)
# M3 — doctor module exposes run(); M4 — output envelope is implemented.
import echo_doctor as doc
check("M3 doctor exposes run()", hasattr(doc, "run"))
import echo_output as out
env = out.envelope("created", {"path": "p.md"})
check("M4 envelope shape", env.get("ok") is True and env.get("action") == "created")
print(f"\n{len(failures)} failure(s)" if failures else "\nall scaffold interface checks passed")
return 1 if failures else 0
if __name__ == "__main__":
raise SystemExit(main())