1 Commits

Author SHA1 Message Date
Jason Stedwell 227e26c8db 2.2.0 — echo-mcp: the containerized MCP server
Build and Push Docker Image / build (push) Successful in 23s
ECHO as 14 typed MCP tools (streamable HTTP, stateless JSON, bearer auth, open
/health) wrapping the 2.1.1 *_op cores in-process:

- mcp-server/app.py: FastMCP app; duplicate gate + offline queueing surface as
  DATA (merge_into/force are parameters, never blind retries); recall packs
  excerpts into budget_chars by score; get_note is traversal-guarded with
  section/max_chars; patch_note enriches invalid-target errors with the note's
  actual headings; log_session wraps the session-end bundle (heartbeat-last);
  ECHO_MCP_TOOLS=core exposes only the six daily drivers; all MCP writes
  serialize in-process; startup fails fast on missing env.
- Dockerfile (legacy format — no BuildKit on the CI runner), python:3.12-slim,
  healthcheck probes 127.0.0.1; .dockerignore keeps the context lean.
- .gitea/workflows/docker-build.yml: standard image build; PORT redeploy
  trigger targets the echo-mcp container explicitly (repo name is echo).
- deploy.unraid.yml: br0/auto-IP, /data volume, vault adjacency
  (ECHO_BASE=http://10.2.0.35:27123), secrets as SECRET: refs.
- SKILL.md: prefer the echo_* tools when the connector is present; CLI recipes
  stay as the fallback. Spec header marked BUILT (tool count corrected to 14).
- eval/test_mcp_server.py: e2e over real streamable HTTP (health/auth/
  initialize/tools-list + capture->gate->merge->recall->log_session); skips
  cleanly when the SDK is absent. All seven suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 23:01:47 -05:00
13 changed files with 761 additions and 6 deletions
+11
View File
@@ -0,0 +1,11 @@
# Only mcp-server/ + the plugin scripts reach the image; keep the context lean.
.git
*.plugin
dist/
CODEX/
docs/
eval/
echo-icon*
*.pdf
*.html
__pycache__/
+50
View File
@@ -0,0 +1,50 @@
name: Build and Push Docker Image
on:
push:
branches: [main]
workflow_dispatch:
jobs:
build:
# Runs on the forgerunner host: bundled Docker CLI + mounted /var/run/docker.sock.
runs-on: host
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: registry.alwisp.com
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build and Push
run: |
IMAGE="registry.alwisp.com/${{ gitea.repository }}"
GIT_SHA="$(git rev-parse --short HEAD)"
COMMIT_COUNT="$(git rev-list --count HEAD)"
docker build \
--label org.alwisp.git-sha="${{ gitea.sha }}" \
--label org.alwisp.version="v2.${COMMIT_COUNT}" \
--label org.alwisp.repo="${{ gitea.repository }}" \
-t "${IMAGE}:latest" .
docker push "${IMAGE}:latest"
# Dangling-only prune: removes untagged leftovers from previous builds. Never
# removes the tagged :latest or any image referenced by a running container.
- name: Prune dangling images on host
if: always()
run: docker image prune -f 2>/dev/null || true
- name: Trigger PORT redeploy
if: success()
run: |
# Repo is 'echo' but the container is 'echo-mcp' — target it explicitly.
curl -fsS -X POST https://port.alwisp.com/hooks/gitea \
-H "X-Deploy-Token: ${{ secrets.WEBHOOK_SECRET }}" \
-H "Content-Type: application/json" \
-d '{"container":"echo-mcp"}' || echo "PORT redeploy trigger failed (non-fatal)"
+34
View File
@@ -1,5 +1,39 @@
# Changelog
## 2.2.0
### Added — echo-mcp: the containerized MCP server
ECHO's operations are now reachable as **typed MCP tools** from any surface with
an MCP connector (Claude Code, CoWork, claude.ai) — no Python-capable shell, no
`$ECHO` path resolution, no output re-parsing. New `mcp-server/` + `Dockerfile`
in this repo; deployed on ALPHA as the `echo-mcp` container behind
`https://echomcp.alwisp.com` (streamable HTTP, stateless JSON, bearer auth,
open `/health` for Docker/Kuma).
- **14 tools** wrapping the 2.1.1 `*_op` cores: `echo_load`, `echo_recall`
(score-packed `budget_chars`), `echo_resolve`, `echo_get_note`
(`section`/`max_chars`, traversal-guarded), `echo_get_scope`/`echo_set_scope`,
`echo_health` (`deep` runs the linter), `echo_capture` (gate as data —
`merge_into`/`force` are parameters), `echo_link` (paths OR resolvable names),
`echo_append_note`/`echo_patch_note` (invalid-target errors include the note's
actual headings), `echo_triage_inbox` (list/preview/apply in one tool),
`echo_reflect`, `echo_log_session` (the session-end bundle, heartbeat-last).
- **Tool profiles**: `ECHO_MCP_TOOLS=core` exposes only the six daily drivers.
- **Vault adjacency**: the container talks to the Obsidian REST API's HTTP
binding on the same box (`ECHO_BASE=http://10.2.0.35:27123`) — vault ops stop
depending on Cloudflare/NPM/DNS; the public chain only fronts the `echomcp`
ingress. Server-side offline queue at `/data`.
- **All MCP writes serialize** through the server process; the vault advisory
lock still coordinates with CLI clients.
- SKILL.md: prefer the `echo_*` tools when present; CLI recipes are the fallback.
- New suite `eval/test_mcp_server.py` (skips without the SDK): health/auth/
initialize/tools-list + the capture→gate→merge→recall→log_session flow over
real streamable HTTP.
The plugin itself is unchanged apart from the SKILL.md note — the container
vendors `skills/echo-memory/scripts/` at build time (one canonical tree).
## 2.1.1
### Changed — Phase 0 of the MCP build: every high-level op returns an envelope
+26
View File
@@ -0,0 +1,26 @@
# echo-mcp — containerized MCP server over the ECHO vault (docs/MCP-SERVER-SPEC.md).
# LEGACY Dockerfile format on purpose: the git.alwisp.com CI runner has no BuildKit
# (no `# syntax=` line, no RUN --mount).
FROM python:3.12-slim
WORKDIR /app
COPY mcp-server/requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
# The canonical plugin scripts ARE the server's ops layer — vendored at build time,
# never a second source tree.
COPY echo-memory.plugin.src/skills/echo-memory/scripts /app/scripts
COPY mcp-server/app.py /app/app.py
ENV ECHO_STATE_DIR=/data \
ECHO_MCP_PORT=8765 \
PYTHONUNBUFFERED=1
VOLUME /data
EXPOSE 8765
# Probe 127.0.0.1, NOT localhost (::1-vs-IPv4 lesson from cpas/memer/breedr).
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD python3 -c "import urllib.request;urllib.request.urlopen('http://127.0.0.1:8765/health', timeout=4)" || exit 1
CMD ["python3", "/app/app.py"]
+2 -1
View File
@@ -1,4 +1,4 @@
# echo-memory — v2.1.1
# echo-memory — v2.2.0
Persistent memory for Claude / CoWork sessions via the **ECHO** Obsidian vault, driven over the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api). The skill makes direct REST calls through a bundled validated client (`scripts/echo.py`). The whole toolchain is **pure Python** (stdlib only), so it runs identically on Windows, macOS, and Linux — no bash, no platform-specific `date`.
@@ -421,6 +421,7 @@ From the credential-free harness (`eval/run_eval.py` against the deterministic m
| Version | Highlights |
|---------|-----------|
| **2.2.0** | **echo-mcp — the containerized MCP server.** ECHO as 14 typed MCP tools from any connector-capable surface (Claude Code, CoWork, claude.ai): `mcp-server/` + Dockerfile in-repo, deployed on ALPHA as `echo-mcp` behind `https://echomcp.alwisp.com` (streamable HTTP, stateless JSON, bearer auth, open `/health`). Talks to the Obsidian REST API directly on the LAN (`10.2.0.35:27123`) — one client round trip per tool call, all vault chatter host-local. Duplicate gate & offline queueing surface as data; `ECHO_MCP_TOOLS=core` trims the surface to six tools; writes serialize server-side. New `eval/test_mcp_server.py` e2e suite. Full spec: `docs/MCP-SERVER-SPEC.md`. |
| **2.1.1** | **MCP Phase 0 — return-not-print.** Every high-level op gains a core `*_op` function returning an envelope dict with a stdout-purity guarantee (helper chatter → stderr); CLI verbs become thin wrappers with identical output and exit codes. Duplicate gate and offline queueing become data (`action: duplicate-gate` / `queued: true`). New `eval/test_ops_api.py` contract suite. This is the seam the 2.2 containerized MCP server wraps. |
| **2.1.0** | **Quick-wins train: cheaper sessions, durable capture, one-call session end.** (1) **`load --brief`** — a token-budgeted cold-start digest (Fact/Pattern in full, last ~10 Observations, scope+freshness, last session's key sections, Agent-Log lines, inbox *count*; `ECHO_LOAD_BUDGET` default ~8000 chars) now injected by the SessionStart hook, replacing the full six-file dump that grew unboundedly; all `load` reads are fetched in parallel. (2) **Offline capture durability**`capture` on an unreachable vault queues the *whole operation* as one semantic record; `flush` replays it **through capture** so routing/gate/aliasing re-run against the current index; a gate stop on replay is kept + flagged, never landed blind; `ensure_daily_log`/update-path writes ride the queue too. (3) **`session-end`** — one call (one lock) writes session log → Agent-Log line → reflect proposals → optional scope switch → **heartbeat last as the commit marker**; dry-run by default; `ECHO_NOW` pins the HHMM. Also fixes the `__main__` twin-module trap so helper-module `EchoError`s exit with their intended codes. +19 end-to-end checks. |
| **2.0.0** | **Packaging & structure major — memory behavior unchanged.** Skills-only: the legacy `commands/` directory is deleted (breaking for pre-skills clients; the 1.6.0-verified skills are the sole entry points). Artifact policy: the repo tracks only the `echo-memory.plugin` pointer; the 15 historical versioned zips leave the tree and versioned builds ship as **Gitea releases** (one per `v<version>` tag) from `v2.0.0` on; `.gitignore` blocks `*.plugin` except the pointer. README gains the "packaging for other agent runtimes" port note (build-target rule, never a second source tree). |
+28
View File
@@ -0,0 +1,28 @@
# echo-mcp — PORT deploy manifest (ALPHA / br0). See docs/MCP-SERVER-SPEC.md §8.
# Container name deliberately differs from the repo (echo): the repo ships the
# plugin AND this server; only the server is a container.
name: echo-mcp
image: registry.alwisp.com/jason/echo:latest
network: br0
ip: auto
ports: [] # br0 static IP — app serves :8765 directly
volumes:
- /mnt/user/appdata/echo-mcp:/data # outbox + read cache (+ backups/history, v1.1)
env:
# Vault adjacency: the Obsidian Local REST API's HTTP binding on the same box —
# never the public echoapi.alwisp.com hairpin (spec §8; cleartext stays on the LAN).
ECHO_BASE: http://10.2.0.35:27123
ECHO_OWNER: Jason Stedwell
ECHO_STATE_DIR: /data
ECHO_MCP_PORT: "8765"
ECHO_MCP_TOOLS: full
# Secrets from the PORT secret store — values never appear in this file or chat.
ECHO_KEY: SECRET:echo-vault-key
ECHO_MCP_TOKEN: SECRET:echo-mcp-token
health_check_path: /health
health_check_port: 8765
webui: https://echomcp.alwisp.com
proxy:
forward_port: 8765
forward_scheme: http
websockets: true
+4 -4
View File
@@ -1,9 +1,9 @@
# echo-mcp — MCP Server Build Spec (containerized)
> Status: **spec for a future build session** (written 2026-07-28 against v1.5.1;
> revised same day: **remote container architecture**, operator decision — heavy
> lifting belongs in a deployed container, not on any one machine).
> Companion plan for the other nine review items: `docs/IMPROVEMENT-PLANS.md`.
> Status: **BUILT — shipped as 2.2.0, 2026-07-28** (`mcp-server/app.py`, 14 tools —
> the count below saying 13 undercounted the append/patch pair; e2e suite
> `eval/test_mcp_server.py`). This document remains the design record; §7.2 is the
> live v1.1 backlog. Companion plan: `docs/IMPROVEMENT-PLANS.md`.
>
> **Prerequisites before starting this build:**
> 1. The `session-end` verb (IMPROVEMENT-PLANS #7) — the MCP tool wraps it.
Binary file not shown.
@@ -1,6 +1,6 @@
{
"name": "echo-memory",
"version": "2.1.1",
"version": "2.2.0",
"homepage": "https://git.alwisp.com/jason/echo",
"repository": "https://git.alwisp.com/jason/echo",
"description": "Persistent memory via the ECHO Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /echo-* commands.",
@@ -52,6 +52,19 @@ Executable logic ships under `scripts/` — **pure Python**, so the whole toolch
- `scripts/check_routing.py` — verifies the routing docs stay in sync with `routing.json` (dev/CI; offline)
- `scripts/bootstrap.py` / `scripts/migrate.py` — deterministic vault setup/repair and schema migration
## MCP tools first (when the echo-mcp connector is available)
When this session has the **`echo_*` MCP tools** (the deployed echo-mcp server —
`echo_load`, `echo_recall`, `echo_resolve`, `echo_get_note`, `echo_get_scope`/`echo_set_scope`,
`echo_health`, `echo_capture`, `echo_link`, `echo_append_note`/`echo_patch_note`,
`echo_triage_inbox`, `echo_reflect`, `echo_log_session`), **prefer them over the CLI
for every operation they cover** — typed calls, structured results, no path
resolution or quoting. The procedures in this skill (reconcile at load, search-first,
scope discipline, third person, preview-before-apply) are unchanged and apply to both
surfaces. A `duplicate-gate` tool result is the same contract as capture exit 76:
`merge_into` or confirmed `force`, never a blind retry. The CLI recipes below are the
fallback for hosts without the connector or when the server is unreachable.
## Bundled Tooling (prefer over raw curl)
All paths below are under `${CLAUDE_PLUGIN_ROOT}/skills/echo-memory/`. **Invoke with `python3`** (on Windows where that isn't on PATH, use `python` or `py -3`).
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""test_mcp_server.py — echo-mcp (2.2) end-to-end against the mock vault.
Spawns mcp-server/app.py + mock_olrapi and drives the real streamable-HTTP MCP
protocol: health (open), auth (401), initialize, tools/list, and the memory-day
tools/call flow (capture -> duplicate gate as DATA -> merge_into -> recall ->
get_note -> log_session with heartbeat commit).
Requires the `mcp` package in the interpreter that runs the SERVER. Set
ECHO_MCP_PYTHON to a venv python that has it; otherwise the current interpreter
is tried and the suite SKIPS (exit 0) when the SDK is absent — the other suites
don't depend on it.
Run: python test_mcp_server.py [--port 8862] [--mcp-port 8767]
"""
from __future__ import annotations
import argparse
import json
import os
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
APP = HERE.parent / "mcp-server" / "app.py"
KEY = "test-key-not-a-real-secret"
TOKEN = "local-test-token"
failures = []
def check(name, cond, detail=""):
print(f"{'ok ' if cond else 'FAIL'} {name}" + (f" -- {detail}" if not cond else ""))
if not cond:
failures.append(name)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--port", type=int, default=8862)
ap.add_argument("--mcp-port", type=int, default=8767)
a = ap.parse_args()
mock_base = f"http://127.0.0.1:{a.port}"
mcp_url = f"http://127.0.0.1:{a.mcp_port}/mcp"
server_py = os.environ.get("ECHO_MCP_PYTHON") or sys.executable
probe = subprocess.run([server_py, "-c", "import mcp"], capture_output=True)
if probe.returncode != 0:
print("SKIP: `mcp` SDK not installed for the server interpreter "
"(set ECHO_MCP_PYTHON to a venv python that has it)")
return 0
def http(method, url, body=None, headers=None, timeout=30):
data = body.encode() if isinstance(body, str) else body
req = urllib.request.Request(url, data=data, method=method, headers=headers or {})
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return r.status, r.read().decode("utf-8", "replace")
except urllib.error.HTTPError as e:
return e.code, e.read().decode("utf-8", "replace")
except Exception as e: # noqa: BLE001
return 0, str(e)
_id = [0]
def rpc(method, params):
_id[0] += 1
st, body = http("POST", mcp_url, json.dumps(
{"jsonrpc": "2.0", "id": _id[0], "method": method, "params": params}),
headers={"Content-Type": "application/json",
"Accept": "application/json, text/event-stream",
"Authorization": f"Bearer {TOKEN}"})
return st, (json.loads(body) if body.strip().startswith("{") else {})
def call(name, args):
st, d = rpc("tools/call", {"name": name, "arguments": args})
res = d.get("result") or {}
payload = {}
if res.get("content"):
try:
payload = json.loads(res["content"][0]["text"])
except Exception: # noqa: BLE001
payload = {}
return res.get("isError", False), payload
mock = subprocess.Popen([sys.executable, str(HERE / "mock_olrapi.py"), "--port", str(a.port)],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
srv = None
try:
for _ in range(50):
try:
urllib.request.urlopen(f"{mock_base}/__debug__reset", data=b"", timeout=1)
break
except Exception:
time.sleep(0.1)
http("PUT", f"{mock_base}/vault/_agent/echo-vault.md",
"---\nschema_version: 4\n---\n# marker\n",
headers={"Authorization": f"Bearer {KEY}"})
env = dict(os.environ, ECHO_BASE=mock_base, ECHO_KEY=KEY, ECHO_MCP_TOKEN=TOKEN,
ECHO_MCP_PORT=str(a.mcp_port), ECHO_STATE_DIR=tempfile.mkdtemp(),
ECHO_TODAY="2026-07-28")
srv = subprocess.Popen([server_py, str(APP)], env=env,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
for _ in range(60):
st, _b = http("GET", f"http://127.0.0.1:{a.mcp_port}/health", timeout=2)
if st == 200:
break
time.sleep(0.5)
st, body = http("GET", f"http://127.0.0.1:{a.mcp_port}/health")
try:
hb = json.loads(body)
except Exception: # noqa: BLE001
hb = {}
check("health is open and green", st == 200 and hb.get("ok") is True
and hb.get("vault_reachable") is True, body[:200])
st, _b = http("POST", mcp_url, "{}", headers={"Content-Type": "application/json"})
check("MCP endpoint requires the bearer token (401)", st == 401, str(st))
st, d = rpc("initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
"clientInfo": {"name": "eval", "version": "0"}})
check("initialize answers with serverInfo",
d.get("result", {}).get("serverInfo", {}).get("name") == "echo-mcp", json.dumps(d)[:200])
st, d = rpc("tools/list", {})
tools = [t["name"] for t in d.get("result", {}).get("tools", [])]
check("tools/list exposes the full profile (14 tools)", len(tools) == 14, str(tools))
e, b = call("echo_capture", {"title": "Vera Lumen", "kind": "person",
"body": "CTO at Fluxcorp, met at the summit."})
check("capture create over MCP", not e and b.get("action") == "created"
and b.get("path") == "resources/people/vera-lumen.md", json.dumps(b))
e, b = call("echo_capture", {"title": "Vera Lumen Jr", "kind": "person"})
check("duplicate gate is data, not isError", not e
and b.get("action") == "duplicate-gate" and b.get("candidates"), json.dumps(b))
e, b = call("echo_capture", {"title": "Vera Lumen Jr", "kind": "person",
"merge_into": "vera-lumen", "body": "Follow-up."})
check("merge_into resolves the gate as an update", not e and b.get("action") == "updated",
json.dumps(b))
e, b = call("echo_recall", {"query": "fluxcorp", "budget_chars": 900})
check("recall over MCP finds by body term", not e and any(
h["path"] == "resources/people/vera-lumen.md" for h in b.get("primary", [])),
json.dumps(b)[:300])
e, b = call("echo_get_note", {"path": "resources/people/vera-lumen.md"})
check("get_note returns frontmatter + content", not e
and b.get("frontmatter", {}).get("type") == "person", json.dumps(b)[:200])
e, b = call("echo_get_note", {"path": "../etc/passwd"})
check("get_note rejects path traversal", not b.get("ok"))
e, b = call("echo_log_session", {"slug": "mcp-e2e", "hhmm": "2345", "apply": True,
"log_body": "---\ntype: session-log\n---\n# S\n\n## Goal\ne2e\n"})
check("log_session commits with the heartbeat", not e
and b.get("steps", {}).get("heartbeat") == "ok", json.dumps(b))
e, b = call("echo_capture", {"title": "Bad Kind", "kind": "wizard"})
check("unknown kind rejected with the valid list", not b.get("ok")
and "wizard" in b.get("error", ""), json.dumps(b))
print(f"\n{len(failures)} failure(s)" if failures else "\nall mcp-server tests passed")
return 1 if failures else 0
finally:
if srv:
srv.terminate()
mock.terminate()
if __name__ == "__main__":
raise SystemExit(main())
+417
View File
@@ -0,0 +1,417 @@
#!/usr/bin/env python3
"""echo-mcp — the containerized MCP server over the ECHO ops layer. [2.2]
Streamable-HTTP (stateless JSON) MCP server exposing the plugin's `*_op` cores
(Phase 0, shipped 2.1.1) as typed tools. Runs next to the Obsidian REST API on
the same box, so every vault round-trip behind a tool call is LAN-local.
Spec: docs/MCP-SERVER-SPEC.md. Highlights implemented here:
* bearer-token auth on everything except /health (constant-time compare);
* tool results = the op envelopes, returned as structured JSON;
* duplicate gate / offline queueing are DATA (never protocol errors), so the
model deliberately chooses merge_into/force instead of blind-retrying;
* tool profiles: ECHO_MCP_TOOLS=core exposes only the six daily drivers
(schema tokens cost context on every surface that lists them);
* result budgets: echo_recall(budget_chars), echo_get_note(section/max_chars);
* per-write serialization (one process mediates all MCP writes, so a plain
lock removes the advisory-lock race window for server-mediated writes);
* startup validation: fail fast (crash-loop visibly) on missing env.
Deliberately NOT here in v1 (spec §7.2 backlog): entity-index TTL cache — the
atomic_index_update correctness fix depends on a FRESH re-read under the lock,
so caching idx_mod.load would reintroduce the clobber race it closed. LAN
adjacency + the warm keep-alive pool make the read ~ms anyway.
Env: ECHO_BASE, ECHO_KEY, ECHO_MCP_TOKEN (required) · ECHO_OWNER ·
ECHO_MCP_PORT=8765 · ECHO_MCP_TOOLS=full|core · ECHO_STATE_DIR=/data
"""
from __future__ import annotations
import contextlib
import hmac
import json
import os
import re
import subprocess
import sys
import threading
from pathlib import Path
# --- locate the plugin scripts (image layout first, repo layout for local dev) ---
_HERE = Path(__file__).resolve().parent
for _cand in (_HERE / "scripts",
_HERE.parent / "echo-memory.plugin.src" / "skills" / "echo-memory" / "scripts"):
if _cand.is_dir():
SCRIPTS = _cand
break
else: # pragma: no cover
sys.exit("echo-mcp: cannot locate the ECHO scripts directory")
sys.path.insert(0, str(SCRIPTS))
# --- startup validation: a misdeployed container must crash-loop visibly ---------
_missing = [k for k in ("ECHO_BASE", "ECHO_KEY", "ECHO_MCP_TOKEN") if not os.environ.get(k)]
if _missing:
sys.exit(f"echo-mcp: missing required env: {', '.join(_missing)} "
"— check the PORT template / secret refs")
# Modules aliased *_mod: several tool functions below share a module's name
# (echo_recall, echo_reflect, ...) and would shadow it at module scope otherwise.
import echo # noqa: E402
import echo_doctor as doctor_mod # noqa: E402
import echo_index as idx_mod # noqa: E402
import echo_ops as ops_mod # noqa: E402
import echo_queue as queue_mod # noqa: E402
import echo_recall as recall_mod # noqa: E402
import echo_reflect as reflect_mod # noqa: E402
import echo_session as session_mod # noqa: E402
import echo_triage as triage_mod # noqa: E402
from mcp.server.fastmcp import FastMCP # noqa: E402
from starlette.requests import Request # noqa: E402
from starlette.responses import JSONResponse, Response # noqa: E402
from starlette.middleware.base import BaseHTTPMiddleware # noqa: E402
PORT = int(os.environ.get("ECHO_MCP_PORT", "8765"))
TOKEN = os.environ["ECHO_MCP_TOKEN"]
PROFILE = os.environ.get("ECHO_MCP_TOOLS", "full").strip().lower()
KINDS = sorted(idx_mod.KIND_FOLDER)
mcp = FastMCP(
"echo-mcp",
instructions=(
"Persistent memory over the operator's ECHO Obsidian vault. Use echo_load at "
"the start of substantive sessions, echo_recall/echo_resolve to read, "
"echo_capture as the default write, and echo_log_session to end a session. "
"A duplicate-gate result is a decision point, not an error — merge_into or "
"(confirmed-distinct) force. Write about the operator in third person."),
host="0.0.0.0",
port=PORT,
stateless_http=True,
json_response=True,
)
WRITE_LOCK = threading.Lock() # all MCP-path writes serialize through this process
def _guard(write: bool, fn, *args, **kw) -> dict:
"""Run an op core with stdout redirected to stderr (belt-and-braces stream
purity) and EchoError mapped to an actionable error envelope, not a protocol
error. Write ops serialize on WRITE_LOCK."""
lock = WRITE_LOCK if write else contextlib.nullcontext()
try:
with lock, contextlib.redirect_stdout(sys.stderr):
return fn(*args, **kw)
except RuntimeError as exc: # EchoError (either module instance) subclasses it
code = getattr(exc, "code", 1)
err = {"ok": False, "error": str(exc), "code": code}
if code == 78:
err["hint"] = ("server deployment is missing vault credentials — operator: "
"check the PORT template env (SECRET: refs)")
elif code == 44:
err["code_name"] = "not-found"
elif "unreachable" in str(exc):
err["hint"] = ("vault unreachable (Obsidian/REST plugin likely down) — "
"proceed without memory; writes queue durably")
return err
_SAFE_PATH = re.compile(r"^[^/][^\0]*$")
def _check_path(path: str) -> str | None:
if not path or path.startswith("/") or ".." in path.split("/") or not _SAFE_PATH.match(path):
return "invalid path: must be vault-relative, no leading '/' and no '..'"
return None
def _frontmatter(text: str) -> dict:
out: dict = {}
if text.startswith("---"):
end = text.find("\n---", 3)
for ln in text[3:end if end != -1 else len(text)].splitlines():
m = re.match(r"^([A-Za-z_][\w-]*):\s*(.*)$", ln)
if m:
out[m.group(1)] = m.group(2).strip().strip('"').strip("'")
return out
# ------------------------------------------------------------------ tools -------
CORE_TOOLS = {"echo_load", "echo_recall", "echo_capture", "echo_triage_inbox",
"echo_log_session", "echo_health"}
def tool(fn):
"""Register `fn` as an MCP tool unless the core profile excludes it."""
if PROFILE == "core" and fn.__name__ not in CORE_TOOLS:
return fn
return mcp.tool()(fn)
@tool
def echo_load(brief: bool = True) -> dict:
"""Cold-start memory orientation — call at the start of a substantive session.
brief=true (default) returns a token-budgeted digest plus structured sections;
brief=false returns the six raw sections. Also flushes writes queued offline."""
return _guard(True, echo.load_op, brief)
@tool
def echo_recall(query: str, limit: int = 6, budget_chars: int = 4000,
include_linked: bool = True) -> dict:
"""Search memory: ranked matches for a topic/person/project PLUS their linked
neighbourhood. Excerpts are packed into budget_chars by score — call
echo_get_note for any hit's full content. Freshness/status-aware ranking."""
limit = max(1, min(int(limit), 20))
budget = max(500, min(int(budget_chars), 20000))
env = _guard(False, recall_mod.recall_op, query, limit)
if not env.get("ok"):
return env
if not include_linked:
env["linked"] = []
used = 0
for hit in (env.get("primary") or []) + (env.get("linked") or []):
ex = hit.get("excerpt") or ""
room = max(0, budget - used)
if len(ex) > room:
hit["excerpt"] = ex[:room] + ("… (truncated — echo_get_note for full)" if room else "")
hit["truncated"] = True
used += len(hit.get("excerpt") or "")
return env
@tool
def echo_resolve(mention: str) -> dict:
"""Resolve a name/mention to its canonical vault note (alias-aware), or get
did-you-mean candidates. Call before creating any note by hand; echo_capture
does this automatically."""
return _guard(False, ops_mod.resolve_op, mention)
@tool
def echo_get_note(path: str, section: str = "", max_chars: int = 8000) -> dict:
"""Read one vault note (vault-relative path). section='Status' returns only that
## section. Content over max_chars is truncated with a marker."""
bad = _check_path(path)
if bad:
return {"ok": False, "error": bad}
def _read() -> dict:
status, body = echo.request("GET", echo.vault_url(path))
if status == 404:
return {"ok": False, "code_name": "not-found", "path": path,
"error": f"{path}: not found"}
echo.check(status, body, f"get {path}")
text = body.decode(errors="replace")
fm = _frontmatter(text)
if section:
text = echo.extract_heading(text, section)
if not text:
return {"ok": False, "path": path,
"error": f"no '## {section}' section in {path}"}
truncated = len(text) > max_chars
return {"ok": True, "path": path, "frontmatter": fm,
"content": text[:max_chars] + ("\n… (truncated)" if truncated else ""),
"truncated": truncated}
return _guard(False, _read)
@tool
def echo_get_scope() -> dict:
"""The operator's active scope + freshness. If sessions_since >= 3, treat the
recorded scope as suspect and confirm with the operator before working."""
return _guard(False, echo.scope_show_op)
@tool
def echo_set_scope(scope: str) -> dict:
"""Switch the active scope atomically (archives the prior scope to Scope History
and stamps freshness). Use when the session's work diverges from the recorded
scope."""
return _guard(True, echo.scope_set_op, scope)
@tool
def echo_health(deep: bool = False) -> dict:
"""ECHO readiness: config, vault reachability, auth, bootstrap/schema, and the
offline-queue depth. deep=true also runs the full vault-invariant linter."""
env = _guard(False, doctor_mod.run_op)
try:
env["outbox_depth"] = len(queue_mod.pending())
env["needs_attention"] = len(queue_mod.needs_attention())
except Exception: # noqa: BLE001
pass
if deep and env.get("ok"):
r = subprocess.run([sys.executable, str(SCRIPTS / "vault_lint.py")],
capture_output=True, text=True,
env=dict(os.environ), timeout=120)
env["lint_exit"] = r.returncode
env["lint_report"] = (r.stdout or r.stderr)[-6000:]
return env
@tool
def echo_capture(title: str, kind: str = "", body: str = "", tags: list[str] = [],
aliases: list[str] = [], sources: list[str] = [], status: str = "",
date: str = "", domain: str = "business", merge_into: str = "",
force: bool = False, dry_run: bool = False) -> dict:
"""The default memory write: routes by kind (person, company, concept, reference,
meeting, project, area, semantic, episodic, working, skill, decision), stamps
complete frontmatter, indexes, auto-links, and writes the Agent-Log line — one
call. Omit kind for an inbox capture. If the result is action='duplicate-gate',
do NOT retry blindly: call again with merge_into=<candidate slug> to update the
existing entity, or force=true only after confirming they are genuinely distinct.
Offline, the whole capture queues durably (queued=true)."""
if kind and kind not in KINDS:
return {"ok": False, "error": f"unknown kind '{kind}' — one of: {', '.join(KINDS)}"}
return _guard(True, ops_mod.capture_op, kind or None, title,
body_text=body or "", status_v=status, aliases=aliases,
sources=sources, tags=tags, date=date or None, domain=domain,
inbox=not kind, dry_run=dry_run, force=force,
merge_into=merge_into or None)
@tool
def echo_link(a: str, b: str) -> dict:
"""Add reciprocal '## Related' links between two notes. Accepts vault paths or
resolvable entity names (aliases work)."""
def _to_path(x: str) -> str | None:
if "/" in x:
return x if x.endswith(".md") else x + ".md"
nmap = idx_mod.name_map(idx_mod.load())
return nmap.get(idx_mod.slugify(x))
def _do() -> dict:
pa, pb = _to_path(a), _to_path(b)
if not pa or not pb:
missing = a if not pa else b
return {"ok": False, "error": f"'{missing}' resolves to no note — "
"pass a vault path or a known entity name"}
return ops_mod.link_op(pa, pb)
return _guard(True, _do)
@tool
def echo_append_note(path: str, line: str) -> dict:
"""Append one line to a note, idempotently (skipped if the exact line already
exists). For inbox lines, Agent-Log entries, Observations bullets."""
bad = _check_path(path)
if bad:
return {"ok": False, "error": bad}
def _do() -> dict:
rc = echo.cmd_append(path, line)
return {"ok": rc == 0, "action": "append", "path": path}
return _guard(True, _do)
@tool
def echo_patch_note(path: str, operation: str, target_type: str, target: str,
content: str) -> dict:
"""Targeted edit: append/prepend/replace under a heading, frontmatter field, or
block. HEADING TARGETS must be the full '::'-delimited path from the top-level
heading (e.g. 'Operator Preferences::Fact / Pattern') — on an invalid target the
error includes the note's actual headings. replace overwrites the section."""
bad = _check_path(path)
if bad:
return {"ok": False, "error": bad}
def _do() -> dict:
rc = echo.cmd_patch(path, operation, target_type, target,
echo.temp_file(content.encode("utf-8")))
return {"ok": rc == 0, "action": f"patch:{operation}", "path": path,
"target": target}
env = _guard(True, _do)
if not env.get("ok") and "HTTP 400" in str(env.get("error", "")) and target_type == "heading":
st, body = echo.request("GET", echo.vault_url(path),
headers={"Accept": "application/vnd.olrapi.document-map+json"})
if st == 200:
try:
headings = [h.get("heading") or h for h in
json.loads(body).get("headings", [])][:40]
env["available_headings"] = headings
env["hint"] = "use one of available_headings verbatim as the Target"
except Exception: # noqa: BLE001
pass
return env
@tool
def echo_triage_inbox(proposals: list[dict] = [], apply: bool = False) -> dict:
"""Inbox triage. No proposals => structured listing of captures (line, date, text,
age_days). With proposals (reflect schema + optional 'line'): apply=false previews
the routing; apply=true routes via capture and writes the processing-log audit.
List first, propose, preview, then apply only after the operator confirms."""
if not proposals:
return _guard(False, triage_mod.list_op)
return _guard(True, triage_mod.route_op, proposals, apply)
@tool
def echo_reflect(proposals: list[dict], apply: bool = False) -> dict:
"""Session-reflection proposals: validate -> classify against the entity index ->
preview (apply=false) -> apply (routes each via capture). Never apply without the
operator's go-ahead; never invent memories to have something to save."""
return _guard(True, reflect_mod.apply_op, proposals, apply)
@tool
def echo_log_session(slug: str, log_body: str, agent_log_line: str = "",
scope: str = "", reflect: list[dict] = [],
apply: bool = False, hhmm: str = "") -> dict:
"""End a substantive session in ONE call: session log -> Agent-Log line -> reflect
proposals -> optional scope switch -> heartbeat LAST (the commit marker).
apply=false previews the plan. Pass hhmm (local time, e.g. '1430') so the log
filename sorts truthfully."""
bundle = {"slug": slug, "log_body": log_body}
if agent_log_line:
bundle["agent_log_line"] = agent_log_line
if scope:
bundle["scope"] = scope
if reflect:
bundle["reflect"] = reflect
def _do() -> dict:
prev = os.environ.get("ECHO_NOW")
try:
if hhmm:
os.environ["ECHO_NOW"] = hhmm
return session_mod.session_end_op(bundle, apply=apply)
finally:
if hhmm:
if prev is None:
os.environ.pop("ECHO_NOW", None)
else:
os.environ["ECHO_NOW"] = prev
return _guard(True, _do)
# ------------------------------------------------------------- health + auth ----
@mcp.custom_route("/health", methods=["GET"])
async def health(_request: Request) -> JSONResponse:
"""Unauthenticated liveness for Docker HEALTHCHECK + Kuma. No vault data."""
st, _ = echo.request("GET", echo.vault_url("_agent/echo-vault.md"))
try:
outbox = len(queue_mod.pending())
except Exception: # noqa: BLE001
outbox = -1
return JSONResponse({"ok": True, "vault_reachable": st != 0,
"outbox_depth": outbox, "profile": PROFILE})
class BearerAuth(BaseHTTPMiddleware):
async def dispatch(self, request, call_next):
if request.url.path == "/health":
return await call_next(request)
auth = request.headers.get("authorization", "")
if not hmac.compare_digest(auth, f"Bearer {TOKEN}"):
return Response(status_code=401)
return await call_next(request)
def main() -> None:
import uvicorn
app = mcp.streamable_http_app()
app.add_middleware(BearerAuth)
print(f"echo-mcp: serving on :{PORT} (profile={PROFILE}, vault={echo.BASE})",
file=sys.stderr)
uvicorn.run(app, host="0.0.0.0", port=PORT, log_level="info")
if __name__ == "__main__":
main()
+4
View File
@@ -0,0 +1,4 @@
# echo-mcp server deps. The PLUGIN stays pure-stdlib; the dependency budget for the
# container is deliberately tiny: the official MCP SDK (brings starlette/uvicorn/
# pydantic) and nothing else.
mcp>=1.9,<2