diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b2625d..a0c5ff8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,32 @@ # Changelog +## 2.0.0-alpha.4 — Phase 4 (shared-write hardening + group lenses) + +### Added +- **Bounded lock retry on index writes.** `vault_lock` gains `retries`/`backoff` + (linear); `atomic_index_update` and the recall-index upkeep now retry acquisition + (`CHORUS_LOCK_RETRIES`, default 4 / `CHORUS_LOCK_BACKOFF`, default 0.5s). Index + lock holds are sub-second, so a retrying writer virtually always acquires; only + after exhausted retries does it proceed unlocked — loudly — because an + already-written note must never be stranded unindexed. +- **Duplicate-gate attribution.** Gate candidates now carry the existing note's + `author`; the STOP message reads `candidate: acme-rockets -> … (score 1.0, + created by jason)`, so a blocked member knows whose entity they collided with. +- **`recall --author `** — filter hits and linked context to one member's + notes (a lens, not a wall); recall briefs and `--json` summaries now surface the + `author:` stamp alongside updated/status. +- **`load --all-members`** — appends a compact group-wide orientation: every + member's active scope + last-session heartbeat ("what has the team been doing"). +- `references/operating-contract.md` concurrency section rewritten for N members + (namespacing as first defense; member-tagged idempotent appends; retried index + lock; member-attributed lock owners). +- Eval: the two-member end-to-end suite is now committed as + `eval/test_multimember.py` (26 checks, incl. the four Phase-4 behaviors). + +### Fixed +- `load --all-members` roster parsing accepts folders as trailing-slash entries in + `files` (the shape the REST API actually returns) as well as a `folders` key. + ## 2.0.0-alpha.3 — Phase 3 (per-member namespacing — schema 5) The core of the group conversion: one shared vault, per-member namespaces for all diff --git a/CHORUS-PLAN.md b/CHORUS-PLAN.md index 4ea0179..4eaa002 100644 --- a/CHORUS-PLAN.md +++ b/CHORUS-PLAN.md @@ -209,10 +209,12 @@ up between phases — there is no ECHO-vault adoption path; CHORUS vaults start fresh). Docs: vault-layout.md, routing-map.md, bootstrap.md rewritten. **Deliverable:** a real multi-member vault, bootstrapped from empty. -### Phase 4 — Shared-write hardening (1–2 days) -Required lock on index writes with bounded retry/backoff, gate messages carry -author, `recall --author`, `load --all-members`, concurrency docs -(operating-contract.md) rewritten for N writers. +### Phase 4 — Shared-write hardening (1–2 days) — DONE +Lock on index writes with bounded retry/backoff (unlocked fallback only after +exhausted retries, loudly — a written note must never go unindexed), gate +messages carry the existing entity's author, `recall --author`, +`load --all-members`, concurrency docs (operating-contract.md) rewritten for +N writers. Two-member e2e suite committed as `eval/test_multimember.py`. ### Phase 5 — Docs, onboarding & eval (2–3 days) SKILL.md full rewrite (operator → member voice; drop the "single owner" diff --git a/README.md b/README.md index 833967c..754d5a3 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ -# chorus-memory — v2.0.0-alpha.3 +# chorus-memory — v2.0.0-alpha.4 > **CHORUS is a group-based fork of [ECHO](https://git.alwisp.com/jason/echo)** — one shared > Obsidian vault serving a group of members instead of a single operator. The conversion > plan and settled design decisions live in [CHORUS-PLAN.md](CHORUS-PLAN.md). The pre-fork -> ECHO v1.5.1 state is preserved at the `echo-fork-point` tag. **Phases 1–3 (through per-member +> ECHO v1.5.1 state is preserved at the `echo-fork-point` tag. **Phases 1–4 (through shared-write > namespacing) are done**; the body of this README below the fold still describes the > inherited single-operator behavior and is rewritten in Phase 5. > Deployment endpoint: `https://chorusapi.mpm.to`. CHORUS is a **clean break** from diff --git a/chorus-memory.plugin b/chorus-memory.plugin index 6176a54..14afe84 100644 Binary files a/chorus-memory.plugin and b/chorus-memory.plugin differ diff --git a/chorus-memory.plugin.src/.claude-plugin/plugin.json b/chorus-memory.plugin.src/.claude-plugin/plugin.json index 7aea9a1..e0541af 100644 --- a/chorus-memory.plugin.src/.claude-plugin/plugin.json +++ b/chorus-memory.plugin.src/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "chorus-memory", - "version": "2.0.0-alpha.3", + "version": "2.0.0-alpha.4", "description": "Group-based persistent memory via the shared CHORUS Obsidian vault over the Obsidian Local REST API. Cross-platform Python client: one-call capture/resolve/recall/link/triage over an entity index, hybrid BM25 + graph recall spanning entities + sessions/journal (recency/status-aware), a pre-write duplicate gate, complete-frontmatter capture, session hooks that self-fire load/reflect, offline write-ahead queue, lock-guarded concurrency, linter-enforced routing, and /chorus-* commands.", "author": { "name": "Jason Stedwell" diff --git a/chorus-memory.plugin.src/commands/chorus-load.md b/chorus-memory.plugin.src/commands/chorus-load.md index 8a2c199..3eebfe9 100644 --- a/chorus-memory.plugin.src/commands/chorus-load.md +++ b/chorus-memory.plugin.src/commands/chorus-load.md @@ -2,17 +2,18 @@ description: Load CHORUS memory — cold-start context read (profile, scope, latest session, today, inbox) --- -Use the **chorus-memory** skill to load memory now. Run the cold-start **Loading procedure** from `SKILL.md`: the six orientation reads (marker, operator-preferences, current-context, heartbeat, today's daily note, inbox) in **one call**, then do the load-time **reconcile** (inbox-depth + scope-drift) and surface it in a single line. +Use the **chorus-memory** skill to load memory now. Run the cold-start **Loading procedure** from `SKILL.md`: the eight orientation reads (marker, group profile, your preferences/context/heartbeat, today's shared daily note, your inbox, the group inbox) in **one call**, then do the load-time **reconcile** (inbox-depth + scope-drift) and surface it in a single line. ```bash CHORUS="${CLAUDE_PLUGIN_ROOT}/skills/chorus-memory/scripts/chorus.py" [ -f "$CHORUS" ] || CHORUS=$(ls /sessions/*/mnt/.remote-plugins/*/skills/chorus-memory/scripts/chorus.py 2>/dev/null | head -1) # CoWork sandbox fallback python3 "$CHORUS" load +# add --all-members for a group-wide view (every member's scope + last session) # Windows: use `python` or `py -3` if `python3` is not on PATH. ``` -`load` prints all six sections (404s on today's note / inbox are shown as absent, not errors) and flags an un-bootstrapped vault. Follow up with `chorus.py scope show` if you need the sessions-since-switch count, and a project search if a specific project is in play. +`load` prints all eight sections (and self-onboards your member namespace on a first visit) (404s on today's note / inbox are shown as absent, not errors) and flags an un-bootstrapped vault. Follow up with `chorus.py scope show` if you need the sessions-since-switch count, and a project search if a specific project is in play. -**If `load` prints `NOT CONFIGURED` (exit 78)**, this machine has no usable key file yet. Don't proceed with memory — follow **First run** in `SKILL.md`: tell the operator CHORUS isn't configured, ask for their chorus-memory config file (owner/endpoint/key), install it with `chorus.py config import ` (or `config set`), then re-run `load`. +**If `load` prints `NOT CONFIGURED` (exit 78)**, this machine has no usable key file yet. Don't proceed with memory — follow **First run** in `SKILL.md`: tell the operator CHORUS isn't configured, ask for their chorus-memory config file (group/member/endpoint/key), install it with `chorus.py config import ` (or `config set`), then re-run `load`. Do not narrate the reads. End with a one-line orientation: who/what/where the active scope is, and any reconcile prompt. diff --git a/chorus-memory.plugin.src/commands/chorus-recall.md b/chorus-memory.plugin.src/commands/chorus-recall.md index 1fe514a..e938c50 100644 --- a/chorus-memory.plugin.src/commands/chorus-recall.md +++ b/chorus-memory.plugin.src/commands/chorus-recall.md @@ -12,7 +12,7 @@ Run the one-call recall — it resolves the term against the entity index, searc ```bash CHORUS="${CLAUDE_PLUGIN_ROOT}/skills/chorus-memory/scripts/chorus.py" [ -f "$CHORUS" ] || CHORUS=$(ls /sessions/*/mnt/.remote-plugins/*/skills/chorus-memory/scripts/chorus.py 2>/dev/null | head -1) # CoWork sandbox fallback -python3 "$CHORUS" recall "$ARGUMENTS" +python3 "$CHORUS" recall "$ARGUMENTS" # add --author to filter to one member's notes # Windows: use `python` or `py -3` if `python3` is not on PATH. ``` diff --git a/chorus-memory.plugin.src/skills/chorus-memory/SKILL.md b/chorus-memory.plugin.src/skills/chorus-memory/SKILL.md index c8d543d..ae3486a 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/SKILL.md +++ b/chorus-memory.plugin.src/skills/chorus-memory/SKILL.md @@ -64,7 +64,7 @@ All paths below are under `${CLAUDE_PLUGIN_ROOT}/skills/chorus-memory/`. **Invok ```bash CHORUS="${CLAUDE_PLUGIN_ROOT}/skills/chorus-memory/scripts/chorus.py" # call as: python3 "$CHORUS" ... [ -f "$CHORUS" ] || CHORUS=$(ls /sessions/*/mnt/.remote-plugins/*/skills/chorus-memory/scripts/chorus.py 2>/dev/null | head -1) # CoWork sandbox fallback -python3 "$CHORUS" load # cold-start: the 8 orientation reads in one call +python3 "$CHORUS" load [--all-members] # cold-start: the 8 orientation reads (+ optional every-member scope/last-session view) python3 "$CHORUS" get # 404 -> exit 44 python3 "$CHORUS" ls ; python3 "$CHORUS" map # listing / document-map python3 "$CHORUS" search @@ -166,7 +166,7 @@ Keep the reconcile to a single short line to the operator (e.g. "3 inbox capture **If a specific topic/project/person is in play**, pull its connected context in one call: ```bash -python3 "$CHORUS" recall "" # the matching notes AND their one-hop neighbourhood +python3 "$CHORUS" recall "" [--author ] # matching notes + one-hop neighbourhood (optionally one member's only) ``` `recall` resolves the term against the entity index, searches, and expands one hop along `## Related` links and `source_notes` — so you get the project note *plus* its linked decisions, people, and prior sessions, not an isolated file. (If you only need the canonical path, `resolve ""` is the O(1) lookup.) diff --git a/chorus-memory.plugin.src/skills/chorus-memory/references/operating-contract.md b/chorus-memory.plugin.src/skills/chorus-memory/references/operating-contract.md index 56ec1b9..0b6fc4b 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/references/operating-contract.md +++ b/chorus-memory.plugin.src/skills/chorus-memory/references/operating-contract.md @@ -35,11 +35,12 @@ You are an agent operating against an Obsidian vault that functions as a shared, Assume clients may operate without filesystem access, through the Obsidian Local REST API. Keep paths predictable, frontmatter parseable, titles stable, and all state stored in notes rather than hidden conversation memory. Structure must stay portable: a fresh, empty vault is brought fully online by the plugin's bootstrap (`references/bootstrap.md`), so nothing essential should depend on files existing in the vault ahead of time. -## Concurrency (shared vault) +## Concurrency (N members, one vault) -The vault is a **shared** substrate — Claude Code, CoWork, and other REST API clients may operate on it concurrently. The REST API offers no transactions, so writers coordinate cooperatively: +The vault is a **group** substrate — every member's Claude Code and CoWork sessions may operate on it concurrently, from different machines. The REST API offers no transactions, so safety is layered structurally: -- Single-line, overwrite-style files (`_agent/members/<member>/heartbeat.md`, `current-context.md::Scope`) and append targets (`inbox.md`, `## Agent Log`) assume **one writer at a time**. Two sessions writing at once can clobber or duplicate. -- Before a burst of writes in a session that may overlap another, take the advisory lock (`chorus.py lock <id>` → `_agent/locks/vault.lock`) and release it at session end. The lock is read-back-confirmed and cooperative with a TTL (stale locks are reclaimable); it is a courtesy, not a hard mutex. -- Idempotent append (read-before-POST, whole-line match, via `chorus.py append`) is the second line of defense against duplicate lines from retries or overlapping sessions. +- **Namespacing is the first defense (schema 5).** All churn-prone single-writer state is per member (`_agent/members/<member>/{preferences,current-context,heartbeat,inbox}.md`, `_agent/sessions/<member>/`), so members never contend on each other's pointers by construction. Only your own sessions write your namespace. +- **What remains shared-write** is designed for interleaving: the daily `## Agent Log` and the group inbox take **member-tagged, whole-line-idempotent appends** (`- YYYY-MM-DD [member]: …` via `chorus.py append` — read-before-POST, exact-line match), so concurrent appends interleave without clobbering or duplicating; shared entity notes take member-tagged dated blocks the same way. +- **The machine indexes** (`_agent/index/*.json`, full-file read-modify-write) are guarded by the advisory lock with **bounded retry/backoff** (`CHORUS_LOCK_RETRIES`, default 4): index writes hold the lock sub-second, so a retrying writer virtually always acquires it, and the update re-reads the index fresh inside the critical section so concurrent captures merge instead of clobber. Only after exhausted retries does it proceed unlocked — loudly — because a note that was already written must never be stranded unindexed. +- The advisory lock (`chorus.py lock <id>` → `_agent/locks/vault.lock`) is read-back-confirmed and cooperative with a TTL (stale locks are reclaimable). Lock owner ids carry the member (`<member>-<client>-<pid>`), so contention is attributable to a person. For a burst of writes to *shared* files that may overlap another member's session, take it manually and release at session end; it is a courtesy, not a hard mutex. - Status-check every write. A write that returns `>= 400` did **not** land — surface it rather than assuming success. diff --git a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus.py b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus.py index 259d28a..f10c773 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus.py +++ b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus.py @@ -31,7 +31,7 @@ Config (group/member/endpoint/key are machine-local — see chorus_config; the p CHORUS_TODAY YYYY-MM-DD to use as "today" (pass the conversation's currentDate) Usage: - chorus.py load # cold-start: the 8 orientation reads in one call + chorus.py load [--all-members] # cold-start: the 8 orientation reads (+ optional group-wide view) chorus.py get <path> # print file contents (404 -> exit 44) chorus.py map <path> # document-map JSON (headings/blocks/frontmatter) chorus.py ls <dir> # directory listing JSON @@ -626,12 +626,14 @@ def cmd_scope(subcommand: str, text: str | None = None, as_json: bool = False) - return 0 -def cmd_load() -> int: +def cmd_load(all_members: bool = False) -> int: """Cold-start orientation: the canonical 8 reads in one call — the marker, the GROUP profile, then MY member state (preferences/context/heartbeat/inbox), the shared daily note, and the group inbox. 404s on today's daily note and the inboxes are normal (printed as absent, not errors). If the vault is - bootstrapped but MY namespace is missing, self-onboard it.""" + bootstrapped but MY namespace is missing, self-onboard it. With + `all_members`, append a compact group-wide view: every member's active scope + and last-session pointer ("what has the team been doing").""" if not chorus_config.is_configured(_CFG): print("chorus: NOT CONFIGURED — this machine has no usable CHORUS key file yet.") print(f"Expected at: {chorus_config.config_path()}") @@ -725,6 +727,34 @@ def cmd_load() -> int: for f in files[:5]: print(f" {member_sessions_dir()}/{f}") print() + # Group-wide orientation: one compact line-pair per member (scope + last + # session). A lens over everyone's namespaces — read-only, opt-in. + if all_members and not offline: + st, body = request("GET", vault_url("_agent/members/")) + roster = [] + if st == 200: + try: + payload = json.loads(body) + # Folders may arrive under "folders" or as trailing-slash entries + # in "files" (both shapes exist in the wild — cf. vault_lint.list_dir). + entries = list(payload.get("files", [])) + list(payload.get("folders", [])) + roster = sorted({e[:-1] for e in entries if e.endswith("/")}) + except json.JSONDecodeError: + roster = [] + print(f"===== group: all members ({len(roster)}) =====") + for m in roster: + me = " (me)" if m == MEMBER else "" + st_c, b_c = request("GET", vault_url(member_path("current-context.md", m))) + scope = "" + if st_c == 200: + scope = " ".join(extract_heading(b_c.decode(errors="replace"), "Scope").split()) + scope = re.sub(r"<!--.*?-->", "", scope).strip()[:140] + st_h, b_h = request("GET", vault_url(member_path("heartbeat.md", m))) + last = b_h.decode(errors="replace").strip()[:120] if st_h == 200 else "(no sessions yet)" + print(f" {m}{me}") + print(f" scope: {scope or '(not set)'}") + print(f" last: {last}") + print() if offline: print("NOTE: vault unreachable — context above is last-known-good cache; writes this " "session will be queued and synced when the vault returns.") @@ -780,7 +810,9 @@ def cmd_config(args) -> int: def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(description="Validated cross-platform CHORUS vault client") sub = parser.add_subparsers(dest="cmd", required=True) - sub.add_parser("load") + p = sub.add_parser("load") + p.add_argument("--all-members", action="store_true", + help="append a group-wide orientation: every member's scope + last session") sub.add_parser("flush") # H2: replay writes queued during a prior outage sub.add_parser("doctor") # M3: one-call readiness check sub.add_parser("get").add_argument("path") @@ -819,6 +851,7 @@ def build_parser() -> argparse.ArgumentParser: # High-level operations (entity index + cross-links + recall). See chorus_ops.py. sub.add_parser("resolve").add_argument("mention", nargs="+") p = sub.add_parser("recall"); p.add_argument("query", nargs="+"); p.add_argument("--limit", type=int, default=6) + p.add_argument("--author") # filter hits to one member's notes (a lens, not a wall) p.add_argument("--json", action="store_true") # structured hits + neighbourhood p = sub.add_parser("link"); p.add_argument("a"); p.add_argument("b") p.add_argument("--json", action="store_true") @@ -845,7 +878,7 @@ def main(argv: list[str] | None = None) -> int: args = build_parser().parse_args(argv) try: if args.cmd == "load": - return cmd_load() + return cmd_load(all_members=args.all_members) if args.cmd == "doctor": import chorus_doctor return chorus_doctor.run() @@ -918,7 +951,8 @@ def main(argv: list[str] | None = None) -> int: if args.cmd == "resolve": return chorus_ops.resolve(" ".join(args.mention)) if args.cmd == "recall": - return chorus_ops.recall(args.query, limit=args.limit, as_json=args.json) + return chorus_ops.recall(args.query, limit=args.limit, as_json=args.json, + author=args.author) if args.cmd == "link": return chorus_ops.link(args.a, args.b, as_json=args.json) return chorus_ops.capture( diff --git a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_concurrency.py b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_concurrency.py index a582685..c7e7b24 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_concurrency.py +++ b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_concurrency.py @@ -23,12 +23,19 @@ from __future__ import annotations import contextlib import os import sys +import time from pathlib import Path sys.path.insert(0, str(Path(__file__).resolve().parent)) import chorus # noqa: E402 import chorus_index as idx_mod # noqa: E402 +# Bounded retry for index-write locks (Phase 4 hardening). With N members writing, +# lock holds are sub-second, so a short backoff nearly always wins the lock; the +# unlocked fallback (loud, fresh-re-read) survives only a genuinely stuck peer. +INDEX_LOCK_RETRIES = max(0, int(os.environ.get("CHORUS_LOCK_RETRIES", "4"))) +INDEX_LOCK_BACKOFF = float(os.environ.get("CHORUS_LOCK_BACKOFF", "0.5")) + def auto_owner() -> str: """A stable-per-process owner id: member + client tag + pid, so lock contention @@ -41,16 +48,24 @@ def auto_owner() -> str: @contextlib.contextmanager -def vault_lock(owner: str | None = None, required: bool = False): +def vault_lock(owner: str | None = None, required: bool = False, + retries: int = 0, backoff: float = INDEX_LOCK_BACKOFF): """Hold the advisory lock for a write burst; release on exit (incl. exceptions). - Yields `held: bool`. The lock is cooperative (read-back-confirmed, TTL-reclaimable), - so if another session holds it we do NOT block: with `required=False` we yield + Yields `held: bool`. The lock is cooperative (read-back-confirmed, TTL-reclaimable). + `retries` > 0 re-attempts acquisition with linear backoff (attempt N sleeps + N*backoff) — with multiple members writing, lock holds are sub-second, so a short + retry almost always acquires. If it still can't: with `required=False` we yield `held=False` and let the caller proceed (advisory) or warn; with `required=True` we raise. Release is best-effort and never masks the body's own exception. """ owner = owner or auto_owner() rc = chorus.cmd_lock(owner, quiet=True) + attempt = 0 + while rc != 0 and attempt < retries: + attempt += 1 + time.sleep(backoff * attempt) + rc = chorus.cmd_lock(owner, quiet=True) held = rc == 0 if not held and required: raise chorus.ChorusError(f"vault_lock: could not acquire lock for '{owner}' (held elsewhere)", 75) @@ -69,12 +84,13 @@ def atomic_index_update(mutate, owner: str | None = None): entry: take the lock, **re-read the index fresh** (not a stale in-memory copy), run the mutator, save, release. Returns the freshly-read-and-saved index. - The fresh re-read inside the critical section is the core of the fix — even if the - advisory lock can't be taken (another session active), reading immediately before - the save collapses the read-modify-write window to two calls instead of spanning the - whole capture, so an unlocked update is still far safer than the prior code. + The fresh re-read inside the critical section is the core of the fix. Acquisition + retries with bounded backoff (INDEX_LOCK_RETRIES) — in a group vault the lock is + only ever held for sub-second index writes, so retries nearly always acquire; the + unlocked path (fresh re-read + loud warning) survives only a genuinely stuck peer, + chosen over hard failure so a note that was already written never goes unindexed. """ - with vault_lock(owner) as held: + with vault_lock(owner, retries=INDEX_LOCK_RETRIES) as held: index = idx_mod.load() mutate(index) idx_mod.save(index) diff --git a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_ops.py b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_ops.py index bbe8cb3..2a655ec 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_ops.py +++ b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_ops.py @@ -73,11 +73,11 @@ def link(a_path: str, b_path: str, as_json: bool = False) -> int: # ----------------------------------------------------------------- recall ----- -def recall(query, limit: int = 8, as_json: bool = False) -> int: +def recall(query, limit: int = 8, as_json: bool = False, author: str | None = None) -> int: """Hybrid lexical (BM25) + graph recall — implemented in chorus_recall. Kept here as the stable public entrypoint (chorus.py and /chorus-recall call chorus_ops.recall).""" import chorus_recall # lazy: only pulls the BM25 modules when recall is actually run - return chorus_recall.recall(query, limit=limit, as_json=as_json) + return chorus_recall.recall(query, limit=limit, as_json=as_json, author=author) # ----------------------------------------------------------- agent log ------- @@ -112,6 +112,21 @@ def ensure_daily_log(line: str) -> None: # ----------------------------------------------------------------- capture --- +def _note_author(path: str) -> str: + """Best-effort author: of an existing note — so gate messages can say WHOSE + entity a capture collided with. Never raises; empty string on any miss.""" + if not path: + return "" + try: + status, body = chorus.request("GET", chorus.vault_url(path)) + if status == 200: + m = re.search(r"(?m)^author:\s*(\S+)", body.decode(errors="replace")[:2000]) + return m.group(1) if m else "" + except Exception: # noqa: BLE001 + pass + return "" + + def _build_note(fm_type: str, status_v: str, today_s: str, title: str, body_text: str, sources: list[str], aliases: list[str] | None = None, tags: list[str] | None = None) -> str: @@ -245,7 +260,8 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = gate_hits = [] if not existing_reachable and not force and not dry_run: gate_hits = [{"slug": s, "path": c.get("path"), "title": c.get("title"), - "kind": c.get("kind"), "score": sc} + "kind": c.get("kind"), "score": sc, + "author": _note_author(c.get("path", ""))} for s, c, sc in idx_mod.gate_candidates(index, title, kind=kind, threshold=DUP_GATE)] if gate_hits: @@ -259,7 +275,8 @@ def capture(kind: str | None, title: str, file_arg: str | None, status_v: str = print(f"STOP: '{title}' likely already exists — not creating a duplicate.", file=real_stdout) for c in gate_hits: - print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']})", + by = f", created by {c['author']}" if c.get("author") else "" + print(f" candidate: {c['slug']} -> {c['path']} (score {c['score']}{by})", file=real_stdout) print(" re-run with --merge-into <slug> to update the existing entity, " "or --force to create anyway.", file=real_stdout) diff --git a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_recall.py b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_recall.py index fe1757e..2564ed3 100644 --- a/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_recall.py +++ b/chorus-memory.plugin.src/skills/chorus-memory/scripts/chorus_recall.py @@ -316,7 +316,7 @@ def update_note(path: str, text: str) -> None: if not _indexable(path): return # only entity notes are part of the recall corpus import chorus_concurrency - with chorus_concurrency.vault_lock(): + with chorus_concurrency.vault_lock(retries=chorus_concurrency.INDEX_LOCK_RETRIES): ix = load_index() # fresh read inside the lock ix.add(path, text) # raw text: add() strips + extracts meta save_index(ix) @@ -385,6 +385,9 @@ def _doc_summary(path: str, text: str) -> dict: ms = _FM_STATUS.search(text[:2000]) if ms: out["status"] = ms.group(1) + ma = re.search(r"(?m)^author:\s*(\S+)", text[:2000]) + if ma: + out["author"] = ma.group(1) body = strip_frontmatter(text) status = re.search(r"(?ms)^##\s+Status\s*$(.*?)(?=^##\s|\Z)", body) if status and status.group(1).strip(): @@ -414,6 +417,8 @@ def _brief(path: str, score: float | None = None, via: str | None = None) -> Non stamps.append(f"updated: {info['updated']}") if info.get("status"): stamps.append(f"status: {info['status']}") + if info.get("author"): + stamps.append(f"author: {info['author']}") if stamps: print("_" + " · ".join(stamps) + "_") # staleness is visible, not guessed if info.get("excerpt"): @@ -421,7 +426,7 @@ def _brief(path: str, score: float | None = None, via: str | None = None) -> Non # ------------------------------------------------------------------- entrypoint -def recall(query, limit: int = 8, as_json: bool = False) -> int: +def recall(query, limit: int = 8, as_json: bool = False, author: str | None = None) -> int: q = " ".join(query) if isinstance(query, list) else query today_s = chorus.today() index = idx_mod.load() @@ -464,6 +469,19 @@ def recall(query, limit: int = 8, as_json: bool = False) -> int: # --- graph layer ---------------------------------------------------------- neighbours = expand_graph(hits, nmap, base, max_hops=MAX_HOPS) + # --- optional member filter (--author): keep only notes whose frontmatter + # author matches. Group memory is shared, so this is a lens, not a wall. + if author: + cand = hits + [p for p, _ in neighbours] + atexts = chorus.read_many(cand) + + def _by(p: str) -> bool: + m = re.search(r"(?m)^author:\s*(\S+)", (atexts.get(p) or "")[:2000]) + return bool(m) and m.group(1) == author + + hits = [p for p in hits if _by(p)] + neighbours = [(p, sv) for p, sv in neighbours if _by(p)] + if as_json: import chorus_output texts = chorus.read_many(hits + [p for p, _ in neighbours[: 2 * limit]]) diff --git a/eval/test_multimember.py b/eval/test_multimember.py new file mode 100644 index 0000000..2c47e49 --- /dev/null +++ b/eval/test_multimember.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""test_multimember.py — two members share one vault (end-to-end vs the mock). + +Covers the schema-5 group behavior: jason bootstraps; alice self-onboards +via load; both capture; the daily log interleaves member-tagged lines; scopes are +independent; triage sees member+group inboxes; lint is clean; schema is 5.""" +import os +import subprocess +import sys +import time +import urllib.request +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +SCRIPTS = REPO / "chorus-memory.plugin.src" / "skills" / "chorus-memory" / "scripts" +PORT = 8807 +BASE = f"http://127.0.0.1:{PORT}" +fails = [] + + +def check(name, cond, detail=""): + print(("ok " if cond else "FAIL") + f" {name}" + (f" — {detail}" if detail and not cond else "")) + if not cond: + fails.append(name) + + +def run(member, script, *args, stdin=None): + env = {k: v for k, v in os.environ.items() if not k.startswith("CHORUS_")} + env.update({"CHORUS_BASE": BASE, "CHORUS_KEY": "k", "CHORUS_GROUP": "MPM", + "CHORUS_MEMBER": member, "CHORUS_TODAY": "2026-07-21"}) + return subprocess.run([sys.executable, str(SCRIPTS / script), *args], + capture_output=True, text=True, env=env, input=stdin) + + +def vault(path): + try: + return urllib.request.urlopen(f"{BASE}/vault/{path}", timeout=2).read().decode() + except Exception: + return None + + +mock = subprocess.Popen([sys.executable, str(REPO / "eval/mock_olrapi.py"), "--port", str(PORT)], + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) +for _ in range(50): + try: + urllib.request.urlopen(f"{BASE}/__debug__reset", data=b"", timeout=1) + break + except Exception: + time.sleep(0.1) + +try: + # --- jason stands up the vault --- + r = run("jason", "bootstrap.py") + check("jason bootstraps", r.returncode == 0, r.stderr[:200]) + marker = vault("_agent/chorus-vault.md") or "" + check("marker is schema 5", "schema_version: 5" in marker, marker[:120]) + check("group profile seeded + jason rostered", + "- jason (joined 2026-07-21)" in (vault("_agent/memory/semantic/group-profile.md") or "")) + check("jason anchors exist", vault("_agent/members/jason/preferences.md") is not None + and "author: jason" in vault("_agent/members/jason/preferences.md")) + check("group inbox seeded", vault("inbox/captures/group.md") is not None) + + # --- alice self-onboards via load --- + r = run("alice", "chorus.py", "load") + check("alice load self-onboards", "self-onboarding: member 'alice'" in r.stdout, r.stdout[:600]) + check("alice anchors created", vault("_agent/members/alice/preferences.md") is not None) + profile = vault("_agent/memory/semantic/group-profile.md") or "" + check("alice rostered in group profile", "- alice (joined 2026-07-21)" in profile, profile[-200:]) + r = run("alice", "chorus.py", "load") + check("second alice load does NOT re-onboard", "self-onboarding" not in r.stdout) + check("load reads group profile + both inboxes", + "group-profile" in r.stdout and "my-inbox" in r.stdout and "group-inbox" in r.stdout) + + # --- both capture; daily log interleaves member-tagged lines --- + run("jason", "chorus.py", "capture", "Acme Rockets", "--kind", "company") + run("alice", "chorus.py", "capture", "Widget Live", "--kind", "project") + daily = vault("journal/daily/2026-07-21.md") or "" + check("daily log tags jason", "[jason]: created company" in daily, daily[-300:]) + check("daily log tags alice", "[alice]: created project" in daily, daily[-300:]) + check("captures carry authors", + "author: jason" in (vault("resources/companies/acme-rockets.md") or "") + and "author: alice" in (vault("projects/active/widget-live.md") + or vault("projects/incubating/widget-live.md") or "")) + + # --- alice updates jason's entity: dated bullet attributes alice --- + run("alice", "chorus.py", "capture", "Acme Rockets", "--kind", "company", "-", + stdin="Alice met their CTO.\n") + acme = vault("resources/companies/acme-rockets.md") or "" + check("cross-member update attributed", "- 2026-07-21 [alice]: Alice met their CTO." in acme, + acme[-300:]) + + # --- scopes are independent (raw files: the basic mock's heading-PATCH only + # appends, so real replace semantics live in test_patch_semantics; here we + # prove the writes land in EACH member's own file and nowhere else) --- + r1 = run("jason", "chorus.py", "scope", "set", "fiber buildout") + r2 = run("alice", "chorus.py", "scope", "set", "widget launch") + jctx = vault("_agent/members/jason/current-context.md") or "" + actx = vault("_agent/members/alice/current-context.md") or "" + check("scope set succeeds per member", r1.returncode == 0 and r2.returncode == 0, + r1.stderr[:150] + r2.stderr[:150]) + check("scopes independent", "fiber buildout" in jctx and "widget launch" in actx + and "widget launch" not in jctx and "fiber buildout" not in actx) + + # --- member inbox + group inbox; triage lists both --- + run("jason", "chorus.py", "capture", "look into starlink backhaul", "--inbox") + run("jason", "chorus.py", "append", "inbox/captures/group.md", + "- 2026-07-21 [jason]: someone should test the new proxy") + r = run("alice", "chorus.py", "triage", "--list") + check("alice's triage sees the group item", "test the new proxy" in r.stdout, r.stdout[:400]) + check("alice's triage does NOT see jason's personal item", + "starlink" not in r.stdout, r.stdout[:400]) + + # --- session logs namespaced; lint clean --- + run("jason", "chorus.py", "put", "_agent/sessions/jason/2026-07-21-0900-fiber.md", "-", + stdin="---\ntype: session-log\nstatus: done\ncreated: 2026-07-21\nupdated: 2026-07-21\n" + "tags: [session]\nagent_written: true\nauthor: jason\nsource_notes: []\n---\n# Fiber session\n") + # --- Phase 4: gate names the existing entity's author --- + r = run("alice", "chorus.py", "capture", "Acme Rockets Inc", "--kind", "company") + check("gate blocks near-duplicate", r.returncode == 76, f"rc={r.returncode}") + check("gate names the author", "created by jason" in r.stdout, r.stdout[:300]) + + # --- Phase 4: recall --author filters to one member's notes --- + r = run("jason", "chorus.py", "recall", "Acme", "--author", "jason") + check("recall --author keeps jason's note", "acme-rockets" in r.stdout, r.stdout[:300]) + r = run("jason", "chorus.py", "recall", "Acme", "--author", "alice") + check("recall --author filters out others", "acme-rockets" not in r.stdout.split("Linked context")[0].split("Primary")[-1], r.stdout[:400]) + r = run("jason", "chorus.py", "recall", "Acme") + check("recall surfaces author stamp", "author: jason" in r.stdout, r.stdout[:400]) + + # --- Phase 4: load --all-members group view --- + run("jason", "chorus.py", "put", "_agent/members/jason/heartbeat.md", "-", + stdin="_agent/sessions/jason/2026-07-21-0900-fiber.md @ 2026-07-21T14:00:00Z\n") + r = run("alice", "chorus.py", "load", "--all-members") + check("load --all-members lists both", "group: all members (2)" in r.stdout + and "alice (me)" in r.stdout and "jason" in r.stdout, r.stdout[-600:]) + check("all-members shows jason's heartbeat", "2026-07-21-0900-fiber" in r.stdout, + r.stdout[-600:]) + + run("jason", "sweep.py", "--apply") # index the seeded anchors (documented bootstrap→sweep flow) + run("jason", "chorus.py", "link", "_agent/memory/semantic/group-profile.md", + "resources/companies/acme-rockets.md") + r = run("jason", "vault_lint.py") + check("lint clean on two-member vault", r.returncode == 0, r.stdout[-500:]) +finally: + mock.terminate() + +print() +sys.exit(1 if fails else print("all multimember tests passed") or 0)