forked from jason/echo
Add CHORUS-PLAN.md — ECHO to group-based memory conversion plan
Full review of the ECHO v1.5.1 architecture plus the phased plan to convert the plugin to multi-member (group) memory: identity model, schema-5 member namespacing, concurrency posture, rename map, and the five settled design decisions (shared rollups, group-visible prefs, shared-key first, /chorus-* commands, per-member + group inboxes). Deployment endpoint: chorusapi.mpm.to. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+248
@@ -0,0 +1,248 @@
|
|||||||
|
# CHORUS — Group-Based Memory: Conversion Plan
|
||||||
|
|
||||||
|
**Fork of:** `jason/echo` (ECHO v1.5.1, schema 4) → `jason/chorus`
|
||||||
|
**Goal:** convert the single-operator ECHO memory plugin into CHORUS — one shared
|
||||||
|
Obsidian vault serving a **group of members**, with per-member identity,
|
||||||
|
attribution, and isolation of the churn-prone per-person state, while keeping the
|
||||||
|
shared knowledge graph (projects, areas, resources, decisions) communal.
|
||||||
|
|
||||||
|
**Naming metaphor:** ECHO is one voice reflected back; CHORUS is many voices in
|
||||||
|
one performance.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What the review found
|
||||||
|
|
||||||
|
ECHO is ~4,800 LOC of pure-stdlib Python (18 scripts) + a large doc surface
|
||||||
|
(SKILL.md ~456 lines, 6 references, 8 commands, 15 scaffold seeds). The
|
||||||
|
architecture is already favorable for this conversion:
|
||||||
|
|
||||||
|
- **Identity is barely wired in.** `echo.py` loads `OWNER` from config but never
|
||||||
|
actually consumes it — attribution today is purely doc-driven ("write in third
|
||||||
|
person about Jason"). Adding a `member` identity breaks nothing.
|
||||||
|
- **Routing is data-driven.** `routing.json` is the single source of truth for
|
||||||
|
"what may be written where"; `vault_lint.py` and `check_routing.py` enforce it.
|
||||||
|
Per-member paths are new route rows, not new code paths.
|
||||||
|
- **The vault is self-describing.** Bootstrap/migrate/sweep + `schema_version`
|
||||||
|
in the marker give us a clean upgrade lever (schema 4 → 5).
|
||||||
|
|
||||||
|
The hard part is **not** identity plumbing — it's the five shared single-writer
|
||||||
|
artifacts that make no sense (or clobber) with multiple concurrent users:
|
||||||
|
|
||||||
|
| Artifact | Today | Problem in a group |
|
||||||
|
|---|---|---|
|
||||||
|
| `_agent/memory/semantic/operator-preferences.md` | One profile | Whose preferences? |
|
||||||
|
| `_agent/context/current-context.md` | One active scope for the vault | Members work on different things simultaneously |
|
||||||
|
| `_agent/heartbeat/last-session.md` | One "last session" pointer (PUT overwrite) | Race: lost updates; meaningless shared |
|
||||||
|
| `inbox/captures/inbox.md` | One shared inbox (POST append) | Append races; triage can't tell whose capture |
|
||||||
|
| `journal/daily/YYYY-MM-DD.md` `## Agent Log` | One shared heading | Duplicate-heading races; unattributed lines |
|
||||||
|
|
||||||
|
Also single-user by design: session filenames carry no author
|
||||||
|
(`YYYY-MM-DD-HHMM-<slug>.md` — two members can collide on the same minute),
|
||||||
|
frontmatter has no `author:` field anywhere, the offline queue is per-machine
|
||||||
|
with no member attribution, and the advisory lock owner is a session id, not a
|
||||||
|
person. Concurrency is advisory-only — `vault_lock()` warns and **proceeds** if
|
||||||
|
it can't acquire (echo_concurrency.py:78–81), which is acceptable for one person
|
||||||
|
on two machines but not for N people.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Target design
|
||||||
|
|
||||||
|
### 2.1 Identity model
|
||||||
|
|
||||||
|
- Config gains a `member` field:
|
||||||
|
`{ "group": "...", "member": "jason", "endpoint": "https://chorusapi.mpm.to", "key": "..." }`.
|
||||||
|
**Deployment endpoint: `https://chorusapi.mpm.to`** — the CHORUS vault gets
|
||||||
|
its own Obsidian + Local REST API instance exposed there (separate from
|
||||||
|
ECHO's `echoapi.alwisp.com`; the personal ECHO vault is untouched). The
|
||||||
|
endpoint stays machine-local/baked config, never in source, same as ECHO.
|
||||||
|
`owner` → `group` (with `owner` accepted as a legacy alias). Resolution order
|
||||||
|
unchanged: baked > env (`CHORUS_MEMBER` etc.) > config file. Baked per-user
|
||||||
|
builds (`build.py --bake-key --label alice`) already exist and become the
|
||||||
|
primary **onboarding** mechanism: one artifact per member, member id baked in.
|
||||||
|
- Every agent-written note gets `author: <member>` stamped in frontmatter
|
||||||
|
(`echo_ops._build_note`), added to the canonical frontmatter block, to
|
||||||
|
`KIND_REQUIRED_FM` for agent-written kinds, linted by `vault_lint`, and
|
||||||
|
backfilled by `sweep --apply` (existing notes → the migrating member).
|
||||||
|
- Lock owner and offline-queue entries carry the member id
|
||||||
|
(`auto_owner()` → `f"{member}-{client}-{pid}"`), so contention and replay are
|
||||||
|
attributable.
|
||||||
|
|
||||||
|
**Auth reality check:** the Obsidian Local REST API has **one bearer key per
|
||||||
|
vault**. Phase 1–4 ship with a shared key and honor-system attribution (fine for
|
||||||
|
a trusted team). Real per-member credentials/revocation would need a thin auth
|
||||||
|
proxy in front of `chorusapi.mpm.to` that maps per-member tokens → the vault
|
||||||
|
key and can inject/verify the member id. That's deliberately **out of scope**
|
||||||
|
here but the config shape (`member` + per-member baked artifacts) is designed so
|
||||||
|
the proxy slots in later without another migration.
|
||||||
|
|
||||||
|
### 2.2 Vault layout changes (schema 5)
|
||||||
|
|
||||||
|
Per-member state moves under a member namespace; shared knowledge stays where it
|
||||||
|
is.
|
||||||
|
|
||||||
|
```
|
||||||
|
_agent/
|
||||||
|
├── chorus-vault.md ← renamed marker (schema_version: 5, managed_by: chorus-memory-plugin)
|
||||||
|
├── members/
|
||||||
|
│ └── <member>/
|
||||||
|
│ ├── preferences.md ← was operator-preferences.md (one per member)
|
||||||
|
│ ├── current-context.md ← per-member scope + Scope History
|
||||||
|
│ ├── heartbeat.md ← per-member last-session pointer
|
||||||
|
│ └── inbox.md ← per-member quick-capture inbox
|
||||||
|
├── memory/
|
||||||
|
│ └── semantic/
|
||||||
|
│ └── group-profile.md ← NEW: shared facts about the group itself
|
||||||
|
├── sessions/
|
||||||
|
│ └── <member>/YYYY-MM-DD-HHMM-<slug>.md ← member-namespaced session logs
|
||||||
|
└── (context/, memory/, health/, index/, locks/, templates/, skills/ unchanged — shared)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Shared and unchanged:** `projects/`, `areas/`, `resources/`, `decisions/`,
|
||||||
|
`journal/`, semantic/episodic/working memory (now with `author:`), the entity
|
||||||
|
index, recall index, health notes.
|
||||||
|
- **Daily Agent Log stays one shared heading** (it's the group's timeline) but
|
||||||
|
every line is member-prefixed: `- HH:MM [alice] did X`. `ensure_daily_log`
|
||||||
|
keeps its GET→POST-heading→PATCH flow; idempotent whole-line append plus the
|
||||||
|
member prefix makes interleaving safe and attributed. (Per-member daily notes
|
||||||
|
were considered and rejected — the whole point of a group vault is one shared
|
||||||
|
timeline.)
|
||||||
|
- **Inbox:** per-member capture files (`_agent/members/<id>/inbox.md`) kill the
|
||||||
|
append race and make triage member-aware, **plus** one shared
|
||||||
|
`inbox/captures/group.md` for unowned "someone should look at this" items
|
||||||
|
(member-prefixed lines; routed by whoever triages first). Triage covers both:
|
||||||
|
accepted items route to shared homes; preference-type items route to *that
|
||||||
|
member's* `preferences.md`. Processing-log lines carry the member id.
|
||||||
|
- `resources/people/<member>.md` continues to exist for every member — the
|
||||||
|
public "who is this person" note, distinct from their private-ish
|
||||||
|
`preferences.md` (how the agent should behave *for them*).
|
||||||
|
|
||||||
|
### 2.3 Cold-start load (per member)
|
||||||
|
|
||||||
|
`chorus.py load` reads: marker → **group-profile** → **my** preferences → **my**
|
||||||
|
current-context → **my** heartbeat/last-session (fallback: my sessions dir) →
|
||||||
|
today's shared daily note → **my** inbox depth → **group inbox** depth. Optional `--all-members` flag
|
||||||
|
for a group-wide orientation (e.g., "what has the team been doing"). Reconcile
|
||||||
|
(inbox aging, scope drift) runs against the member's own state only.
|
||||||
|
|
||||||
|
### 2.4 Concurrency posture
|
||||||
|
|
||||||
|
Namespacing removes ~all of the contention (each member's churn-prone files are
|
||||||
|
theirs alone). What remains shared-write:
|
||||||
|
|
||||||
|
- **Entity + recall indexes** — keep `vault_lock` + `atomic_index_update`, but
|
||||||
|
flip the lock to **required** (`required=True`) on index writes: block/retry
|
||||||
|
briefly instead of warn-and-proceed. TTL reclaim unchanged.
|
||||||
|
- **Daily note / shared notes** — idempotent member-prefixed appends; PATCH to
|
||||||
|
headings is last-writer-wins per line, which is acceptable for log lines.
|
||||||
|
- **Duplicate gate** — unchanged and global (dedup across the group is the
|
||||||
|
*feature*: two members shouldn't create parallel notes for the same client).
|
||||||
|
Gate messages name the `author` of the existing candidate.
|
||||||
|
|
||||||
|
### 2.5 Recall / reflection
|
||||||
|
|
||||||
|
- Corpus unchanged (shared graph is the point). `recall` gains an optional
|
||||||
|
`--author <member>` filter and surfaces `author:` in hits.
|
||||||
|
- `reflect` proposals route preference items to the current member's
|
||||||
|
`preferences.md`; everything else routes to shared homes with `author:`
|
||||||
|
stamped. The Stop-hook nudge state stays per-machine (already is).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Renaming map (echo → chorus)
|
||||||
|
|
||||||
|
| Category | From | To |
|
||||||
|
|---|---|---|
|
||||||
|
| Plugin | `echo-memory` (plugin.json, skill dir, SKILL.md name) | `chorus-memory` |
|
||||||
|
| Commands | `/echo-{load,save,recall,reflect,triage,sweep,health,doctor}` | `/chorus-*` |
|
||||||
|
| Scripts | `echo*.py` (16 modules + imports everywhere) | `chorus*.py` |
|
||||||
|
| Exception | `EchoError` | `ChorusError` |
|
||||||
|
| Env vars | `ECHO_OWNER/BASE/KEY/CONFIG/TODAY/VERIFY/LOCK_TTL/TIMEOUT/WORKERS/STATE_DIR/CLIENT/DUP_GATE/REFLECT_MIN_TURNS/FRESH_HALF_LIFE` | `CHORUS_*` (+ new `CHORUS_MEMBER`); `ECHO_*` read as fallback aliases in `chorus_config` for one major version |
|
||||||
|
| Config file | `~/.claude/echo-memory/config.json` | `~/.claude/chorus-memory/config.json` (import path: `config import` accepts an old echo config) |
|
||||||
|
| State dir | `~/.echo-memory/` | `~/.chorus-memory/` |
|
||||||
|
| Marker | `_agent/echo-vault.md` (hardcoded in 8 files + routing.json) | `_agent/chorus-vault.md`; probe checks new name **then** old (adoption path) |
|
||||||
|
| Hooks | `hooks.json` paths ×2 incl. CoWork fallback glob | updated paths |
|
||||||
|
| Docs/branding | README, SKILL.md, references, scaffold seeds, icons | rewrite pass (cosmetic but large) |
|
||||||
|
|
||||||
|
`operator-preferences.md` is renamed as part of the **layout** change
|
||||||
|
(§2.2), not the branding pass.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Phased plan
|
||||||
|
|
||||||
|
### Phase 0 — Baseline (½ day)
|
||||||
|
Repo hygiene on the fork: strip stale versioned `.plugin` artifacts from the
|
||||||
|
tree (move to Gitea releases per ROADMAP-2.0), confirm `eval/` suites green
|
||||||
|
as-is, tag the fork point.
|
||||||
|
|
||||||
|
### Phase 1 — Mechanical rename (1–2 days)
|
||||||
|
Everything in §3, with back-compat shims (env aliases, marker dual-probe,
|
||||||
|
config import from echo paths). All existing tests pass under the new names.
|
||||||
|
No behavior change. **Deliverable:** installable `chorus-memory` plugin that is
|
||||||
|
ECHO-with-a-new-name.
|
||||||
|
|
||||||
|
### Phase 2 — Identity plumbing (1–2 days)
|
||||||
|
`member` in config (+ `CHORUS_MEMBER`, baked-tier support, `config set --member`),
|
||||||
|
`MEMBER` global in `chorus.py`, `author:` stamping in `_build_note`, canonical
|
||||||
|
frontmatter + `KIND_REQUIRED_FM` + lint + sweep backfill, member-aware lock
|
||||||
|
owner and queue entries, `doctor` reports resolved member. Vault layout
|
||||||
|
untouched. **Deliverable:** every new write is attributed; single-user behavior
|
||||||
|
otherwise identical.
|
||||||
|
|
||||||
|
### Phase 3 — Per-member namespacing (3–5 days — the core)
|
||||||
|
Schema 5. New `LEAVES` + seeds (`members/<id>/…`, `group-profile.md`),
|
||||||
|
member-parameterized bootstrap (bootstraps *your* member subtree on first load
|
||||||
|
if the vault exists but your namespace doesn't — new members self-onboard),
|
||||||
|
`cmd_load`/`cmd_scope`/heartbeat/inbox/triage/reflect retargeted to member
|
||||||
|
paths, session logs under `sessions/<member>/`, member-prefixed Agent Log
|
||||||
|
lines, `routing.json` route rows (with `<member>` pattern segment) + retired
|
||||||
|
old paths, lint updates, `migrate.py` 4→5 (moves Jason's existing
|
||||||
|
operator-preferences/current-context/heartbeat/inbox/sessions into
|
||||||
|
`members/jason/…`, stamps `author: jason` vault-wide via sweep). Docs:
|
||||||
|
vault-layout.md, routing-map.md, bootstrap.md rewritten.
|
||||||
|
**Deliverable:** a real multi-member vault; the live ECHO vault migrates in
|
||||||
|
place.
|
||||||
|
|
||||||
|
### Phase 4 — Shared-write hardening (1–2 days)
|
||||||
|
Required lock on index writes with bounded retry/backoff, gate messages carry
|
||||||
|
author, `recall --author`, `load --all-members`, concurrency docs
|
||||||
|
(operating-contract.md) rewritten for N writers.
|
||||||
|
|
||||||
|
### Phase 5 — Docs, onboarding & eval (2–3 days)
|
||||||
|
SKILL.md full rewrite (operator → member voice; drop the "single owner"
|
||||||
|
safety rule, replace with group rules: don't edit another member's
|
||||||
|
`preferences.md` except via their session; shared notes are fair game),
|
||||||
|
8 command docs, README. Onboarding runbook: per-member baked builds. Extend
|
||||||
|
`eval/`: two-member mock scenarios (concurrent capture, interleaved daily-log
|
||||||
|
appends, index race, cross-member dedup gate, member-scoped load) + rerun the
|
||||||
|
retrieval/durability metrics. New icon.
|
||||||
|
|
||||||
|
**Total: roughly 9–15 working days**, shippable at every phase boundary
|
||||||
|
(1 = rename, 2 = attribution, 3 = groups).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Decisions (settled with Jason, 2026-07-21)
|
||||||
|
|
||||||
|
1. **Journal rollups — shared.** One group rollup per period
|
||||||
|
(`journal/weekly/YYYY-Www.md` etc.) with member-attributed bullets inside.
|
||||||
|
No per-member rollup trees.
|
||||||
|
2. **Preference privacy — group-visible, documented.** Member `preferences.md`
|
||||||
|
files live in the shared vault and are readable by every member (one vault,
|
||||||
|
one key — no read ACLs possible). SKILL.md documents this explicitly;
|
||||||
|
anything a member considers private belongs in their own personal ECHO
|
||||||
|
vault, not CHORUS.
|
||||||
|
3. **Auth proxy — out of scope; shared key first.** Phases 1–5 ship with the
|
||||||
|
shared vault key and honor-system attribution. The per-member token proxy
|
||||||
|
(namespace write ACL, server-side audit, per-member revocation) is a later
|
||||||
|
standalone project; the config shape (`member` + per-member baked
|
||||||
|
artifacts) already accommodates it with zero plugin changes.
|
||||||
|
4. **Commands — `/chorus-*`.** Full-prefix names
|
||||||
|
(`/chorus-load`, `/chorus-save`, …), consistent with the ECHO pattern and
|
||||||
|
collision-free if both plugins are installed.
|
||||||
|
5. **Inboxes — per-member + shared group inbox.** Each member gets
|
||||||
|
`_agent/members/<id>/inbox.md`; a shared `inbox/captures/group.md` holds
|
||||||
|
unowned items (member-prefixed lines, idempotent appends), routed by
|
||||||
|
whoever triages first. Both are probed at load and covered by triage.
|
||||||
Reference in New Issue
Block a user