docs: add VitePress documentation site
- 22 content pages across Guide, Concepts, and Reference sections - Custom indigo/cyan theme with Lucide icons and Mermaid diagrams - GitHub Actions workflow for GitHub Pages deployment - Live preview: https://mempalace-docs.netlify.app/
This commit is contained in:
@@ -0,0 +1,246 @@
|
||||
# API Reference
|
||||
|
||||
Comprehensive parameter-level documentation for all public Python APIs.
|
||||
|
||||
## `mempalace.searcher`
|
||||
|
||||
### `search(query, palace_path, wing=None, room=None, n_results=5)`
|
||||
|
||||
CLI-oriented search that prints results to stdout.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `query` | `str` | — | Search query text |
|
||||
| `palace_path` | `str` | — | Path to ChromaDB palace directory |
|
||||
| `wing` | `str` | `None` | Filter by wing name |
|
||||
| `room` | `str` | `None` | Filter by room name |
|
||||
| `n_results` | `int` | `5` | Maximum number of results |
|
||||
|
||||
**Raises:** `SearchError` if palace not found or query fails.
|
||||
|
||||
---
|
||||
|
||||
### `search_memories(query, palace_path, wing=None, room=None, n_results=5) → dict`
|
||||
|
||||
Programmatic search returning a dict. Used by the MCP server.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `query` | `str` | — | Search query text |
|
||||
| `palace_path` | `str` | — | Path to ChromaDB palace directory |
|
||||
| `wing` | `str` | `None` | Filter by wing name |
|
||||
| `room` | `str` | `None` | Filter by room name |
|
||||
| `n_results` | `int` | `5` | Maximum number of results |
|
||||
|
||||
**Returns:**
|
||||
```python
|
||||
{
|
||||
"query": str,
|
||||
"filters": {"wing": str | None, "room": str | None},
|
||||
"results": [
|
||||
{
|
||||
"text": str, # verbatim drawer content
|
||||
"wing": str, # wing name
|
||||
"room": str, # room name
|
||||
"source_file": str, # original file basename
|
||||
"similarity": float, # 0.0 to 1.0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
On error: `{"error": str, "hint": str}`
|
||||
|
||||
---
|
||||
|
||||
## `mempalace.layers`
|
||||
|
||||
### `class Layer0(identity_path=None)`
|
||||
|
||||
Identity layer (~50 tokens). Reads from `~/.mempalace/identity.txt`.
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `render()` | `str` | Identity text or default message |
|
||||
| `token_estimate()` | `int` | Approximate token count (`len(text) // 4`) |
|
||||
|
||||
---
|
||||
|
||||
### `class Layer1(palace_path=None, wing=None)`
|
||||
|
||||
Essential story layer (~500–800 tokens). Auto-generated from top drawers.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `MAX_DRAWERS` | `int` | Max moments in wake-up (15) |
|
||||
| `MAX_CHARS` | `int` | Hard cap on L1 text (3200) |
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `generate()` | `str` | Compact L1 text grouped by room |
|
||||
|
||||
---
|
||||
|
||||
### `class Layer2(palace_path=None)`
|
||||
|
||||
On-demand retrieval layer (~200–500 tokens per call).
|
||||
|
||||
| Method | Parameters | Returns |
|
||||
|--------|-----------|---------|
|
||||
| `retrieve(wing, room, n_results=10)` | Wing/room filters | Formatted drawer text |
|
||||
|
||||
---
|
||||
|
||||
### `class Layer3(palace_path=None)`
|
||||
|
||||
Deep semantic search layer (unlimited depth).
|
||||
|
||||
| Method | Parameters | Returns |
|
||||
|--------|-----------|---------|
|
||||
| `search(query, wing=None, room=None, n_results=5)` | Query + optional filters | Formatted result text |
|
||||
| `search_raw(query, wing=None, room=None, n_results=5)` | Query + optional filters | List of result dicts |
|
||||
|
||||
---
|
||||
|
||||
### `class MemoryStack(palace_path=None, identity_path=None)`
|
||||
|
||||
Unified 4-layer interface.
|
||||
|
||||
| Method | Parameters | Returns | Description |
|
||||
|--------|-----------|---------|-------------|
|
||||
| `wake_up(wing=None)` | Optional wing | `str` | L0 + L1 context (~170–900 tokens) |
|
||||
| `recall(wing, room, n_results=10)` | Filters | `str` | L2 on-demand retrieval |
|
||||
| `search(query, wing, room, n_results=5)` | Query + filters | `str` | L3 deep search |
|
||||
| `status()` | — | `dict` | All layer status info |
|
||||
|
||||
---
|
||||
|
||||
## `mempalace.knowledge_graph`
|
||||
|
||||
### `class KnowledgeGraph(db_path=None)`
|
||||
|
||||
Default path: `~/.mempalace/knowledge_graph.sqlite3`
|
||||
|
||||
#### Write Methods
|
||||
|
||||
| Method | Parameters | Returns | Description |
|
||||
|--------|-----------|---------|-------------|
|
||||
| `add_entity(name, entity_type='unknown', properties=None)` | Name, type, props dict | `str` (entity ID) | Add or update entity node |
|
||||
| `add_triple(subject, predicate, obj, valid_from, valid_to, confidence, source_closet, source_file)` | See below | `str` (triple ID) | Add relationship triple |
|
||||
| `invalidate(subject, predicate, obj, ended=None)` | Entity names, end date | — | Mark relationship as ended |
|
||||
|
||||
**`add_triple` parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `subject` | `str` | — | Source entity name |
|
||||
| `predicate` | `str` | — | Relationship type |
|
||||
| `obj` | `str` | — | Target entity name |
|
||||
| `valid_from` | `str` | `None` | Start date (YYYY-MM-DD) |
|
||||
| `valid_to` | `str` | `None` | End date |
|
||||
| `confidence` | `float` | `1.0` | Confidence score 0.0–1.0 |
|
||||
| `source_closet` | `str` | `None` | Link to verbatim memory |
|
||||
| `source_file` | `str` | `None` | Original source file |
|
||||
|
||||
#### Query Methods
|
||||
|
||||
| Method | Parameters | Returns |
|
||||
|--------|-----------|---------|
|
||||
| `query_entity(name, as_of=None, direction='outgoing')` | Entity name, date filter, direction | `list[dict]` |
|
||||
| `query_relationship(predicate, as_of=None)` | Relationship type, date filter | `list[dict]` |
|
||||
| `timeline(entity_name=None)` | Optional entity filter | `list[dict]` |
|
||||
| `stats()` | — | `dict` with entities, triples, predicates |
|
||||
| `seed_from_entity_facts(entity_facts)` | Dict of entity facts | — |
|
||||
|
||||
**`query_entity` direction values:** `"outgoing"` (entity→?), `"incoming"` (?→entity), `"both"`
|
||||
|
||||
---
|
||||
|
||||
## `mempalace.palace_graph`
|
||||
|
||||
### `build_graph(col=None, config=None) → (nodes, edges)`
|
||||
|
||||
Build the palace graph from ChromaDB metadata.
|
||||
|
||||
**Returns:**
|
||||
- `nodes`: `dict` of `{room: {wings: list, halls: list, count: int, dates: list}}`
|
||||
- `edges`: `list` of `{room, wing_a, wing_b, hall, count}`
|
||||
|
||||
---
|
||||
|
||||
### `traverse(start_room, col=None, config=None, max_hops=2) → list`
|
||||
|
||||
BFS graph traversal from a room across wings.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `start_room` | `str` | — | Room slug to start from |
|
||||
| `max_hops` | `int` | `2` | Max connection depth |
|
||||
|
||||
**Returns:** `[{room, wings, halls, count, hop, connected_via}]` (max 50)
|
||||
|
||||
---
|
||||
|
||||
### `find_tunnels(wing_a=None, wing_b=None, col=None, config=None) → list`
|
||||
|
||||
Find rooms spanning multiple wings.
|
||||
|
||||
**Returns:** `[{room, wings, halls, count, recent}]` (max 50)
|
||||
|
||||
---
|
||||
|
||||
### `graph_stats(col=None, config=None) → dict`
|
||||
|
||||
**Returns:** `{total_rooms, tunnel_rooms, total_edges, rooms_per_wing, top_tunnels}`
|
||||
|
||||
---
|
||||
|
||||
## `mempalace.dialect`
|
||||
|
||||
### `class Dialect(entities=None, skip_names=None)`
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `entities` | `dict[str, str]` | Full name → 3-letter code mapping |
|
||||
| `skip_names` | `list[str]` | Names to skip (fictional characters, etc.) |
|
||||
|
||||
#### Class Methods
|
||||
|
||||
| Method | Parameters | Returns |
|
||||
|--------|-----------|---------|
|
||||
| `from_config(config_path)` | JSON file path | `Dialect` instance |
|
||||
|
||||
#### Instance Methods
|
||||
|
||||
| Method | Parameters | Returns | Description |
|
||||
|--------|-----------|---------|-------------|
|
||||
| `compress(text, metadata=None)` | Plain text + optional metadata dict | `str` | AAAK-formatted summary |
|
||||
| `encode_entity(name)` | Entity name | `str \| None` | 3-letter entity code |
|
||||
| `encode_emotions(emotions)` | List of emotion strings | `str` | Compact emotion codes |
|
||||
| `compress_file(path, output=None)` | Zettel JSON path | `str` | Compress zettel file |
|
||||
| `compress_all(dir, output=None)` | Zettel directory | `str` | Compress all zettels |
|
||||
| `save_config(path)` | Output path | — | Save entity mappings |
|
||||
| `compression_stats(original, compressed)` | Both texts | `dict` | Compression ratio stats |
|
||||
|
||||
#### Static Methods
|
||||
|
||||
| Method | Parameters | Returns |
|
||||
|--------|-----------|---------|
|
||||
| `count_tokens(text)` | Any text | `int` |
|
||||
|
||||
---
|
||||
|
||||
## `mempalace.config`
|
||||
|
||||
### `class MempalaceConfig()`
|
||||
|
||||
Reads from `~/.mempalace/config.json` and environment variables.
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `palace_path` | `str` | `~/.mempalace/palace` | ChromaDB storage path |
|
||||
| `collection_name` | `str` | `mempalace_drawers` | ChromaDB collection name |
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `init()` | Create config directory and default files |
|
||||
@@ -0,0 +1,95 @@
|
||||
# Benchmarks
|
||||
|
||||
Curated summary of MemPalace benchmark results. For the full 725-line progression with every experiment, see [`benchmarks/BENCHMARKS.md`](https://github.com/milla-jovovich/mempalace/blob/main/benchmarks/BENCHMARKS.md) in the repository.
|
||||
|
||||
## The Core Finding
|
||||
|
||||
MemPalace's benchmarked raw baseline stores the source text and searches it with ChromaDB's default embeddings. No extraction layer or summarization step is required for that baseline.
|
||||
|
||||
**And it scores 96.6% on LongMemEval.**
|
||||
|
||||
## LongMemEval Results
|
||||
|
||||
| Mode | R@5 | LLM Required | Cost/query |
|
||||
|------|-----|-------------|------------|
|
||||
| Raw ChromaDB | **96.6%** | None | $0 |
|
||||
| Hybrid v3 + rerank | 99.4% | Haiku | ~$0.001 |
|
||||
| Palace + rerank | 99.4% | Haiku | ~$0.001 |
|
||||
| **Hybrid v4 + rerank** | **100%** | Haiku | ~$0.001 |
|
||||
|
||||
The 96.6% raw score requires no API key, no cloud, and no LLM at any stage. The 100% result uses optional Haiku reranking.
|
||||
|
||||
### Per-Category Breakdown (Raw, 96.6%)
|
||||
|
||||
| Question Type | R@5 | Count |
|
||||
|---------------|-----|-------|
|
||||
| Knowledge update | 99.0% | 78 |
|
||||
| Multi-session | 98.5% | 133 |
|
||||
| Temporal reasoning | 96.2% | 133 |
|
||||
| Single-session user | 95.7% | 70 |
|
||||
| Single-session preference | 93.3% | 30 |
|
||||
| Single-session assistant | 92.9% | 56 |
|
||||
|
||||
### Held-Out Validation
|
||||
|
||||
**98.4% R@5** on 450 questions that hybrid_v4 was never tuned on — confirming the improvements generalize.
|
||||
|
||||
## Comparison vs Published Systems
|
||||
|
||||
| System | LongMemEval R@5 | API Required | Cost |
|
||||
|--------|----------------|--------------|------|
|
||||
| **MemPalace (hybrid)** | **100%** | Optional | Free |
|
||||
| Supermemory ASMR | ~99% | Yes | — |
|
||||
| **MemPalace (raw)** | **96.6%** | **None** | **Free** |
|
||||
| Mastra | 94.87% | Yes | API costs |
|
||||
| Hindsight | 91.4% | Yes | API costs |
|
||||
| Mem0 | ~85% | Yes | $19–249/mo |
|
||||
|
||||
## Other Benchmarks
|
||||
|
||||
### ConvoMem (Salesforce, 75K+ QA pairs)
|
||||
|
||||
| System | Score |
|
||||
|--------|-------|
|
||||
| **MemPalace** | **92.9%** |
|
||||
| Gemini (long context) | 70–82% |
|
||||
| Block extraction | 57–71% |
|
||||
| Mem0 (RAG) | 30–45% |
|
||||
|
||||
On this benchmark, MemPalace materially outperforms the Mem0 result cited in the comparison table.
|
||||
|
||||
### LoCoMo (1,986 multi-hop QA pairs)
|
||||
|
||||
| Mode | R@10 | LLM |
|
||||
|------|------|-----|
|
||||
| Hybrid v5 + Sonnet rerank (top-50) | **100%** | Sonnet |
|
||||
| bge-large + Haiku rerank (top-15) | 96.3% | Haiku |
|
||||
| Hybrid v5 (top-10, no rerank) | **88.9%** | None |
|
||||
| Session, no rerank (top-10) | 60.3% | None |
|
||||
|
||||
### MemBench (ACL 2025, 8,500 items)
|
||||
|
||||
**80.3% R@5** overall. Strongest categories: aggregative (99.3%), comparative (98.4%), lowlevel_rec (99.8%).
|
||||
|
||||
## Reproducing Results
|
||||
|
||||
All benchmarks are reproducible with public datasets:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/milla-jovovich/mempalace.git
|
||||
cd mempalace
|
||||
pip install chromadb pyyaml
|
||||
|
||||
# Download LongMemEval data
|
||||
curl -fsSL -o /tmp/longmemeval_s_cleaned.json \
|
||||
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json
|
||||
|
||||
# Run raw baseline (96.6%, no API key needed)
|
||||
python benchmarks/longmemeval_bench.py /tmp/longmemeval_s_cleaned.json
|
||||
```
|
||||
|
||||
::: tip
|
||||
Results are deterministic. Same data + same script = same result every time. Every result JSONL file contains every question, every retrieved document, every score.
|
||||
:::
|
||||
|
||||
For complete reproduction instructions, benchmark integrity notes, and the full score progression, see the [full benchmark documentation](https://github.com/milla-jovovich/mempalace/blob/main/benchmarks/BENCHMARKS.md).
|
||||
@@ -0,0 +1,156 @@
|
||||
# CLI Commands
|
||||
|
||||
All commands accept `--palace <path>` to override the default palace location.
|
||||
|
||||
## `mempalace init`
|
||||
|
||||
Detect rooms from your folder structure and set up the palace.
|
||||
|
||||
```bash
|
||||
mempalace init <dir>
|
||||
mempalace init <dir> --yes # non-interactive mode
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `<dir>` | Project directory to scan |
|
||||
| `--yes` | Auto-accept all detected entities |
|
||||
|
||||
What it does:
|
||||
1. Scans for people and projects in file content
|
||||
2. Detects rooms from folder structure
|
||||
3. Creates `~/.mempalace/` config directory
|
||||
4. Saves detected entities to `<dir>/entities.json`
|
||||
|
||||
## `mempalace mine`
|
||||
|
||||
Mine files into the palace.
|
||||
|
||||
```bash
|
||||
mempalace mine <dir>
|
||||
mempalace mine <dir> --mode convos
|
||||
mempalace mine <dir> --mode convos --extract general
|
||||
mempalace mine <dir> --wing myapp
|
||||
```
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `<dir>` | — | Directory to mine |
|
||||
| `--mode` | `projects` | `projects` for code/docs, `convos` for chat exports |
|
||||
| `--wing` | directory name | Wing name override |
|
||||
| `--agent` | `mempalace` | Agent name tag |
|
||||
| `--limit` | `0` (all) | Max files to process |
|
||||
| `--dry-run` | — | Preview without filing |
|
||||
| `--extract` | `exchange` | `exchange` or `general` (for convos mode) |
|
||||
| `--no-gitignore` | — | Don't respect .gitignore |
|
||||
| `--include-ignored` | — | Always scan these paths even if ignored |
|
||||
|
||||
## `mempalace search`
|
||||
|
||||
Find anything by semantic search.
|
||||
|
||||
```bash
|
||||
mempalace search "query"
|
||||
mempalace search "query" --wing myapp
|
||||
mempalace search "query" --wing myapp --room auth
|
||||
mempalace search "query" --results 10
|
||||
```
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `"query"` | — | What to search for |
|
||||
| `--wing` | all | Filter by wing |
|
||||
| `--room` | all | Filter by room |
|
||||
| `--results` | `5` | Number of results |
|
||||
|
||||
## `mempalace split`
|
||||
|
||||
Split concatenated transcript mega-files into per-session files.
|
||||
|
||||
```bash
|
||||
mempalace split <dir>
|
||||
mempalace split <dir> --dry-run
|
||||
mempalace split <dir> --min-sessions 3
|
||||
mempalace split <dir> --output-dir ~/split-output/
|
||||
```
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `<dir>` | — | Directory with transcript files |
|
||||
| `--output-dir` | same dir | Write split files here |
|
||||
| `--dry-run` | — | Preview without writing |
|
||||
| `--min-sessions` | `2` | Only split files with N+ sessions |
|
||||
|
||||
## `mempalace wake-up`
|
||||
|
||||
Show L0 + L1 wake-up context (~170–900 tokens).
|
||||
|
||||
```bash
|
||||
mempalace wake-up
|
||||
mempalace wake-up --wing driftwood
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--wing` | Project-specific wake-up |
|
||||
|
||||
## `mempalace compress`
|
||||
|
||||
Compress drawers using AAAK Dialect.
|
||||
|
||||
```bash
|
||||
mempalace compress --wing myapp
|
||||
mempalace compress --wing myapp --dry-run
|
||||
mempalace compress --config entities.json
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--wing` | Wing to compress (default: all) |
|
||||
| `--dry-run` | Preview without storing |
|
||||
| `--config` | Entity config JSON file |
|
||||
|
||||
## `mempalace status`
|
||||
|
||||
Show what's been filed — drawer count, wing/room breakdown.
|
||||
|
||||
```bash
|
||||
mempalace status
|
||||
```
|
||||
|
||||
## `mempalace repair`
|
||||
|
||||
Rebuild palace vector index from stored data. Fixes segfaults after database corruption.
|
||||
|
||||
```bash
|
||||
mempalace repair
|
||||
```
|
||||
|
||||
Creates a backup at `<palace_path>.backup` before rebuilding.
|
||||
|
||||
## `mempalace hook`
|
||||
|
||||
Run hook logic for Claude Code / Codex integration.
|
||||
|
||||
```bash
|
||||
mempalace hook run --hook stop --harness claude-code
|
||||
mempalace hook run --hook precompact --harness claude-code
|
||||
mempalace hook run --hook session-start --harness codex
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
|--------|--------|-------------|
|
||||
| `--hook` | `session-start`, `stop`, `precompact` | Hook name |
|
||||
| `--harness` | `claude-code`, `codex` | Harness type |
|
||||
|
||||
## `mempalace instructions`
|
||||
|
||||
Output skill instructions to stdout.
|
||||
|
||||
```bash
|
||||
mempalace instructions init
|
||||
mempalace instructions search
|
||||
mempalace instructions mine
|
||||
mempalace instructions help
|
||||
mempalace instructions status
|
||||
```
|
||||
@@ -0,0 +1,81 @@
|
||||
# Contributing
|
||||
|
||||
PRs welcome. MemPalace is open source and we welcome contributions of all sizes — from typo fixes to new features.
|
||||
|
||||
## Getting Started
|
||||
|
||||
```bash
|
||||
git clone https://github.com/milla-jovovich/mempalace.git
|
||||
cd mempalace
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
All tests must pass before submitting a PR. Tests should run without API keys or network access.
|
||||
|
||||
## Running Benchmarks
|
||||
|
||||
```bash
|
||||
# Quick test (20 questions, ~30 seconds)
|
||||
python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json --limit 20
|
||||
|
||||
# Full benchmark (500 questions, ~5 minutes)
|
||||
python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json
|
||||
```
|
||||
|
||||
See [Benchmarks](/reference/benchmarks) for data download instructions.
|
||||
|
||||
## PR Guidelines
|
||||
|
||||
1. Fork the repo and create a feature branch: `git checkout -b feat/my-thing`
|
||||
2. Write your code
|
||||
3. Add or update tests if applicable
|
||||
4. Run `pytest tests/ -v` — everything must pass
|
||||
5. Commit with clear [conventional commits](https://www.conventionalcommits.org/):
|
||||
- `feat: add Notion export format`
|
||||
- `fix: handle empty transcript files`
|
||||
- `docs: update MCP tool descriptions`
|
||||
- `bench: add LoCoMo turn-level metrics`
|
||||
6. Push to your fork and open a PR against `main`
|
||||
|
||||
## Code Style
|
||||
|
||||
- **Formatting**: [Ruff](https://docs.astral.sh/ruff/) with 100-char line limit
|
||||
- **Naming**: `snake_case` for functions/variables, `PascalCase` for classes
|
||||
- **Docstrings**: on all modules and public functions
|
||||
- **Type hints**: where they improve readability
|
||||
- **Dependencies**: minimize — ChromaDB + PyYAML only. Don't add new deps without discussion.
|
||||
|
||||
## Good First Issues
|
||||
|
||||
Check the [Issues](https://github.com/milla-jovovich/mempalace/issues) tab:
|
||||
|
||||
- **New chat formats** — add import support for Cursor, Copilot, or other AI tool exports
|
||||
- **Room detection** — improve pattern matching in `room_detector_local.py`
|
||||
- **Tests** — increase coverage, especially for `knowledge_graph.py` and `palace_graph.py`
|
||||
- **Entity detection** — better name disambiguation in `entity_detector.py`
|
||||
- **Docs** — improve examples, add tutorials
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
If you're planning a significant change, open an issue first. Key principles:
|
||||
|
||||
- **Verbatim first** — never summarize user content. Store exact words.
|
||||
- **Local first** — everything runs on the user's machine. No cloud dependencies.
|
||||
- **Zero API by default** — core features must work without any API key.
|
||||
- **Palace structure matters** — wings, halls, and rooms aren't cosmetic — they drive a 34% retrieval improvement.
|
||||
|
||||
## Community
|
||||
|
||||
- [Discord](https://discord.com/invite/ycTQQCu6kn)
|
||||
- [GitHub Issues](https://github.com/milla-jovovich/mempalace/issues) — bug reports and feature requests
|
||||
- [GitHub Discussions](https://github.com/milla-jovovich/mempalace/discussions) — questions and ideas
|
||||
|
||||
## License
|
||||
|
||||
MIT — your contributions will be released under the same license.
|
||||
@@ -0,0 +1,249 @@
|
||||
# MCP Tools Reference
|
||||
|
||||
Detailed parameter schemas for all 19 MCP tools.
|
||||
|
||||
## Palace — Read Tools
|
||||
|
||||
### `mempalace_status`
|
||||
|
||||
Palace overview: total drawers, wing and room counts, AAAK spec, and memory protocol.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ total_drawers, wings, rooms, palace_path, protocol, aaak_dialect }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_list_wings`
|
||||
|
||||
List all wings with drawer counts.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ wings: { "wing_name": count } }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_list_rooms`
|
||||
|
||||
List rooms within a wing (or all rooms if no wing given).
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `wing` | string | No | Wing to list rooms for |
|
||||
|
||||
**Returns:** `{ wing, rooms: { "room_name": count } }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_get_taxonomy`
|
||||
|
||||
Full wing → room → drawer count tree.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ taxonomy: { "wing": { "room": count } } }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_search`
|
||||
|
||||
Semantic search. Returns verbatim drawer content with similarity scores.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `query` | string | **Yes** | What to search for |
|
||||
| `limit` | integer | No | Max results (default: 5) |
|
||||
| `wing` | string | No | Filter by wing |
|
||||
| `room` | string | No | Filter by room |
|
||||
|
||||
**Returns:** `{ query, filters, results: [{ text, wing, room, source_file, similarity }] }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_check_duplicate`
|
||||
|
||||
Check if content already exists in the palace before filing.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `content` | string | **Yes** | Content to check |
|
||||
| `threshold` | number | No | Similarity threshold 0–1 (default: 0.9) |
|
||||
|
||||
**Returns:** `{ is_duplicate, matches: [{ id, wing, room, similarity, content }] }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_get_aaak_spec`
|
||||
|
||||
Returns the AAAK dialect specification.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ aaak_spec: "..." }`
|
||||
|
||||
---
|
||||
|
||||
## Palace — Write Tools
|
||||
|
||||
### `mempalace_add_drawer`
|
||||
|
||||
File verbatim content into the palace. Identical content (same deterministic drawer ID) is silently skipped. For similarity-based duplicate detection before filing, use `mempalace_check_duplicate`.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `wing` | string | **Yes** | Wing (project name) |
|
||||
| `room` | string | **Yes** | Room (aspect: backend, decisions, etc.) |
|
||||
| `content` | string | **Yes** | Verbatim content to store |
|
||||
| `source_file` | string | No | Where this came from |
|
||||
| `added_by` | string | No | Who is filing (default: "mcp") |
|
||||
|
||||
**Returns:** `{ success, drawer_id, wing, room }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_delete_drawer`
|
||||
|
||||
Delete a drawer by ID. Irreversible.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `drawer_id` | string | **Yes** | ID of the drawer to delete |
|
||||
|
||||
**Returns:** `{ success, drawer_id }`
|
||||
|
||||
---
|
||||
|
||||
## Knowledge Graph Tools
|
||||
|
||||
### `mempalace_kg_query`
|
||||
|
||||
Query entity relationships with time filtering.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `entity` | string | **Yes** | Entity to query (e.g. "Max", "MyProject") |
|
||||
| `as_of` | string | No | Date filter — only facts valid at this date (YYYY-MM-DD) |
|
||||
| `direction` | string | No | `outgoing`, `incoming`, or `both` (default: `both`) |
|
||||
|
||||
**Returns:** `{ entity, as_of, facts: [{ direction, subject, predicate, object, valid_from, valid_to, current }], count }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_kg_add`
|
||||
|
||||
Add a fact to the knowledge graph.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `subject` | string | **Yes** | The entity doing/being something |
|
||||
| `predicate` | string | **Yes** | Relationship type (e.g. "loves", "works_on") |
|
||||
| `object` | string | **Yes** | The entity being connected to |
|
||||
| `valid_from` | string | No | When this became true (YYYY-MM-DD) |
|
||||
| `source_closet` | string | No | Closet ID where this fact appears |
|
||||
|
||||
**Returns:** `{ success, triple_id, fact }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_kg_invalidate`
|
||||
|
||||
Mark a fact as no longer true.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `subject` | string | **Yes** | Entity |
|
||||
| `predicate` | string | **Yes** | Relationship |
|
||||
| `object` | string | **Yes** | Connected entity |
|
||||
| `ended` | string | No | When it stopped being true (default: today) |
|
||||
|
||||
**Returns:** `{ success, fact, ended }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_kg_timeline`
|
||||
|
||||
Chronological timeline of facts.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `entity` | string | No | Entity to get timeline for (omit for full timeline) |
|
||||
|
||||
**Returns:** `{ entity, timeline: [{ subject, predicate, object, valid_from, valid_to, current }], count }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_kg_stats`
|
||||
|
||||
Knowledge graph overview.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ entities, triples, current_facts, expired_facts, relationship_types }`
|
||||
|
||||
---
|
||||
|
||||
## Navigation Tools
|
||||
|
||||
### `mempalace_traverse`
|
||||
|
||||
Walk the palace graph from a room. Find connected ideas across wings.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `start_room` | string | **Yes** | Room to start from |
|
||||
| `max_hops` | integer | No | How many connections to follow (default: 2) |
|
||||
|
||||
**Returns:** `[{ room, wings, halls, count, hop, connected_via }]`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_find_tunnels`
|
||||
|
||||
Find rooms that bridge two wings.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `wing_a` | string | No | First wing |
|
||||
| `wing_b` | string | No | Second wing |
|
||||
|
||||
**Returns:** `[{ room, wings, halls, count, recent }]`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_graph_stats`
|
||||
|
||||
Palace graph overview: nodes, tunnels, edges, connectivity.
|
||||
|
||||
**Parameters:** None
|
||||
|
||||
**Returns:** `{ total_rooms, tunnel_rooms, total_edges, rooms_per_wing, top_tunnels }`
|
||||
|
||||
---
|
||||
|
||||
## Agent Diary Tools
|
||||
|
||||
### `mempalace_diary_write`
|
||||
|
||||
Write to your personal agent diary.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `agent_name` | string | **Yes** | Your name — each agent gets its own wing |
|
||||
| `entry` | string | **Yes** | Diary entry (in AAAK format recommended) |
|
||||
| `topic` | string | No | Topic tag (default: "general") |
|
||||
|
||||
**Returns:** `{ success, entry_id, agent, topic, timestamp }`
|
||||
|
||||
---
|
||||
|
||||
### `mempalace_diary_read`
|
||||
|
||||
Read recent diary entries.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `agent_name` | string | **Yes** | Your name |
|
||||
| `last_n` | integer | No | Number of recent entries (default: 10) |
|
||||
|
||||
**Returns:** `{ agent, entries: [{ date, timestamp, topic, content }], total, showing }`
|
||||
@@ -0,0 +1,129 @@
|
||||
# Module Map
|
||||
|
||||
Complete source file reference for the MemPalace codebase.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
mempalace/
|
||||
├── README.md ← project documentation
|
||||
├── mempalace/ ← core package
|
||||
│ ├── cli.py ← CLI entry point
|
||||
│ ├── mcp_server.py ← MCP server (19 tools)
|
||||
│ ├── knowledge_graph.py ← temporal entity graph
|
||||
│ ├── palace_graph.py ← room navigation graph
|
||||
│ ├── dialect.py ← AAAK compression
|
||||
│ ├── miner.py ← project file ingest
|
||||
│ ├── convo_miner.py ← conversation ingest
|
||||
│ ├── searcher.py ← semantic search
|
||||
│ ├── layers.py ← 4-layer memory stack
|
||||
│ ├── onboarding.py ← guided setup
|
||||
│ ├── config.py ← configuration loading
|
||||
│ ├── normalize.py ← chat format converter
|
||||
│ ├── entity_detector.py ← auto-detect people/projects
|
||||
│ ├── entity_registry.py ← entity code registry
|
||||
│ ├── room_detector_local.py ← room detection from directories
|
||||
│ ├── general_extractor.py ← 5-type memory extraction
|
||||
│ ├── split_mega_files.py ← transcript splitting
|
||||
│ ├── spellcheck.py ← optional spell checking
|
||||
│ ├── hooks_cli.py ← hook logic
|
||||
│ ├── instructions_cli.py ← skill instructions
|
||||
│ └── version.py ← version string
|
||||
├── benchmarks/ ← reproducible benchmark runners
|
||||
│ ├── BENCHMARKS.md ← full results + methodology
|
||||
│ ├── longmemeval_bench.py ← LongMemEval runner
|
||||
│ ├── locomo_bench.py ← LoCoMo runner
|
||||
│ ├── membench_bench.py ← MemBench runner
|
||||
│ └── convomem_bench.py ← ConvoMem runner
|
||||
├── hooks/ ← Claude Code auto-save hooks
|
||||
│ ├── mempal_save_hook.sh ← save every N messages
|
||||
│ └── mempal_precompact_hook.sh ← emergency save
|
||||
├── examples/ ← usage examples
|
||||
│ ├── basic_mining.py
|
||||
│ ├── convo_import.py
|
||||
│ ├── mcp_setup.md
|
||||
│ └── gemini_cli_setup.md
|
||||
├── tests/ ← test suite
|
||||
├── assets/ ← logo + brand
|
||||
└── pyproject.toml ← package config
|
||||
```
|
||||
|
||||
## Core Modules
|
||||
|
||||
### `cli.py` — CLI Entry Point
|
||||
|
||||
Argparse-based CLI with subcommands: `init`, `mine`, `split`, `search`, `compress`, `wake-up`, `repair`, `status`, `hook`, `instructions`. Dispatches to the corresponding module.
|
||||
|
||||
### `mcp_server.py` — MCP Server
|
||||
|
||||
JSON-RPC over stdin/stdout. Implements the MCP protocol with 19 tools covering palace read/write, knowledge graph, navigation, and agent diary operations. Includes the Memory Protocol and AAAK Spec in status responses.
|
||||
|
||||
### `searcher.py` — Semantic Search
|
||||
|
||||
Two functions: `search()` for CLI output and `search_memories()` for programmatic use. Both query ChromaDB with optional wing/room filters and return verbatim drawer content with similarity scores.
|
||||
|
||||
### `layers.py` — Memory Stack
|
||||
|
||||
Four classes (`Layer0` through `Layer3`) and the unified `MemoryStack`. Layer 0 reads identity, Layer 1 auto-generates from top drawers, Layer 2 does filtered retrieval, Layer 3 does semantic search.
|
||||
|
||||
### `knowledge_graph.py` — Temporal KG
|
||||
|
||||
SQLite-backed entity-relationship graph with temporal validity windows. Supports add, invalidate, query, timeline, and stats. Auto-creates entities on triple insertion.
|
||||
|
||||
### `palace_graph.py` — Navigation Graph
|
||||
|
||||
Builds a graph from ChromaDB metadata where nodes = rooms and edges = tunnels (rooms spanning multiple wings). Supports BFS traversal and tunnel finding.
|
||||
|
||||
### `dialect.py` — AAAK Compression
|
||||
|
||||
Lossy abbreviation system with entity encoding, emotion detection, topic extraction, and flag identification. Works on both plain text and structured zettel data.
|
||||
|
||||
## Ingest Modules
|
||||
|
||||
### `miner.py` — Project Ingest
|
||||
|
||||
Scans project directories for code and doc files. Respects `.gitignore`. Files content as drawers tagged with wing/room metadata.
|
||||
|
||||
### `convo_miner.py` — Conversation Ingest
|
||||
|
||||
Imports conversation exports (Claude, ChatGPT, Slack, Markdown, plaintext). Chunks by exchange pair. Supports `exchange` and `general` extraction modes.
|
||||
|
||||
### `normalize.py` — Format Converter
|
||||
|
||||
Converts 5 chat formats to a standard transcript format before mining.
|
||||
|
||||
### `general_extractor.py` — Memory Type Extraction
|
||||
|
||||
Classifies conversation content into decisions, preferences, milestones, problems, and emotional context.
|
||||
|
||||
## Detection Modules
|
||||
|
||||
### `entity_detector.py` — Entity Detection
|
||||
|
||||
Scans file content to auto-detect people and projects using regex patterns and heuristics.
|
||||
|
||||
### `entity_registry.py` — Entity Registry
|
||||
|
||||
Manages entity name → code mappings for AAAK dialect.
|
||||
|
||||
### `room_detector_local.py` — Room Detection
|
||||
|
||||
Detects rooms from folder structure during `mempalace init`.
|
||||
|
||||
## Utility Modules
|
||||
|
||||
### `config.py` — Configuration
|
||||
|
||||
Loads settings from `~/.mempalace/config.json` and environment variables.
|
||||
|
||||
### `split_mega_files.py` — Transcript Splitting
|
||||
|
||||
Splits concatenated transcripts into per-session files based on session boundary detection.
|
||||
|
||||
### `onboarding.py` — Guided Setup
|
||||
|
||||
Interactive setup wizard for `mempalace init`. Generates AAAK bootstrap and wing config.
|
||||
|
||||
### `spellcheck.py` — Spell Checking
|
||||
|
||||
Optional spell checking utility (requires `autocorrect` package).
|
||||
@@ -0,0 +1,121 @@
|
||||
# Python API
|
||||
|
||||
High-level overview of the key Python interfaces you'd use to integrate MemPalace into your application.
|
||||
|
||||
## Search
|
||||
|
||||
The primary way to query the palace programmatically.
|
||||
|
||||
```python
|
||||
from mempalace.searcher import search_memories
|
||||
|
||||
results = search_memories(
|
||||
query="why did we switch to GraphQL",
|
||||
wing="myapp", # optional filter
|
||||
room="architecture", # optional filter
|
||||
n_results=5,
|
||||
)
|
||||
|
||||
# Results structure:
|
||||
# {
|
||||
# "query": "...",
|
||||
# "filters": {"wing": "myapp", "room": "architecture"},
|
||||
# "results": [
|
||||
# {"text": "...", "wing": "...", "room": "...", "source_file": "...", "similarity": 0.89}
|
||||
# ]
|
||||
# }
|
||||
```
|
||||
|
||||
## Memory Stack
|
||||
|
||||
The 4-layer memory system with a unified interface.
|
||||
|
||||
```python
|
||||
from mempalace.layers import MemoryStack
|
||||
|
||||
stack = MemoryStack() # uses default paths from MempalaceConfig
|
||||
|
||||
# Wake-up: L0 (identity) + L1 (essential story)
|
||||
context = stack.wake_up(wing="myapp") # ~170-900 tokens
|
||||
|
||||
# On-demand: L2 retrieval
|
||||
recall = stack.recall(wing="myapp", room="auth", n_results=10)
|
||||
|
||||
# Deep search: L3 semantic search
|
||||
results = stack.search("pricing change", wing="myapp")
|
||||
|
||||
# Status
|
||||
status = stack.status()
|
||||
```
|
||||
|
||||
## Knowledge Graph
|
||||
|
||||
Temporal entity-relationship graph built on SQLite.
|
||||
|
||||
```python
|
||||
from mempalace.knowledge_graph import KnowledgeGraph
|
||||
|
||||
kg = KnowledgeGraph() # uses default path: ~/.mempalace/knowledge_graph.sqlite3
|
||||
|
||||
# Write
|
||||
kg.add_entity("Kai", entity_type="person")
|
||||
kg.add_triple("Kai", "works_on", "Orion", valid_from="2025-06-01")
|
||||
kg.invalidate("Kai", "works_on", "Orion", ended="2026-03-01")
|
||||
|
||||
# Read
|
||||
facts = kg.query_entity("Kai", as_of="2026-01-15", direction="both")
|
||||
relationships = kg.query_relationship("works_on")
|
||||
timeline = kg.timeline("Orion")
|
||||
stats = kg.stats()
|
||||
```
|
||||
|
||||
## Palace Graph
|
||||
|
||||
Room-based navigation graph built from ChromaDB metadata.
|
||||
|
||||
```python
|
||||
from mempalace.palace_graph import build_graph, traverse, find_tunnels, graph_stats
|
||||
|
||||
# Build the graph
|
||||
nodes, edges = build_graph()
|
||||
|
||||
# Navigate
|
||||
path = traverse("auth-migration", max_hops=2)
|
||||
tunnels = find_tunnels(wing_a="wing_code", wing_b="wing_team")
|
||||
stats = graph_stats()
|
||||
```
|
||||
|
||||
## AAAK Dialect
|
||||
|
||||
Lossy compression for token density at scale.
|
||||
|
||||
```python
|
||||
from mempalace.dialect import Dialect
|
||||
|
||||
# Basic
|
||||
dialect = Dialect()
|
||||
text = "We decided to use GraphQL because REST was too chatty for our dashboard."
|
||||
compressed = dialect.compress(text)
|
||||
|
||||
# With entity mappings
|
||||
dialect = Dialect(entities={"Alice": "ALC", "Bob": "BOB"})
|
||||
compressed = dialect.compress(text, metadata={"wing": "myapp"})
|
||||
|
||||
# From config
|
||||
dialect = Dialect.from_config("entities.json")
|
||||
|
||||
# Stats
|
||||
stats = dialect.compression_stats(text, compressed)
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
```python
|
||||
from mempalace.config import MempalaceConfig
|
||||
|
||||
config = MempalaceConfig()
|
||||
print(config.palace_path) # ~/.mempalace/palace
|
||||
print(config.collection_name) # mempalace_drawers
|
||||
```
|
||||
|
||||
For detailed parameter documentation, see [API Reference](/reference/api-reference).
|
||||
Reference in New Issue
Block a user