Files

T

Igor Lins e Silva f20a1a30fe docs(website): align mempalaceofficial.com with honest benchmarks

Part of #875. Bring the VitePress site into line with the new README
and the reproducibility scorecard: drop category-error comparisons,
drop retracted claims, retain only metrics and caveats that survive
audit.

website/index.md
 - New tagline matches README (local-first, verbatim, pluggable backend,
   96.6% R@5 raw, zero API calls).
 - Replace the "MemPalace hybrid 100% / Supermemory ~99% / Mastra
   94.87% / Mem0 ~85%" comparison table with a single honest table
   showing MemPalace's own retrieval-recall numbers (raw 96.6%,
   hybrid v4 held-out 98.4%). Add an explicit sentence explaining why
   we no longer publish a cross-system table on the landing page
   (retrieval recall vs QA accuracy are different metrics).
 - Soften the "ChromaDB-powered vector search" feature blurb to be
   backend-agnostic, since the retrieval layer is pluggable.

website/reference/benchmarks.md
 - Full rewrite of the retrieval-recall tables. No more "100%"
   headline; honest held-out 98.4% R@5 replaces it. Added the
   model-agnostic rerank result (99.2% R@5 / 100% R@10 with
   minimax-m2.7 via Ollama) to show the pipeline is not Haiku-specific.
 - Drop the LoCoMo "Hybrid v5 + Sonnet rerank (top-50) 100%" row.
   With per-conversation session counts of 19-32 and top_k=50, the
   retrieval stage returns every session by construction — the number
   measures an LLM's reading comprehension, not retrieval.
 - Drop the cross-system comparison tables. Link out to each project's
   own research page (Mastra, Mem0, Supermemory) for their published
   numbers and metric definitions.
 - Rewrite reproduction commands to use the correct repository and
   demonstrate the new --llm-backend ollama flag.

website/concepts/the-palace.md
 - Remove the "+34%" row / paragraph. Wing/room filtering is standard
   metadata filtering in the vector store, not a novel retrieval
   mechanism — the April-7 note already retracted that framing; this
   finishes the retraction on the website where it had remained.

website/guide/searching.md
 - Same treatment for "34% retrieval improvement". Reframe as
   operational scoping, not a novel boost.

website/reference/contributing.md
 - Update the "palace structure matters" bullet to reflect the same
   framing: scoping-not-magic.

website/concepts/knowledge-graph.md
 - Replace the MemPalace-vs-Zep feature matrix with a short "related
   work" note that links to Zep's own documentation for authoritative
   details on their deployment model. Avoids claims we cannot verify
   at source.

2026-04-14 21:37:45 -03:00

3.0 KiB

Raw Blame History

Contributing

PRs welcome. MemPalace is open source and we welcome contributions of all sizes — from typo fixes to new features.

Getting Started

git clone https://github.com/MemPalace/mempalace.git
cd mempalace
pip install -e ".[dev]"

Running Tests

pytest tests/ -v

All tests must pass before submitting a PR. Tests should run without API keys or network access.

Running Benchmarks

# Quick test (20 questions, ~30 seconds)
python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json --limit 20

# Full benchmark (500 questions, ~5 minutes)
python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json

See Benchmarks for data download instructions.

PR Guidelines

Fork the repo and create a feature branch: git checkout -b feat/my-thing
Write your code
Add or update tests if applicable
Run pytest tests/ -v — everything must pass
Commit with clear conventional commits:
- feat: add Notion export format
- fix: handle empty transcript files
- docs: update MCP tool descriptions
- bench: add LoCoMo turn-level metrics
Push to your fork and open a PR against main

Code Style

Formatting: Ruff with 100-char line limit
Naming: snake_case for functions/variables, PascalCase for classes
Docstrings: on all modules and public functions
Type hints: where they improve readability
Dependencies: minimize — ChromaDB + PyYAML only. Don't add new deps without discussion.

Good First Issues

Check the Issues tab:

New chat formats — add import support for Cursor, Copilot, or other AI tool exports
Room detection — improve pattern matching in room_detector_local.py
Tests — increase coverage, especially for knowledge_graph.py and palace_graph.py
Entity detection — better name disambiguation in entity_detector.py
Docs — improve examples, add tutorials

Architecture Decisions

If you're planning a significant change, open an issue first. Key principles:

Verbatim first — never summarize user content. Store exact words.
Local first — everything runs on the user's machine. No cloud dependencies.
Zero API by default — core features must work without any API key.
Palace structure is scoping, not magic — wings, halls, and rooms act as metadata filters in the underlying vector store. They make scoping predictable when a palace holds many unrelated projects; they are not a novel retrieval mechanism.

Community

Discord
GitHub Issues — bug reports and feature requests
GitHub Discussions — questions and ideas

License

MIT — your contributions will be released under the same license.

3.0 KiB Raw Blame History