docs: #875 follow-up — repo surfaces + reproduction URLs + CHANGELOG
Remaining in-repo surfaces carrying the same retracted or broken claims as the public pages fixed in the previous two commits. CONTRIBUTING.md - "Palace structure matters ... 34% retrieval improvement" → reframed as scoping (same rewording applied to the website equivalents). benchmarks/BENCHMARKS.md - Add a prominent "Important caveat" block at the top of the "Comparison vs Published Systems" table explaining that R@5 (retrieval recall) and QA accuracy are different metrics, with citations to Mastra, Mem0, and Supermemory's own published methodology pages. Annotate the specific competitor rows whose numbers are QA accuracy, not retrieval recall. - Annotate the `hybrid v4 + rerank 100%` row to note that the 99.4 → 100 step was tuned on 3 specific wrong answers (already disclosed further down in the doc under "Benchmark Integrity"); the honest hybrid figure is held-out 98.4%. - Fix the broken clone URL — `aya-thekeeper/mempal` no longer points at anything; now `MemPalace/mempalace`. benchmarks/README.md + benchmarks/HYBRID_MODE.md - Same clone-URL fix applied. CHANGELOG.md - Add a ### Documentation entry under [Unreleased] v3.3.0 that names #875 and summarises the scope of the rewrite.
This commit is contained in:
@@ -196,9 +196,9 @@ python benchmarks/longmemeval_bench.py data/longmemeval_s_cleaned.json --mode hy
|
||||
|
||||
```bash
|
||||
# Setup
|
||||
git clone -b ben/benchmarking https://github.com/aya-thekeeper/mempal.git
|
||||
cd mempal
|
||||
pip install chromadb
|
||||
git clone https://github.com/MemPalace/mempalace.git
|
||||
cd mempalace
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Download data
|
||||
mkdir -p /tmp/longmemeval-data
|
||||
|
||||
Reference in New Issue
Block a user