fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (#1090)

Opening chroma.sqlite3 via Python's sqlite3.connect() against a live
ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next
PersistentClient call — the same failure mode tracked at #1090.

_fix_blob_seq_ids runs unconditionally on every make_client() call, so
every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite
open → corrupt → segfault cycle on palaces that have already completed
the 0.6.x → 1.5.x seq_id migration.

Guard with a .blob_seq_ids_migrated marker file in the palace directory:
- If marker exists, return immediately — skip sqlite entirely
- After successful migration (or confirmation that no BLOBs remain),
  write the marker so subsequent opens take the fast path
- Palaces that never had BLOB seq_ids also get the marker on first open,
  so they too avoid the redundant sqlite open after that
- Already-migrated palaces can touch the marker manually to opt in

Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh
palace; second call returns immediately because marker exists. 1094
existing tests pass.
This commit is contained in:
jp
2026-04-24 13:35:07 -07:00
parent 0d9929c0dd
commit bc24aa14e2
+20
View File
@@ -159,6 +159,9 @@ def _pin_hnsw_threads(collection) -> None:
logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
def _fix_blob_seq_ids(palace_path: str) -> None:
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
@@ -168,10 +171,19 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
type INTEGER) is not compatible with SQL type BLOB".
Must run BEFORE PersistentClient is created (the compactor fires on init).
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
database leaves state that segfaults the next PersistentClient call. After
the migration has run once successfully, a marker file is written so
subsequent opens skip the sqlite connection entirely. Already-migrated
palaces can touch the marker manually to opt into the fast path.
"""
db_path = os.path.join(palace_path, "chroma.sqlite3")
if not os.path.isfile(db_path):
return
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
if os.path.isfile(marker):
return
try:
with sqlite3.connect(db_path) as conn:
for table in ("embeddings", "max_seq_id"):
@@ -189,6 +201,14 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
conn.commit()
except Exception:
logger.exception("Could not fix BLOB seq_ids in %s", db_path)
return
# Write marker whether or not rows needed migration — the palace is now
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
# sqlite3.connect() entirely.
try:
open(marker, "a").close()
except OSError:
logger.exception("Could not write migration marker %s", marker)
# ---------------------------------------------------------------------------