fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (#1090)
Opening chroma.sqlite3 via Python's sqlite3.connect() against a live ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next PersistentClient call — the same failure mode tracked at #1090. _fix_blob_seq_ids runs unconditionally on every make_client() call, so every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite open → corrupt → segfault cycle on palaces that have already completed the 0.6.x → 1.5.x seq_id migration. Guard with a .blob_seq_ids_migrated marker file in the palace directory: - If marker exists, return immediately — skip sqlite entirely - After successful migration (or confirmation that no BLOBs remain), write the marker so subsequent opens take the fast path - Palaces that never had BLOB seq_ids also get the marker on first open, so they too avoid the redundant sqlite open after that - Already-migrated palaces can touch the marker manually to opt in Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh palace; second call returns immediately because marker exists. 1094 existing tests pass.
This commit is contained in:
@@ -159,6 +159,9 @@ def _pin_hnsw_threads(collection) -> None:
|
||||
logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
|
||||
|
||||
|
||||
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
|
||||
|
||||
|
||||
def _fix_blob_seq_ids(palace_path: str) -> None:
|
||||
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
|
||||
|
||||
@@ -168,10 +171,19 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
|
||||
type INTEGER) is not compatible with SQL type BLOB".
|
||||
|
||||
Must run BEFORE PersistentClient is created (the compactor fires on init).
|
||||
|
||||
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
|
||||
database leaves state that segfaults the next PersistentClient call. After
|
||||
the migration has run once successfully, a marker file is written so
|
||||
subsequent opens skip the sqlite connection entirely. Already-migrated
|
||||
palaces can touch the marker manually to opt into the fast path.
|
||||
"""
|
||||
db_path = os.path.join(palace_path, "chroma.sqlite3")
|
||||
if not os.path.isfile(db_path):
|
||||
return
|
||||
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
|
||||
if os.path.isfile(marker):
|
||||
return
|
||||
try:
|
||||
with sqlite3.connect(db_path) as conn:
|
||||
for table in ("embeddings", "max_seq_id"):
|
||||
@@ -189,6 +201,14 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
|
||||
conn.commit()
|
||||
except Exception:
|
||||
logger.exception("Could not fix BLOB seq_ids in %s", db_path)
|
||||
return
|
||||
# Write marker whether or not rows needed migration — the palace is now
|
||||
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
|
||||
# sqlite3.connect() entirely.
|
||||
try:
|
||||
open(marker, "a").close()
|
||||
except OSError:
|
||||
logger.exception("Could not write migration marker %s", marker)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user