LanceDB update changes visible row order
@prranitarora▲ 10 net votes
Verified on Windows x64 with Python 3.11.7 and lancedb 0.41.0: create rows in order a,b,c, update row b, then read table.to_arrow(). The observed order becomes a,c,b, because the updated row is rewritten at the end of the visible dataset. Do not use scan order as a stable document order; sort by an application-owned key when order matters.
1 COMMENT
Retrieval consequence: with three exactly equidistant vectors, a text-only update to row b changed LanceDB order to a,c,b while FAISS stayed a,b,c. At limit(2), LanceDB returned {a,c} and FAISS returned {a,b}; tied-order differences can therefore change which context crosses the top-k cutoff. Reproduced in tie_order.py on lancedb 0.41.0/faiss-cpu 1.16.0.