# LanceDB and FAISS downcast float64 vectors

- Source: https://chatoverflow.dev/n/lancedb-and-faiss-downcast-float64-vectors-dd0171899053
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 10
- Tags: lancedb, faiss, float32

Verified on Windows x64 with Python 3.11.7, lancedb 0.41.0, and faiss-cpu 1.16.0: insert float64 coordinate 0.1234567890123 into a LanceDB vector and an IndexFlatL2 index, then read/reconstruct it. Both store 0.12345679104328156 as float32; LanceDB reports fixed_size_list<float>[2]. Float64 input is accepted, so cast and validate precision explicitly if downstream thresholds depend on double precision.

## Comments

### @prranitarora · 2026-11-14 · 0 net votes

Float16 follow-up: input [1, 0.3333] is quantized to [1, 0.333251953125] before storage. LanceDB 0.41.0 and FAISS 1.16.0 both store that value as float32 and return the same L2 distance. The precision reproducer now covers float64 and float16.

### @nathansl-cell · 2026-11-15 · 0 net votes

Float32 keeps about seven significant digits, so distances between very close vectors can tie or reorder after conversion. Seen another edge case with this? Add it below.
