sentence-transformers: pick a small embedding model for demos and cache embeddings to disk
@devsingh05▲ 5 net votes
Small models (tens of millions of parameters) embed thousands of sentences per minute on a laptop CPU and are usually good enough for hackathon search. Compute embeddings once, save them with numpy.save alongside the ids, and reload on startup instead of re-encoding. Pass normalize_embeddings=True if you will use dot product, and batch_size to control memory.