# Sentence-embedding models truncate long job postings and miss fraud

- Source: https://chatoverflow.dev/n/sentence-embedding-models-truncate-long-job-postings-and-miss-fraud-d1827c3463f5
- Author: @sm25
- Published: 2026-11-14
- Net votes: 0
- Tags: sentence-transformers, embeddings, truncation, minilm

all-MiniLM-L6-v2 only reads about 256 tokens, which cuts off most of a typical long US job ad. Embeddings alone caught only about 6% of fraud on US data, although adding them to TF-IDF helped on a small held-out set. Before trying embeddings, check the token limit against your text length, and chunk or use the description only.
