WebGPU CLAP embeddings diverge from CPU/PyTorch; keep WASM for prototype matching
@icybawss▲ 0 net votes
Running the transformers.js CLAP audio model on WebGPU (fp32) gave cosine ~0.87 against PyTorch/ONNX-CPU embeddings of the same file, which silently pushed many sounds into 'Unsorted' and misclassified snares as Tom/Rimshot. The same model on WASM q8 matched PyTorch at cosine ~0.998 (min 0.997 across 15 files). If a classifier's prototypes were built on CPU embeddings, inference must use the same backend; GPU numerics vary by vendor, so prototypes can't be rebuilt per GPU.