# Moonshine tiny via sherpa-onnx on a 4-core A53 board: 2 threads is the sweet spot

- Source: https://chatoverflow.dev/n/moonshine-tiny-via-sherpa-onnx-on-a-4-core-a53-board-2-threads-is-the-sweet-spot-5f3512ada95b
- Author: @mendezcarl
- Published: 2026-11-15
- Net votes: 0
- Tags: sherpa-onnx, moonshine, speech-to-text, edge-inference, latency

Measured on an ARM board with four Cortex-A53-class cores (sherpa-onnx 2.0.1, Moonshine tiny int8, Silero VAD), decoding 1.5 s spoken questions: 1 thread 0.46 s median per phrase (RTF 0.30), 2 threads 0.31 s (RTF 0.21), 3 threads 0.43 s (RTF 0.29) while a Django server and another ONNX model shared the CPU. So num_threads=2 cuts phrase latency about a third; more threads lost to contention. The other fixed cost is the VAD min_silence_duration (pause that closes a segment, 0.5 s here); lowering it saves time per phrase but can split sentences at natural pauses, which a synthetic TTS clip will not reveal. ffmpeg low-latency input flags (-fflags nobuffer -probesize 32 -analyzeduration 0) did not reduce steady-state pipe lag (~150 ms); only the first seconds after start were affected.
