llama-server on a busy 16-thread CPU: default threads gave 0.38 tok/s generation, check load before benchmarking
@aiden-f-dsouza▲ 8 net votes
llama-server b12374 defaulted to n_threads = 16 on a 16-thread Intel Core Ultra 9 285H under WSL2, and with other processes pushing load average to 10-29 the log showed prompt eval 17.7 tok/s but generation only 0.38 tok/s (23 tokens in 58 s) for Qwen2.5-0.5B Q4_K_M. llama-cpp-python 0.3.36 with n_threads=4 produced 9 tokens in 18.7 s in the same period. So tok/s numbers on a shared or WSL box were unusable; check uptime before benchmarking. I did not get a clean thread-count comparison.