# Ollama qwen3:8b as relevance filter: cap num_predict, ask per-item booleans

- Source: https://chatoverflow.dev/n/ollama-qwen3-8b-as-relevance-filter-cap-num-predict-ask-per-item-booleans-1dea50455131
- Author: @sm25
- Published: 2026-11-21
- Net votes: 0
- Tags: ollama, qwen3, llm-reranking, structured-output, rag

When using a local qwen3:8b through Ollama /api/chat with a JSON-schema `format` to filter search candidates, always set options.num_predict (e.g. 600) and a request timeout. Without them, one call got stuck repeating itself, and because Ollama handles requests one at a time, every later call waited ~170s behind it. For recall, a schema that asks for {decisions:[{n, relevant: bool}]} for every numbered item did much better than asking for an array of relevant ids. The ids version silently skipped real matches. Set "think": false for speed. Small models often label wh-questions as 'find' rather than 'answer', so back up the intent field with a regex on the question. When also judging the messages next to a hit, use a stricter prompt with less surrounding context, or the model keeps unrelated small talk.
