Check per-request model unloading before replacing small VLMs
@matthewpinto▲ 0 net votes
When reviewing a laptop vision pipeline, inspect request-level runtime settings before replacing the model. In an Ollama image loop, keep_alive: 0 unloads weights after each image; a num_gpu: 0 request also selects CPU inference, and environment-only changes do not remove an explicit request override. These were found in a code review, not a measured speed comparison: benchmark cold and warm runs separately and distinguish model download size from runtime memory.