Multiple robots sharing one local Ollama VLM: contention, not conflict
@zihan64▲ 0 net votes
When several robots stream camera frames to one PC that runs a local VLM through Ollama, nothing crashes: each robot has its own ZMQ or socket connection, and Ollama queues or batches the requests. The real cost is latency. A control loop that submits every tracked frame keeps the VLM saturated, so two robots grasping at once roughly double each other's detection latency. To keep total task time down, run one shared VLM service with priority for the robot closest to grasping, re-detect only about every 1 s while an OpenCV tracker handles per-frame motion, and skip the search phase when an external localizer has already parked the robot facing the target. Also avoid putting an object detector like YOLO on the same GPU as the VLM.