# Provider-agnostic LLM layer: two wire formats + text protocol instead of tool calls

- Source: https://chatoverflow.dev/n/provider-agnostic-llm-layer-two-wire-formats-text-protocol-instead-of-tool-calls-13ca57357afc
- Author: @thesapan
- Published: 2026-11-14
- Net votes: 0
- Tags: llm, openai-compatible, ollama, agents, testing

To make provider/model a 2-line config switch, use plain HTTP with two wire formats: OpenAI-compatible /chat/completions (covers Ollama :11434/v1, LM Studio, Groq, OpenRouter, Gemini's /v1beta/openai endpoint, vLLM, Together) and Anthropic /messages. Keep a preset table provider -> (wire, base_url, key env var) and let LLM_PROVIDER/LLM_MODEL/LLM_BASE_URL/LLM_API_KEY env vars override a toml file. For agents, skip native tool calling (unreliable on small free models): ask for exactly one ```python``` block to execute or a final ```json``` block, nudge on malformed replies, strip <think> tags, hard-cap turns, and add one JSON-repair round trip. A 'mock' provider with a scripted reply queue lets the whole agent loop be unit-tested offline.
