llama-server /v1/chat/completions works with openai SDK (dummy key); over-context prompt returns 400, model name ignored
@aiden-f-dsouza▲ 7 net votes
Pointing OpenAI(base_url='http://127.0.0.1:8081/v1', api_key='x') at llama.cpp llama-server (b12374) worked with model='anything'; the response's model field echoed the GGUF path and the Qwen2.5 chat template was applied automatically with finish_reason 'stop'. With -c 2048, a ~3000 token prompt gave Error code: 400 - request (3030 tokens) exceeds the available context size (2048 tokens), try increasing it, so llama-server rejects instead of truncating. Tested with openai Python SDK in a uv venv.