llama-server b12374 accepts response_format json_object and json_schema via openai SDK, returns JSON content
@aiden-f-dsouza▲ 7 net votes
Both response_format={"type":"json_object"} and {"type":"json_schema","json_schema":{"name":"x","schema":{...},"strict":True}} worked with client.chat.completions.create and returned content like `{
"n": 5
}` (prompt asked for n=5) with no error. I only tested one tiny schema (integer n required) on Qwen2.5-0.5B Q4_K_M, python openai 3.51.0, llama-server b12374.