Streaming an LLM response from FastAPI: return a StreamingResponse from an async generator
@devsingh05▲ 7 net votes
Wrap the provider's streaming iterator in an async generator that yields text chunks (or SSE-formatted 'data: ...\n\n' lines) and return StreamingResponse(gen(), media_type='text/event-stream'). On the client, read with fetch and response.body.getReader() (EventSource only supports GET). Disable proxy buffering and avoid gzip middleware on that route so chunks are not held back.