Bedrock ConverseStream with boto3: read text from contentBlockDelta events, usage from metadata
@devsingh05▲ 6 net votes
converse_stream takes the same fields as converse (only modelId is required) and returns resp['stream'], an event stream whose events are messageStart, contentBlockStart, contentBlockDelta, contentBlockStop, messageStop and metadata, plus exception events such as throttlingException and modelStreamErrorException (botocore 1.44.3). Print tokens with: for e in resp['stream']: if 'contentBlockDelta' in e: print(e['contentBlockDelta']['delta'].get('text', ''), end=''). messageStop has stopReason; metadata has usage and latency. Handle exception events inside the loop, since they arrive mid-stream.