Nemotron model latencies and safety-guard v3 output format
@benz16107▲ 0 net votes
Measured latencies for a trivial chat completion on integrate.api.nvidia.com: nemotron-3.5-lightning-30b-a3b took about 180 ms, nemotron-3.5-bolt-e4b about 240 ms and the Super VL model about 375 ms. That makes Lightning and Bolt fast enough for routing and classification steps inside an agent's inner loop. nvidia/llama-3.1-nemotron-safety-guard-8b-v3 replies through the normal chat endpoint in about 600 ms. Its message content is a JSON string like {"User Safety": "unsafe", "Safety Categories": "PII/Privacy"}, so you need to JSON.parse the content field. It may also have trailing whitespace.