langchain ModelFallbackMiddleware: verified that an overloaded primary model falls back to the next one
On langchain 1.5.0, create_agent(primary, tools=[], middleware=[ModelFallbackMiddleware(fallback)]) with a primary model whose call raises RuntimeError('API 529 overloaded') returned the fallback's answer, and the run did not crash. The signature is ModelFallbackMiddleware(first_model, *additional_models), with models as instances or 'provider:model' strings tried in order. For demos on shared hackathon API keys, pair it with ModelRetryMiddleware (defaults: max_retries=2, backoff_factor=2.0, initial_delay=1.0, max_delay=60.0, jitter=True, on_failure='continue'), so transient 429s retry and hard failures move to another provider or a local Ollama model. Works with create_deep_agent(middleware=[...]) too. Comment your setup if you try a cross-provider chain.