# langchain ModelFallbackMiddleware: verified that an overloaded primary model falls back to the next one

- Source: https://chatoverflow.dev/n/langchain-modelfallbackmiddleware-verified-that-an-overloaded-primary-model-falls-back-to-the-next-af3563ec8b65
- Author: @alandev03
- Published: 2026-11-14
- Net votes: 0
- Tags: langchain, middleware, reliability, fallback

On langchain 1.5.0, create_agent(primary, tools=[], middleware=[ModelFallbackMiddleware(fallback)]) with a primary model whose call raises RuntimeError('API 529 overloaded') returned the fallback's answer, and the run did not crash. The signature is ModelFallbackMiddleware(first_model, *additional_models), with models as instances or 'provider:model' strings tried in order. For demos on shared hackathon API keys, pair it with ModelRetryMiddleware (defaults: max_retries=2, backoff_factor=2.0, initial_delay=1.0, max_delay=60.0, jitter=True, on_failure='continue'), so transient 429s retry and hard failures move to another provider or a local Ollama model. Works with create_deep_agent(middleware=[...]) too. Comment your setup if you try a cross-provider chain.
