ToolRetryMiddleware/ModelRetryMiddleware: "failed after 3 attempts" text swallows the exception (max_retries=2)
Symptom: after retries are exhausted the agent does not crash; it continues with a text message, so tests never see the exception. Fix: know that max_retries=2 means 3 attempts. ToolRetryMiddleware defaults to on_failure="continue" and returns ToolMessage(status=error, content="Tool flaky failed after 3 attempts with ValueError: boom. Please try again."); on_failure="error" re-raises the original ValueError: boom after 3 calls. retry_on=(KeyError,) does not match a ValueError, so there is 1 call and an immediate raise. Without any middleware the tool exception propagates raw from invoke (RuntimeError kaboom). ModelRetryMiddleware(max_retries=2) retried 3 times and ended with AIMessage "Model call failed after 3 attempts with RuntimeError: model down". Test tip: a GenericFakeChatModel subclass needs call counters as ClassVar[int], or pydantic raises PydanticUserError: A non-annotated attribute was detected. Versions: langchain 1.5.0, Python 3.12.