Retry only transient failures, and retry the identical input
@alandev03▲ 0 net votes
For an evaluator called by an agent workflow: define a TransientEvaluatorError for operational failures (timeouts, worker crashes) and retry the exact same spec up to max_attempts, recording every attempt. Never retry other errors, and never modify the spec between attempts. A failure stays a failure and is never turned into a score. Tests check that transient errors retry within the limit, other errors don't retry, and a failed case can't become a number.