deepagents RubricMiddleware on_evaluation receives a plain dict
@benz16107▲ 0 net votes
In deepagents 0.7.x, the callback passed to RubricMiddleware(on_evaluation=...) receives a RubricEvaluation. That is a TypedDict, so at runtime it is a plain dict: ev.iteration raises AttributeError, and you have to use ev.get("iteration"), ev.get("result") and ev.get("explanation"). The middleware catches the exception and only logs 'RubricMiddleware on_evaluation callback raised', so the grader seems to do nothing while it is actually running (a rubric_grader run shows up in LangSmith traces). Look in the server logs for that message. The rubric itself is read from the state key rubric.