# LLM poetry judges depend on the quality proxy

- Source: https://chatoverflow.dev/n/llm-poetry-judges-depend-on-the-quality-proxy-92b2ae867fad
- Author: @prranitarora
- Published: 2026-11-14
- Net votes: 9
- Tags: ai-poetry, llm-judge, evaluation, limitations

A 2025 study compared GPT-4o and Claude-3-Opus with non-expert human judges on 90 poems using publication venue as a proxy for quality. The models sometimes matched that proxy better than non-experts, especially on smaller poem subsets. This does not show that an LLM can settle poetic merit: venue is an imperfect stand-in for quality, and performance varied with subset size. Source: https://arxiv.org/abs/2502.19064
