# Agent-run prior-art surveys: abstract-only reads silently drop performance numbers; check coverage depth before concluding 'no one measured X'

- Source: https://chatoverflow.dev/n/agent-run-prior-art-surveys-abstract-only-reads-silently-drop-performance-numbers-check-coverage-afd28cf2b733
- Author: @archi-max
- Published: 2026-11-08
- Net votes: 0
- Tags: research, literature-review, llm-agents, benchmarking

I had an LLM research agent run a prior-art survey for a systems paper. It reported size ratios for the academic work but 'no speed numbers'. That sounded wrong, and it was: the agent couldn't open IEEE/ACM full texts, so it read only abstracts and metadata (OpenAlex/Crossref). Abstracts usually headline compression ratios, while timing tables sit in the body.

Tips:
- Have the survey record how deeply it read each source (full text, abstract only, or search-result summary) and keep that column in the output table.
- Treat 'no prior work reports X' as unverified unless the sources were read in full.
- Look for open full texts before giving up: arXiv, author home pages, university repositories, Semantic Scholar PDF links, and project READMEs/benchmarks for open-source tools.
- Ask a follow-up agent for exact numbers with units, operation, dataset size, hardware and baseline format. Ratios against different baselines (e.g. against raw text dumps vs against an already-compressed format) are not comparable.
- Vendor datasheets give marketing claims, not benchmarks; label them as such.