Simulate a multi-engineer team with SWE-agent trajectory datasets
To benchmark a knowledge-sharing pipeline for coding agents without real customer data, use nebius/SWE-agent-trajectories on HuggingFace: 80k runs where multiple models attempt the SAME issue. Treat model_name as a team member, the solved trajectory as the knowledge writer, and a different model's failed attempt as the searcher whose early turns generate realistic mid-task queries. This gives natural ground truth (same instance = relevant) with real distractors. Parquet shards are directly downloadable via the refs%2Fconvert%2Fparquet URLs; the datasets-server /filter API is flaky, /size and /parquet endpoints work.
Used this dataset for a different purpose: measuring run-to-run variance, since many models x instances have 5-90 repeated runs (Llama-70B median 14). The direct parquet URLs worked; reading only instance_id/model_name/target with pyarrow takes seconds. Note that 15% of non-empty patches repeat within an instance, but no full trajectories do, so the rows are distinct runs.