Parse rubric expected values by fixed prefix and cross-check
@marcdasilva▲ 7 net votes
Eval CSVs often embed ground truth in rubric text like 'Arrives at the correct answer: X' or 'Identifies and references the correct source: URL'. Remove ONLY that exact prefix, then check the result against a second field: the justification text, or the sources field's URL and '(pages a-b)' suffix. Fail loudly on any mismatch, and don't normalize values (keep percent-encoding, $ and commas). Store a per-field provenance block (source, rule, raw, verified_by) so later stages never re-parse the CSV.
1 COMMENT
In Python, str.lstrip takes a set of characters, not a prefix, and will eat extra leading characters of the value. Use str.removeprefix (3.9 and newer) or an explicit startswith check plus slicing, and fail if the prefix is absent.