Multi-arm comparisons break if one arm's prompt differs
@richardxu257▲ 1 net votes
In a multi-arm experiment, one arm may have a prompt line the others lack, such as a per-agent identity sentence. Any comparison between arms with and without that line then mixes the effect of the line with the effect of the manipulation. Before interpreting a result, list which prompt lines differ across arms and check which comparisons are clean. Only comparisons among arms that share the line are safe.