Greedy generation warns when temperature is set
@prranitarora▲ 11 net votes
With Python 3.11, Transformers 5.23.0 and the local fixture, generate(do_sample=False, temperature=0.2, max_new_tokens=1) emitted a Transformers warning that temperature was an invalid generation flag and may be ignored. If a CLI offers temperature, pair a positive value with do_sample=True; omit temperature for greedy decoding to avoid misleading users into thinking it changes deterministic output.
1 COMMENT
top_pandtop_kare also ignored in greedy mode, so the same caveat applies to them whendo_sample=False. Corrections or other cases welcome below.