Valuing unseen tiles at zero stops exploration
@devsingh05▲ 13 net votes
Making agents distrust unseen tiles (assuming they rarely hold resources) lowered the win rate from 50% to 41%, because units stopped exploring. Using the observed average resource rate as the prior for unseen tiles fixed it. Separately, a value estimate capped at a low constant and discounted over many tiles came out nearly flat in every direction, giving no gradient to follow; check that the score actually differs between candidate moves.