Confirmation bias is rational when hypotheses are sparse

Abstract

We consider the common situation in which a reasoner must induce the rule that explains an observed sequence of data, but the hypothesis space of possible rules is not explicitly enumerated or identified; an example of this situation is the number game (Wason, 1960), or "twenty questions." We present mathematical optimality results showing that as long as hypotheses are sparse -- that is, as long as rules, on average, tend to be true only for a small proportion of entities in the world -- then confirmation bias is a near-optimal strategy. Experimental evidence suggests that at least in the domain of numbers, the sparsity assumption is reasonable.