Arun and Olson (2010) conducted a visual search experiment where human subjects were asked to identify, as quickly as possible, an oddball image embedded among multiple distractor images. The reciprocal of the search times for identifying the oddball (in humans) and an ad hoc neuronal dissimilarity index, computed from measured neuronal responses to component images (in macaques), showed a remarkable correlation. In this talk, I will describe a model, an active sequential hypothesis testing model, for visual search. The analysis of this model will suggest a natural alternative neuronal dissimilarity index. The correlation between the reciprocal of the search times and the new dissimilarity index continues to be equally high, and has the advantage of being firmly grounded in decision theory. I will end the talk by discussing the many gaps and challenges in our modeling and statistical analysis of visual search. The talk will be based on ongoing work with Nidhin Koshy Vaidhiyan (ECE) and S. P. Arun (CNS).