Nice point by Seth Godin on how to see if someone really knows what they're talking about. I see it in my field too---people judging by the extent to which one can carry on an extemporaneous conversation about the subject at hand, revealing months of thought about the topic at hand, years of study of the background issues. On the other hand, we also need --- sorely, and in my opinion, even in technical fields --- people with breadth---non-superficial breadth, to be sure. How does one spot that?
Wine, Physics, and Song
Howard Barnum's blog on art, music, culture, science, public affairs, philosophy, and life
Another way of telling depth is to see to what extent people can describe what might cause them to change their beliefs. E.g., the extent to which people can explain circumstances under which they would change their beliefs about the correctness of quantum mechanics, about the theory of evolution, about string theory, about climate change, etc.
The breadth question is an interesting one. You can't just poll them on a range of topics; you might, by chance, hit topics they just happen not to know much about. von Neumann purportedly claimed that you could fail anyone on a reasonable graduate exam on mathematics, simply by exploiting weaknesses in their knowledge. I believe he asked Ulam to construct such an exam which he (von Neumann) would fail; I don't know the outcome. In any case, if that's true of von Neumann, it's true of anyone.
Nice point on belief change, Michael, and one that goes beyond just assessing depth of knowledge to assessing intellectual and moral seriousness---or at least, reasonableness. Most people probably don't get to the point of deep knowledge of a subject without a degree of seriousness, of course---but perhaps surprisingly, that isn't always accompanied by reasonableness.
On breadth, how appropriate to mention von Neumann---one of my favorite examples of non-superficial depth, at least within mathematics and the sciences.