Deutsch, Popper, Gelman and Shalizi, with a side of Mayo, on Bayesian ideas, models and fallibilism in the philosophy of science and in statistics (I)

A few years back, when I reviewed David Deutsch's The Beginning of Infinity for Physics Today (see also my short note on the review at this blog), I ended up spending a fair amount of time revisiting an area of perennial interest to me: the philosophy of science, and the status of Popper's falsificationist and anti-inductive view of scientific reasoning. I tend to like the view that one should think of scientific reasoning in terms of coherently updating subjective probabilities, which might be thought of as Bayesian in a broad sense. (Broad because it might be more aligned with Richard Jeffrey's point of view, in which any aspect of one's probabilities might be adjusted in light of experience, rather than a more traditional view on which belief change is always and only via conditioning the probabilities of various hypotheses on newly acquired data, with one's subjective probabilities of data given the hypotheses never adjusting.) I thought Deutsch didn't give an adequate treatment of this broadly Bayesian attitude toward scientific reasoning, and wrote:

Less appealing is Deutsch and Popper’s denial of the validity of inductive reasoning; if this involves a denial that evidence can increase the probability of general statements such as scientific laws, it is deeply problematic. To appreciate the nature and proper role of induction, one should also read such Bayesian accounts as Richard Jeffrey’s (Cambridge University Press, 2004) and John Earman’s (MIT Press, 1992).

Deutsch and Popper also oppose instrumentalism and physical reductionism but strongly embrace fallibilism. An instrumentalist believes that particular statements or entities are not literally true or real, but primarily useful for deriving predictions about other matters. A reductionist believes that they have explanations couched in the terms of some other subject area, often physics. Fallibilism is the view that our best theories and explanations are or may well be false. Indeed many of the best have already proved not to be strictly true. How then does science progress? Our theories approximate truth, and science replaces falsified theories with ones closer to the truth. As Deutsch puts it, we “advance from misconception to ever better misconception.” How that works is far from settled. This seems to make premature Deutsch’s apparent dismissal of any role for instrumentalist ideas, and his neglect of pragmatist ones, according to which meaning and truth have largely to do with how statements are used and whether they are useful.

Thanks to Brad DeLong I have been reading a very interesting paper from a few years back by Andrew Gelman and Cosma Shalizi, "Philosophy and the practice of Bayesian statistics" that critiques the Bayesian perspective on the philosophy of science from a broadly Popperian---they say "hypothetico-deductive"---point of view, that embraces (as did Popper in his later years) fallibilism (in the sense of the quote from my review above).  They are particularly concerned to point out that the increasing use of Bayesian methods in statistical analysis should not necessarily be interpreted as supporting a Bayesian viewpoint on the acquisition of scientific knowledge more generally.  That point is well taken; indeed I take it to be similar to my point in this post that the use of classical methods in statistical analysis need not be interpreted as supporting a non-Bayesian viewpoint on the acquisition of knowledge.  From this point of view, statistical analysis, whether formally Bayesian or "classical" is an input to further processes of scientific reasoning; the fact that Bayesian or classical methods may be useful at some stage of statistical analysis of the results of some study or experiment does not imply that all evaluation of the issues being investigated must be done by the same methods.  While I was most concerned to point out that use of classical methods in data analysis does not invalidate a Bayesian (in the broad sense) point of view toward how the results of that analysis should be integrated with the rest of our knowledge, Gelman and Shalizi's point is the mirror image of this.  Neither of these points, of course, is decisive for the "philosophy of science" question of how that broader integration of new experience with knowledge should proceed.

Although it is primarily concerned to argue against construing the use of  Bayesian methods in data analysis as supporting a Bayesian view of scientific methods more generally, Gelman and Shalizi's paper does also contain some argument against Bayesian, and more broadly "inductive", accounts of scientific method, and in favor of a broadly Popperian, or what they call "hypothetico-deductive" view.  (Note that they distinguish this from the "hypothetico-deductive" account of scientific method which they associate with, for instance, Carl Hempel and others, mostly in the 1950s.)

To some extent, I think this argument may be reaching a point that is often reached when smart people, indeed smart communities of people,  discuss, over many years, fundamental issues like this on which they start out with strong differences of opinion:  positions become more nuanced on each side, and effectively closer, but each side wants to keep the labels they started with, perhaps in part as a way of wanting to point to the valid or partially valid insights that have come from "their" side of the argument (even if they have come from the other side as well in somewhat different terms), and perhaps also as a way of wanting to avoid admitting having been wrong in "fundamental" ways.  For example, one sees insights similar to those in the work of Richard Jeffrey and others from a "broadly Bayesian" perspective, about how belief change isn't always via conditionalization using fixed likelihoods, also arising in the work of the "hypothetico-deductive" camp, where they are used against the simpler "all-conditionalization-all-the-time" Bayesianism.  Similarly, probably Popperian ideas played a role in converting some  "relatively crude" inductivists to more sophisticated Bayesian or Jefferian approach.  (Nelson Goodman's "Fact, Fiction, and Forecast", with its celebrated "paradox of the grue emeralds", probably played this role a generation or two later.)  Roughly speaking, the "corroboration" of hypotheses of which Popper speaks, involves not just piling up observations compatible with the hypothesis (a caricature of "inductive support") but rather the passage of stringent tests. In the straight "falsification"  view of Popper, these are stringent because there is a possibility they will generate results inconsistent with the hypothesis, thereby "falsifying" it; on the view which takes it as pointing toward a more Bayesian view of things (I believe I once read something by I.J.Good in which he said that this was the main thing to be gotten from Popper), this might be relaxed to the statement that there are outcomes that are very unlikely if the hypothesis is true, thereby having the potential, at least, of leading to a drastic lowering of the posterior probability of the hypothesis (perhaps we can think of this as a softer version of falsification) if observed.  The posterior probability given that such an outcome is observed of course does not depend only on the prior probability of the hypothesis and the probability of the data conditional on the hypothesis---it also depends on many other probabilities.  So, for instance, one might also want such a test to have the property that "it would be difficult (rather than easy) to get an accordance between data x and H (as strong as the one obtained) if H were false (or specifiably flawed)".  The quote is from this post on Popper's "Conjectures and Refutations" by philosopher of science D. G. Mayo, who characterizes it as part of "a modification of Popper".  ("The one obtained" refers to an outcome in which the hypothesis is considered to pass the test.)  I view the conjunction of these two aspects of a test of a hypothesis or theory as rather Bayesian in spirit.  (I do not mean to attribute this view to Mayo.)
I'll focus later---most likely in a follow-up post---on Gelman and Shalizi's direct arguments against inductivism and more broadly Bayesian approaches to scientific methodology and the philosophy of science.  First I want to focus on a point that bears on these questions but arises in their discussion of Bayesian data analysis.  It is that in actual Bayesian statistical data analysis "the prior distribution is one of the assumptions of the model and does not need to represent the statistician's personal degree of belief in alternative parameter values".  They go on to say "the prior is connected to the data, so is potentially testable".  It is presumably just this sort of testing that Matt Leifer was referring to when he wrote (commenting on my earlier blog entry on Bayesian methods in statistics)

"What I often hear from statisticians these days is that it is good to use Bayesian methods, but classical methods provide a means to check the veracity of a proposed Bayesian method. I do not quite understand what they mean by this, but I think they are talking at a much more practical level than the abstract subjective vs. frequentist debate in the foundations of probability, which obviously would not countenance such a thing.

The point Gelman and Shalizi are making is that the Bayesian prior being used for data analysis may not capture "the truth", or more loosely, since they are taking into account the strong possibility that no model under consideration is literally true, that it may not adequately capture those aspects of the truth one is interested in---for example, may not be good at predicting things one is interested in. Hence one wants to do some kind of test of whether or not the model is acceptable. This can be based on using the Bayesian posterior distribution as a model to be tested further, typically with classical tests such as "pure significance tests".
As Matthew's comment above might suggest, those of us of more Bayesian tendencies, who might agree that the particular family of priors---and potential posteriors---being used in data analysis (qua "parameter fitting" where perhaps we think of the prior distribution as the (higher-level) "parameter" being fit) might well not "contain the truth", might be able to take these tests of the model, even if done using some classical statistic, as fodder for further, if perhaps less formal, Bayesian/Jeffreysian reasoning about what hypotheses are likely to do a good job of predicting what is of interest.

One of the most interesting things about Gelman and Shalizi's paper is that they are thinking about how to deal with "fallibilism" (Popper's term?), in particular, inference about hypotheses that are literally false but useful. This is very much in line with recent discussion at various blogs of the importance of models in economics, where it is clear that the models are so oversimplified as to be literally false, but nonetheless they may prove predictively useful.  (The situation is complicated, however, by the fact that the link to prediction may also be relatively loose in economics; but presumably it is intended to be there somehow.)  It is not very clear how Popperian "falsificationism" is supposed to adapt to the fact that most of the hypotheses that are up for falsification are already known to be false. Probably I should go back and see what Popper had to say on that score, later in his career when he had embraced fallibilism. (I do recall that he tried introducing a notion of "verisimilitude", i.e. some kind of closeness to the truth, and that the consensus seems to have been---as Gelman and Shalizi point out in a footnote---that this wasn't very successful.)  It seems to that a Bayesian might want to say one is reasoning about the probability of statements like "the model is a good predictor of X in circumstances Y", " the model does a good job capturing how W relates to Z" , and so forth. It is perhaps statements like these that are really being tested when one does the " pure significance tests" advocated by Gelman and Shalizi when they write things like "In designing a good test for model checking, we are interested in finding particular errors which, if present, would mess up particular inferences, and devise a test statistic which is sensitive to this sort of mis-specification."

As I said above, I hope to take up Gelman and Shalizi's more direct arguments (in the cited paper) against "inductivism" (some of which I may agree with) and Bayesianism sensu lato as scientific methodology in a later post. I do think their point that the increasing use of Bayesian analysis in actual statistical practice, such as estimation of models by calculating a posterior distribution over model parameters beginning with some prior, via formal Bayesian conditioning, does not necessarily tell in favor of a Bayesian account of scientific reasoning generally, is important. In fact this point is important for those who do hold such a loosely Bayesian view of scientific reasoning:  most of us do not wish to get stuck with interpreting such priors as the full prior input to the scientific reasoning process.  There is always implicit the possibility that such a definite specification is wrong, or, when it is already known to be wrong but thought to be potentially useful for some purposes nonetheless, "too wrong to be useful for those purposes".

A thought on resistance to Bayesian statistics

I'm not a statistician, and as a quantum theorist of a relatively abstract sort, I've done little actual data analysis.  But because of my abstract interests, the nature of probability and its use in making inferences from data are of great interest.  I have some relatively ill-informed thoughts on why the "classical statistics" community seems to have been quite resistant to "Bayesian statistics", at least for a while, that may be of interest, or at least worth logging for my own reference. Take this post in the original (?) spirit of the term "web log", rather than as a polished piece of the sort many blogs, functioning more in the spirit of online magazines, seem to aim at nowadays.

The main idea is this.  Suppose doing Bayesian statistics is thought of as actually adopting a prior which specifies, say, one's initial estimate of the probabilities of several hypotheses, and then, on the basis of the data, computing the posterior probability of the hypotheses.  In other words, what is usually called "Bayesian inference". That may be a poor way of presenting the results of an experiment, although it is a good way for individuals to reason about how the results of the experiment should affect their beliefs and decisions.  The problem is that different users of the experimental results, e.g. different readers of a published study, may have different priors.  What one would like is rather to present these users with a statistic, that is, some function of the data, much more succinct than simply publishing the data themselves, but just as useful, or almost as useful, in making the transition from prior probabilities to posterior probabilities, that is, of updating one's beliefs about the hypotheses of interest, to take into account the new data. Of course, for a compressed version of the data (a statistic) to be useful, it is probably necessary that the users share certain basic assumptions about the nature of the experiment.  These assumptions might involve the probabilities of various experimental outcomes, or sets of data, if various hypotheses are true (or if a parameter takes various values), i.e., the likelihood function;  they might also involve a restriction on the class of priors for which the statistic is likely to be useful.  These should be spelled out, and, if it is not obvious, how the statistic can be used in computing posterior probabilities should be spelled out as well.

It seems to me likely that many classical or "frequentist" statistics may be used in such a way; but, quite possibly, classical language, like saying that statistical inference leads to "acceptance" or "rejection" of hypotheses, tends to obscure this more desirable use of the statistic as a potential input to the computation of posterior probabilities.  In fact, I think people tend to have a natural tendency to want some notion of what the posterior probability of a hypothesis is; this is one source of the erroneous tendency, still sometimes found among the public, to confuse confidence levels with probabilities.  Sometimes an advocacy of classical statistical tests may go with an ideological resistance to the computation of posterior probabilities, but I suppose not always.  It also seems likely that in many cases, publishing actual Bayesian computations may be a good alternative to classical procedures, particularly if one is able to summarize in a formula what the data imply about posterior probabilities, for a broad enough range of priors that many or most users would find their prior beliefs adequately approximated by them.  But in any case, I think it is essential, in order to properly understand the meaning of reports of classical statistical tests, to understand how they can be used as inputs to Bayesian inference.  There may be other issues as well, e.g. that in some cases classical tests may make suboptimal use of the information available in the data.  In other words, they may not provide a sufficient statistic: a function of the data that contains all the information available in the data, about some random variable of interest (say, whether a particular hypothesis is true or not). Of course whether or not a statistic is sufficient will depend on how one models the situation.

Most of this is old hat, but it is worth keeping in mind, especially as a Bayesian trying to understand what is going on when "frequentist" statisticians get defensive about general Bayesian critiques of their methods.

A question about measurement in Bohmian quantum mechanics

I was disturbed by aspects of Craig Callender's post "Nothing to see here," on the uncertainty principle, in the New York Times' online philosophy blog "The Stone," and I'm pondering a response, which I hope to post here soon.  But in the process of pondering, some questions have arisen which I'd like to know the answers to.  Here are a couple:

Callender thinks it is important that quantum theory be formulated in a way that does not posit measurement as fundamental.  In particular he discusses the Bohmian variant of quantum theory (which I might prefer to describe as an alternative theory) as one of several possibilities for doing so.  In this theory, he claims,

Uncertainty still exists. The laws of motion of this theory imply that one can’t know everything, for example, that no perfectly accurate measurement of the particle’s velocity exists. This is still surprising and nonclassical, yes, but the limitation to our knowledge is only temporary. It’s perfectly compatible with the uncertainty principle as it functions in this theory that I measure position exactly and then later calculate the system’s velocity exactly.

While I've read Bohm's and Bell's papers on the subject, and some others, it's been a long time in most cases, and this theory is not something I consider very promising as physics even though it is important as an illustration of what can be done to recover quantum phenomena in a somewhat classical theory (and of the weird properties one can end up with when one tries to do so).  So I don't work with it routinely.  And so I'd like to ask anyone, preferably more expert than I am in technical aspects of the theory, though not necessarily a de Broglie-Bohm adherent, who can help me understand the above claims, in technical or non-technical terms, to chime in in the comments section.

I have a few specific questions.  It's my impression that in this theory, a "measurement of position" does not measure the pre-existing value of the variable called, in the theory, "position".  That is, if one considers a single trajectory in phase space (position and momentum, over time), entering an apparatus described as a "position measurement apparatus", that apparatus does not necessarily end up pointing to, approximately, the position of the particle when it entered the apparatus.

Question 1:  Is that correct?

A little more discussion of Question 1.  On my understanding, what is claimed is, rather, something like: that if one has a probability distribution over particle positions and momenta and a "pilot wave" (quantum wave function) whose squared amplitude agrees with these distributions (is this required in both position and momentum space? I'm guessing so), then the probability (calculated using the distribution over initial positions and momenta, and the deterministic "laws of motion" by which these interact with the "pilot wave" and the apparatus) for the apparatus to end up showing position in a given range, is the same as the integral of the squared modulus of the wavefunction, in the position representation, over that range.  Prima facie, this could be achieved in ways other than having the measurement reading being perfectly correlated with the initial position on a given trajectory, and my guess is that in fact it is not achieved in that way in the theory.    If that were so it seems like the correlation should hold whatever the pilot wave is.  Now, perhaps that's not a problem, but it makes the pilot wave feel a bit superfluous to me, and I know that it's not, in this theory.  My sense is that what happens is more like:  whatever the initial position is, the pilot wave guides it to some---definite, of course---different final position, but when the initial distribution is given by the squared modulus of the pilot wave itself, then the distribution of final positions is given by the squared modulus of the (initial, I guess) pilot wave.

But if the answer to question 1 is "Yes", I have trouble understanding what Callender means by "I measure position exactly".  Also, regardless of the answer to Question 1, either there is a subtle distinction being made between measuring "perfectly accurately" and measuring "exactly" (in which case I'd like to know what the distinction is), or these sentences need to be reformulated more carefully.  Not trying to do a gotcha on Callender here, just trying to understand the claim, and de Broglie Bohm.

My second question relates to Callender's statement that:

It’s perfectly compatible with the uncertainty principle as it functions in this theory that I measure position exactly and then later calculate the system’s velocity exactly

Question 2: How does this way of ascertaining the system's velocity differ from the sort of "direct measurement" that is, presumably, subject to the uncertainty principle? I'm guessing that by the time one has enough information (possibly about further positions?) to calculate what the velocity was, one can't do with it the sorts of things that one could have done if one had known the position and velocity simultaneously.  But this depends greatly on what it would mean to "have known" the position and/or velocity, which --- especially if the answer to Question 1 was "Yes"--- seems a rather subtle matter.

So, physicists and other readers knowledgeable on these matters (if any such exist), your replies with explanations, or links to explanations, of these points would be greatly appreciated.  And even if you don't know the answers, but know de Broglie-Bohm well on a technical level... let's figure this out!  (My guess is that it's well known, and indeed that the answer to Question 1 in particular is among the most basic things one learns about this interpretation...)

Nagel's Mind and Cosmos, Objective Value, Delong and Blackburn

I have trouble understanding why critics of Thomas Nagel's Mind and Cosmos are coming down so hard on his belief that value statements---particularly ethical ones, can (some of them, at any rate) be objectively true or false.  I'll consider two examples here.  Brad DeLong's objection seems to me based primarily on his continued mistaken view that Nagel views his reason as infallible.  It's therefore not specific to the case of moral or other value judgments.  Simon Blackburn's objections are more interesting because they are more specific to value judgments, and better address Nagel's actual position.

Brad DeLong seems to think that Nagel's juxtaposition of reasoning in the form modification of a belief about the direction one is driving in, because of its inconsistency with newly acquired evidence, with reasoning like Nagel's "I oppose the abolition of the inheritance tax... because I recognize that the design of property rights should be sensitive not only to autonomy but also to fairness..." is self-evidently ridiculous.  Says Brad:

"I do wonder: Does Gene Callahan have any idea what he has committed himself to when he endorses Thomas Nagel's claim that Nagel has transcendent direct access to truths of objective reality? I think not:

Thomas Nagel: [ (HB's) ellipsis here, in place of a typo by Brad that repeated part of his own introduction, quoted above, to this quote...] I decide, when the sun rises on my right, that I must be driving north instead of south... because I recognize that my belief that I am driving south is inconsistent with that observation, together with what I know about the direction of rotation of the earth. I abandon the belief because I recognize that it could not be true.... I oppose the abolition of the inheritance tax... because I recognize that the design of property rights should be sensitive not only to autonomy but also to fairness...

Game, set, match, and tournament!"

That last sentence, which is Brad's, seems revealing of a mindset that sometimes creeps into his writing in his blog, less aimed at truth than at victory in some argumentative competition. I like a lot of what he does on his blog, but that attitude, and the related one that reads like an attempt to exhibit his hip and with-it-ness by using internet jargon that the unhip like me have to google ("self-pwnage", which Callahan is said to have committed), are not so appealing. The "transcendent direct access" I have already argued is mostly a straw-man of Brad's own creation, Nagel's point being primarily that (as says immediately following what Brad has quoted) "As the saying goes, I operate in the space of reasons." One aspect of operating in the space of reasons is trying to preserve some consistency between ones various beliefs; that seems to be the nub of the driving example (but we should not forget the important point that there is more than just deductive logic going on here... we have to decide which of the contradictory beliefs to give up). And we are also to some extent doing so (preserving consistency) in the case of the inheritance tax, though the full argument in this case is likely to be much more involved and less clear than in the case of the driving example. Nagel is arguing that we try to square our beliefs about the particular case of the inheritance tax with general beliefs that we (may) hold about how social institutions like property rights should be designed. Focusing on this consistency issue, though, can --- in both factual and ethical situations --- obscure the essential role of factors other than mere consistency in the process of reasoning about what beliefs to hold. As I mentioned in earlier posts, Nagel gives this somewhat short shrift, notably by not discussing inductive reasoning much, though he's clear about the fact that it's needed. But it's remarkable that DeLong---who I would guess shares Nagel's views on the inheritance tax, and possibly even his reasons (although he may also find some strength in arguments involving "social welfare functions) should think that this passage grounds an immediate declaration of victory. I guess it's because he wrongly thinks the issue is about "direct transcendent access".

Even more remarkable is philosopher Simon Blackburn's very similar reaction---if, as I am guessing, his example of "why income distribution in the US is unjust" is prompted in part by Nagel's reference to the inheritance tax. There are points I agree with in Blackburn's article, but then there is this:

According to Nagel, Darwinians can explain, say, why we dislike pain and seek to minimize bringing it about for ourselves and for others we love. But, Nagel thinks, for the Darwinian, its “real badness” can be no part of the explanation of why we are averse to it. So it is another mystery how real badness and other real normative properties enter our minds. Nagel here manifests his founding membership of a peculiar and fortunately local philosophical subculture that thrives by resolutely dismissing the resources of the alternative, Humean picture, which sees our judgement that pain is a bad thing as a useful expression of our natural aversion to it. All he says about this is that it “denies that value judgements can be true in their own right”, which he finds implausible. He is silent about why he thinks this, perhaps wisely, if only because nobody thinks that value judgements are true in their own right. The judgement that income distribution in the US is unjust, for instance, is not true in its own right. It is true in virtue of that fact that after decades of lobbying, chief executives of major companies earn several hundred times the income of their rank-and-file workers. It is true because of natural facts.

Parenthetically, but importantly: I agree with Blackburn's characterization of Nagel as believing that the "real badness"
of pain cannot be a main part of a Darwinian explanation of our aversion to pain. And I disagree with this belief of Nagel's.

However, I don't know what's so peculiar and local about resolutely dismissing (sometimes with plenty of discussion, though one virtue of Nagel's book is that it is short, so a point like this may not get extensive discussion) the Humean view here that this badness is just "natural aversion".  But in any case, Blackburn's discussion of his example is truly weird.  It seems reasonable to view a statement like "income distribution in the US is unjust" as true both because of the "natural facts" Blackburn cites, which explain how it has come to be what it is, and because of the component where the actual "values" come in, which give reasons for our belief that this high degree of inequality, is in fact unjust.  True, according to some theories of justice, e.g. a libertarian one, the genesis of a pattern of income and wealth distribution may be germane to whether or not it is just.  Blackburn might be adducing such an explanation, since he mentions "lobbying" as a cause (and not, say "hard work").  But if so, he still hasn't explained: what's wrong with lobbying?  Why does it cast doubt on the justice of the resulting outcome?  What Nagel means by value judgements being true "in their own right" is not likely that every statement with a value component, like Blackburn's about US income and wealth distribution, is true in and of itself and no reasons can be given for it.  What I think he means is that at some point, probably at many different points, there enters into our beliefs about matters of value an element of irreducible judgement that something is right or wrong, good or bad, and that this is objective, not just a matter of personal taste or "natural aversion".  What Blackburn's statement reads most like, due to his emphasis on "natural facts", is an attempt to substitute the causal factors leading to US income distribution being what it is, for the moral and political considerations---quite involved, perhaps subtle, and certainly contentious---that have led many to judge that it should not be what it is.  It's quite clear from Nagel's discussion of the inheritance tax what he thinks some of those considerations are: "autonomy and fairness". I just don't understand how someone could think that Blackburn's discussion of why US income distribution is unjust is better than an account in terms of concepts like autonomy and fairness---the sort of account that Nagel would obviously give. I've gotten some value from parts of Blackburn's work, even parts of this article, but this part---if this reading is correct---seems monumentally misguided.  Or does he think that the rest of the explanation is that human beings just have a "natural aversion" to income distribution that is as unequal, or perhaps as influenced by lobbying, as the US's currently is.  But you might think that a cursory look at a large part of the Republican party in the US would have disabused him of that notion.

Perhaps I'm being excessively snarky here...advocates, like Blackburn, of the natural aversion view would probably argue that it needs to be supplemented and modified by reasoning.... perhaps it is just that the "irreducibly moral", as opposed to the deductive/analogical reasoning component, of this process, is still just a matter of natural aversion.  I would think more Hobbesian considerations would come into play as well, but that is a matter for (you may be sorry to hear) another post.