Free will and retrocausality at Cambridge II: Conspiracy vs. Retrocausality; Signaling and Fine-Tuning

Expect (with moderate probability) substantial revisions to this post, hopefully including links to relevant talks from the Cambridge conference on retrocausality and free will in quantum theory, but for now I think it's best just to put this out there.

Conspiracy versus Retrocausality

One of the main things I hoped to straighten out for myself at the conference on retrocausality in Cambridge was whether the correlation between measurement settings and "hidden variables" involved in a retrocausal explanation of Bell-inequality-violating quantum correlations are necessarily "conspiratorial", as Bell himself seems to have thought.  The idea seems to be that correlations between measurement settings and hidden variables must be due to some "common cause" in the intersection of the backward light cones of the two.  That is, a kind of "conspiracy" coordinating the relevant hidden variables that can affect the meausrement outcome with all sorts of intricate processes that can affect which measurement is made, such as those affecting your "free" decision as to how to set a polarizer, or, in case you set up a mechanism to control the polarizer setting according to some apparatus reasonably viewed as random ("the Swiss national lottery machine" was the one envisioned by Bell), the functioning of this mechanism.  I left the conference convinced once again (after doubts on this score had been raised in my mind by some discussions at New Directions in the Philosophy of Physics 2013) that the retrocausal type of explanation Price has in mind is different from a conspiratorial one.

Deflationary accounts of causality: their impact on retrocausal explanation

Distinguishing "retrocausality" from "conspiratorial causality" is subtle, because it is not clear that causality makes sense as part of a fundamental physical theory.   (This is a point which, in this form, apparently goes back to Bertrand Russell early in this century.  It also reminds me of David Hume, although he was perhaps not limiting his "deflationary" account of causality to causality in physical theories.)  Causality might be a concept that makes sense at the fundamental level for some types of theory, e.g. a version ("interpretation") of quantum theory that takes measurement settings and outcomes as fundamental, taking an "instrumentalist" view of the quantum state as a means of calculating outcome probabilities giving settings, and not as itself real, without giving a further formal theoretical account of what is real.  But in general, a theory may give an account of logical implications between events, or more generally, correlations between them, without specifying which events cause, or exert some (perhaps probabilistic) causal influence on others.  The notion of causality may be something that is emergent, that appears from the perspective of beings like us, that are part of the world, and intervene in it, or model parts of it theoretically.  In our use of a theory to model parts of the world, we end up taking certain events as "exogenous".  Loosely speaking, they might be determined by us agents (using our "free will"), or by factors outside the model.  (And perhaps "determined" is the wrong word.)   If these "exogenous" events are correlated with other things in the model, we may speak of this correlation as causal influence.  This is a useful way of speaking, for example, if we control some of the exogenous variables:  roughly speaking, if we believe a model that describes correlations between these and other variables not taken as exogenous, then we say these variables are causally influenced by the variables we control that are correlated with them.  We find this sort of notion of causality valuable because it helps us decide how to influence those variables we can influence, in order to make it more likely that other variables, that we don't control directly, take values we want them to.  This view of causality, put forward for example in Judea Pearl's book "Causality", has been gaining acceptance over the last 10-15 years, but it has deeper roots.  Phil Dowe's talk at Cambridge was an especially clear exposition of this point of view on causality (emphasizing exogeneity of certain variables over the need for any strong notion of free will), and its relevance to retrocausality.

This makes the discussion of retrocausality more subtle because it raises the possibility that a retrocausal and a conspiratorial account of what's going on with a Bell experiment might describe the same correlations, between the Swiss National lottery machine, or whatever controls my whims in setting a polarizer, all the variables these things are influenced by, and the polarizer settings and outcomes in a Bell experiment, differing only in the causal relations they describe between these variables.  That might be true, if a retrocausalist decided to try to model the process by which the polarizer was set.  But the point of the retrocausal account seems to be that it is not necessary to model this to explain the correlations between measurement results.  The retrocausalist posits a lawlike relation of correlation between measurement settings and some of the hidden variables that are in the past light cone of both measurement outcomes.  As long as this retrocausal influence does not influence observable past events, but only the values of "hidden", although real, variables, there is nothing obviously more paradoxical about imagining this than about imagining----as we do all the time---that macroscopic variables that we exert some control over, such as measurement settings, are correlated with things in the future.   Indeed, as Huw Price has long (I have only recently realized for just how long) been pointing out, if we believe that the fundamental laws of physics are symmetric with respect to time-reversal, then it would be the absence of retrocausality, if we dismiss its possibility, and even if we accept its possibility to the limited extent needed to potentially explain Bell correlations, its relative scarcity, that needs explaining.  Part of the explanation, of course, is likely that causality, as mentioned above, is a notion that is useful for agents situated within the world, rather than one that applies to the "view from nowhere and nowhen" that some (e.g. Price, who I think coined the term "nowhen") think is, or should be,  taken by fundamental physical theories.  Therefore whatever asymmetries---- these could be somewhat local-in-spacetime even if extremely large-scale, or due to "spontaneous" (i.e. explicit, even if due to a small perturbation) symmetry-breaking --- are associated with our apparently symmetry-breaking experience of directionality of time may also be the explanation for why we introduce the causal arrows we do into our description, and therefore why we so rarely introduce retrocausal ones.  At the same time, such an explanation might well leave room for the limited retrocausality Price would like to introduce into our description, for the purpose of explaining Bell correlations, especially because such retrocausality does not allow backwards-in-time signaling.

Signaling (spacelike and backwards-timelike) and fine-tuning. Emergent no-signaling?

A theme that came up repeatedly at the conference was "fine-tuning"---that no-spacelike-signaling, and possibly also no-retrocausal-signaling, seem to require a kind of "fine-tuning" from a hidden variable model that uses them to explain quantum correlations.  Why, in Bohmian theory, if we have spacelike influence of variables we control on physically real (but not necessarily observable) variables, should things be arranged just so that we cannot use this influence to remotely control observable variables, i.e. signal?  Similarly one might ask why, if we have backwards-in-time influence of controllable variables on physically real variables, things are arranged just so that we cannot use this influence to remotely control observable variables at an earlier time?  I think --- and I think this possibility was raised at the conference --- that a possible explanation, suggested by the above discussion of causality, is that for macroscopic agents such as us, with usually-reliable memories, some degree of control over our environment and persistence over time, to arise, it may be necessary that the scope of such macroscopic "observable" influences be limited, in order that there be a coherent macroscopic story at all for us to tell---in order for us even be around to wonder about whether there could be such signalling or not.  (So the term "emergent no-signalling" in the section heading might be slightly misleading: signalling, causality, control, and limitations on signalling might all necessarily emerge together.) Such a story might end up involving thermodynamic arguments, about the sorts of structures that might emerge in a metastable equilibrium, or that might emerge in a dynamically stable state dependent on a temperature gradient, or something of the sort.  Indeed, the distribution of hidden variables (usually, positions and/or momenta) according to the squared modulus of the wavefunction, which is necessary to get agreement of Bohmian theory with quantum theory and also to prevent signaling (and which does seem like "fine-tuning" inasmuch as it requires a precise choice of probability distribution over initial conditions), has on various occasions been justified by arguments that it represents a kind of equilibrium that would be rapidly approached even if it did not initially obtain.  (I have no informed view at present on how good these arguments are, though I have at various times in the past read some of the relevant papers---Bohm himself, and Sheldon Goldstein, are the authors who come to mind.)

I should mention that at the conference the appeal of such statistical/thermodynamic  arguments for "emergent" no-signalling was questioned---I think by Matthew Leifer, who with Rob Spekkens has been one of the main proponents of the idea that no-signaling can appear like a kind of fine-tuning, and that it would be desirable to have a model which gave a satisfying explanation of it---on the grounds that one might expect "fluctuations" away from the equilibria, metastable structures, or steady states, but we don't observe small fluctuations away from no-signalling---the law seems to hold with certainty.  This is an important point, and although I suspect there are  adequate rejoinders, I don't see at the moment what these might be like.

Free will and retrocausality in the quantum world, at Cambridge. I: Bell inequalities and retrocausality

I'm in Cambridge, where the conference on Free Will and Retrocausality in the Quantum World, organized (or rather, organised) by Huw Price and Matt Farr will begin in a few hours.  (My room at St. Catherine's is across from the chapel, and I'm being serenaded by a choir singing beautifully at a professional level of perfection and musicality---I saw them leaving the chapel yesterday and they looked, amazingly, to be mostly junior high school age.)  I'm hoping to understand more about how "retrocausality", in which effects occur before their causes, might help resolve some apparent problems with quantum theory, perhaps in ways that point to potentially deeper underlying theories such as a "quantum gravity". So, as much for my own use as anyone else's, I thought perhaps I should post about my current understanding of this possibility.

One of the main problems or puzzles with quantum theory that Huw and others (such as Matthew Leifer, who will be speaking) think retrocausality may be able to help with, is the existence of Bell-type inequality violations. At their simplest, these involve two spacelike-separated regions of spacetime, usually referred to as "Alice's laboratory" and "Bob's laboratory", at each of which different possible experiments can be done. The results of these experiments can be correlated, for example if they are done on a pair of particles, one of which has reached Alice's lab and the other Bob's, that have previously interacted, or were perhaps created simultaneously in the same event. Typically in actual experiments, these are a pair of photons created in a "downconversion" event in a nonlinear crystal.  In a "nonlinear"  optical process photon number is not conserved (so one can get a "nonlinearity" at the level of a Maxwell's equation where the intensity of the field is proportional to photon number; "nonlinearity" refers to the fact that the sum of two solutions is not required to be a solution).  In parametric downconversion, a photon is absorbed by the crystal which emits a pair of photons in its place, whose energy-momentum four-vectors add up to that of the absorbed photon (the process does conserve energy-momentum).   Conservation of angular momentum imposes correlations between the results of measurements made by "Alice" and "Bob" on the emitted photons. These are correlated even if the measurements are made sometime after the photons have separated far enough that the changes in the measurement apparatus that determine which component of polarization it measures (which we'll henceforth call the "polarization setting"), on one of the photons, are space-like separated from the measurement process on the other photon, so that effects of the polarization setting in Alice's laboratory, which one typically assumes can propagate only forward in time, i.e. in their forward light-cone, can't affect the setting or results in Bob's laboratory which is outside of this forward light-cone.  (And vice versa, interchanging Alice and Bob.)

Knowledge of how their pair of photons were prepared (via parametric downconversion and propagation to Alice and Bob's measurement sites) is encoded in a "quantum state" of the polarizations of the photon pair.  It gives us, for any pair of polarization settings that could be chosen by Alice and Bob, an ordinary classical joint probability distribution over the pair of random variables that are the outcomes of the given measurements.  We have different classical joint distributions, referring to different pairs of random variables, when different pairs of polarization settings are chosen.   The Bell "paradox" is that there is no way of introducing further random variables that are independent of these polarization settings, such that for each pair of polarization settings, and each assignment of values to the further random variables, Alice and Bob's measurement outcomes are independent of each other, but when the further random variables are averaged over, the experimentally observed correlations, for each pair of settings, are reproduced. In other words, the outcomes of the polarization measurements, and in particular the fact that they are correlated, can't be "explained" by variables uncorrelated with the settings. The nonexistence of such an explanation is implied by the violation of a type of inequality called a "Bell inequality". (It's equivalent to to such a violation, if "Bell inequality" is defined generally enough.)

How I stopped worrying and learned to love quantum correlations

One might have hoped to explain the correlations by having some physical quantities (sometimes referred to as "hidden variables") in the intersection of Alice and Bob's backward light-cone, whose effects, propagating forward in their light-cone to Alice and Bob's laboratories, interact their with the physical quantities describing the polarization settings to produce---whether deterministically or stochastically---the measurement outcomes at each sites, with their observed probabilities and correlations. The above "paradox" implies that this kind of "explanation" is not possible.

Some people, such as Tim Maudlin, seem to think that this implies that quantum theory is "nonlocal" in the sense of exhibiting some faster-than-light influence. I think this is wrong. If one wants to "explain" correlations by finding---or hypothesizing, as "hidden variables"---quantities conditional on which the probabilities of outcomes, for all possible measurement settings, factorize, then these cannot be independent of measurement settings. If one further requires that all such quantities must be localized in spacetime, and that their influence propagates (in some sense that I'm not too clear about at the moment, but that can probably be described in terms of differential equations---something like a conserved probability current might be involved) locally and forward in time, perhaps one gets into inconsistencies. But one can also just say that these correlations are a fact. We can have explanations of these sorts of fact---for example, for correlations in photon polarization measurements, the one alluded to above in terms of energy-momentum conservation and previous interaction or simultaneous creation---just not the sort of ultra-classical one some people wish for.

Retrocausality

It seems to me that what the retrocausality advocates bring to this issue is the possibility of something that is close to this type of classical explanation. It may allow for the removal of these types of correlation by conditioning on physical quantities. [Added July 31: this does not conflict with Bell's theorem, for the physical quantities are not required to be uncorrelated with measurement settings---indeed, being correlated with the measurement settings is to be expected if there is retrocausal influence from a measurement setting to physical quantities in the backwards light-cone of the measurement setting.] And unlike the Bohmian hidden variable theories, it hopes to avoid superluminal propagation of the influence of measurement settings to physical quantities, even unobservable ones.  It does this, however, by having the influence of measurement settings pursue a "zig-zag" path from Alice to Bob: in Alice's backward light-cone back to the region where Alice and Bob's backward light-cones intersect, then forward to Bob's laboratory. What advantages might this have over superluminal propagation? It probably satisfies some kind of spacetime continuity postulate, and seems more likely to be able to be Lorentz-invariant. (However, the relation between formal Lorentz invariance and lack of superluminal propagation is subtle, as Rafael Sorkin reminded me at breakfast today.)