Don't Birnbaumize that Experiment my Friend*

(A)  "It is not uncommon to see statistics texts argue that in frequentist theory one is faced with the following dilemma: either to deny the appropriateness of conditioning on the precision of the tool chosen by the toss of a coin[i], or else to embrace the strong likelihood principle which entails that frequentist sampling distributions are irrelevant to inference once the data are obtained.  This is a false dilemma … The 'dilemma' argument is therefore an illusion". (Cox and Mayo 2010, p. 298)

The “illusion” stems from the sleight of hand I have been explaining in the Birnbaum argument—it starts with Birnbaumization.
(B) A reader wrote in that he awaits approval of my argument by either Sir David Cox or Christian Robert ; I cannot vouchsafe for Robert, unless he has revised his first impression in his October 6, 2011 blog (as I hope he has). For in that blog post Robert says

“If Mayo’s frequentist stance leads her to take the sampling distribution into account at all times, this is fine within her framework. But I do not see how this argument contributes to invalidate Birnbaum’s proof.”


I am taking sampling distributions into account because Birnbaum’s “proof” is supposed to be relevant for a sampling theorist!   If it is not relevant for a sampling theorist (my error statistician) then there is no breakthrough and there is no special interest in the result (given that Bayesians already have the LP, as do the likelihoodists.)[ii] It is only because principles that are already part of the sampling theorist’s steady diet are alleged to entail the LP (in Birbaum's argument) that Savage declared that, once made aware of Birnbaum’s result, he doubted people would stop at the LP appetizer, but would instead go all the way to consuming the full Bayesian omelet!


Robert’s remark is just the tip of the iceberg that reveals a deep misunderstanding of sampling theory.  (Although I prefer error statistics, I will use sampling theory for this post.)   Even if Robert has corrected himself, as I very much hope he has, other readers may be under the same illusion. I had paused to clarify this point in my October 20, 2011 post.

Model Validation and the LP-(Long Playing Vinyl Record)

A Bayesian acquaintance writes:

Although the Birnbaum result is of primary importance for sampling theorists, I'm still interested in it because many Bayesian statisticians think that model checking violates the likelihood principle, as if this principle is a fundamental axiom of Bayesian statistics.  

But this is puzzling for two reasons. First, if the LP does not preclude testing for assumptions (and he is right that it does not[i]), then why not simply explain that rather than appeal to a disproof of something that actually never precluded model testing?   To take the disproof of the LP as grounds to announce: “So there! Now even Bayesians are free to test their models” would seem only to ingrain the original fallacy*.

The 3 stages of the acceptance of novel truths

 There is an often-heard slogan about the stages of the acceptance of novel truths:

First people deny a thing. 

Then they belittle it. 

Then they say they knew it all along.

I don’t know who was first to state it in one form or another.  Here’s Schopenhauer with a slightly different variant:

"All truth passes through three stages: First, it is ridiculed; Second, it is violently opposed; and Third, it is accepted as self-evident."        - Arthur Schopenhauer

After recently presenting my paper criticizing the Birnbaum result on the likelihood principle (LP)[1] the reception of my analysis seems somewhere around stage two, in some cases, moving into stage three (see my blogposts of December 6 and 7, 2011).

But it is time to make good on my promise to return to concerns of those (at least in the blogosphere), who were or are still at the first stage of denial (or Schopenhauer's second stage of violent opposition).  Doing so will advance our goal of drilling deeply into some fundamental, puzzling misunderstandings of frequentist error statistical (or sampling) theory.  

Putting the Brakes on the Breakthrough Part I*

I am going to post a FIRST draft (for a brief presentation next week in Madrid).  [I thank David Cox for the idea!] I expect errors, and I will be very grateful for feedback!  This is part I; part II will be posted tomorrow.  These posts may disappear once I've replaced them with a corrected draft.  I'll then post the draft someplace.

  If you wish to share queries/corrections please post as a comment or e-mail: error@vt.edu.  (ignore Greek symbols that are not showing correctly, I await fixes by Elbians.) Thanks much!

ONE: A Conversation between Sir David Cox and D. Mayo (June, 2011)

Toward the end of this exchange, the issue of the Likelihood Principle (LP)[1] arose:

COX: It is sometimes claimed that there are logical inconsistencies in frequentist theory, in particular surrounding the strong Likelihood Principle (LP). I know you have written about this, what is your view at the moment.

MAYO: What contradiction?