Interview with Dr. Henry Bauer - Part 1

It seems to me that most experimental parapsychologists spend a lot of their time trying to falsify null hypotheses. And those hypotheses are falsifiable in principle, and if they are falsified it proves that there's something anomalous going on. Does the fact that they're trying to falsify the null hypothesis rather than a psi hypothesis make the process any less scientific?

[Laird]

Falsification is about whether your hypothesis reflects what you would see if your idea is true or whether it reflects what you would see if your hypothesis is false. This is different than the distinction between an alternative and a null hypothesis.

I haven't yet read the references you've provided, and am not sure whether/when I'll do that, so perhaps in the meantime (if you choose to - no pressure if you're not inclined) you could elucidate this difference with a concrete example in the field at issue: parapsychology.

Let's take, for example, psychokinesis (or whichever phenomenon you prefer). Perhaps you could suggest an example of both a "falsifying" test and a "proving" test (Steve's labels), as well as what the alternative and null hypotheses would be in each case.

I haven't yet read the references you've provided, and am not sure whether/when I'll do that, so perhaps in the meantime (if you choose to - no pressure if you're not inclined) you could elucidate this difference with a concrete example in the field at issue: parapsychology.

Let's take, for example, psychokinesis (or whichever phenomenon you prefer). Perhaps you could suggest an example of both a "falsifying" test and a "proving" test (Steve's labels), as well as what the alternative and null hypotheses would be in each case.

An example of using confirming tests would be the Global Consciousness Project. The alternative and null hypotheses would be that the eggs will or will not show anomalous deviations associated with global event.

http://noosphere.princeton.edu/results.html

An example of using a disconfirming test would be "Remarkable Correspondences Between Ganzfeld Mentation and Target Content - A Psychical or Psychological Effect?". The alternative and null hypotheses would be that remarkable correspondences will or will not be produced at a lower rate in the decoy group.

http://w3.psychology.su.se/staff/jwd/cs06westerlund.pdf

Linda

[Laird]

The alternative and null hypotheses would be that remarkable correspondences will or will not be produced at a lower rate in the decoy group.

That's a very long paper to read, and I am unlikely to do so in the near future. But let me ask you this: how is your characterisation of the alternative and null hypotheses, and, especially, the related analysis, functionally different from swapping those hypotheses, such that instead of your original statement, we had this: "The alternative and null hypotheses would be that remarkable correspondences will or will not be produced at a higher rate in the non-decoy group"? Note that the only changes I have made are "lower" into "higher" and "decoy" into "non-decoy".

That's a very long paper to read, and I am unlikely to do so in the near future. But let me ask you this: how is your characterisation of the alternative and null hypotheses, and, especially, the related analysis, functionally different from swapping those hypotheses, such that instead of your original statement, we had this: "The alternative and null hypotheses would be that remarkable correspondences will or will not be produced at a higher rate in the non-decoy group"? Note that the only changes I have made are "lower" into "higher" and "decoy" into "non-decoy".

I don't think they are functionally different.

Note that I put it that way (making the mention of the decoy explicit) so you wouldn't have to ask me "what has that to do with the 'no psi' condition?", given that you may or may not look at the paper. In this case, there is an explicit measure of the 'no psi' condition (as well as the psi condition). If that is confusing, imagine running the experiment looking only at whether remarkable correspondences are produced at all in the decoy group. 

Linda

[Laird]

I don't think they are functionally different.

OK, and so would you then accept that the alternative framing I posed would be in your terms a "confirming" test, and that therefore this example could be described either as a "disconfirming" or as a "confirming" test, given that functionally both framings are the same, and that therefore you haven't provided a definitive example of a(n exclusively) "disconfirming" test? If not, why not?

[Edit: In the above I ought to have used "hypothesis" rather than "test", since that's what we were discussing. As became clear later in this thread, Linda seemed from the start to be unclear as to which of "hypothesis" and "test" she intended her "disconfirming" versus "confirming" distinction to apply, and it seems I succumbed here to that confusion]

OK, and so would you then accept that the alternative framing I posed would be in your terms a "confirming" test, and that therefore this example could be described either as a "disconfirming" or as a "confirming" test, given that functionally both framings are the same, and that therefore you haven't provided a definitive example of a "disconfirming" test? If not, why not?

Why not? There may not be an actual example. I don't remember off the top of my head if there are examples of a disconfirming test which doesn't also include a confirming test among parapsychology studies (there are very few disconfirming tests to begin with). It's simpler to do what I suggested earlier - just drop the part of the study which includes a confirming test in order to understand what 'disconfirmation only' looks like.
 
My second example - imagine running the study looking only at whether remarkable correspondences would be produced at all in the decoy group - would be an example of a disconfirming test.

Another example of disconfirming and confirming tests can be found in Wiseman's and Sheldrake's experiments with JayTee. Pam's parents noted that the dog would go to the door for no apparent reason and stay there for a while, and that would be their signal that Pam was returning home.

Wiseman looked at whether or not this signal was present when Pam was not returning home (disconfirming test). Sheldrake looked at whether the dog was at the door more when Pam was returning home (confirming test). Note that in the actual studies, Sheldrake did not examine how his test would perform when Pam was not returning home. Wiseman did look at both conditions - "was the signal present?" when Pam was not returning home and was returning home. Like the previous example, imagine that Wiseman looked only at whether the signal was present when Pam was not returning home, for a definitive example of a disconfirming test.

[Edit: In the above I ought to have used "hypothesis" rather than "test", since that's what we were discussing. As became clear later in this thread, Linda seemed from the start to be unclear as to which of "hypothesis" and "test" she intended her "disconfirming" versus "confirming" distinction to apply, and it seems I succumbed here to that confusion]

There shouldn't be confusion. The paper I referenced refers specifically to hypothesis tests, including describing a confirming test (+Htest) and a disconfirming test (-Htest). I also referred to tests in my initial post. Then, once you made it clear that you had not read the papers (and so, could potentially miss this point), I made sure that I explicitly referred to confirming and disconfirming tests of the hypotheses for the remainder of our discussion.

Linda

[Laird]

It's simpler to do what I suggested earlier - just drop the part of the study which includes a confirming test in order to understand what 'disconfirmation only' looks like.

I don't see how this is, as Steve suggested, more scientifically rigorous. For example:

My second example - imagine running the study looking only at whether remarkable correspondences would be produced at all in the decoy group - would be an example of a disconfirming test.

I don't get it. How is a test in isolation better than a comparative test?

Wiseman looked at whether or not this signal was present when Pam was not returning home (disconfirming test).

No. Wiseman constructed an arbitrary criterion: if Jaytee went to the door at any time before Pam was returning home, then the trial was counted as a failure.

Sheldrake looked at whether the dog was at the door more when Pam was returning home (confirming test). Note that in the actual studies, Sheldrake did not examine how his test would perform when Pam was not returning home.

No again. Rupert Sheldrake compared the scenarios of "Pam returning" to "Pam not returning" - and you have admitted that such a comparative test could be either described as "falsifying" or "confirmatory", because functionally both are identical - whereas...

Wiseman did look at both conditions - "was the signal present?" when Pam was not returning home and was returning home.

....Wiseman weaselled his way around this by stipulating that if at any time prior to Pam returning home Jaytee went to the window, then the test was a failure. This is the height of biasing the odds in your favour. He at least accepted that the data he got in his experiment were comparable with the data Rupert Sheldrake got.

Like the previous example, imagine that Wiseman looked only at whether the signal was present when Pam was not returning home, for a definitive example of a disconfirming test.

I can imagine it. Here's what I'm imagining: an utterly shitty test. Without a comparison to the scenario when Pam was returning home, it would be utterly stupid to conclude anything from solely the results when Pam was not returning home.

Basically, I see no argument or evidence to back up Steve's claim that parapsychology has some unique difference to the rest of science with respect to "proving" rather than "falsifying". The only examples you, Linda, have given are poor ones, since they avoid comparisons - and comparisons are how we find out which alternative is more likely. You have admitted that where there is a comparison, the test cannot be definitively described as either "confirmatory" or "falsifying" because both are functionally identical - I would like you now to admit that non-comparative tests - where comparative tests are possible - are inferior. Can you do that?

My feeling is that if there is a problem with repeatability of psi effects, then that makes it harder to "characterise" psi by a falsifiable hypothesis. (If the hypothesis is falsified, it might just be because of the repeatability problem.) But that could just be because we have so little understanding of psi. If we knew more, the repeatability problem might go away. Or it might be more fundamental for some reason.

[Laird]

My feeling is that if there is a problem with repeatability of psi effects, then that makes it harder to "characterise" psi by a falsifiable hypothesis. (If the hypothesis is falsified, it might just be because of the repeatability problem.) But that could just be because we have so little understanding of psi. If we knew more, the repeatability problem might go away. Or it might be more fundamental for some reason.

Fair enough, Chris, but you say that as something like an aside, right? Because I can't see how it bears directly on the proposition Steve made that parapsychology is unique in trying to "prove" rather than "falsify" hypotheses.