Principles of Curiosity

[Laird]

In the “Uri Geller – What do you think” thread I compare two documents both written by Targ and Puthoff on the Geller experiments: one original report for the CIA and then the Nature paper.

http://psiencequest.net/forums/thread-16...ml#pid2930

They differed in that the actual target words had sometimes been changed from one to the other, and in reporting which trials Uri Geller had actually not offered a guess for.

Thanks. I suspected that that was what you were referring to. Yes, this is very "questionable" behaviour. That said, it seems to me to be more cosmetic than fundamental. e.g. Whether the target word was "bunch" or "grapes", the drawings coincided to a remarkable degree - including in the number of grapes. I don't think that that correspondence can be wished away on the basis of the target word having been changed between diary notes and official paper.

OK. I'm using 'degrees of freedom' to include practices around the statistical analysis... Which data to include, and even the type of statistical analysis applied etc.

Yes, I agree that can be a problem, but it should be possible to address it by pre-registration, including specification of the statistical analysis to be used.

But I think if that is addressed, and if the statistical evidence is adequate, then the smallness of the effect is no reason to reject a phenomenon per se.

In the “Uri Geller – What do you think” thread I compare two documents both written by Targ and Puthoff on the Geller experiments: one original report for the CIA and then the Nature paper.

http://psiencequest.net/forums/thread-16...ml#pid2930

They differed in that the actual target words had sometimes been changed from one to the other, and in reporting which trials Uri Geller had actually not offered a guess for.

Obviously the inconsistency in reporting whether Geller made a guess is a serious flaw.

I'm not so sure about the target words. It seems to me the targets were the pictures, not the words, especially given that the dictionary method was only used for the first four trials. They didn't even bother to record the dictionary words for the third and fourth (though it's clear from the report that the third picture didn't start out as a devil, but only turned into one as it was drawn).

I didn't understand what Alex was referring to there. He did make it sound as though it was a particular study. I suspect he was talking about the idea that parapsychologists have been aware for some time of the issues plaguing conventional psychology at the moment, and have taken a lead in developing remedies. I think that view is shared by moderate sceptics such as Caroline Watt.

Actually, I think that is illustrated by the paper from 2009 posted by David Bailey on Skeptiko:
https://www.edvul.com/pdf/VulHarrisWinki...S-2009.pdf

It pointed out that published functional MRI studies were overestimating correlations between MRI data and personality measures, because researchers were actually selecting a subset of data that showed high correlations, calculating averages from that subset and discarding the rest of the data. I'll be happy to be proved wrong with a counter-example, but I don't think that would have happened in parapsychology, even at that time.

I don't think many 'skeptics' (or proponents) are interested in 'small effect size psi' justified by mathematical significance. To paraphrase, there are lies, damn lies, and Radin analysis.

MAlf:

Are you saying that "small" effects don't matter, or are unconvincing? Or rather that "most" people just don't care about them?

Not sure I agree with any of these, but wanted to understand your point.

Thanks. I suspected that that was what you were referring to. Yes, this is very "questionable" behaviour. That said, it seems to me to be more cosmetic than fundamental. e.g. Whether the target word was "bunch" or "grapes", the drawings coincided to a remarkable degree - including in the number of grapes. I don't think that that correspondence can be wished away on the basis of the target word having been changed between diary notes and official paper.

I'm not attempting to wish away the correspondences: I think they're there for all to see, but I disagree that it is merely cosmetic. At some point while writing up their work for Nature, Targ & Puthoff must've made a conscious decision to change the data in the paper. The word is the seed for the target drawing, and by changing that Targ & Puthoff have shrunk the semantic gap between word and drawing, thus diminishing how much leeway the person doing the drawing actually had.

Not that I want to get into another discussion about this. My point was merely that Alex's assertion that the SRI work was beyond question was just plain wrong, and I wondered how would he react if he knew that the Nature paper was not wholly accurate.

MAlf:

Are you saying that "small" effects don't matter, or are unconvincing? Or rather that "most" people just don't care about them?

Not sure I agree with any of these, but wanted to understand your point.

Follow the subsequent exchange between Chris and me where we arrive at some consensus of the issues. Also check out the 'pushing false positives' link I posted #39.

http://theness.com/neurologicablog/index...positives/

An article by someone who thinks Bayesian analysis requires no more assumptions than frequentist analysis, doesn't inspire confidence. 

An article by someone who thinks Bayesian analysis requires no more assumptions than frequentist analysis, doesn't inspire confidence. 

There is some discussion around this in the comments.

There is some discussion around this in the comments.

To my mind, Novella's response in the comments underlines the fact that he doesn't really understand this. It's not just a single prior probability you have to specify for Bayesian analysis, but a whole prior probability distribution.