Statistical Significance

What a nerdy debate about p-values shows about science — and how to fix it

Possible impacts on the sort of studies that make up much of psychical research?

What a nerdy debate about p-values shows about science — and how to fix it

Possible impacts on the sort of studies that make up much of psychical research?

From memory, from an interview on Youtube somewhere, Dean Radin said that some of his studies had a P-Value of less than 0.5, meaning that it went beyond the usual standard for there being something to PSI phenomena. However I could be wrong and misremembered that for the 6 or 7 sigma results that he was getting.

What a nerdy debate about p-values shows about science — and how to fix it

Possible impacts on the sort of studies that make up much of psychical research?

From memory, from an interview on Youtube somewhere, Dean Radin said that some of his studies had a P-Value of less than 0.5, meaning that it went beyond the usual standard for there being something to PSI phenomena. However I could be wrong and misremembered that for the 6 or 7 sigma results that he was getting.

I think he would probably have said 0.05, not 0.5, as that's the usual standard. 

But hard-line sceptics want to be much more radical than just tightening the p-value criterion by a factor of 10, the proposal mentioned in this article. Bayesian statistics allows them to specify their own estimates of the a priori unlikelihood of any particular phenomenon, and of course hard-line sceptics think psi phenomena are very unlikely indeed.

Not being a maths type myself, I find that this kind of statistical nit-picking effectively restricts the debate to a small circle of maths types. Linda and others used that tactic for years on the Skeptiko forum. Far from clarifying and quantifying the data, it is used to promote doubt and uncertainty. Then you get statisticians arguing with statisticians, arguments about who's the most qualified and the original discussion goes down the toilet. Debate over.

Not being a maths type myself, I find that this kind of statistical nit-picking effectively restricts the debate to a small circle of maths types. Linda and others used that tactic for years on the Skeptiko forum. Far from clarifying and quantifying the data, it is used to promote doubt and uncertainty. Then you get statisticians arguing with statisticians, arguments about who's the most qualified and the original discussion goes down the toilet. Debate over.

Human nature being what it is, some people will use technical concepts as a means of obfuscation rather than communication. But it shouldn't be so, because the concepts are generally easy enough to understand, even if the numbers are sometimes difficult to calculate.

For example, in psi experiments, the p value is usually just the probability that the observed result (or a more extreme result) would occur by chance, even if psi didn't exist.

What concerns me is that "by chance" becomes the criteria for all psi events. What I mean is that the assumption is made that if it could have happened by chance then we have to conclude that it DID happen by chance. That just ain't necessarily so, to paraphrase the song.

What concerns me is that "by chance" becomes the criteria for all psi events. What I mean is that the assumption is made that if it could have happened by chance then we have to conclude that it DID happen by chance. That just ain't necessarily so, to paraphrase the song.

If people do argue that, it's really a negation of the statistical approach, because the point of calculating statistics like the p value is to help us to judge how likely it is that it happened by chance (though unfortunately the p value doesn't itself equate to that likelihood).

Not being a maths type myself, I find that this kind of statistical nit-picking effectively restricts the debate to a small circle of maths types. Linda and others used that tactic for years on the Skeptiko forum. Far from clarifying and quantifying the data, it is used to promote doubt and uncertainty. Then you get statisticians arguing with statisticians, arguments about who's the most qualified and the original discussion goes down the toilet. Debate over.

Human nature being what it is, some people will use technical concepts as a means of obfuscation rather than communication. But it shouldn't be so, because the concepts are generally easy enough to understand, even if the numbers are sometimes difficult to calculate.

For example, in psi experiments, the p value is usually just the probability that the observed result (or a more extreme result) would occur by chance, even if psi didn't exist.

What concerns me is that "by chance" becomes the criteria for all psi events. What I mean is that the assumption is made that if it could have happened by chance then we have to conclude that it DID happen by chance. That just ain't necessarily so, to paraphrase the song.

If people do argue that, it's really a negation of the statistical approach, because the point of calculating statistics like the p value is to help us to judge how likely it is that it happened by chance (though unfortunately the p value doesn't itself equate to that likelihood).

So basically you are saying what I just claimed was wrong with the assumption, aren't you? If something proves that it could easily have happened by chance does it prove that it actually did happen by chance? I don't believe that it does.

An example: I have a dream that the national soccer team of New Zealand win the next world cup. I write it down and give it to a journalist and a lawyer in case I need to prove the date of my predictive dream. Astonishingly, New Zealand come through and win the World Cup and my dream is vindicated. Immediately you will get statisticians saying that, because there are so many billions of people in the world having dreams and a large proportion of them are aware of the soccer World Cup, then it is statistically likely that someone would have that dream.

Now, my question is: does that prove that my dream was the result of pure chance or does it prove that chance was one of the options as an explanation? From what I can see, a sceptic would say: of course it was chance, case closed.

Not being a maths type myself, I find that this kind of statistical nit-picking effectively restricts the debate to a small circle of maths types. Linda and others used that tactic for years on the Skeptiko forum. Far from clarifying and quantifying the data, it is used to promote doubt and uncertainty. Then you get statisticians arguing with statisticians, arguments about who's the most qualified and the original discussion goes down the toilet. Debate over.

Here is the problem as "the skeptics" see it... In more or less plain English. Not too much maths:

http://theness.com/neurologicablog/index...ical-sins/

I intend to listen to this over the weekend:

https://audioboom.com/posts/2731537-rati...y-research

No, I'm not saying that if something could have happened by chance then it must have happened by chance, or anything like that. I'm just saying that statistics helps us to judge the likelihood that it happened by chance.