(2018-02-02, 10:47 AM)Chris Wrote: [ -> ]I see that three more people have put their names to the letter - Dr Linda Schultz, Dr Rickard Carlsson and Dr Stefan Schmukle.
Rickard Carlsson is a senior lecturer in the Department of Psychology at Linnaeus University in Sweden, and Stefan Schmukle is a professor in the Department of Psychology at the University of Leipzig.
(2018-02-02, 10:47 AM)Chris Wrote: [ -> ]Thanks. Prof. Schimmack previously circulated a draft of the letter for comment, including to some of those who had commented on his blog. I see that three more people have put their names to the letter - Dr Linda Schultz, Dr Rickard Carlsson and Dr Stefan Schmukle.
My own comments on the first draft of the letter were as follows:
[elided]
Thanks for sharing that, Chris. It all seems pretty reasonable to me. Not being particularly statistically literate, it is hard for me to contribute to this conversation. Lacking sufficient statistical literacy, I can only take at face value the claims of various skeptical folk that, e.g., the data lack plausibility because they are "too good to be true" given the supposed effect size. On the other hand, if Daryl Bem was fudging the data in the way he would need to have to have achieved these "too good to be true" results, then I think that that which you contribute here is key:
(2018-02-02, 10:47 AM)Chris Wrote: [ -> ]I think if it were possible to come up with a definite "recipe" by which the observed decline effect and the statistical properties 1-3 could be generated from chance data, then your case would become much stronger - though after looking at the dates in the data file I was left with the impression that there wasn't really time to fit in the kind of numbers of unreported participants that would be needed in the scenario you suggested.
If the letter had been in its present form, I'd also have commented that - while Schimmack's scenario of discarding pilot experiments if unsuccessful and continuing them if successful would, in qualitative terms, produce a decline effect - I'm not convinced it would produce a decline effect which quantitatively matched the observed one.
I agree that (hopefully paraphrasing you accurately) if these results are to be definitively accepted to be a result of - even inadvertent - biased choices such that the paper deserves to be retracted, then a plausible model (i.e. with sufficient detail) needs to be provided as to how this occurred.
Tbf, the replication rate right now for statistics significance is around 10%.
(2018-01-28, 12:39 AM)Chris Wrote: [ -> ]I don't know how interested people are, but the discussion is continuing.
Linda has compared the statistics of Bem's experiment 101 (published in 2003) and of the first 50 participants in his experiment 5 (published in 2011), and has concluded that - as suspected previously - they are the same. (There is a factor of 10 difference in one statistic, but I assume that's a typo.)
Because 101 involved 6 classes of images, but the description of 5 made it sound as though there was only one class of images plus controls, Linda says "I think this may be sufficient evidence to consider calling for a retraction."
https://replicationindex.wordpress.com/2...mment-3486
I'd like to ask Linda (assuming she is the same Linda who posts here as fls) to clarify one of her criticisms of Daryl Bem's paper. I did ask this previously on Prof. Schimmack's blog, but got no response. Possibly my request was missed.
Linda wrote:
Experiment 7
The description of this experiment is different from the initial report, which included strongly negative and erotic pictures. Either Bem neglected to include the results from 146 of the subjects, or neglected to include all the trials from each subject.
Linda provided a link to what she referred to as the "original report" of Experiments 5 and 6, namely a modified version of a presentation given at the 46th Annual Convention of the Parapsychlogical Association in 2003, entitled "Precognitive Habituation: Replicable Evidence for a Process of Anomalous Cognition":
https://pdfs.semanticscholar.org/8033/f0...0b2e73.pdf
But this can't be the initial report of Experiment 7, as the data file shows that this experiment was conducted 18 months after the presentation. I wonder if Linda was referring to a different report of Experiment 7, or if she was mistaken in thinking that Experiment 7 was included in the 2003 report.
I agree. Now that the data (with dates) from Feeling the Future has been made available, Bem does not appear to have included any of the supraliminal experiments (including his experiment on the retroactive induction of boredom) from 2003 in Feeling the Future.
Linda
(2018-02-05, 12:05 PM)fls Wrote: [ -> ]I agree. Now that the data (with dates) from Feeling the Future has been made available, Bem does not appear to have included any of the supraliminal experiments (including his experiment on the retroactive induction of boredom) from 2003 in Feeling the Future.
Thank you for clarifying that.
The supraliminal experiment published in 2003 is apparently the one mentioned in the "file drawer" section of "Feeling the Future".
I just thought I'd post a link here to Larry Dossey's 2015 article in Explore, entitled "Why Are Scientists Afraid of Daryl Bem?"
http://www.explorejournal.com/article/S1...1/fulltext
It's really about the (scientific) politics of the controversy over Bem's work, and not about the details of the work itself. As one would expect, it makes no attempt to be balanced, but sides wholeheartedly with Bem against his critics. But it does have a useful list of 91 references, which includes contributions on both sides of the argument, as well as material related to the general debate between proponents and sceptics.
Also, I see that there is a new article on Dr Schimmack's blog, presenting a revised version (with parenthetical comments) of a paper he published in 2012, criticising "Feeling the Future" because he would have expected more non-significant results on the basis of the reported effect sizes:
https://replicationindex.wordpress.com/2...gic-index/
At the start, he comments:
"Aside from Alcock’s demonstration of a nearly perfect negative correlation between effect sizes and sample sizes and my demonstration of insufficient variance in Bem’s p-values, Francis’s article and my article remain the only article that question the validity of Bem’s origina findings. Other articles have shown that the results cannot be replicated, but I showed that the original results were already too good to be true. This blog post explains, how I did it."
For comparison, the version of Schimmack's paper published in 2012 is here:
http://citeseerx.ist.psu.edu/viewdoc/dow...1&type=pdf
Alcock's comment about the negative correlation between effect size and sample size is in an online article here:
https://www.csicop.org/specialarticles/s...the_future
And Francis's paper is here:
https://link.springer.com/article/10.375...012-0227-9
(2018-03-01, 09:23 AM)Chris Wrote: [ -> ]At the start, he comments:
"Aside from Alcock’s demonstration of a nearly perfect negative correlation between effect sizes and sample sizes and my demonstration of insufficient variance in Bem’s p-values, Francis’s article and my article remain the only article that question the validity of Bem’s origina findings. Other articles have shown that the results cannot be replicated, but I showed that the original results were already too good to be true. This blog post explains, how I did it."
I'm afraid the emboldened statement is a bit misleading. While some replication attempts have failed, Bem et al. claim in their meta-analysis paper that overall the replication attempts show a significant effect - as I've pointed out a couple of times in comments on Dr Schimmack's blog.
I do think if someone is calling for a paper to be retracted because questionable research practices are suspected, there is a responsibility to be scrupulously fair and accurate in presenting the evidence.