Call for retraction of "Feeling the Future"

54 Replies, 9373 Views

(2018-03-01, 09:23 AM)Chris Wrote: At the start, he comments:
"Aside from Alcock’s demonstration of a nearly perfect negative correlation between effect sizes and sample sizes and my demonstration of insufficient variance in Bem’s p-values, Francis’s article and my article remain the only article that question the validity of Bem’s origina findings. Other articles have shown that the results cannot be replicated, but I showed that the original results were already too good to be true. This blog post explains, how I did it."

I'm afraid the emboldened statement is a bit misleading. While some replication attempts have failed, Bem et al. claim in their meta-analysis paper that overall the replication attempts show a significant effect - as I've pointed out a couple of times in comments on Dr Schimmack's blog. 

I do think if someone is calling for a paper to be retracted because questionable research practices are suspected, there is a responsibility to be scrupulously fair and accurate in presenting the evidence.
[-] The following 2 users Like Guest's post:
  • Silence, Typoz
In his letter calling for the retraction of Bem's paper, Prof. Schimmack said that there was no decline effect in attempted replications of the work by Galak and Maier.

Looking at the graph in the letter I'm not entirely convinced there's no decline - particularly when it's compared with the one he posted in a blog article showing Maier's data alone. But apart from that, I really wonder whether a more complete investigation of the replication data would necessarily bear this conclusion out. Schooler reported a very strong decline effect [significant at p< 0.00017], albeit with a much longer scale than seen in Bem's data, in a conceptual replication which he called "Temporally Reversed Implicit Perceptual Priming":
https://www.bial.com/imagem/Bolsa19506_21022014.pdf

Schooler can be heard talking about this work, and also about Bem's paper (for which he was a reviewer) in this talk from 2011, at about 37 minutes:
https://www.youtube.com/watch?v=0Tdiu5kwjKs&t=77s

[Edit: Having listened to a bit more of the talk, Schooler also found a significant decline effect [p=0.05] in the other protocol - retrocausal practice used to predict the result of spinning a roulette wheel.]
[-] The following 1 user Likes Guest's post:
  • Typoz
(2018-03-06, 10:17 PM)Chris Wrote: Schooler can be heard talking about this work, and also about Bem's paper (for which he was a reviewer) in this talk from 2011, at about 37 minutes:
https://www.youtube.com/watch?v=0Tdiu5kwjKs&t=77s

[Edit: Having listened to a bit more of the talk, Schooler also found a significant decline effect [p=0.05] in the other protocol - retrocausal practice used to predict the result of spinning a roulette wheel.]

Although it's quite long, I'd recommend that video from 2011 to anyone who is interested in the controversy over Bem's experiments. The participants are Bem, Schooler and Sam Moulton, who started as a psi enthusiast but became a sceptic when he was unable to get significant results. His early experiments were apparently an influence on Bem's work.

Each of the participants gives a presentation, and then there is a discussion, followed by questions from the audience. Bem's presentation is on the "Feeling the Future" work, Schooler's is on Bem's work and the decline effect (including his own conceptual replications of some of Bem's experiments), and Moulton's is on his own work and his criticisms of Bem's. He disbelieves Bem's results because he thinks such effects would have been noticed before (though the same argument seems to apply to his own work) and because the number of successful studies is too good to be true. He thinks this points to selective reporting, perhaps in conjunction with optional stopping. He is also concerned about one-tailed tests (I'm not sure why) and multiple comparisons.
There's an interesting exchange in that video at around 1 hour 10 mins, where Bem mentions Moulton's work. Moulton asks why his failed replication wasn't mentioned in Bem's paper, and Bem explained that the editor of the journal only wanted published work to be cited in the paper. Moulton then asks "Who publishes negative results?" to which Bem starts to reply everyone in parapsychology before Moulton cuts him off, saying "I don't want to publish nagetive results, it's not interesting."

This demonstrates a kind of culture shock between parapsychology and mainstream science - academic parapsychology (mostly) takes the approach that negative results are evidential, whereas science seems to think that somehow doesn't apply to them. Moulton's odd insistence that negative results were only evidence when he needed them to be, and could otherwise be ignored was very telling.
[-] The following 2 users Like ersby's post:
  • Typoz, Doug
(2018-03-07, 06:33 PM)ersby Wrote: There's an interesting exchange in that video at around 1 hour 10 mins, where Bem mentions Moulton's work. Moulton asks why his failed replication wasn't mentioned in Bem's paper, and Bem explained that the editor of the journal only wanted published work to be cited in the paper. Moulton then asks "Who publishes negative results?" to which Bem starts to reply everyone in parapsychology before Moulton cuts him off, saying "I don't want to publish nagetive results, it's not interesting."

This demonstrates a kind of culture shock between parapsychology and mainstream science - academic parapsychology (mostly) takes the approach that negative results are evidential, whereas science seems to think that somehow doesn't apply to them. Moulton's odd insistence that negative results were only evidence when he needed them to be, and could otherwise be ignored was very telling.

Yes - and Richard Wiseman later complained that the journal that published Bem's paper refused to publish a paper on failed replications of Bem's work, on the basis that it didn't publish replications, whether successful or not:
https://www.newscientist.com/article/dn2...cognition/

In the circumstances, the second part of Wiseman's comment, "It’s a problem in psychology, and it’s a particular problem in parapsychology", seems rather inappropriate.
[-] The following 1 user Likes Guest's post:
  • Sciborg_S_Patel

  • View a Printable Version
Forum Jump:


Users browsing this thread: 2 Guest(s)