Watt's analysis of judges' ratings in her 2014 dream precognition study

In another thread, fls has raised the question of bias owing to subjects not completing the prescribed number of trials in Caroline Watt's 2014 dream precognition study (Journal of Parapsychology, 78(1): 115-125).

The same paper raises another interesting question. The overall hit rate based on the target or decoy most highly ranked by the judges was significant, even after the inclusion of the less successful trials performed by the drop-outs, at 30.6% for a p value of 0.04 (one-tailed).

Watt also did some post hoc analysis based on the judges' percentage ratings of targets and decoys, which reflected their similarity with the subjects' dreams. She made three comparisons: (1) between the ratings of targets, and the ratings of decoys, (2) between the highest ratings in trials for which the target was chosen, and the highest ratings in trials for which a decoy was chosen, and (3) between the ratio of the highest ratings to the average of other ratings in trials for which the target was chosen, and the same thing for trials in which a decoy was chosen. She used the Mann-Whitney test, which compares two sample sets by combining them into a single ranked set, and checking whether the average ranks differ significantly between the two original sets. None of these tests yielded a significant result, and Watt concluded that this didn't support the precognition hypothesis, and might indicate that non-psi factors were at work.

I'm not convinced that the first of these comparisons tells us very much. Watt quotes a p value of p=0.16, which was certainly very different from the original p value based on hit rate - 0.015. But the former was based on a two-tailed test, whereas the latter was one-tailed. If one-tailed tests are used for both, and if the results from drop-outs are included in the latter, the p values are not so very different - 0.08 and 0.04. (Note that revised figures for the Mann-Whitney tests weren't included in Watt's note on the drop-out question.)

In contrast, the differences for the second and third Mann-Whitney tests are nowhere near significant. This seems paradoxical, and does seem hard to reconcile with a precognitive effect. As the Mann-Whitney test is based on rankings, and is often discussed in terms of median values, I wondered whether it might be insensitive to a precognitive effect that manifested itself only in a small percentage of trials - say 10%. But some simple model calculations suggest that's not the case. I also wondered whether, if there were systematic differences between the size of the ratings given by the two judges, that might decrease the significance of the Mann-Whitney statistic (there was only one judge per trial in this study). Watt doesn't say that she tested for such differences, and I think in principle they could affect the results. But I find it difficult to believe they could explain such a large divergence between the different measures of success.

However, it seems to me that this is exactly what we should expect to see if the mechanism were not precognition, but a psychokinetic effect influencing the random selection of the target. I don't think that would tend to produce any differences in comparisons (2) and (3) - though I'm happy to be corrected if that's wrong. This raises the interesting possibility that Watt's post hoc comparisons are not pointing to a non-psi explanation, but a different modality of psi. It would be interesting to know whether this idea could be tested using data from other precognition studies.

I think it's interesting to contrast the idea that Watt may have set out to find precognition and instead found psychokinesis, with Peter Bancel's conclusion that the Global Consciousness Project set out to find psychokinesis and instead found telepathy/precognition on the part of the experimenter.

Obviously, as Watt's findings are only marginally significant, the idea can only be very tentative. But in conjunction with the GCP analysis, it might suggest that it's not helpful to use the traditional psi categories when trying to interpret these studies.

There is a recurring theme through parapsychology that telepathy and psychokinesis and both forms of information exchange an so are maybe the same thing. Trying to tease the two apart is very tricky. I'll try and dig up a few papers in the next day or two about this.

There is a recurring theme through parapsychology that telepathy and psychokinesis and both forms of information exchange an so are maybe the same thing. Trying to tease the two apart is very tricky. I'll try and dig up a few papers in the next day or two about this.

Thanks. I'd still be interested in any relevant papers you know of.

Having thought about it a bit more, it strikes me that if there is some degree of equivalence between telepathy/precognition and psychokinesis, it's perhaps not so surprising that the Watt experiment and the GCP would work out as they did - if we imagine that psi obeys some kind of "least action" principle, in which changes in information are in some sense minimised in the process of satisfying the experimental hypothesis.

In the GPC, on the one hand we have a set of events, each specified by a start time and an end time, and on the other a whole mass of random data produced second-by-second by several dozen random number generators. The experimental hypothesis is that during the events, some statistic calculated from all those data will deviate from its chance value. In terms of modifying the relevant information, by far the most economical way of producing such a deviation would be just to modify the start time and the end time of the events, rather than modifying all the data produced by the random number generators.

In the Watt experiment, things aren't so clear-cut, but on the one hand we have a set of judges' ratings for each trial, in the form of percentages expressing how closely the dream report matched the four target and decoy images. On the other, for each trial there was a randomly chosen number between 1 and 4, specifying which of the four images was the target. The hypothesis is that the target will be the highest rated image more often than would happen by chance. Here also, it seems the most economical way of confirming the hypotheses would be simply to modify the target indicators to agree with the highest rating, rather than altering the percentage ratings.

One awkward thing is that much the same considerations would apply if a fraudulent parapsychologist were editing his/her data to achieve confirmation of the hypothesis. Not that I'm suggesting for a moment that anything like that happened in either of these cases. But unfortunately this kind of "least action" principle could all too easily mimic the appearance of fraud.

Well, I thought it was a recurring theme through parapsychlogy, but when I came to look, it has been mostly Chris Roe who's written about it.

Roe, Holt, Simmonds, Considering the Sender as a PK Agent in Ganzfeld ESP Studies, Journal of Parapsychology Vol 67, Spring 2003, pp 129-145

And

Roe, Holt, A Further Consideration of the Sender as a PK Agent in Ganzfeld ESP Studies, Proceedings of the Parapsychological Association Convention 2004

Both tested the idea that, since a sender seemed to be favourable in Ganzfeld experiments perhaps the sender's thought-transference could be thought of as a kind of PK. They put 20 descriptors for a target set into a computer which would chose some randomly during a ganzfeld experiment. I'm unsure if this is necessarily how PK works, but it's an interesting idea.

Roe continued to explore the similarities between ESP and PK in other papers:

Roe, C. A., Davey, R., & Stevens, P. (2003). Are ESP and PK aspects of a unitary phenomenon? A preliminary test of the relationship between ESP and PK. Journal of Parapsychology.
Roe, C. A., Davey, R., & Stevens, P. (in press). Are ESP and PK aspects of a unitary phenomenon? A further test of the relationship between ESP and PK. Journal of the Society for Psychical Research.
Roe, C. A., Davey, R., & Stevens, P. (unpub.). Arousal and performance at ESP and PK tasks using a common protocol. Submitted to the European Journal of Parapsychology.

I haven't read the three above, but I have read the next one:

Roe, Davey, Stevens, Experimenter Effects In Laboratory Tests Of ESP And PK Using A Common Protocol, Proceedings of the Parapsychological Association Convention 2004

In this the ESP and PK experiments are much more similar. Both involve a greyhound race run on a computer. In one protocol, the subject is asked to try and force one greyhound to win (ie, PK, using an RNG to decide the movement of the greyhounds) and in the other the subject was asked to predict which had won a race the computer had already run (ie, ESP using a psuedorandom set of numbers) with the subject viewing the race as feedback on their prediction.

After all this, I don't think he's come to any real definitive conclusion on the matter.

Thanks for posting these references. 

I suppose the question of PK versus precognition is also central to the discussions of Decision Augmentation Theory, though I haven't read much about that, and I don't really understand how it's meant to account for experiments where all the decisions are randomised (unless the experimental protocol itself is supposed to be paranormally determined).

While reading the Star Gate Archive book, I found a passage which also addressed the question about whether different manifestations of psi are the actually the same thing. I thought it worth posting here.

Taken from The Star Gate Archives, p24

1. Precognition is most likely the only form of anomalous cognition, as clairvoyance, telepathy, and micro-PK can be subsumed within precognition. It appears impossible to close the door to information coming from a future point in space-time (Marwaha & May, 2016).
   
2. Precognitionis a purely local phenomenon as once an informational signal emerging from a distant space-time point comes to the “vicinity” of the percipient, the perception of the information is occurring in close to real time (Marwaha & May, 2016, p. 81).
   
3. Quantum mechanics may not play a role in psi phenomena as: (i) classically one cannot obtain informationfrom a quantum correlation; (ii) we cannot do so evein in four-dimensional space (three spatial and one time dimension) where we live; (iii) in higher dimensional space, however temporal entablement (correlation) might be able to carry information; and (iv) information from a correlation will still require a signal to interact with the brain/consciousness (Marwaha & May, 2015d, p.91).
   

References mentioned are

Marwaha, S.B., & May, E.C. (2015d). A refutation of the dualist perspective in psi research. Journal of Consciousness Studies, 22(5-6), 70-95

and

Marwaha, S.B., & May, E.C. (2016). Precognition: The only form of psi? Journal of Consciousness Studies, 23 (3-4), 76-100

While reading the Star Gate Archive book, I found a passage which also addressed the question about whether different manifestations of psi are the actually the same thing. I thought it worth posting here.

Taken from The Star Gate Archives, p24

1. Precognition is most likely the only form of anomalous cognition, as clairvoyance, telepathy, and micro-PK can be subsumed within precognition. It appears impossible to close the door to information coming from a future point in space-time (Marwaha & May, 2016).
   
2. Precognitionis a purely local phenomenon as once an informational signal emerging from a distant space-time point comes to the “vicinity” of the percipient, the perception of the information is occurring in close to real time (Marwaha & May, 2016, p. 81).
   
3. Quantum mechanics may not play a role in psi phenomena as: (i) classically one cannot obtain informationfrom a quantum correlation; (ii) we cannot do so evein in four-dimensional space (three spatial and one time dimension) where we live; (iii) in higher dimensional space, however temporal entablement (correlation) might be able to carry information; and (iv) information from a correlation will still require a signal to interact with the brain/consciousness (Marwaha & May, 2015d, p.91).
   

References mentioned are

Marwaha, S.B., & May, E.C. (2015d). A refutation of the dualist perspective in psi research. Journal of Consciousness Studies, 22(5-6), 70-95

and

Marwaha, S.B., & May, E.C. (2016). Precognition: The only form of psi? Journal of Consciousness Studies, 23 (3-4), 76-100

Thanks for these too. This seems an interesting angle, about which I know very little.

I do feel that a theory based on pure precognition raises rather bewildering metaphysical questions. If the idea is that the experimenter subconsciously makes psi-based choices that lead to positive results, wouldn't that require paranormal knowledge of a range of different outcomes, based on different possible choices? But precognition seems only to offer knowledge of a single outcome. Or is the idea that one can have precognitive knowledge of all the possible alternative futures?

Does anyone here know how May's theory works?

Thanks for these too. This seems an interesting angle, about which I know very little.

I do feel that a theory based on pure precognition raises rather bewildering metaphysical questions. If the idea is that the experimenter subconsciously makes psi-based choices that lead to positive results, wouldn't that require paranormal knowledge of a range of different outcomes, based on different possible choices? But precognition seems only to offer knowledge of a single outcome. Or is the idea that one can have precognitive knowledge of all the possible alternative futures?

Does anyone here know how May's theory works?

I grasped the nettle and read the second of the references mentioned by ersby, which is available here:
http://www.patriziotressoldi.it/cmssimpl...nali16.pdf

It seems the idea is that:
(1) The person has precognitive access to the whole range of possible future events. Not too much is said about this in the paper, and the relationship with experimenter psi isn't discussed, but there's an anecdote about an associative precognitive remote viewing in which the subject, despite having scored a "hit", refused to view the target.
(2) Precognition applies not only to cases in which the subject later perceives the information, but also to cases where the subject remains blind and only the experimenter perceives it. There's even a reference to an experiment where the results were analysed by computer, and no human being ever perceived the target. The authors acknowledge that precognitively reading the "mind" of a computer is "seemingly implausible", but argue that it can't be ruled out.

Probably Decision Augmentation Theory is discussed more fully elsewhere, but I have to say I find this paper a bit unconvincing. It's one thing to say that effect sizes are similar for clairvoyance and precognition, and that it would be simpler to think of it all as precognition, but shouldn't there be a more positive argument? I can see the appeal of attributing all psi phenomena to precognition where the same mind perceives the target in the future, but if that has to be widened to include precognitive telepathy and precognitive clairvoyance, I can't see that just putting "precognitive" in front of everything simplifies things much. Precognitive perception in general seems more problematical than perception of present or past events.

I also think that although - as the authors point out - it's very difficult to exclude the precognitive element, that doesn't mean to say it has to be the explanation. And conversely, it's very difficult to see how precognition could explain positive results in appropriately designed psychokinesis experiments. Indeed, the results of Watt that I started this thread with do seem to point to PK rather than precognition.

On Skeptiko, someone posted the abstract of a Parapsychological Association abstract by Westerlund et al. on "Remarkable Correspondences between Ganzfeld Mentation and Target Content ...". The text of a related paper in the Journal of Parapsychology is here:
https://w3.psychology.su.se/staff/jwd/cs...erlund.pdf

One aspect of this study is that the "remarkable correspondences" (short sequences in which the subject's thoughts matched the part of the target video that was playing at the time - or the corresponding part of a decoy video) were rated for impressiveness, and there was found to be no statistically significant difference between the correspondences with targets and those with decoys. That might be consistent with the findings in Watt's precognition studies. However, the ratings were about 10% higher on average for the targets, so the lack of significance may just be the result of the low statistical power of the study.