My Psychic Told Me to Date a Psychopath

“Risk of bias” is the term used in the field, so it’s the appropriate phrasing. 

https://www.ncbi.nlm.nih.gov/books/NBK13...pf-fm1.pdf

The research showing the effects of bias are based on measuring the difference when the risk of bias is present vs. when it is absent. Trying to identify whether or not a specific result was altered by, for example, a lack of blinding turns out not to work. For example, a) you take some positive results, try to figure out how a lack of blinding may have affected the results, go looking for that effect, and then fail to find that effect. Or b) you perform the same study with blinding and without blinding, and find a difference between the two. It turns out that even though you supposedly failed to find an effect from the lack of blinding (i.e. you failed to find a bias in a)), a bias was present as demonstrated in b). So a risk of bias has been demonstrated over and over again to be enough to bias the results. 

And this has held in Parapsychology research as well. For example, even though Beischel et. al. try to identify and eliminate all the ways in which unblinded assessments of mediumship readings could bias the results, they still find marked differences in the same assessments depending upon whether the assessment is blinded or not.

Linda

So you don't accept the work of Andy Paquette, who did extensive recording of dreams, which showed psi phenomena including precognition?

Well I think there's a difference between accepting that something was going on, even something that involved Psi, and questioning whether it involves future information coming back from the past in the sense of a reverse arrow of time.

As Max notes in this thread, there are other ways of conceiving time.

So taking a stance of, “well yeah, there may be problems, but unless you prove it’s a false positive it’s okay for me to believe it’s a true positive,” is unjustified by the evidence. 

But there has been discussion of problems that would make research doubtful due to bad study design - I've already mentioned the criticism Max has made about Radin's work as an example, there was also the numerous issues proponents took with Diane Powell's work on Skeptiko. And that's just off the top of my head. IIRC Iyace had discussed this as well in the past, perhaps even presented a paper at a conference.

Bias is certainly very dangerous.

It can lead to such absurd assertions as "Where there is a risk of bias, most positive results will turn out to be false"!

It does lead to an interesting problem - what do you do when there is reason to doubt a positive result? It seems to me most lay people figure something is valid because it has applicability.

This is part of why I turned more toward a personal investigation of paranormal phenomenon, and am less worried about a study that will lead to general acceptance of Psi in academia.

“Risk of bias” is the term used in the field, so it’s the appropriate phrasing. 

https://www.ncbi.nlm.nih.gov/books/NBK13...pf-fm1.pdf

Linda - just to be clear about one thing: twice in the past you've indicated you'd prefer me not to respond to your posts, and twice I've said I won't do respond to yours provided you don't respond to mine either. But regardless of that you've responded to mine anyway - most recently here, where you quote what I've said (though without using the "quote" facility):
https://psiencequest.net/forums/thread-p...8#pid32168

So in future I am going to follow your lead, and respond to your posts where I feel it's appropriate.

Regarding the phrase "risk of bias," of course I didn't say it was inappropriate to use it. People can use any phrase they like here, so long as what they say makes sense. (Though of course the "field" you mention is medicine, not parapsychology.)

What I objected to was the claim that when risk of bias is present, then "the “positive” results turn out to be false most of the time." Of course, that depends how high the risk of bias is.

The link you just provided actually leads to a set of criteria for assessing whether the risk of bias in a particular medical study is high or low. And really that is exactly what I am saying - that if criticisms have been made of a study, we should try to assess how strong their potential effect will be. Ultimately we want to know whether flaws in a study are capable of explaining the observations. If they aren't, granted we have a flawed study, but we still have an unexplained effect that has been observed, and that merits further investigation. It's not appropriate just to disregard it.

What I objected to was the claim that when risk of bias is present, then "the “positive” results turn out to be false most of the time." Of course, that depends how high the risk of bias is.

Not only the magnitude of the risk of bias. Also its direction. Positive, negative, or going off in some tangential direction for example.

It does lead to an interesting problem - what do you do when there is reason to doubt a positive result? It seems to me most lay people figure something is valid because it has applicability.

This is part of why I turned more toward a personal investigation of paranormal phenomenon, and am less worried about a study that will lead to general acceptance of Psi in academia.

For the first, I would say when we're dealing with statistics a positive result wouldn't be absolute proof of anything, even in a (hypothetical) flawless experiment. It could always be a false positive just by chance. So we should always doubt positive results to some extent, or at least refrain from drawing absolute conclusions from them.

I can understand people wanting to follow the personal path instead (and if one did a personal scientific study, at least one could be sure there was no fraud!). But to my mind, if we think science is capable of uncovering the truth of these matters (or some of it), then there is a big incentive to persuade the scientific community that it's worth investigating, in terms both of resources and of the calibre of researchers who would be attracted to the field. Equally, if the positive results of past experiments are the result of bias, error and fraud, then I should prefer to know that. I hope we'll get to the point in my lifetime when experimental parapsychology will be providing some clear answers one way or the other - clear enough not just for the interested minority, but for the scientific community at large.

For the first, I would say when we're dealing with statistics a positive result wouldn't be absolute proof of anything, even in a (hypothetical) flawless experiment. It could always be a false positive just by chance. So we should always doubt positive results to some extent, or at least refrain from drawing absolute conclusions from them.

I can understand people wanting to follow the personal path instead (and if one did a personal scientific study, at least one could be sure there was no fraud!). But to my mind, if we think science is capable of uncovering the truth of these matters (or some of it), then there is a big incentive to persuade the scientific community that it's worth investigating, in terms both of resources and of the calibre of researchers who would be attracted to the field. Equally, if the positive results of past experiments are the result of bias, error and fraud, then I should prefer to know that. I hope we'll get to the point in my lifetime when experimental parapsychology will be providing some clear answers one way or the other - clear enough not just for the interested minority, but for the scientific community at large.

Oh I agree we should continue the investigation of anomalous phenomena, I just meant for myself as a layperson there's little to be done beyond some networking assistance and a donation here or there. As such my remaining years on this blue marble are better spent on trying to understand the phenomena on a more personal journey.

That said, there's a saying that Philosophy ends up burying its undertakers, I figure the same will ultimately be true for parapsychology. The world is becoming more, not less, primed for acceptance of Psi after all.

Well I think there's a difference between accepting that something was going on, even something that involved Psi, and questioning whether it involves future information coming back from the past in the sense of a reverse arrow of time.

As Max notes in this thread, there are other ways of conceiving time.

Personally I tend to view precognition in some respects (but not all) as analogous to such things as weather forecasting or predicting the next solar eclipse. Of course we tend to view the latter (eclipses) as a very precise science, predicting the weather somewhat less so. But even in those cases there might be scope for the unexpected to occur, for example a visit by a stray comet could conceivably change the behaviour of eclipses at some point, or a volcanic or solar eruption could change the weather (among many other factors). I tend to view precognition as a little like that, a sort of projected rather than actual future. Though it seems to involve factors (other dimensions perhaps) which we don't have in our physical models of the world. The interplay of consciousness and free will is another angle. It is a complex area.

Mainly, I don't view the future as fixed, so in that sense there would not be anything there to actually send a message back in time.

But there has been discussion of problems that would make research doubtful due to bad study design - I've already mentioned the criticism Max has made about Radin's work as an example, there was also the numerous issues proponents took with Diane Powell's work on Skeptiko. And that's just off the top of my head. IIRC Iyace had discussed this as well in the past, perhaps even presented a paper at a conference.

I don’t disagree that there is occasional discussion of some of the flaws in individual research studies and the resultant articles and claims. My concern was that there is little engagement with issues related to validity, quality and bias. And the examples you gave weren’t about addressing those issues.

Iyace’s response was similar to others here, when I brought up these issues. He would say that I was talking about medicine, not parapsychology. But the research and researchers which contribute to this field come from many different areas - education, sociology, psychology, physical education, statistics, etc. as well as medicine. The only reason I often reference articles aimed at medical research is because medicine leads the social sciences in having a formal system of assessment of risk of bias, so review articles are more readily accessible. Again, the idea that these issues don’t apply to parapsychology is part of what I’m railing against.

You may be thinking of Johann and Maaneli, who did write a paper on this. And again, despite being a paper on the strength of the evidence for psi, they made no use of the extensive literature on the subject of validity, bias and quality. Instead, they assessed validity by referring to their intuitions, made up quality ratings based on some irrelevant and trivial characteristics, and claimed that even if studies are flawed, the effects still need to be explained, which entirely misses the point.

The point of bringing up these issues is that research has shown that any and all of them can create the appearance of an effect, even very large effects, when no effect is present. Any of them can lead to a ganzfeld study which shows a hit rate of 36%, when 25% is expected due to chance, for example. So when we are faced with that positive result, in the presence of known selective reporting, we can’t say that the effect is unexplained, because “selective reporting” would explain the effect. Do we know the size of the effect of selective reporting in this specific case? No, because we didn’t measure it. The way to measure the effect of selective reporting, in the above ganzfeld study, would be to also perform the study in the absence of selective reporting and compare the hit rates. But we’re not interested in measuring “selective reporting”, we’re interested in finding out if we can measure “psi”. So why not just perform the study in the absence of selective reporting to begin with, so we can get closer to measuring psi?

And that’s all this is about, for me. I’m interested in psi. But when studies are performed in the presence of all these various biases, all we are measuring are the effects of bias. “Feeling the Future” turned out to be a master class in how to make researcher degrees of freedom generate the appearance of an effect. So the results tell us nothing about what precognition might look like. Instead they tell us what testing multiple hypotheses and post hoc hypothesis generation looks like. If proponents and parapsychologists start to take these issues seriously, instead of putting people like Bem and Radin on a pedestal, it not only has the effect of starting to find out what psi actually looks like (and I suspect it looks very different from where our wishful thinking has led us), it also brings broader scientific interest and acceptance to the field.

Linda