Psience Quest

(2017-09-15, 03:40 AM)Arouet Wrote: [ -> ]It's been awhile since I've looked into the Stevenson/Tucker work in detail.  (I lost my Kobo Glo which had the notes I took on the Tucker book!  Cry).  I should give it another read if we're going to be talking about it a lot.  And I don't think they did a terrible job.  In many cases they probably did the best they could in the circumstances.  

But I'll give you one general example of a reliability risk that affects many of the accounts.  In those accounts, the researchers only get to the case a long time after it began.  Interviewing people about events that took place long ago are fraught with risk.  This is not the fault of the researchers, it is an inherent risk of bias when dealing with these kinds of cases.

I recall questioning their use of stats with regard to birthmarks but I want to relook at the material before commenting on this.  

Another question that came to mind when I was reading the book, is that when the children refer to themselves in the first person referring to their potential ancestors, I couldn't help but think of my own son who around the age of these kids (he's 7 now but I'm talking 2-5) he regularly identified with certain characters and referred to himself in the first person regarding them.  In my son's case most notably Batman at one time and Mario at another time.  When he saw these characters in on tv or in a video game, or heard us mention them he would say "that's me!, that's my _____, etc."  Now, I am not making the claim that I have concluded this is part of what may be going on in these accounts but its an issue I can't recall being covered in this research (though I may have missed it so if anyone has a reference please let me know).

Another issue I have with regard to risk of bias is that interviewing children is also fraught with risk.  It is no easy task to interview a child in a non-leading manner, especially when the early conversations would have been with family members and friends who have no training in the matter.  Even when someone is properly trained it is a very difficult task.  This is a risk that is inherent in the task.  In many cases there would have been little the researchers could have done to have avoided this risk.  

So don't mistake my saying there are risks of bias as my saying the work was shoddy.  From what I recall I didn't find it shoddy, but I did find risk of bias/error.  

There's more to say but I think this is good to get the discussion started.

Getting tired so will have to respond to the rest of your post tomorrow.

I understand the risk that comes with not getting to the cases as soon as the child initially says something, as do Tucker and as did Stevenson.

As far as the birthmarks go, I'm not particularly familiar with that part of the research - but nonetheless, the few cases I've read involving them have been a mix of what I found to be impressive and not so impressive, or more likely to be due to coincidence.

For the "children refer to themselves in the first person" thing, I just don't see how that would explain the research, and I know you noted that... but I think it's self explanatory enough that it wouldn't shock me if the researchers hadn't touched on it (and they may have, I don't recall but I haven't read nearly all the books the two of them have published - only a couple among other articles and interviews). It's one thing for a kid to say, "Look! That's me! I'm batman!" or something of that ilk, like a famous character, super hero, or video game character, and of course an entirely different thing when a child says something about being a random person from another location while simultaneously listing a number of facts about that person that they should or would have no access to, if they even had the capacity to remember the information if it was ever told to them. And I know you said that you're not making the claim that this is part of what may be going on in these cases, but I would think these reasons are obvious enough that maybe the researchers devoted time to other possible explanations or reasoning first. Again, I'm also not certain that they've never addressed it, though off the top of my head I can't recall that I've personally read such a thing. 

Interviewing children is, of course, going to be "risky" - but this is, as with most things, best addressed specifically with regards to an actual case rather than broadly. This doesn't really hold a lot of water for me when there are thousands of cases and I don't think that the method of questioning by scientifically cautious and methodologically careful researchers who have been at this for awhile is especially likely to be suspect. They are aware of the difficulties of doing the research, and I believe do their best to conduct the research in as unbiased a way as possible. 

Of course there is risk of bias/error, as there is with everything that any human is directly involved in. I wouldn't say it is extra fraught with that risk - of course, because the subjects are also human beings, there is more subjectivity and risk, but I don't think "fraught with" is accurate, and I think that because the researchers appreciate the importance of their work and know how important their methodology is, they try their best to control for it, which it seems you've acknowledged. 

In spite of what you said, I think that an actual reading case by case makes it fairly difficult to assign some form of bias, coincidence, luck, or fraud to the majority of the cases, especially the stronger ones, which aren't lacking in number. Speaking generally has its place but won't move the needle much for me or others who are familiar with a large number of cases and the researchers' works and noted caveats.

(2017-09-17, 12:36 AM)fls Wrote: [ -> ]Yeah, you're probably right.

I have to admit that over the last several years, as I've looked at the research in greater detail, I've moved more towards skepticism. I thought it would be the other way around (like the OP). But there it is. I think part of it is because I've seen what happens in medicine, where weak evidence almost never holds up and even good evidence is regularly overturned by excellent evidence. I just can't bring myself to trust that weak evidence is going to be different in the case of psi.

I still think there's a chance something novel and odd or psi-like is going on. I'm interested in what kind of research designs will tease this out. But I don't know that there's much point to talking about what we think psi is until we know to what extent we're just talking about happenstance, cognitive biases, associative memory (intuition) and a sprinkling of misadventure.

Linda

How do happenstance, cognitive biases, associative memory or misadventure explain an enormous body of cases at large? This is my issue - it's a personal, subjective bar you are setting, and then calling the evidence weak. We may at this point be talking past one another, but what I'm trying to articulate is that inferences can be drawn from evidence that isn't as perfectly sound or materially available as those used in the hard sciences - and those inferences aren't unreasonable, or irrational, or due to some fantasy or hopeful wishing. Nor are they based on "weak" evidence. 

This is different than medicine in a lot of ways, and while you've highlighted the similarities, the differences are extremely important to keep in mind. We don't understand memory, we don't understand consciousness, at any fundamental level whatsoever. While in medicine there are many subtopics and things we don't understand, we at least have a basis or foundation off of which to work, so that you can get good evidence of something, and then get even better evidence to overturn that good evidence, as you noted. For the reincarnation research, we don't have any foundation. We only have the studies, which are not small in number, by Stevenson, Tucker, and others. At this point I'm not sure what you'd suggest might "overturn" it. Of course, when that good evidence is found, it's taken to support a certain view. When the excellent evidence is found, it might overturn that view, or support it further. But the possible future existence of such "excellent" evidence does not preclude anyone from considering the "good" evidence to be legitimate or supportive of some idea. They don't look at the evidence and say, "Oh, well, someday we might find evidence that is better than this and supports the opposite or a different view than the one this evidence supports, so we probably shouldn't think much of this evidence". They take the good evidence for what it is and continue to research, and if they find evidence in the future that's even better, they apply it at that time. And certainly, there are many cases where that "good evidence" hasn't been overturned - that is hardly a universal thing. 

The methodology is always important, but it has to be within reason. Setting the bar so high that a type of research can hardly achieve it due to the very nature is patently unreasonable, and like I said, trying to put it into a box. Putting something in a box that we just don't understand doesn't seem to be a reasonable thing to do to me. Further, and I'll repeat myself ad nauseum on this point - it's not just about showing the existence of something like reincarnation (or something similar, whatever you want to call it) or psi; it's equally, and perhaps even more, about showing the weaknesses of reductionism. That matters too, very much, and in my eyes would have a different evidentiary value and require a distinct analysis. And I think there is very much a point in discussing the evidence to that effect, keeping in mind a reasonable standard for methodology and an equally reasonable approach to the research and its results, given the nature of the subject. I don't mean to say, "Well, let's take it easy on reincarnation or psi." But to expect the same stringency as a hard science or something akin to it for something like the Stevenson research is to miss the point.

(2017-09-17, 01:54 AM)fls Wrote: [ -> ]If you think that a particular phenomenon has better evidence in its favor than the Ganzfeld experiments, please share. 

I have a feeling I may regret this, but - the reason I started the thread on the Global Consciousness Project is that (according to the organisers) for the formal series of experiments, all the details including the statistical hypotheses were fixed in advance, an extremely high overall level of statistical significance was reached (Z=7.31), and (pace the general concerns Max has raised about the behaviour of the random number generators) no one has suggested a conventional mechanism by which the observed correlations, which are absent from the data as a whole, could be produced.

It's an odd state of affairs. Nearly always, the sceptical comment is simply that the GCP involves retrospectively looking for patterns in the data at the times of global events. As that is the opposite of what they say they are doing, that tells us more about the poor quality of sceptical comment than about the GCP.

(2017-09-17, 08:22 AM)Chris Wrote: [ -> ]It's an odd state of affairs. Nearly always, the sceptical comment is simply that the GCP involves retrospectively looking for patterns in the data at the times of global events. As that is the opposite of what they say they are doing, that tells us more about the poor quality of sceptical comment than about the GCP.

Unfortunately even J. E. Kennedy falls into this trap, writing of "The intrinsically post hoc nature of the analyses" in his review of "Handbook for the 21st Century":
http://jeksite.org/psi/handbook_review.htm

But he adds:
"I have not delved into the complex statistical issues to develop an informed opinion about the validity of the claimed results. If there are psi effects in these data, I expect that experimenter effects will remain the most parsimonious explanation for the foreseeable future."

(2017-09-17, 02:53 AM)Dante Wrote: [ -> ]First and foremost, this doesn't sound very realistic as far as applying it to the cases goes. This is what I was talking about as far as setting the bar unrealistically high. You can try to fit it into a methodological box - but I'm not sure that what you've laid out here is really a reasonable thing to expect in terms of researching a case.

I'm not setting the bar unrealistically high. I'm talking about looking at where it is now and about where it would need to be in order to establish that the idea of reincarnation may be true. "The bar" is simply a recognition of where we are at in terms of methodological biases, cognitive biases, happenstance, misadventure, etc. - none of which are what we mean by "reincarnation".

Linda

(2017-09-17, 03:44 AM)Dante Wrote: [ -> ]How do happenstance, cognitive biases, associative memory or misadventure explain an enormous body of cases at large? This is my issue - it's a personal, subjective bar you are setting, and then calling the evidence weak. We may at this point be talking past one another, but what I'm trying to articulate is that inferences can be drawn from evidence that isn't as perfectly sound or materially available as those used in the hard sciences - and those inferences aren't unreasonable, or irrational, or due to some fantasy or hopeful wishing. Nor are they based on "weak" evidence. 

This is different than medicine in a lot of ways, and while you've highlighted the similarities, the differences are extremely important to keep in mind. We don't understand memory, we don't understand consciousness, at any fundamental level whatsoever. While in medicine there are many subtopics and things we don't understand, we at least have a basis or foundation off of which to work, so that you can get good evidence of something, and then get even better evidence to overturn that good evidence, as you noted. For the reincarnation research, we don't have any foundation. We only have the studies, which are not small in number, by Stevenson, Tucker, and others. At this point I'm not sure what you'd suggest might "overturn" it. Of course, when that good evidence is found, it's taken to support a certain view. When the excellent evidence is found, it might overturn that view, or support it further. But the possible future existence of such "excellent" evidence does not preclude anyone from considering the "good" evidence to be legitimate or supportive of some idea. They don't look at the evidence and say, "Oh, well, someday we might find evidence that is better than this and supports the opposite or a different view than the one this evidence supports, so we probably shouldn't think much of this evidence". They take the good evidence for what it is and continue to research, and if they find evidence in the future that's even better, they apply it at that time. And certainly, there are many cases where that "good evidence" hasn't been overturned - that is hardly a universal thing. 

The methodology is always important, but it has to be within reason. Setting the bar so high that a type of research can hardly achieve it due to the very nature is patently unreasonable, and like I said, trying to put it into a box. Putting something in a box that we just don't understand doesn't seem to be a reasonable thing to do to me. Further, and I'll repeat myself ad nauseum on this point - it's not just about showing the existence of something like reincarnation (or something similar, whatever you want to call it) or psi; it's equally, and perhaps even more, about showing the weaknesses of reductionism. That matters too, very much, and in my eyes would have a different evidentiary value and require a distinct analysis. And I think there is very much a point in discussing the evidence to that effect, keeping in mind a reasonable standard for methodology and an equally reasonable approach to the research and its results, given the nature of the subject. I don't mean to say, "Well, let's take it easy on reincarnation or psi." But to expect the same stringency as a hard science or something akin to it for something like the Stevenson research is to miss the point.

These factors have often explained huge bodies of cases. For example, the entire practice of medicine up until the late 19th/early 20th century was essentially explained by these factors, almost none of which survived once evidence was sought and examined. How many millions of people think homeopathy is helping them, when the research has shown these are the only factors in play?

None of this is some subjective standard I've come up with on my own - these are the factors, which when tested against, have caused vast swathes of supposed effects to disappear. Even just blinded evaluation (the suggestion I made earlier) has had a huge effect. I'm not comfortable ignoring that.

The problem is that what you describe in your second paragraph (as what "they don't do") is exactly what scientists do do. If the evidence is weak, you're not likely to run with the idea knowing that fair to good evidence is almost never forthcoming. That's not to say that some scientists won't remain interested and continue to perform research. But if weak evidence hasn't been generally persuasive, then more of the same isn't going to help. I'm more interested in finding ways to rise above that - invest in the kind of research that provides a stronger evidentiary level.

When it comes to claiming that it offers disproof of reductionism, you are in the same boat. You still need good evidence to disprove something. I don't know what you mean about putting something into a box - what box? And I'm not talking about holding it to the standards of a hard science. Psi should be very amenable to the processes which are demonstrably valid and reliable in the social sciences/medicine.  

Linda

(2017-09-17, 08:22 AM)Chris Wrote: [ -> ]I have a feeling I may regret this, but - the reason I started the thread on the Global Consciousness Project is that (according to the organisers) for the formal series of experiments, all the details including the statistical hypotheses were fixed in advance, an extremely high overall level of statistical significance was reached (Z=7.31), and (pace the general concerns Max has raised about the behaviour of the random number generators) no one has suggested a conventional mechanism by which the observed correlations, which are absent from the data as a whole, could be produced.

It's an odd state of affairs. Nearly always, the sceptical comment is simply that the GCP involves retrospectively looking for patterns in the data at the times of global events. As that is the opposite of what they say they are doing, that tells us more about the poor quality of sceptical comment than about the GCP.

I'm not sure how these results are supposed to be psi-like. How would they be connected to reincarnation, for example?

Linda

(2017-09-17, 10:05 AM)Max_B Wrote: [ -> ]But Chris, it's so wide open... what is such a wide open study supposed to tell us...?

Is it any wonder people don't take the study seriously, and don't take much time to bother understanding it before dismissing the experiment, when researchers don't bother to improve their experiment... Selecting from an apparently unlimited range of events with flexible timescales which have already happened, and won't say how they make the selection. Who take no steps to shield their nose-based RNG devices, or improve their devices power supply quality. Who don't run controls with non XORed data to find correlations. It's typical of the Radin/ION's junk experiments I've mentioned elsewhere.

Radin/ION's 'junk experiments' - where the authors showed they dealt with your concerns about said experiments, yet you're still claiming they're junk?

(2017-09-17, 01:54 AM)fls Wrote: [ -> ]So far, they seem to have the same sorts of things in common. 
If you think that a particular phenomenon has better evidence in its favor than the Ganzfeld experiments, please share. 
Linda

I'd rather not try and find the one example that can't be refuted. Not that it doesn't exist (for me), but that there will always be a crack in any example of anything. And since I've spent enough time talking about the individual size and shape of those cracks, however small, I'm talking about a different, larger view of it.  

My point really was that there is also a great weight cast by virtue of the vast array of examples, the shear number and variety of them, all pointing at different aspects of the nature of things, all reinforcing the same basic theme of non-physicality. 

In this view, it's not about the merits of a particular one, but the weight of the whole. And whether it is reasonable that by some odd quirk of fate, the evidence for each would have a different flaw that would discount it. Seems like a long and therefore unlikely string of apparent coincidences to me.

(2017-09-17, 01:54 AM)fls Wrote: [ -> ]So far, they seem to have the same sorts of things in common. 
If you think that a particular phenomenon has better evidence in its favor than the Ganzfeld experiments, please share. 
Linda

What things do you refer to?

And could I ask: given the non-physical nature of much of this stuff, theoretically what would constitute a compelling example for you? Pick a field of non-physical psi. Any one.