Psience Quest

(2017-10-23, 07:40 AM)Chris Wrote: [ -> ]But maybe there's something wrong with his test of the hypothesis too.

Indeed. There is. Something very wrong.

 Thank you Laird for being specific, as that was very helpful in letting me see where/how we were simply talking past each other.

First, much of what you said is where we are in agreement. 

Popper's idea of falsifiability, which asks whether an idea can be falsified, in principle, is not in play here. Parapsychology produces ideas which are falsifiable, and so is not distinguishable from 'science' on that basis. And I don't think that was what Steve was referring to, but you can ask him (I don't think he was claiming that parapsychologists' ideas were not falsifiable in principle, only that they weren't attempting to falsify them in practice). So "falsify" doesn't refer to "falsifiability in principle".

Of course any hypothesis can be framed as a positive and as a negative. What you described is the basic process of forming your alternative hypothesis and a null hypothesis. No disagreement here - that's Science 101. So "falsify" doesn't refer to having a null hypothesis.

And of course, both kinds of tests - "p implies q" or "not q implies not p" - have the potential to be false or true, so "falsify" also does not refer to whether a test has the potential to return an answer of "false". 

And I also agree that you cannot imply "p implies q" from "not p implies not q", but no one would suggest otherwise, since that is logically invalid. I suggested that "p implies q" is implied (or confirmed) when "not q implies not p" is found to be true. So "falsify" also does not refer to a test which does not have the ability to confirm the hypothesis.

So if we are agreed that none of those things are pertinent with respect to "falsify" or "disconfirmation", what was I (and other scientists) talking about? I gave the answer in my first post by linking to two papers which you've only just now read (partially), and have decided are irrelevant. So you can see why I have found this an uphill struggle.  But at least, we can maybe be on the same page from this point.

Linda

(2017-10-23, 02:57 AM)Laird Wrote: [ -> ]You seem to agree with me about this. So, let's move on to whether the distinction can apply to hypothesis testing. Of this, in your initial response to me, you wrote, of trying to test whether "p implies q" as per the paper you referenced:

[fls:] You are attempting to confirm the idea when you choose to look at "p". Your result may be "q" (alternative hypothesis confirmed) or "not q" (null hypothesis confirmed). 

You are attempting to falsify the idea when you choose to look at "not q". Your result may be "not p" (alternative hypothesis confirmed) or "p" (null hypothesis confirmed).

However, as both the paper and logic dictate, whilst, yes, starting by looking at the consequent "not q" can potentially falsify the hypothesis (in the case that the antecedent turns out to be "p"), starting by looking at the antecedent "p" can also potentially falsify the hypothesis (in the case that the consequent turns out to be "not q").

Both tests can potentially falsify the hypothesis.

Yes. Looking at the p/q paper, I'm puzzled by fls's description of looking at "p" as confirming and looking at "not q" as falsifying. In fact, as her comments indicate, both of them either can (tend to) confirm the hypothesis "p implies q", or else can falsify it.

In the experiments described in the paper, there's a set of cards which have either "p" or "not p" on one side, and either "q" or "not q" on the other, and they are displayed to the subject so that some have the "p side" showing and some the "q side". The point of the experiment seems to be that in order to test the hypothesis "p implies q", it's necessary to look at all the cards showing "p" and all the cards showing "not q", but that only about 10% of the subjects can work that out. 

But there's no difference between the action of looking at a "p" card and looking at a "not q" card. In each case, the result of looking may be to falsify the hypothesis by a counter-example. Or the result may be to tend to support the hypothesis because the card is consistent with it. And if all the cards with "p" and all the cards with "not q" showing are consistent with the hypothesis, then the hypothesis is confirmed.

(2017-10-23, 03:58 PM)fls Wrote: [ -> ]So if we are agreed that none of those things are pertinent with respect to "falsify" or "disconfirmation", what was I (and other scientists) talking about? I gave the answer in my first post by linking to two papers which you've only just now read (partially), and have decided are irrelevant. So you can see why I have found this an uphill struggle.  But at least, we can maybe be on the same page from this point.

Thanks for your generous response, Linda. Am about to run out of battery charge so will have to respond to you in a few hours when I have the chance to power back up.

Thanks for your post, Linda.

Throughout it you suggest that different types of falsifiability are not in play. I think I understand what you are getting at here - I think you are saying that those types of falsifiability are not what you meant as referenced in those papers. That's fine, I just want to clarify that I do think that falsifiability, including in the Popperian sense (as best I understand it) is in play in the simple process of hypothesis testing given a null and alternative hypothesis. I think we would agree on this though.

(2017-10-23, 03:58 PM)fls Wrote: [ -> ]And of course, both kinds of tests - "p implies q" or "not q implies not p" - have the potential to be false or true, so "falsify" also does not refer to whether a test has the potential to return an answer of "false".

And I also agree that you cannot imply "p implies q" from "not p implies not q", but no one would suggest otherwise, since that is logically invalid. I suggested that "p implies q" is implied (or confirmed) when "not q implies not p" is found to be true. So "falsify" also does not refer to a test which does not have the ability to confirm the hypothesis.

I really hate, when we're so close to full agreement, to have to say, "Argh, that's still not quite right", but...

...there are a few errors in what you write above. You write that the two tests are "p implies q" and "not q implies not p" - whereas those are (sort of - see next point) rather potential outcomes of the tests, not the tests themselves. Why "sort of"? Because, at all times, including in particular re your second case ("not q implies not p"), we are not testing what either "q" or "not q" imply about p, but rather the reverse: what p (or not p) imply about q. Sometimes we start with "not q", but it is still the consequent, not the antecedent.

[Edit: Ouch. I was badly confused there, to the point that it's embarrassing. Once again: by turning over a card with "p" or "not p" on one side, and "q" or "not q" on the other, we can't say the outcome of our test involves anything to do with implication, only conjunction.]

I'm glad we agree that "not p implies not q" doesn't entail "p implies q". I think from what I wrote above, you can see that I don't agree that "not q implies not p" can ever be found to be true (at least in the scenario at issue: that of the first paper), because q (or not q) is only ever the consequent in these tests [Edit: really because we can't find anything to do with implication, only conjunction]

But let's say that it could. In that case: yes, you are correct that "not q implies not p" logically entails "p implies q", but this is really only because of a quirk of the definition of logical implication, in which a false antecedent always produces a true result, regardless of the value of the consequent (I could elaborate on what I mean but I won't unless it particularly interests you or anybody else). I don't think that in "everyday" logic we would accept this entailment, and I especially don't think that we would accept this entailment as proof in a scientific experiment.

And those are about all the bones I have to pick with your post. The rest of it is all very productively agreeable. :-)

OK, so, to the more important part of your post:

(2017-10-23, 03:58 PM)fls Wrote: [ -> ]So if we are agreed that none of those things are pertinent with respect to "falsify" or "disconfirmation", what was I (and other scientists) talking about? I gave the answer in my first post by linking to two papers which you've only just now read (partially), and have decided are irrelevant.

I think the second paper could definitely be relevant in certain scientific contexts - and it even provides examples of especially medical contexts in which it could be relevant. The sort of hypothesis testing or rule discovery in which it deals seems to be about more complex cases in which we are trying to discover which rule out of a universe of rules is applicable.

For example, it describes "the rule discovery task" as follows:

'There is a class of objects with which you are concerned; some of the objects have a particular property of interest and others do not. The task of rule discovery is to determine the set of characteristics that differentiate those with this target property from those without it'.

I would argue that these sort of tasks are not very (if at all) relevant in the simpler, perhaps more "standard" cases we've been discussing (at least in parapsychology): those in which a straightforward, falsifiable hypothesis is proposed, and is then tested in an experiment framed in terms of null and alternative hypotheses, in which the null is then either rejected or not rejected on a statistical basis.

I think too that this paper anyway says something key about "disconfirmation" (in these more complex cases of rule discovery etc):

'[It] is very important to distinguish between two different senses of "seeking disconfirmation." One sense is to examine instances that you predict will not have the target property. The other sense is to examine instances you most expect to falsify, rather than verify, your hypothesis. This distinction has not been well recognized in past analyses, and confusion between the two senses of disconfirmation has figured in at least two published debates'.

In other words: sometimes, examining an instance that you predict will have the target property, which in one sense is "seeking confirmation", can, in another sense, be "disconfirming". The paper goes into more detail as to when this is the case.

Possibly, this is why the paper prefers to distinguish between +Htests and -Htests, and +Ttests and -Ttests, rather than calling these "confirming" or "disconfirming" tests.

(2017-10-24, 01:10 AM)Laird Wrote: [ -> ]I think the second paper could definitely be relevant in certain scientific contexts - and it even provides examples of especially medical contexts in which it could be relevant.

It might be helpful to list (at least some of) these. Quoting directly from the paper:

1. 'For example, suppose that you hypothesize that a certain combination of home environment, genetic conditions, and physical health distinguishes schizophrenic individuals from others. It would be natural to select someone diagnosed as schizophrenic and check whether the hypothesized conditions were present. We will call this a positive target test (+Ttest), because you select an instance known to be in the target set. Similarly, you could examine the history of someone judged not to be schizophrenic to see if the hypothesized conditions were present. We call this a negative target test (-Ttest)'.
   
2. 'It is also interesting to consider the relations between Ttests and Htests. In some situations, it may be more natural to think about one or the other. In an epidemiological study, for example, cases often come presorted as T or T [the second T is supposed to have a horizontal bar above it to indicate exclusion from the group "T" --Laird] (e.g., diagnosed victims of disease vs. normal individuals). In an experimental study, on the other hand, the investigator usually determines the presence or absence of hypothesized factors and thus membership in H or H [again, the second H is supposed to have a horizontal bar above it] (e.g., treatment vs. control group).'
   
3. 'For example, suppose a team of meteorologists wants to test whether certain weather conditions are sufficient to produce tornadoes. The team can look for tornadoes where the hypothesized conditions exist (+Htests) or they can test for the conditions where tornadoes have not occurred (-Ttests). [...]  The meteorologists can test whether the hypothesized weather conditions are necessary for tornadoes by looking at conditions where tornadoes are sighted (+Ttests) or by looking for tornadoes where the hypothesized conditions are lacking (-Htests).'
   

If you (Linda) think I've done you an injustice by apparently arbitrarily excluding this sort of hypothesis testing from consideration, and instead focussing on the type of statistical hypothesis testing based on a null and alternative hypothesis, then I would understand.

(2017-10-24, 01:10 AM)PLaird Wrote: [ -> ]I really hate, when we're so close to full agreement, to have to say, "Argh, that's still not quite right", but...

If you hate it, why go to such lengths to find something to disagree with?

Quote:...there are a few errors in what you write above. You write that the two tests are "p implies q" and "not q implies not p" - whereas those are (sort of - see next point) rather potential outcomes of the tests, not the tests themselves.

I don't disagree. We've been talking all along about whether the results agree with a hypothesis or not, so we've essentially been talking about outcomes, rather than strictly what is or is not implied.

Quote:I think from what I wrote above, you can see that I don't agree that "not q implies not p" can ever be found to be true (at least in the scenario at issue: that of the first paper), because q (or not q) is only ever the consequent in these tests.

But let's say that it could. In that case: yes, you are correct that "not q implies not p" logically entails "p implies q", but this is really only because of a quirk of the definition of logical implication, in which a false antecedent always produces a true result, regardless of the value of the consequent (I could elaborate on what I mean but I won't unless it particularly interests you or anybody else). I don't think that in "everyday" logic we would accept this entailment, and I especially don't think that we would accept this entailment as proof in a scientific experiment.

That's an odd way to put it. Why use name-calling to dismiss it? I could just as easily say that it is a quirk of the definition of logical implication that a true consequent always produces a true result, regardless of the value of the antecedent, so why would anyone would accept "p implies q" based on this entailment?

Quote:I think the second paper could definitely be relevant in certain scientific contexts - and it even provides examples of especially medical contexts in which it could be relevant. The sort of hypothesis testing or rule discovery in which it deals seems to be about more complex cases in which we are trying to discover which rule out of a universe of rules is applicable.

For example, it describes "the rule discovery task" as follows:

'There is a class of objects with which you are concerned; some of the objects have a particular property of interest and others do not. The task of rule discovery is to determine the set of characteristics that differentiate those with this target property from those without it'.

I would argue that these sort of tasks are not very (if at all) relevant in the simpler, perhaps more "standard" cases we've been discussing (at least in parapsychology): those in which a straightforward, falsifiable hypothesis is proposed, and is then tested in an experiment framed in terms of null and alternative hypotheses, in which the null is then either rejected or not rejected on a statistical basis.

I disagree. Very little in parapsychology seems straightforward in comparison to other scientific pursuits.

Quote:I think too that this paper anyway says something key about "disconfirmation" (in these more complex cases of rule discovery etc):

'[It] is very important to distinguish between two different senses of "seeking disconfirmation." One sense is to examine instances that you predict will not have the target property. The other sense is to examine instances you most expect to falsify, rather than verify, your hypothesis. This distinction has not been well recognized in past analyses, and confusion between the two senses of disconfirmation has figured in at least two published debates'.

In other words: sometimes, examining an instance that you predict will have the target property, which in one sense is "seeking confirmation", can, in another sense, be "disconfirming". The paper goes into more detail as to when this is the case.

Possibly, this is why the paper prefers to distinguish between +Htests and -Htests, and +Ttests and -Ttests, rather than calling these "confirming" or "disconfirming" tests.

Thank you for getting that far in the paper.

Linda

I think it bears emphasis that the potential outcomes of the different tests are conclusive falsifications or ambiguous verifications. And that it is "false negatives" (h-tests, t+tests) which provide conclusive falsification that your hypothesis is necessary. "False positives" (h+tests, t-tests) merely show that your hypothesis is not sufficient. If you want to show that psi is necessary (which is probably what is needed to get other scientists to take psi seriously), an emphasis on h- or t+tests would be useful. 

Linda

(2017-10-24, 10:35 PM)fls Wrote: [ -> ]I think it bears emphasis that the potential outcomes of the different tests are conclusive falsifications or ambiguous verifications. And that it is "false negatives" (h-tests, t+tests) which provide conclusive falsification that your hypothesis is necessary. "False positives" (h+tests, t-tests) merely show that your hypothesis is not sufficient. If you want to show that psi is necessary (which is probably what is needed to get other scientists to take psi seriously), an emphasis on h- or t+tests would be useful. 

Well, I don't suppose you'll respond, because you're "not talking to me".

But in helping people to decide whether any of this stuff you're posting here is worth trying to make sense of, I think it would help if you could clarify your previous claims about these p/not p/q/not q experiments.

This is what you wrote originally:

You are attempting to confirm the idea when you choose to look at "p". Your result may be "q" (alternative hypothesis confirmed) or "not q" (null hypothesis confirmed). 

You are attempting to falsify the idea when you choose to look at "not q". Your result may be "not p" (alternative hypothesis confirmed) or "p" (null hypothesis confirmed).

Do you agree, now that it's been explained to you, that looking at "p" and looking at "not q" are exactly similar in their potential effects and consequences? That it's nonsensical to say that one is an attempt to confirm the "idea" and the other is an attempt to falsify it? That the whole point of the paper you referred to was that it was necessary to look at both "p" and "not q" in order to test the hypothesis? In short, that you missed the whole point of that paper when you tried to present looking at "p" and looking at "not q" as alternative tests? And that much of the ensuing confusion on this thread is of your own making?

Linda, responding to both of your posts at once:

(2017-10-24, 10:03 PM)fls Wrote: [ -> ]If you hate it, why go to such lengths to find something to disagree with?

No offence, but I didn't - it all jumped out at me. Why I did it is because I think that - especially on a public forum - mistakes ought to be corrected when they are discovered, whether those mistakes are yours, mine or anybody else's.

(2017-10-24, 10:03 PM)fls Wrote: [ -> ]That's an odd way to put it. Why use name-calling to dismiss it?

Name-calling? Not at all: that which I pointed out is a simple matter of fact and very justifiable.

(2017-10-24, 10:03 PM)fls Wrote: [ -> ]I could just as easily say that it is a quirk of the definition of logical implication that a true consequent always produces a true result, regardless of the value of the antecedent

You could, but you'd be totally missing the point.

(2017-10-24, 10:03 PM)fls Wrote: [ -> ]so why would anyone would accept "p implies q" based on this entailment?

Which entailment? You've lost me here.

(2017-10-24, 10:35 PM)fls Wrote: [ -> ]I think it bears emphasis that the potential outcomes of the different tests are conclusive falsifications or ambiguous verifications. And that it is "false negatives" (h-tests, t+tests) which provide conclusive falsification that your hypothesis is necessary. "False positives" (h+tests, t-tests) merely show that your hypothesis is not sufficient.

I don't take issue with any of that - and I agree that it is all backed up by the paper - except for the word "merely". I don't see why information about necessity is intrinsically more valuable than information about sufficiency. Perhaps you can explain why you do.

(2017-10-24, 10:35 PM)fls Wrote: [ -> ]If you want to show that psi is necessary (which is probably what is needed to get other scientists to take psi seriously), an emphasis on h- or t+tests would be useful.

That's a rather unfortunate phrasing: "show[ing] that psi is necessary". In any case, I think I understand what you mean. I'm not sure though that it is universally the case that psi is claimed to be necessary for various effects to occur. Potentially, there might also be other causes of these effects. And I think that that's fine: it's enough for parapsychology to demonstrate that psi is one of the ways that certain effects can occur i.e. that it is sufficient, even if not necessary, for them.

But perhaps you could anyway suggest some possible -Htests and +Ttests for parapsychological experiments. I can't think of any realistic ones, in particular for +Ttests, but then again I haven't tried very hard. I can think of many scenarios in parapsychology where they aren't realistic options.