Principles of Curiosity

As OP, I'm happy with any discussions that come up here (even tangentially) as a result of the Alex and Brian dialogue. 

WRT to small effect size, it feels "correct" that marginally significant free decisions (when to  stop a line, which line to include) would absolutely result in a small effect size.

As OP, I'm happy with any discussions that come up here (even tangentially) as a result of the Alex and Brian dialogue. 

WRT to small effect size, it feels "correct" that marginally significant free decisions (when to  stop a line, which line to include) would absolutely result in a small effect size.

Yes, but there are two separate issues here. One is what you've called "degrees of freedom", and those have to be curtailed however the data are going to be analysed, and in principle they can be curtailed by suitable pre-registration. (Outright fraud would be another matter, of course.) The other is the question of how to interpret the significance of the results once that's been done - p values, Bayesian analysis (coughs and spits) and so on.

Another comment I’ve read in these articles about underpowered studies is that they can inflate  effect sizes which is another concern with small effect sizes. In other words there is danger that Jon significant results can appear as significant. If ive understood it correctly. 


Also note the article points out that this assumes no other bias is present. If there is bias present, as I understand it, there is the risk that a small effect size is going to be measuring the presence of bias rather than a true effect. Note if we’re just comparing against chance this would legitimately be a non chance result. It just might not be showing psi. This makes that  much more crucial to minimize bias risk.

Another comment I’ve read in these articles about underpowered studies is that they can inflate  effect sizes which is another concern with small effect sizes. In other words there is danger that Jon [non] significant results can appear as significant. If ive understood it correctly. 

The danger of non-significant results appearing significant is the question of false positives we've been discussing. (Assuming that by "significant" you mean "reflecting a real phenomenon", rather than "p<0.05" or whatever.)

I think the inflation of the effect size is something a bit different - that if there is a real effect and we estimate its size only from studies with p<0.05 (or whatever), then we'll end up with an overestimate, because we're selecting studies in which the effect happens to be larger by chance. I'd have thought the answer to that would be simply to estimate the effect size from all the studies that have been done, though. (Publication bias might be a problem there, however.)

Also note the article points out that this assumes no other bias is present. If there is bias present, as I understand it, there is the risk that a small effect size is going to be measuring the presence of bias rather than a true effect. Note if we’re just comparing against chance this would legitimately be a non chance result. It just might not be showing psi. This makes that  much more crucial to minimize bias risk.

Yes, that's what I meant in my last comment about curtailing "degrees of freedom".

I was simply trying to be considerate by not responding to your posts, as you'd said you didn't wish to communicate with me. If you don't care whether I respond to them or not, I'll feel free to do so. If you prefer me not to, I won't - but in that case it will obviously make things easier if you don't respond to discussions I'm already involved in.

I wanted to tell you why you likely wouldn't see a response from me. I wasn't telling you how to behave.

I doubt that I won't respond to discussions you're already involved in. Our areas of interest overlap. And questions come up, like they did here, which I can answer because I'm already familiar with the subject. I did make a mistake in this case, because I meant to be answering jkmac's question about small effect sizes, false positives and low p-values, but I accidentally chose to quote a post by malf which had been a response to you. I will try to be more careful about that (I'm absent-minded though).

Linda

I wanted to tell you why you likely wouldn't see a response from me. I wasn't telling you how to behave.

Thank you for clarifying. There's no difficulty then.

https://sciencefriction.tv/

https://sciencefriction.tv/

$210,000? Maybe PsienceQuest should make its own documentary. I'll be happy to write the script for £25,000.

https://sciencefriction.tv/

Hmmm...why are they picturing someone who isn't a scientist?

I always wonder if anyone is ever persuaded to change their minds in these culture wars. Skeptics present their information in a way which is palatable to Skeptics, and Believers present their information in a way which is palatable to Believers, and each side wonders why the other falls for it. 

Linda