Congratulations. Your Study Went Nowhere.

Congratulations. Your Study Went Nowhere.


By Aaron E. Carroll

Publication bias refers to the decision on whether to publish results based on the outcomes found. With the 105 studies on antidepressants, half were considered “positive” by the F.D.A., and half were considered “negative.” Ninety-eight percent of the positive trials were published; only 48 percent of the negative ones were.

Outcome reporting bias refers to writing up only the results in a trial that appear positive, while failing to report those that appear negative. In 10 of the 25 negative studies, studies that were considered negative by the F.D.A. were reported as positive by the researchers, by switching a secondary outcome with a primary one, and reporting it as if it were the original intent of the researchers, or just by not reporting negative results.

Spin refers to using language, often in the abstract or summary of the study, to make negative results appear positive. Of the 15 remaining “negative” articles, 11 used spin to puff up the results. Some talked about statistically nonsignificant results as if they were positive, by referring only to the numerical outcomes. Others referred to trends in the data, even though they lacked significance. Only four articles reported negative results without spin.

Spin works. A randomized controlled trial found that clinicians who read abstracts in which nonsignificant results for cancer treatments were rewritten with spin were more likely to think the treatment was beneficial and more interested in reading the full-text article.
It gets worse. Research becomes amplified by citation in future papers. The more it’s discussed, the more it’s disseminated both in future work and in practice. Positive studies were cited three times more than negative studies. This is citation bias.

Scary man.  Sure as hell isn't science.  Hard to reconcile when so much of what scientists study has such clear connections to profit making entities.  I'm not sure what's the answer.

Interestingly, the physicians rated the treatments as unlikely to be beneficial. The group without spin rated them slightly more unlikely (but less than the one point difference the researchers were looking for). The studies were also given low ratings (by both groups) on importance and desire to perform a follow-up study. And the studies with spin in the abstract were rated lower in rigor than those without. So the physicians weren’t fooled into thinking ineffective treatments were beneficial because of the spin. Nor did the spin go unnoticed. But I can understand why the researchers didn’t present their results in that light.

Linda

Scary man.  Sure as hell isn't science.  Hard to reconcile when so much of what scientists study has such clear connections to profit making entities.  I'm not sure what's the answer.

My late wife was diagnosed with post traumatic stress and agoraphobia. Before she took her life. She became addicted  to  xanax.
According to the book by  Robert Whitaker entitled, Anatomy of an Epidemic. The trials for the drug were completely cooked. 
In the early stages. People on xanax, did much better than those on placebo. But at the end of the 14 week trial people on placebo, were experiencing  much less anxiety than the group on xanax.
The manufacturer only published  the evidence  from the early part of the trial.

My late wife was diagnosed with post traumatic stress and agoraphobia. Before she took her life. She became addicted  to  xanax.
According to the book by  Robert Whitaker entitled, Anatomy of an Epidemic. The trials for the drug were completely cooked. 
In the early stages. People on xanax, did much better than those on placebo. But at the end of the 14 week trial people on placebo, were experiencing  much less anxiety than the group on xanax.
The manufacturer only published  the evidence  from the early part of the trial.

This is horrifying, although I'm not surprised that this is case, given rampart industry corruption. I wonder if it's the same for all Benzos, as it is for Xanax?

On the drugs forum on Reddit, there are many posts about Benzo addiction, as well as many defending Benzos, probably because they're still in the honeymoon stage, in denial, or something.

This is horrifying, although I'm not surprised that this is case, given rampart industry corruption. I wonder if it's the same for all Benzos, as it is for Xanax?

On the drugs forum on Reddit, there are many posts about Benzo addiction, as well as many defending Benzos, probably because they're still in the honeymoon stage, in denial, or something.

I think the Food.and Drug Administration decision to let pharmaceutical companies run thier own trials. Was an abdication  of Epidemic  proportions.

The kind of bias I have a problem with is characterized by the Wizard of Oz Syndrome. For instance, some parapsychologist asks misleading questions in a survey and publishes a conclusion that agrees with his ideology based on the survey. One, he does not consider alternative theories, and two, the authority of his Ph.D. makes his results gospel for people looking for proof of their same ideology.

Here is an opinion. What are your thoughts? ---

Bad science tends to become evident over time, albeit sometimes after many deaths. Science is a trust we pay for as part of our taxes. It is intended to serve society. But progress is hampered by bad science. If we consider science a service, it seems in the public's best interest to fund better oversight over how our tax dollars are translated to benefit society.

This essay will give you a sense of my point of view: Open Letter to Paranormalists: Limits of science, trust and responsibility

The kind of bias I have a problem with is characterized by the Wizard of Oz Syndrome. For instance, some parapsychologist asks misleading questions in a survey and publishes a conclusion that agrees with his ideology based on the survey. One, he does not consider alternative theories, and two, the authority of his Ph.D. makes his results gospel for people looking for proof of their same ideology.

Here is an opinion. What are your thoughts? ---

Bad science tends to become evident over time, albeit sometimes after many deaths. Science is a trust we pay for as part of our taxes. It is intended to serve society. But progress is hampered by bad science. If we consider science a service, it seems in the public's best interest to fund better oversight over how our tax dollars are translated to benefit society.

This essay will give you a sense of my point of view: Open Letter to Paranormalists: Limits of science, trust and responsibility

Interesting stuff - gonna take some time to go through it but in the immediate I like how you clearly delineate a few important positions re: metaphysics, mental functioning, etc.