How does Peer Review work?

9 Replies, 913 Views

Who gets to read a paper? Who decides ? 

I could of course have ‘googled’ an answer. But this way I might get ‘more’.
Oh my God, I hate all this.   Surprise
[-] The following 1 user Likes Stan Woolley's post:
  • Silence
(2020-07-17, 01:22 PM)Stan Woolley Wrote: Who gets to read a paper? Who decides ? 

I could of course have ‘googled’ an answer. But this way I might get ‘more’.

I think the author invites experts in his field that he is either aware of, or knows (of) personally. I think it's quite complicated 
with all kinds of different benchmarks depending on all sorts of variations in the quality of the study. I bet you're glad you asked now, Stan .... Huh   In other words, I don't know.
[-] The following 3 users Like tim's post:
  • Sciborg_S_Patel, Ninshub, Stan Woolley
(2020-07-17, 05:24 PM)tim Wrote: I think the author invites experts in his field that he is either aware of, or knows (of) personally. I think it's quite complicated 
with all kinds of different benchmarks depending on all sorts of variations in the quality of the study. I bet you're glad you asked now, Stan .... Huh   In other words, I don't know.

I thought there might be a few here that are familiar with the process. I was interested in questions like Who would review Bernardo Kastrup’s papers? Would Bernardo have any say in the process or does he hand it to an independent body? These days it seems to me to be worth asking the question: Is there such a thing as an independent body? 

My five star book reviews are mainly by people that know me or knew me personally. I’m sure it’s not like that.  Wink
Oh my God, I hate all this.   Surprise
[-] The following 3 users Like Stan Woolley's post:
  • tim, Sciborg_S_Patel, Ninshub
(2020-07-17, 05:32 PM)Stan Woolley Wrote: I thought there might be a few here that are familiar with the process. I was interested in questions like Who would review Bernardo Kastrup’s papers? Would Bernardo have any say in the process or does he hand it to an independent body? These days it seems to me to be worth asking the question: Is there such a thing as an independent body? 

My five star book reviews are mainly by people that know me or knew me personally. I’m sure it’s not like that.  Wink
Everyone in the relevant disipline reads the novel paper then proceed to rip holes in it.  Think of the House of Commons. If it stands up to intense scrutiny it gets excepted for publication followed by more hole ripping id est testing. As for Bernardo, other philosophers do that and will argue endlessly (for centuries). It's that simple.
[-] The following 1 user Likes Steve001's post:
  • Stan Woolley
Bernardo has a computer engineering PhD. Regarding his philosophy papers specifically, those also get reviewed depending on the publication.

Even articles like this, where Bernardo and two physicists advocate for Idealism in Sci Am, get reviewed.

Of course there are a variety of issues involved with peer review:

An Era of Doubt: Scientific Misconduct Is Jeopardizing the Quality and Integrity of Journals

Quote:The industry is still grappling with questions about rigor in science. A cottage industry of ethics groups and watchdog organizations such as the Center for Scientific Integrity has risen up to support and scrutinize publication ethics. COPE, the Committee on Publication Ethics for example, has seen their ranks grow from a small group of medical journal editors in 1997 to more than 10,000 members worldwide.

Critics point to new influences like technology and the rise of unscrupulous “predator” publications and call into question the validity of the peer review process and the tremendous pressures on a system based on incentives. Some, like former British Medical Journal Editor Richard Smith, take it a step further and declare that the age of journals is over.

Or more recently:

Peer-Reviewed Scientific Journals Don't Really Do Their Job

Quote:The rapid sharing of pandemic research shows there is a better way to filter good science from bad.
Quote:There’s another way to think about this development, however. Instead of showing (once again) that formal peer review is vital for good science, the last few months could just as well suggest the opposite. To me, at least—someone who’s served as an editor at seven different journals, and editor in chief at two—the recent spate of decisions to bypass traditional peer review gives the lie to a pair of myths that researchers have encouraged the public to believe for years: First, that peer-reviewed journals publish only trustworthy science; and second, that trustworthy science is published only in peer-reviewed journals.
'Historically, we may regard materialism as a system of dogma set up to combat orthodox dogma...Accordingly we find that, as ancient orthodoxies disintegrate, materialism more and more gives way to scepticism.'

- Bertrand Russell


(This post was last modified: 2020-07-17, 10:57 PM by Sciborg_S_Patel.)
[-] The following 4 users Like Sciborg_S_Patel's post:
  • Ninshub, OmniVersalNexus, Typoz, Stan Woolley
(2020-07-17, 10:37 PM)Steve001 Wrote: Everyone in the relevant disipline reads the novel paper then proceed to rip holes in it.  Think of the House of Commons. If it stands up to intense scrutiny it gets excepted for publication followed by more hole ripping id est testing. As for Bernardo, other philosophers do that and will argue endlessly (for centuries). It's that simple.

I’m quite sure it’s not, Steve. Hardly anything is simple nowadays.  Wink
Oh my God, I hate all this.   Surprise
[-] The following 1 user Likes Stan Woolley's post:
  • tim
Quote:"Everyone in the relevant disipline reads the novel paper then proceed to rip holes in it."

That sounds wholly impractical. Surely just a few are involved, rather than everyone.
[-] The following 2 users Like Typoz's post:
  • Silence, tim
(2020-07-17, 10:37 PM)Steve001 Wrote: Everyone in the relevant disipline reads the novel paper then proceed to rip holes in it. 
This is not what happens at all. In reality a handful of people read a paper, and often there is little overlap between the respondents discipline and the discipline of those under review. Some peoples research is so niche, that maybe only a few other people in the world are as knowledgeable about a topic as they are. In addition peer review is a system that is not in itself subject to much scrutiny, and thus often runs afoul of review milling, where the same group of scientists will read the papers of their peers and vice versa and give each other the go ahead and lets not forget that the infamous hermaneutics of quantum gravity paper passed peer review, otherwise known as the Sokal Affair where a completely fake paper successfully passed peer review (and many others as well).

A list of issues with peer review
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/

In short, it's inconsistent, based on undeserved trust, open to bias, slow and expensive as well as open to abuse by plagiarists and others who want to silence their peers by gatekeeping and only works to detect fraud a quarter or so of the time.
[-] The following 3 users Like letseat's post:
  • Silence, Stan Woolley, OmniVersalNexus
(2020-07-25, 05:31 AM)letseat Wrote: In short, it's inconsistent, based on undeserved trust, open to bias, slow and expensive as well as open to abuse by plagiarists and others who want to silence their peers by gatekeeping and only works to detect fraud a quarter or so of the time.

Thanks for that letseat.

From the very useful* article you linked to:
“Famously, it is compared with democracy: a system full of problems but the least worst we have.“

If it’s remotely in as bad a state as ‘democracy’ in the UK, it’s already dead.   Confused

*Richard Smith was editor of the BMJ and chief executive of the BMJ Publishing Group for 13 years.
Oh my God, I hate all this.   Surprise
(This post was last modified: 2020-07-25, 08:28 AM by Stan Woolley.)
[-] The following 1 user Likes Stan Woolley's post:
  • letseat
I really would like this book by above mentioned Richard Smith, but for some reason that I’d be interested in knowing, it’s not cheaply available. (£28 for Paperback via Amazon) Huh


Attached Files Thumbnail(s)
   
Oh my God, I hate all this.   Surprise

  • View a Printable Version
Forum Jump:


Users browsing this thread: 1 Guest(s)