Peer Review
- Thread starter Paul Lee
- Start date
Jim Currie
Member
Hi Paul! nice to see you 'up and running' again.
An interesting proposal - how would you visualise
such a 'check' and how would you prevent 'auditor's bias'?
Jim.
An interesting proposal - how would you visualise
such a 'check' and how would you prevent 'auditor's bias'?
Jim.
Hi Jim, thanks for your kind welcome.
I can only really speak to the process used when refereeing scientific papers; that the paper is sent to many recognised experts, and they raise questions that are addressed to the author. There is an iterative process between the author and each referee, not between referees, and hopefully some consensus can be achieved.
I can only really speak to the process used when refereeing scientific papers; that the paper is sent to many recognised experts, and they raise questions that are addressed to the author. There is an iterative process between the author and each referee, not between referees, and hopefully some consensus can be achieved.
In principle it's a great idea but I'm not sure it is very practical.
This isn't medicine or physics, the number of "experts" available would be very small and it would not therefore be a very difficult matter to work out who the peers were.
Would you as an author submit your article when you know that it would be peer reviewed by someone with whom you had had numerous vigorous disagreements over the same subject matter?
Who would mediate? How long would be allowed... what if there was no consensus, just deadlock? It's a bit of a minefield... but if someone can propose a practical way through it certainly bears consideration.
Obviously we don't want to publish material with errors if it can be avoided.
This isn't medicine or physics, the number of "experts" available would be very small and it would not therefore be a very difficult matter to work out who the peers were.
Would you as an author submit your article when you know that it would be peer reviewed by someone with whom you had had numerous vigorous disagreements over the same subject matter?
Who would mediate? How long would be allowed... what if there was no consensus, just deadlock? It's a bit of a minefield... but if someone can propose a practical way through it certainly bears consideration.
Obviously we don't want to publish material with errors if it can be avoided.
David Gleicher
Member
Philip, et. al.
First, I think the exchange on article reviewing should be moved to a different thread, and separated from discussion David Brown’s article, the merits of which are being discussed in their own right.
I’m in academia and I’ve been interested in this question of article review in ET for some time. I remember a paper way back that had an algebraic error in it that when corrected, as it soon was on the message board, led to the opposite conclusion as the one that author’s paper was all about.
A big advantage, of the research articles on ET is that there is not the kind of space/time limitations that peer-reviewed journals experience. Thus, a much larger percentage of submissions, all the way up to virtually 100% (as is currently the case), can be accepted. Also, highly debatable claims in articles can be challenged via the message board, once the article is posted.
My suggestion would be the assignment of each submission to a reviewer who largely acts as a ‘fact checker’. As in any academic journal a general editor would get a reviewer knowledgeable in the particular area of the submission.
Unlike an academic journal–where issues of originality, contribution to the literature, clarity of writing and so forth are considered--the reviewer would always allow for a re-submission, based on correcting and/or providing references supporting any important assertion, where there are serious questions as to its factual basis. In certain contexts, the writer might simply be invited to state that a particular line of thought is ‘speculative’.
DG
First, I think the exchange on article reviewing should be moved to a different thread, and separated from discussion David Brown’s article, the merits of which are being discussed in their own right.
I’m in academia and I’ve been interested in this question of article review in ET for some time. I remember a paper way back that had an algebraic error in it that when corrected, as it soon was on the message board, led to the opposite conclusion as the one that author’s paper was all about.
A big advantage, of the research articles on ET is that there is not the kind of space/time limitations that peer-reviewed journals experience. Thus, a much larger percentage of submissions, all the way up to virtually 100% (as is currently the case), can be accepted. Also, highly debatable claims in articles can be challenged via the message board, once the article is posted.
My suggestion would be the assignment of each submission to a reviewer who largely acts as a ‘fact checker’. As in any academic journal a general editor would get a reviewer knowledgeable in the particular area of the submission.
Unlike an academic journal–where issues of originality, contribution to the literature, clarity of writing and so forth are considered--the reviewer would always allow for a re-submission, based on correcting and/or providing references supporting any important assertion, where there are serious questions as to its factual basis. In certain contexts, the writer might simply be invited to state that a particular line of thought is ‘speculative’.
DG
Bill Wormstedt
Member
Most of the Titanic authors I'm aware of, do submit their work to other researchers for peer review previous to 'publication', in whatever form the work is finally released.
However, it is a given that these submissions are going to researchers that the author is familiar with and probably has worked with in the past. Researchers who are known to be fair, impartial and detail oriented. Obviously, an author is not going to submit work for peer review to someone who has been antagonistic to that author's work in the past. As Phil says above, that just leads to deadlock and goes nowhere.
However, it is a given that these submissions are going to researchers that the author is familiar with and probably has worked with in the past. Researchers who are known to be fair, impartial and detail oriented. Obviously, an author is not going to submit work for peer review to someone who has been antagonistic to that author's work in the past. As Phil says above, that just leads to deadlock and goes nowhere.
Jim Currie
Member
It occurred to me that there are some pretty basic and particular 'facts' concerning the case of Titanic. Most of these are technical, mathematical or recorded somewhere for posterity and curious researchers. All of them are able to respond to some form of adjustment or correction Therefore a 'Peer Commitee' of like-minded individuals might be useful here. I'm thinking of the example of an algebraic error or similar errors in other mathematics. Perhaps even simple arithmetic errors as I have been guilty of.
Even certain assumptions made due to the absence of knowledge of former practices might be corrected in an otherwise, first class presentation.
On the other hand, there is a great deal of available evidence based on the recollection of individuals. It is the interpretation of this evidence that causes the most controversy. I think the fact that researchers tend to come down on one or other side of an argumentative fence would make it very difficult to set-up a completely unbiased 'Peer Commitee' to deal with this.
Jim.
Even certain assumptions made due to the absence of knowledge of former practices might be corrected in an otherwise, first class presentation.
On the other hand, there is a great deal of available evidence based on the recollection of individuals. It is the interpretation of this evidence that causes the most controversy. I think the fact that researchers tend to come down on one or other side of an argumentative fence would make it very difficult to set-up a completely unbiased 'Peer Commitee' to deal with this.
Jim.
Notwithstanding the practical issues I'm interested to see if this could work.
Whether a contributing author would be willing to subject their work to pre-publication scrutiny and under what ground-rules remains to be seen.
As indicated, there would probably need to be some kind of convenor/editor who was themselves generally regarded as impartial but knowledgeable enough to identify suitable peers.
Issues to consider include whether the author would consent to anonymous review or only by peers known and approved by *them*. Timescales and deadlines would be another factor, especially to avoid deadlock! A further consideration is [if not anonymous] whether peer debate should be fostered or whether it should be restricted to feedback from peer to author via the convenor as go-between as it would be if anonymous review were the chosen approach.
Convoluted explanation but I hope that makes sense.
Whether a contributing author would be willing to subject their work to pre-publication scrutiny and under what ground-rules remains to be seen.
As indicated, there would probably need to be some kind of convenor/editor who was themselves generally regarded as impartial but knowledgeable enough to identify suitable peers.
Issues to consider include whether the author would consent to anonymous review or only by peers known and approved by *them*. Timescales and deadlines would be another factor, especially to avoid deadlock! A further consideration is [if not anonymous] whether peer debate should be fostered or whether it should be restricted to feedback from peer to author via the convenor as go-between as it would be if anonymous review were the chosen approach.
Convoluted explanation but I hope that makes sense.
Samuel Halpern
Member
Interesting discussion indeed.
One of the indicators of well researched paper is the references provided by the author. Another is an acknowledgement list of those who contributed to the paper someway, even if those contributions were nothing more than providing the author with feedback on the content and suggestions for improvement. At least that shows the reader that the author was willing to have others review their work beforehand.
As far putting in place a formal review process is concerned, I kind of like the idea of a convenor/editor who can identify suitable anonymous reviewers willing to provide the anonymous author with constructive feedback. After that, it is the author who must decide what to do with the feedback they get. The author can then inform the convenor/editor what was done regarding the feedback, if anything, and resubmit the revised paper for publication. The editor can then decide to accept or reject it.
One of the indicators of well researched paper is the references provided by the author. Another is an acknowledgement list of those who contributed to the paper someway, even if those contributions were nothing more than providing the author with feedback on the content and suggestions for improvement. At least that shows the reader that the author was willing to have others review their work beforehand.
As far putting in place a formal review process is concerned, I kind of like the idea of a convenor/editor who can identify suitable anonymous reviewers willing to provide the anonymous author with constructive feedback. After that, it is the author who must decide what to do with the feedback they get. The author can then inform the convenor/editor what was done regarding the feedback, if anything, and resubmit the revised paper for publication. The editor can then decide to accept or reject it.
Jim Currie
Member
Sam,
I notice your use of the word 'constructive'.
I agree 100%. Just add 'Objective' as well.
You proposals seem to suggest an almost automatic reference of work to a convenor/editor.
Otherwise the editor would require guidelines concerning the criteria governing whether an article is accepted on first submission or would benefit from being referred to a convenor/editor.
When the amended ( or otherwise ) article is re-submitted; what would be the criteria governing the final decision of the Editor?
I've probably missed something somewhere.
I notice your use of the word 'constructive'.
I agree 100%. Just add 'Objective' as well.
You proposals seem to suggest an almost automatic reference of work to a convenor/editor.
Otherwise the editor would require guidelines concerning the criteria governing whether an article is accepted on first submission or would benefit from being referred to a convenor/editor.
When the amended ( or otherwise ) article is re-submitted; what would be the criteria governing the final decision of the Editor?
I've probably missed something somewhere.
Samuel Halpern
Member
Hi Jim.
>>When the amended ( or otherwise ) article is re-submitted; what would be the criteria governing the final decision of the Editor? <<
The intent is to give the author a chance to address any issues raised and correct any errors pointed out that may have been missed. They don't have to agree with the reviewer on everything, but at least they should provide some explanation if the they choose to simply ignore some suggestions. If they do that, I see no reason why the article should not be published.
>>When the amended ( or otherwise ) article is re-submitted; what would be the criteria governing the final decision of the Editor? <<
The intent is to give the author a chance to address any issues raised and correct any errors pointed out that may have been missed. They don't have to agree with the reviewer on everything, but at least they should provide some explanation if the they choose to simply ignore some suggestions. If they do that, I see no reason why the article should not be published.
Jim Currie
Member
I see what you mean Sam. I was mildly concerned about what would evoke rejection of an article? (apart from the usual grounds of racisism,religeon. politics and obscenity etc).
David Gleicher
Member
Jim,
An article, I think, would be rejected if an author cannot find a basis for factual assertions he/she makes in the article, particularly where there are clear facts that contradict the assertions.
If the author has a problem with the judgment of a reviewer, then in most journals, the general editor decides. So, I agree that a committee just means added bureaucracy.
I don't think it's fair to the general editor to have to make a preliminary decision on all articles submitted, particularly since the research has become more and more specialized, and the editor might not feel qualified to make such a decision in certain cases.
Why not just have the editor assign each article to a reviewer and spread the work to people who have a good knowledge of the area that a particular article covers?
DG
An article, I think, would be rejected if an author cannot find a basis for factual assertions he/she makes in the article, particularly where there are clear facts that contradict the assertions.
If the author has a problem with the judgment of a reviewer, then in most journals, the general editor decides. So, I agree that a committee just means added bureaucracy.
I don't think it's fair to the general editor to have to make a preliminary decision on all articles submitted, particularly since the research has become more and more specialized, and the editor might not feel qualified to make such a decision in certain cases.
Why not just have the editor assign each article to a reviewer and spread the work to people who have a good knowledge of the area that a particular article covers?
DG
Similar threads
- Locked
