Peer review suggestions for ET Research Articles


Status
Not open for further replies.
Mar 20, 2000
3,107
33
323
All:

On another thread, the topic has been derailed by discussion of the feasibility of a peer review or proofing process for ET Research Articles. Perhaps here we can speak to the issue without distorting subjects.

I’ve been drafting a proposal to Phil Hind which would have been sent him today, but I am now adding considerations brought up by Monica Hall which she expressed in the following message posted to the other thread. I have copied it here so that all may read it and I will respond to her points in another post.

Randy


Posted by Monica Hall, ET moderator:

There are a few points I would like to make here, before people polarize any further.

Although ET is highly respected for the integrity of its research, it is not actually an academic forum, not being part of an academic institution. Academic fora are peer-reviewed in refereed journals for a number of reasons, mostly to:

* to check statistical analysis of research
findings
* to verify references and spot plagiarism
* to check on "known" facts

The sort of research undertaken here is very rarely substantiated by analytical research techniques. References, plagiarism and "known" facts can be questioned retrospectively via this very section - as indeed they most often are.

Furthermore, academic peer reviewing is not quite like an editor demanding for facts to be substantiated. It usually involves several reviewers who revise their opinions in the light of their fellow-reviewers comments. It takes a long time. Very often, publication takes up to two years. I'm not sure we want to go there, or indeed it is appropriate for this topic, given the discussion available post-publication in this section.

The other point I would make is that academic peer-reviewers undertake these tasks at work i.e. they are paid to do so. Relying on volunteers actually diminishes the credibility of the reviewers. In academic life, one can ask any known expert to review - s(he) will do so just because it is part of their daily work routine,and necessary for his/her career, which is a vital part of ensuring the unbiased quality of the review (as best one can - it's never perfect) . I don't think we could ensure that here simply because nearly everyone on ET is here in a voluntary, interested capacity. So their time is limited and unpaid.

I think Phil Hind must make the final decision, as some of you have said. However, the Board operates on a day-to-day basis via the Moderators, so if you feel strongly about this issue, I suggest you contact us and we can co-ordinate members' views for Phil.

I can see the arguments both for and against this idea but, principally, I do not think it is an issue which should engender great argument. Whether or not a research article should be critically evaluated before or after publication is largely a matter of its importance to everyday life. Medical research, for example, matters a great deal. The Titanic does not impinge upon everyday life, so research can probably be criticized safely in retrospect, as it is here, already.

I am very happy to discuss this via email with anyone who has views on what is, really, a discussion on how far, academically, we should expect to take Titanic research.
 
Mar 20, 2000
3,107
33
323
Monica Hall wrote:

Academic fora are peer-reviewed in refereed journals for a number of reasons, mostly to:

1) to check statistical analysis of research
findings
2) o verify references and spot plagiarism
3) to check on "known" facts"

That’s basically what a review panel would do here. There is no reason why a website offering the quality of research that can be found in professional journals can’t be operated on an academic model.



Monica: "References, plagiarism and "known" facts can be questioned retrospectively via this very section - as indeed they most often are. "

Many readers who access ET Research Articles via search engines are not members of ET and are therefore unaware of the message board forum. This material is sometimes downloaded for use by students working on school projects, by journalists writing stories and by any number of members of the public who should be assured that what they are reading is true. I am referring to fact-based historical articles and not to editorial/opinion pieces, which would not be subjected to the same scrutiny. Obvious errors of fact or erroneous sources should not be for the general public to find, which would only reflect badly on ET. A peer review process must be established to ensure accuracy. A message board forum is not adequate for correcting material on the main site. Efforts should be made to ensure accuracy of material before it is released to the public.



Monica: "Furthermore, academic peer reviewing is not quite like an editor demanding for facts to be substantiated. It usually involves several reviewers who revise their opinions in the light of their fellow-reviewers comments."

My analogy was based on my working experience. At newspapers and magazines, there are proofreaders/copywriters who report to the editor. But it’s not really so different than an academic journal where there is also an editor who after all, has final "say" on what is published. Any review group elected/appointed here would operate on the same premise as the moderators, with Phil Hind ultimately in charge.



Monica: "It takes a long time. Very often, publication takes up to two years. I'm not sure we want to go there, or indeed it is appropriate for this topic, given the discussion available post-publication in this section."

I am not suggesting a review committee which actually orders an investigation of research methods, etc., before approving a study. But the basic form of academic panel would work very well here to ensure the accuracy of what is released to the public. Historical research in this genre does warrant standards of reviewing that are not in place yet. A post-publication discussion forum does not meet standards that would be considered adequate in any other venue. We need to set up a panel that can work out any problems with a submitted piece before it is published. Then discussion can take place.



Monica: "The other point I would make is that academic peer-reviewers undertake these tasks at work i.e. they are paid to do so. Relying on volunteers actually diminishes the credibility of the reviewers."

That’s not true. There are highly regarded volunteer peer councils or advisory panels functioning at any number of institutions and publications without any compromise in the level of commitment and concern on the part of the committee members. Here at ET we have our own group of unpaid moderators which has a proven track record of excellent management (even in light of last June’s problems).



Monica: "I don't think we could ensure that here simply because nearly everyone on ET is here in a voluntary, interested capacity. So their time is limited and unpaid."

Again, not true. Volunteerism is a remarkably viable process for affecting positive results in many aspects of life, professionally and privately. If the credibility of a voluntary, unpaid group of reviewers on this site can’t be trusted, then the credibility of the voluntary, unpaid group of moderators already in place is similarly in question.



Monica: "I think Phil Hind must make the final decision, as some of you have said. However, the Board operates on a day-to-day basis via the Moderators, so if you feel strongly about this issue, I suggest you contact us and we can co-ordinate members' views for Phil."

Phil Hind absolutely must make the final decision. No moderator can. And I don’t think there is a rule that a paying contributor can’t contact the editor and owner of this site without first having his or her views screened or "co-ordinated" by a moderator. A review panel would report to Phil just as the moderators do now.

Randy
 

Tad G. Fitch

Member
Dec 13, 1999
581
14
263
Randy,

Quote:
"Many readers who access ET Research Articles via search engines are not members of ET and are therefore unaware of the message board forum. This material is sometimes downloaded for use by students working on school projects, by journalists writing stories and by any number of members of the public who should be assured that what they are reading is true. I am referring to fact-based historical articles and not to editorial/opinion pieces, which would not be subjected to the same scrutiny. Obvious errors of fact or erroneous sources should not be for the general public to find, which would only reflect badly on ET. A peer review process must be established to ensure accuracy. A message board forum is not adequate for correcting material on the main site. Efforts should be made to ensure accuracy of material before it is released to the public."

I agree. The overall level of quality of information on ET is absolutely fantastic. However, I have seen many other websites quote or reference articles published on this website which contain mistakes that certainly would have been caught by a review board as you and others have proposed. I know for a fact that students often use this website as a resource for papers as well. Reviewing materials before publishing them here can only improve the quality of this already excellent site.

Randy quote:
"I am not suggesting a review committee which actually orders an investigation of research methods, etc., before approving a study. But the basic form of academic panel would work very well here to ensure the accuracy of what is released to the public. Historical research in this genre does warrant standards of reviewing that are not in place yet. A post-publication discussion forum does not meet standards that would be considered adequate in any other venue. We need to set up a panel that can work out any problems with a submitted piece before it is published. Then discussion can take place."

Very well stated. When I mentioned the possibility of setting up a review board in a similar fashion to that of psychological and medical journals, I was certainly not suggesting that approval and investigation of research techniques take place either. However, I was advocating a similar system of peer review of the articles in general for accuracy of information.

"Monica: "The other point I would make is that academic peer-reviewers undertake these tasks at work i.e. they are paid to do so. Relying on volunteers actually diminishes the credibility of the reviewers."

Randy:
"That’s not true. There are highly regarded volunteer peer councils or advisory panels functioning at any number of institutions and publications without any compromise in the level of commitment and concern on the part of the committee members. Here at ET we have our own group of unpaid moderators which has a proven track record of excellent management (even in light of last June’s problems).


Monica: "I don't think we could ensure that here simply because nearly everyone on ET is here in a voluntary, interested capacity. So their time is limited and unpaid."

Randy:
"Again, not true. Volunteerism is a remarkably viable process for affecting positive results in many aspects of life, professionally and privately. If the credibility of a voluntary, unpaid group of reviewers on this site can’t be trusted, then the credibility of the voluntary, unpaid group of moderators already in place is similarly in question."

I agree. The institutional research review board and the peer council at the university I attended are all-volunteer ones, and my university has a very good reputation in the field for depth, quality and number of research studies published yearly. Volunteer review boards certainly can be of high quality. Those of you who are researchers and regular posters here and moderators are here on a volunteer basis because we all have a shared passion for history and the topic of the Titanic, and look at how well that has turned out for the most part. It has been very successful.

Having majored in psychology and having participated in a few research studies and a thesis, I had to go through the peer review process many times and feel that it is a good system to model, at least on the conceptual level. Again, I would not advocate approval being needed to start an article, or the research methods being investigated and improved, but rather a board of peers to review the articles for overall accuracy before publication on the site. As many of you have said, Phil would absolutely have to make the final decision about the articles.
Kind regards,
Tad
 
Oct 28, 2000
3,242
550
388
Peer review on the academic model is in large part designed as an exclusionary device. Those who do not have academic standing (i.e. have not spent many years and lots of money supporting academe) aren't peers, so are inelligible for review despite the qualifications of their ideas. We are not academics here on the web. Our "peerage" consists of people with an unnatural curiosity for a sidebar event in history.

This site (as the web at large) is nothing more than an open markeplace of ideas, a Titanic bazaar. Just as there are bad vegetables for the unwary in a street market, so there is bad history here on ET. But, the best and freshest ingredients for a gourmet dinner are in the street market. Similarly, our marketplace of ideas has produced some of the best thinking on Titanic ever.

My desire is that ET continue encouranging the publication of ideas as freely and openly as possible. Unlike fruit, there are no "bad" ideas. Even wrong theories serve us well by showing unprofitable directions of research.

However, I believe that each document published on ET (or elsewhere on the web) should have one of those interactive "buttons" that takes the viewer to an open discussion of the specific paper. The often rough-and-tumble arguments in the forum should give the reader a sense of how the paper is perceived by the Titanic community. From that point on, it is up to the reader to come to conclusions...right or wrong.

So, I say let the ideas flow freely and without restriction. But, let the authors beware they will be subject to a barrage of criticism and comment. In the end, may the best ideas win the hearts and minds.

-- David G. Brown
 
Aug 29, 2000
4,562
28
323
Well-expressed, David. Actually we have a very effective system already in place where all members can comment, critique and address the entire message board about articles appearing, 24 hours a day, free of charge, and is monitored for civility by a battalion of moderators on duty in nearly every time zone around the globe. It is also interactive with the author of the piece who can respond immediately to observations and criticisms to all at the same moment. Photos are carefully vetted for copyright infringements. It comes under the topic ET Research Articles and is available to anyone who registers, and for all to read even without registering, right here, right now, on- The ET Message Board.
 
Mar 3, 1998
2,745
261
358
I agree with Dave. Peer review is a vehicle for control and censorship, designed to protect the institution at the cost of the author. Sometimes it works in a benevolent fashion, sometimes not. I like the free exchange of ideas in this forum. Some of my better ideas were inspired during exchanges where I didn't even agree with what was being said.

I would also wonder about the credentials of a peer review board. Not to sound superior, but if I were to submit an article about the technical workings of Titanic's Marconi apparatus, who would check my work for accuracy? Conversely, I would not consider myself the least bit qualified to verify, for example, Phil Gowan's claims about any passenger. There are researchers in this forum whose work is above peer. And what if someone wishes to depart from the proven, to explore the unprovable? If you liked the White Paper that Dave and I put together to explain our grounding theory, you would have been horrified by the reaction Dave received when he presented it for a peer review at Gibbs & Cox. By definition, a peer review has an agenda of some sort, and sometimes novel ideas run counter to a review board's agenda to the detriment of those wanting to look over the horizon.

Naturally, we all have to follow some basic rules of courtesy and decorum in order to have civil discourse. Having an editor to correct typographical errors and bad grammar is also an accepted practice in the publishing world. But a peer review to weigh the validity or correctness of ideas? I won't participate in such a thing. My vote is to let the author take responsibility for what he or she writes, as long as the published work does not violate the standing rules of this forum.

Parks
 
Mar 20, 2000
3,107
33
323
All:

Some of the above posters are getting bogged down by the term "peer," I think. No, we are not an academic group here. But we are offering scholarly work on an historical topic for public consumption, yet no real editorial process is being observed to ensure the accuracy of material before it is published. Phil is one person (though his hats are varied). He is as well-read on Titanica as many of us and does his best when proofing articles. But an editor of a site of this size, which offers a large volume of articles, could stand some help. That’s what this is about. If Phil decides he doesn’t need or want an editorial review board, then he will say so in due course. In my opinion, obviously, we need one. A post-publication discussion forum is all well and good but it should be secondary to a proper editorial process.

Also, we are talking about fact-checking basic (and specialized) information, not censoring. Any writer submitting an article to a print publication would have to have editors check their work. So why should it be different in an online magazine like ET Research?

David wrote: "We are not academics here on the web. Our "peerage" consists of people with an unnatural curiosity for a sidebar event in history."

We are indeed academics, we are historians, we are journalists, we are scientists, we are artists, students, grocers, mailmen, jobless even. We are many things and we have a responsibility to better the product we’re issuing to the public. As to our common cause for being ET members, regardless of the nature of the topic, this site is built around it and is considered the best of its kind. But there is room for improvement.



David: "Unlike fruit, there are no "bad" ideas. Even wrong theories serve us well by showing unprofitable directions of research."

I don’t think any publication should knowingly publish bad ideas. Flawed, erroneous material is not an option. That’s ridiculous. Just throwing any old bone out there isn’t what ET or any responsible publication is about.




David: "The often rough-and-tumble arguments in the forum should give the reader a sense of how the paper is perceived by the Titanic community. From that point on, it is up to the reader to come to conclusions...right or wrong."

As I said before, many people reading ET Articles are not members of the message board. Unless ET Articles are restricted to member access, a review process needs to be put into action to ensure the accuracy of material reaching the general public. We have a responsibility to more than our own membership because the material published here is currently available to any reader through the internet,. We need to act like a responsible, professional organization and that means taking into account the widespread public consumption of ET Articles



Shelley wrote: "Actually we have a very effective system already in place where all members can comment, critique and address the entire message board about articles appearing, 24 hours a day, free of charge, and is monitored for civility by a battalion of moderators on duty in nearly every time zone around the globe."

That sounds like a great advertisement but its not quite true, as many great ads aren’t. You’re ignoring the normal editorial process, which you are certainly familiar with, being a prolific writer and columnist. As I’ve said, a post-publication venue for airing concerns about an article is great but a standard pre-publication editorial review process is needed to ensure accuracy of facts and statistics. Two things are wrong in your statement. First of all, this is not a free site and secondly, moderators aren’t the end-all-be-all. Maybe some of the opposition from moderators to an editorial review board is due to their feeling that their importance will be negated. (Don’t you realize that some of your number would likely be a part of any review panel that’s appointed?)



Parks wrote: "I would also wonder about the credentials of a peer review board. Not to sound superior, but if I were to submit an article about the technical workings of Titanic's Marconi apparatus, who would check my work for accuracy?"

Again, we’re getting lost in the term "peer" which may have been the wrong word to use in the first place. We should be talking about an editorial review board, which is what is being proposed. As to the credentials of a review panel, I’ve addressed that in a proposal to Phil Hind. I believe anyone published here or elsewhere, particularly those who are recognized experts in a given field, should be eligible to be called on by Phil to proof articles. For example, Parks, you would be a contact for Phil for any wireless or forensic research submitted, Shelley would be a contact for reviewing articles touching on fashion, theatre, etc, and the list goes on, according to members’ specialties. As to your own work, Parks, there may well be no one with your particular expertise available to Phil to call on but your article should still be subject to basic proofing for incidental facts, shouldn’t it? If you submitted an article to the New York Times would you question the credentials of the team of editors who would have to proof it before publication?



Parks: "There are researchers in this forum whose work is above peer. And what if someone wishes to depart from the proven, to explore the unprovable?"

No one is above "peer" in the sense that they should be above having their work proof-read for general accuracy before publication. And that’s all we’re proposing. People seem afraid of censorship and so they should be, but that’s not what this is about. I specified in my email to Phil that the team of reviewers should not exhaustively attempt to confirm hard-to-find sources or contact interviewees, etc., and that any recommendations from reviewers as to corrections or changes could be vetoed by Phil if he felt they were trivial, unreasonable, etc. However, any author submitting controversial and/or groundbreaking studies might want to cite sources in footnotes or as chapter notes to substantiate their findings. They would not be required to do so, but Phil or a designated reviewer should have the right to ask for them to be provided - at least privately, as is done in journalism.



Parks: "By definition, a peer review has an agenda of some sort, and sometimes novel ideas run counter to a review board's agenda to the detriment of those wanting to look over the horizon."

Basic accuracy in ET Articles prior to publication is the aim of the "peer" or editorial review board being proposed here. There is no agenda beyond that. This isn’t an attempt at censorship but an endeavor to bring the ET Research editorial process into line with accepted standards at other publications, both print and online, and thereby raise the standard of a product from good to excellent.
 
Jul 9, 2000
58,666
881
563
Easley South Carolina
>>Again, we’re getting lost in the term "peer" which may have been the wrong word to use in the first place.<<

Randy, you may be right on that score, but remember, you're the one who's championing the idea of "Peer review," and you've used that term specifically. If that's what you really want, then you'll have to play by the rules of that game. Anything less is not peer review. If that's unacceptable, then you may wish to rethink your idea as to how to deal with quality control. (Which appears to be the real issue anyway.)
 
Oct 28, 2000
3,242
550
388
Randy -- you understand me perfectly, but we see this quite differently.

I see the overwhelming benefit to a free market in ideas even though it is messy. And, I do see benefit in publishing wrong ideas if for no other reason than to stimulate the creation of rebuttals which contain the truth.

There are many instances in Titanica where the conventional wisdom of the peerage is dead wrong. And, those errors won't be exposed by anything less than the free exchange of ideas. For instance, peer review would hold us to an impossible left turn sideswipe for that is the "official" version of the story.

Frankly, I'm little bothered that some innocents may be "burned" by bad ideas. Humans only learn from their mistakes. Nobody touches a hot stove twice. If someone doesn't know how to research, or is too lazy to do the work, I do not feel it is my responsibility to protect them from their failed education or their own slothfulness.

We will learn the most by letting ideas flow. Let people publish and let the ideas that can't stand up to criticism fall by their own weaknesses. That's real life.

-- David G. Brown
 
Mar 3, 1998
2,745
261
358
I am not going to argue the merits of a "peer review" (or whatever is intended by the term) other than what I have already posted above. I understand the logic behind the suggestion but have also had my own experience with similar processes and found them wanting. It may work for others but it won't for me. If I sound like an idiot through my own incompetence, then that's my responsibility...I don't want anyone else cleaning up my act for me and rob you all of a laugh at my expense.

Does the problem warrant a solution? I find by reading back through the thread that this discussion started over gripes about Molony's article, but is Molony the norm or an aberration? If an aberration, then why is there a need for a reactive measure? Has ET suffered as a result of Molony's (or those like him) alleged incompetence? The way I see it...no. Every time Molony posts an article, a lively debate ensues. A lot of good comes out of these debates even if the overall review is negative, as far as I can tell. If any change is to be made, I would support the recommendation given by someone earlier to attach a link to the end of each article that leads the reader to the forum thread dedicated to the subject so that the reader would be aware of (and even participate in) the ensuing discussion.

That's my two cents for whatever it's worth. I don't envy Phil his decision.

Parks
 
Mar 20, 2000
3,107
33
323
Michael:

I don’t appreciative your tone with me, with all that boldface and italics, as though I can’t read Why must you always be so condescending and negative?

I have put my ideas into a proposal, which was submitted to Phil Hind today. I don’t need to be lectured about the use of the word "peer," which was David Billnitzer’s original term anyway. I was using the term loosely which I already admitted was wrong, as it’s been interpreted literally, hence all this misunderstanding. And I don’t need to rethink my idea as I have had in my mind an editorial board all along.

In your zeal to be smart at my expense, you failed to thoroughly read my last post. I clearly set the record straight that what some of us are proposing is an editorial board made up of ET members/contributors. Instead of accepting the clarification, you’ve conveniently concentrated on my mistaken use of the term "peer."

David:

You definitely have valid moral and philosophical points that are good life lessons. But I’m afraid they really aren’t applicable in the context of publishing, the aim of which should be to present factual data, not to play "tough love" antics, abandoning responsibility to readers, not caring that they have been given wrong information. I say let data be as controversial as you like but let it be fact-based at least.

I think that indeed you should be bothered by innocents burned by bad ideas. If you have the power to give them truth and you choose to give them lies what is that but ill-will and deception?

I’m all for ideas flowing. They can be as full and flowing and as sensational as you want, but let’s put together an inclusive panel that can do what an editorial team should already be doing — and that is making sure that information is authentic and not tossed together, which reflects badly on a site that is otherwise excellent. I want it to be as good as it can be and I thought that by ranging all the many talents we have here, we could male ET Research Articles our Crown Jewels.

But it seems that there is just no way to reach out and do anything here that will better things without causing an avalanche of opposition, even in a case in which it’s so clear that a more professional approach is needed. People are either too set in their ways and don’t want to change or too full of their own importance to support others’ good ideas. (Or, in your case, David, just too darn cynical!
happy.gif
)

Best wishes,
Randy
 
Mar 20, 2000
3,107
33
323
No, Senan Molony is not an aberration. There are other articles with errors on ET Research, including my own. A review board could have caught it. That article is now being used in design history classes. This is not a sour grapes campaign against Senan. And there’s no full-proof way to keep errors from occurring in articles but reviewers could prevent their occurring quite as often.
 
Mar 20, 2000
3,107
33
323
I do believe our editorial board is already at work to judge by the harmlessly funny passage in my post which has now gone missing. Hmmm …. I think I begin to see the root of the opposition from some moderators about an editorial board. Editing is their exclusive right.
 
Mar 18, 2000
1,384
21
313
I see nothing wrong with a review, by knowledgable people, of new articles. Not to say "That's a stupid idea", but to catch small mistakes (we all make 'em), check for clear explanations and such. I've done that on the few things I've written, and have been asked for my 2 cents on other's articles.

With the understanding, of course, that the writer has the final say. Just because someone says "you should change this", doesn't mean the author has to.
 
Jan 21, 2001
144
1
263
Hi all:

My parents have been in town for the holidays, and naturally I have spent the last several days with them and not much on-line. However, now that I see this thread has begun, and as Randy pointed out, the term "peer review" was originally mine, permit me to clarify what I was thinking.

It does seem from the above thread that some are getting hung up on the term "peer." That means different things to different people. When it comes to ET, I think of everybody here as "peers" in this way - we all share an interest in Titanic and related matters, and on that score we are all equals. Everybody has something to contribute, and everybody has something to learn.

I have discovered over the years that my own website has been quoted by newspapers, used in classrooms, footnoted in PhD theses, even been the subject of Sunday sermons! And yet it contains typos, grammatical errors, has had wrong dates, and the like. Fortunately many of those who spot these errors will set me straight with an email, I can correct it accordingly, and life goes on. Some are so-called "experts" (many of you among them) and some are unknown passers-by. Some ask questions which tell me I have not stated my case clearly and need to revise a passage or reword a phrase.

Frankly I was astonished at the suggestion of censorship - and was beginning to wonder about paranoia levels. Then I realized that I had not stated my idea clearly.

My thought here was for something simple, certainly not the Harvard Law Review. *Any* member of ET could volunteer to proof-read articles being submitted to Phil: so-called "newbies" and "experts" alike. No need for special credentials, other than an interest in the subject at hand, and the willingness to proof-read, do some fact-checking, ask for clarification on certain points, offer suggestions about how to make a point clearer - or even ask what the purpose behind the article is in the first place (is it intended to be humorous? an op-ed? a serious piece of academia?). Anybody who has ever written anything for publication knows how helpful it is to get an extra set of eyes (or better, several) to go over it. This is why publishing houses have editors and fact-checkers.

But in all honesty, and if I were Phil, I would be extremely hesitant to have *only* the so-called experts performing this service. Or to even establish a review "board." The process must be seen to be democratic, transparent and be open to anybody. A "board" of "experts" will run the risk of creating the impression that this is a members-only club, and that could quickly spell the end of ET.

Some consideration should be given to what topics are to be proof-read by whom. To use myself as an example, I would not want to be in the position of editing any articles on the Californian controversy. Not that I consider myself an expert, far from it! But I have my own website on that subject, and my taking part in editing another's work on this could potentially be an invitation to accusations of conflict of interest or bias, and put Phil in awkward position. (And I'd rather see someone new to the topic delve into it anyway - it's a great way to learn.)

I didn't see a need to make this too formal of a process. Nobody is applying for a PhD here; nor are we curing leukemia or hepatitis. I was thinking Phil could state something like this:

"I have a received an article for publication. The subject is 'rivets' or 'evacuation procedures' or 'passenger ABC.' The author will remain anonymous until the article is published. I need three or four persons who are willing to proof-read it, critique it, check facts, and are able to return it with their comments by date xxx. When it is published, both the author's and the editors' identities will be made known." That last suggestion hopefully ensures a clean process and makes the process visible to all.

It also puts some responsibility on the proof-readers to read critically. Proof-readers could be helped, perhaps given a set of guidelines or questions to answer, such as:

* Is the intent of the article clear?
* Are facts clearly distinguishable from the author's opinion?
* Are the stated facts easily found, proven, known and knowable?
* Is the math correct? Are the dates accurate? Are sources cited so that others can look up the same information?

Let's not make this more difficult than it has to be. Certainly there's no need to burden Phil with stringent logistics. Nobody is winning prize money and it's not a competition. It's an attempt to ensure accurate and clear communications. It should be fun, informative, open to all and hopefully involve more than the crowd of "usual suspects" (written with tongue-in-cheek).

Looking forward to others' thoughts.

David Billnitzer
 

Tad G. Fitch

Member
Dec 13, 1999
581
14
263
Hello all, how are you? Good I hope. One quick note. While it is apparent that there is a wide-range of opinions here and we have all made it clear why we feel the way we do, please let's keep it civil. No need for hard feelings just because we disagree. It's not worth it and I know that we're all more than capable of having a civil discussion.

Moving on. It does seem as if many people are hung up on the term peer review, perhaps "editorial board" would be a more accurate and less confusing way of putting it. As Dave said, we aren't looking for the Harvard Law Review here, just something to help improve the quality of the articles being published here even more.

Like it or not, or whether it has been intended to be or not, ET is an often quoted resource for enthusiasts and students alike in both research and school work (with there being no way at all for them to know that there are factual errors present), and as such, it should be a goal to make the material as of high of quality and accuracy as possible. No, we cannot make it perfect, but we can make this already excellent website an even better resource, and keep intentionally misleading, or as many silly mistakes from sneaking through as possible.

And before anybody jumps on me for that comment, let me say this: We are all guilty of letting factual mistakes creep into our work from time to time, I have done it many times over myself. Others looking it over is the best way to prevent this, and has helped me out in the past as well. After finishing research or an article, I always send it off to several of my peers for them to check it over before submitting it for publication.

Dave quote:
"* Is the intent of the article clear?
* Are facts clearly distinguishable from the author's opinion?
* Are the stated facts easily found, proven, known and knowable?
* Is the math correct? Are the dates accurate? Are sources cited so that others can look up the same information?"

I agree, I was thinking along similar lines. The goal is certainly not censorship, nor has anyone suggested that. However, I do feel that the process and goals of it should be outlined very thoroughly to make certain that it is as organized and transparent as possible and so no one can be accused of censorship.

Free exchange of ideas and deviating from accepted versions of history when research yields evidence that warrants it should absolutely be encouraged. Maybe I'm wrong and missed something, or maybe it was the misunderstanding about the term "peer review", but I don't recall anyone saying that a group such as this should weigh the validity or correctness of ideas or investigate all the methods of research, or anything like that. That is quite the opposite of what I am advocating.

Bill wrote:
"I see nothing wrong with a review, by knowledgable people, of new articles. Not to say "That's a stupid idea", but to catch small mistakes (we all make 'em), check for clear explanations and such. I've done that on the few things I've written, and have been asked for my 2 cents on other's articles."

Absolutely. That is exactly what I am thinking, and pretty much sums up my thoughts on the matter in far less words than I have been using. John Kerry gets accused of explaining things too much, making things less clear and buried under too many words, and sometimes I share his affliction in discussions such as these. I apologize if I was not clear enough when stating my ideas on the matter.

Dave quote:
"Let's not make this more difficult than it has to be. Certainly there's no need to burden Phil with stringent logistics. Nobody is winning prize money and it's not a competition. It's an attempt to ensure accurate and clear communications. It should be fun, informative, open to all and hopefully involve more than the crowd of "usual suspects" (written with tongue-in-cheek)."

I agree. The process, once defined and agreed upon, should be fairly simple, transparent and straightforward. The way I see it, if done right, everybody wins and nobody loses. Regardless of how good the material already is on this website, it can be made even better and we should always strive for that, particularly with the research articles which are the crown jewel of this great website. I don't believe that simply having a link to a message board to discuss the articles is going to keep a person from believing the information in the article is accurate and being mislead by it or citing inaccurate information, and people such as this may not even know or want to take the time wading through threads on an article, trying to pick out what is reliable and what isn't. That's just my honest opinion about it.

I am looking forward to others' thoughts as well. I hope that this note finds everyone doing well and that all of you have a great day tomorrow.
Kind regards,
Tad Fitch
 
Jul 9, 2000
58,666
881
563
Easley South Carolina
If people are "hung up" on the term "Peer Review" it's because that was the term first mooted. I would also point to the title of this thread which is conspicuously "Peer review suggestions for ET Research Articles " It's the first thing anyone sees coming into this thread. To paraphrase at least one talking head and any number of politicians, "Words have meaning." It shouldn't come as a shock to anyone that when a given term is used that people will tend to take it at face value and respond accordingly. Previously, I asked the following questions;
quote:

1)who do we propose we empanal to make the judgements?
2)Would those people be willing to suspend and put aside any personal biases?
3)Would they have the Time?
4)Would they be willing to take the job?
5)Would they be willing to take the heat?
6)Would said panal be acceptable to all parties concerned?
I would hope that any proposal put forward would answer these specific questions but I've a sense that it's going to be a lot easier said then done and that's at the very core of my concerns.

Understand that I do know something of the peer review process and understand the need for it in scientific circles and academia. It's a neseccery tool but far from perfect. The catch however is that this is not an acedemic forum, and the problems and issues raised by David and Parks are well founded. Especially the spectre of censorship when an article seeks to offer up some unpopular ideas or which dare to think outside the box.

Can you say "Californian?" Can you say "Dr. Ballard?" Can you say "Salvage?" It doesn't take a rocket scientist to see where this can go in light of the often ugly divisions which have existed over those subjects for years. Not just here either but in the Titanic Community at large.

>>Why must you always be so condescending and negative? <<

Why do my questions posted above go unanswered? The problems are real and don't go away. If we're going to have a public discussion over this, then we need *specifics* over what's being discussed as opposed to broadly general statements. You'll notice in my questions that the first question I asked was "Who do we propose we empanal to make the judgements?" Don't take this question lightly as it's a lot more important then some may realize at first blush.

Yes, we have an impressive array of expertise on this forum, No doubt about that. But then all other issues aside, who's going to have that kind of time? The same people who would be best qualified to deal with passenger/crew articles may not be so swift dealing with technical forensics matters and some of the resident rivet counters would be in over their heads dealing with "people" oriented material. Don't take this as a slam because it isn't. It merely reflects the reality the different people take interest in different areas of study at the expense of other areas.

And, these people have lives outside of Titanica and ocean liners as well, which is why I wonder if the best candidates for any sort of editorial review board, or peer review panal would even have the time. Especially when they have to hold down regular jobs to earn the book money and their daily bread.

That's why I'm inclined to acknowladge the idea as great in principle but difficult in execution. We would want it to be easy, but the reality will likely be something very different. Let's not soft pedal that or simply dismiss that.​
 

Paul Rogers

Member
Jun 1, 2000
1,244
14
313
58
West Sussex, UK
I can see merit in the idea of an "editorial board". As an impartial observer, and as someone with no plans to carry out research, I thought I'd try to answer Michael's questions to check my own (and others?) understanding of what is being proposed.

1)who do we propose we empanal to make the judgements?

Volunteers, requested by Phil Hind (or whoever else is tasked to request them). Of course, there would be no "judgements" as such; see below.

2)Would those people be willing to suspend and put aside any personal biases?

There would be no need to put aside personal biases, as only the Hard Facts / sources / etc. would be checked, but not the theories, Soft Facts, opinions, etc. As Bill said, the purpose of the editorial board would be merely to: "...catch small mistakes (we all make 'em), check for clear explanations and such."

3)Would they have the Time?
4)Would they be willing to take the job?


As volunteers would be requested, the only way to answer these questions is to request 'em! I'd suggest a trial period with, say, the next 3 articles published, to see how many volunteers are forthcoming, or to see if the "usual suspects" are always asked to undertake the task (if this is seen as undesirable).

5)Would they be willing to take the heat?
6)Would said panal be acceptable to all parties concerned?


The questions would not be relevant to an editorial board. There would be no "heat" as nothing would be changed or censored. And therefore the panel (of volunteers) would be perfectly acceptable to all concerned.

Have I got the gist, gentlemen?
 

Lee Gilliland

Member
Feb 14, 2003
511
13
183
May I suggest that any person submitting something for peer review be allowed a list of potential reviewers, and that he/she be allowed to pick three? This would block potential difficulties along the lines of personal bias without derailing the review process.
 

Tad G. Fitch

Member
Dec 13, 1999
581
14
263
Paul, I think that you do have the overall gist of what has been discussed. No one is insisting on blocking original ideas or things that deviate from the accepted history, or questioning theories or methods of research. Bill put it best when he described it as being a process to check for and lesson the number of factual mistakes on well-known information (or "hard facts") such as dates, number of passengers aboard, things of that sort and also to make suggestions (which the author would have the option to accept the ideas or not) of ways to make the explanations clearer, etc. This alone would greatly increase the quality of the articles on ET.

Lee wrote:
"May I suggest that any person submitting something for peer review be allowed a list of potential reviewers, and that he/she be allowed to pick three? This would block potential difficulties along the lines of personal bias without derailing the review process."

That's not a bad idea in principle, but remember, our proposal has been to keep the identity of the author unknown to everyone (including the reviewers) except Phil Hind until after their suggestions are forwarded to Phil. This would be done for the same reason as your suggestion, to ensure personal bias does not leak in, and to do so without derailing the review process.

Hope all of you are well.
Regards,
Tad
 
Status
Not open for further replies.

Similar threads

Similar threads