Submited on: 08 Apr 2011 11:22:45 PM GMT
Published on: 09 Apr 2011 06:51:31 AM GMT

The author’s letter is interesting and timey but the views expressed are exaggerated and disproportionate. Peer reviewers are not a bunch of know-nothing individuals who rule over the destiny of authors. In fact, most peer reviewers are authors themselves.

 

In all walks of life, there are certain procedural requirements without which an organized life cannot exist. A group of individuals in the interview committee decide on the fate of the candidates, who as a result of a 30 minute interview that convinces 5-6 people, may well hold on to a job for their life. Examinations at all levels also do not take a majority view of the candidates but instead 2 or 3 examiners sum up the years of hard work put by the candidates. No doubt there are these holes in the processes but no easy alternative is available either. A judge’s decision on the fate of a convict, a jury’s view, and decisions taken by a democratic parliament are thus all arbitrary decisions. These are not identical situations but they drive home the point that procedural requirements should not be done away with without some thinking.

 

I find some questions clearly naïve e.g. “Who is likely to know more about a research topic? A researcher who has spent months or years working on it or a reviewer who has no more than a few hours at the very most to decide its fate?”. But that is the point. Reviewers are expected to pick up on the authors’ subjective influence on the manuscript. The author states that “[Pre-publication review,] far too often does not let the good deserving people in”. I find it hard to believe that a high-quality manuscript would be rejected by every single top quality journal and actually ends up not being published. This view borders on paranoia. Of course, it might have happened sometime but how common is it? Is it really “far too often”? The author states that [pre-publication] peer-review is arbitrary. But so is post-publication review and so is the rating system devised in this website. The only exception is that everyone is entitled to his or her arbitrary views and the hope is that the sum total of all arbitrary views would translate into an accurate reading in the end. That might be fair to all from the author’s perspective but is the right result therefore obtained?   

 

Post-publication review has many pit-falls and certainly so if it replaces the existing system completely. These drawbacks may have been underestimated. Reviewers may not be critical of their colleagues in an open forum. Reviewers may not feel the job rewarding considering that authors are not obliged to take note of their views. If the authors do take note of their views, multiple versions would appear making things even more confusing. Authors often work hard in order to pass the pre-publication peer-review process. Availability of publication freely might make them less diligent when preparing manuscripts. It is likely that many poor articles are never submitted because the process is unlikely to be time effective. In fact, it is obvious from this website that several low quality, poorly reviewed, and in some instances silly or perhaps even defamatory articles have appeared due to abdication of any editorial responsibility by the webmedcentral team whose job is completely unclear, for they neither edit, nor review, but would probably end up charging authors precisely because no service is provided. Reviews can be easily engineered by unscrupulous means. The post-publication review system trusts the authors who have vested interest in generating a higher rating for their manuscripts but for some reason does not trust the pre-publication reviewers, who have no axe to grind.

 

The one drawback correctly mentioned is the time it takes in the process of pre-publication reviews. This must be looked into. The entire process of pre-publication review system needs to improve. It should be emphasized that it might also take a lot of time to sort out the inaccuracies in post-publication reviewed articles.

 

A hybrid system must evolve and is already available. At the very least, editorial review must take place in order to maintain technical robustness. But that would involve work, which the team do not wish to do.

 

However, there are no perfect answers to the questions raised by the author. Post-publication review might succeed and might become profitable. It remains to be seen whether it helps science.    


  • competing interests: The subject has been informally discussed with the author. The author is well known to me and this might have subjectively influenced my writing.
  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science:

    None

  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse
 
Stratified Levels of Publication
Posted by Dr. Abu Dzarr on 14 Apr 2011 06:23:11 AM GMT

Perhaps webmedcentral may want to consider a stratified level of publication. Manuscripts that achieved a certain standard in terms of content and methodology based on post publication reviews are upgraded to a level one publication (aka traditional journal) whereas others remained at level two (entry level).  This more or less mimic pre-publication review but done transparently.  There could be more than two levels.

We are already seeing proliferation of poorly prepared manuscripts in webmedcentral. I suppose there will come a time when serious researchers would not be happy sending in their manuscript if they see their work embedded among poor quality manuscript.  Researchers are trying their best to get their work published in high impact factor journals.  If webmedcentral do not find a mechanism whereby ISI or its like could gauge and quantify impact factors, these researchers may go elsewhere.  Hence quality articles should not get swamped by substandard ones.  If webmedcentral creat some form of stratification, then perhaps ISI could consider a mechanism of awarding impact factor to the various levels individually and this will become a pulling factor for quality articles.

The current popular votes are bias as the voting system is driven more by our curiosity.

Most of us are time constrained and do not have the leisure to pick out the needles in the haystack.  The stratification process also helps.

By having different publication levels, no one gets censured as all manuscript still get published.

  • competing interests: None
  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science:

    Teaching and Research

  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse
 

Very good paper based on reality and motivated for improvements of science and scientific publishing.

I believe that some pre publishing considerations or reviews of papers are necessary, at least as improvement of initial mansucripts, as when young contributors submit. Some comparisons of post publication peer review with pre publication of  initial reviewers can help. It is not necessary to publish names of pre- publication reviewers, but their conclusions about paper and final fate of same paper (citations, impacts, etc.) after certain period can be informative.  

  • competing interests: No
  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science:

    6 years Editor -in -Chief of Folia Anatomica journal (no more existing) and 6 years President -in -past of Yugoslav and following Serbian Anatomical Society

  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse
 
Response to Reviews and Comments
Posted by Mr. Kamal K Mahawar on 27 May 2011 10:21:31 PM GMT

First of all I would like to thank all the commentators and reviewers for taking the time to read my article and posting their views on our portal.

I would like to reassure Dr. Dzarr that WebmedCentral is working on ways to stratify articles and we are in the process of creating a team of respected scientists who would function as faculty. Invitations for our faculty position are in fact currently open. We are aware that platforms such as ours would attract some poorly designed manuscripts by immature authors who would be keen to exploit the freedom of expression. However such authors would only be putting themselves at a disadvantage as there would be little to gain from such a publication exercise. We believe the primary purpose of publication is scientific communication and those seeking to obtain some sort of a mileage out of seeing their names on the internet would not be gaining any favours with the academic community and would only bring disrepute to their credentials. Our publishing model would disadvantage and hence discourage authors from saying something for the sake of doing so.

Dr. Bal, who is also a very close of friend of mine raises several very valid questions and I attempt to satisfy his pertinent queries as below.

 

  1. I have not made such disgraceful remarks about peer reviewers and never will. Peer reviewers (and as Dr. Bal says most of them are themselves authors) are spending time for what can only be described as a scientific exercise and that too without any remuneration. In fact, our model of publishing on WebmedCentral relies totally on post publication peer review. I would urge Dr. Bal not to confuse pre publication peer review (an exercise I do not consider scientific and am against) with peer reviewers (which is basically us all at some point in time).
  2. Dr. Bal says, “procedural requirements should not be done away without some thinking”. I agree entirely. However I have a thought a lot about this and I genuinely believe that the procedure of pre publication peer review is not required for scientific communications in this century. I am not alone in holding this view as a simple google search would reveal and indeed parliament in UK would not be enquiring on this topic either if there wasn’t a reasonable doubt about its efficacy. I am sure Dr. Bal would be aware of the differences between scientific publishing and the analogies he cites which would any practical comparison meaningless.
  3. Dr. Bal misunderstands me when he thinks I am naive in saying who would know more about a manuscript. The point I am trying to make is that we do not need to have a filter before any scientific information comes in public domain. Reviewers typically spend a few hours reading a manuscript, which is usually much lesser than what an author spends writing it. Why then give more weight to the reviewer and let them decide the fate of author’s work? Same review can also be done post publication and all the valid comments/criticisms made. I am also not so naïve as not to understand the drawbacks of post publication peer reviews and scoring systems but the purpose of all this in my view is just to generate a discussion around the manuscript and not help us decide its quality and hence fate. That I believe is a very long term exercise, requiring reproduction and revalidation by generation of scientists. I however do have problems with an arbitrary pre publication peer review which claims to be the gate keeper of science as a gate keeper cannot be unfair and biased.
  4. Dr. Bal makes a just remark when he says that reviewers are likely to be less critical in an open system. My answer to that is, “yes, it is possible”. However I am yet to come across a serious scientist who is so mindful of his/her manners that he/she would not want to set a mistake right. Dr. Bal is a very close of friend of mine. This open forum did not prevent him from expressing his thoughts openly. Why should then a scientist, strong and fearless as we are, not want to express his/her honest views on a poor scientific manuscript? Yes, those who value relationships more than science would be discouraged but not the rest of us. “Transparency” in scientific communications is a good thing Dr. Bal. Truth is not harmful. It can at times be painful though.
  5. On WebmedCentral, only the latest version of the article will be shown avoiding any confusion. Links to all the versions are however kept to keep a scientific trail.
  6. Dr. Bal says that reviewers may not find their jobs rewarding without the power of pre publication peer review. Undoubtedly some reviewers will find it less rewarding if all manuscripts were published feely first. But there will undoubtedly be those who will express their opinions freely irrespective of this power.
  7. It is possible that many poor articles are never submitted in the current system because of the checks in the place but equally precisely because of the importance people place on the articles, these articles would ever be submitted. If there were no scientific gains to be had in publishing, why would one want to publish a poor quality article? Even if one would do so, what potential gains could be had? It would only harm the person’s credibility.
  8. Dr. Bal believes that there are several low quality articles on WebmedCentral. He needs to think of publication of as a means of scientific communication, a scientist wanting to talk to another. I believe that every article on our portal has a unique message to convey. There may be grammatical, formatting and other errors. But the message is still conveyed.
  9. Contrary to what Dr. Bal says, WebmedCentral does not even assume any editorial responsibility. It is clearly stated on our portal. Dr. Bal does not take the wider picture when he accuses us of abdicating our responsibility. Every act of editing and peer review comes with a cost which someone has to bear and which means that a large section of world’s population is not even a part of that academic debate. It is our effort to minimise such costs as much as possible.
  10. Dr. Bal argues how could low quality articles be published in current literature whilst some deserving scientists are denied access. My argument to that is simple. Pre publication peer review passes very arbitrary judgment which varies from journal to journal, editor to editor and reviewer to reviewer. If one was to keep trying, it would not be impossible to eventually find a “peer reviewed” journal that would publish all or part of the manuscript. A large number of manuscripts are published in second, third, or even fourth submission. However a serious scientist presenting a contrary view may not find easy audience amongst prominent journals. Such a scientist may not have time or even want to consider submitting his/her work to much lower impact journals just to get it published. A body of research is never published and hence lost.
  11. Dr. Bal criticises my delay in responding to his comments. I would argue that delays by a peer reviewer deciding the fate of a manuscript are significant as it delays the presentation of information to the rest of the scientific community and should not be compared to that by an author in responding to the comments on his manuscript. In the latter case, it is only the author who is being disadvantaged and it would be up to him/her to decide where to spend his/her time best. Nobody else except me loses out in this process if I do not defend my argument in a timely manner. But then it is surely up to me to decide what the best use of my time is. There could also be an argument made in favour of authors wanting to wait for all potential comments/criticism to appear before responding. In this case as well, if I had responded to Dr. Bal’s earlier comments, I would have had to come back to read his latest ones.
  12. Dr. Bal talks about potential harm from publication of non peer reviewed manuscripts. Dr. Bal should examine his own practice and think how and when he would let any article anywhere in the world change his way of treating his patient and he would find the answer. Very few clinicians would adopt a controversial or new medicine or treatment unless repeatedly verified in trial settings and all these trials have to be approved by ethical bodies as well. I would like to assure Dr. Bal that this would be no more harmful than publication with peer review when people want to publish for academic gains. In our model, there is so little to be gained by the act of publication alone that any scientist wanting to disseminate results not based on factual data or information would only stand to lose it all in lieu of nothing.
  13. Dr. Bal implies that readers should trust information provide in high impact factor more than others. He is probably not aware of the several high profile cases of erroneous, sometimes fraudulent, information published in journals with very high impact factor. I am not so far aware of any fraudulent or erroneous content on WebmedCentral, which Dr. Bal rather dismissively calls a blog.
  14. Time delays in pre publication peer review are more important than in post publication peer review, simply because in the former the fate of a manuscript and the work of a scientist hangs in balance. In the latter, it is just delaying a discussion. Argument has already been presented.
  15. I do not think that comparison of publishing processes with a double blinded trial is a fair one. While the former is an exercise of communication, the latter is a scientific exercise to find truth. Dr. Bal knows the difference between the two.
  16. Selecting an article as a featured article is entirely a decision of WebmedCentral managerial team currently. In near future, our faculty members will assume this responsibility.

Dr. Bal and other authors have been very kind to devote their time on my manuscript. I can only sincerely thank them for their effort. It is possible that I have not been able to answer all their queries, in which case please do not hesitate to ask me for further clarification. I intent to publish a well referenced series on this topic, time permitting, in future.

  • competing interests: As mentioned in the article
  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science: None
  • Publications in the same or a related area of science: Yes
  • References: None
 
Report abuse
 
Response to Mr Mahawar
Posted by Dr. Abhijit Bal on 28 May 2011 10:35:12 AM GMT
  1. I have not accused Mr Mahawar of making disgraceful remarks about peer-reviewers.
  2. Proponents of pre-publication peer-review do not claim that the process is ideal. The UK parliament sought to discuss the issue perhaps in order to reform the system. Mr Mahawar is only right in suggesting alternatives. I am only reviewing his paper.
  3. The question of who knows more is incorrect as that is not the point. If “who knows more” was the point, Mr Mahawar would not have written this response considering that I have had only a cursory interest in the subject while Mr Mahawar has a strong interest and is thus expected to know more. Indeed, if “who knows more” is the point, no kind of review should ever be done even post-publication.
  4. Indeed, I have the confidence in Mr Mahawar so as to be able to review his paper openly and critically. I am not sure if my attitude can therefore be generalized. We do know that author-suggested reviewers are more favourable than editor-suggested reviewers: (Bornmann L, Daniel HD. Do author-suggested reviewers rate submissions more favourably than editor-suggested reviewers? A study on atmospheric chemistry and physics PLoS One 2010; 5(10): e13345). Although author-suggested reviewers are not necessarily identifiable by the authors at the end of the review process (more than one reviewer, no way to ascertain if the same reviewers indeed accepted the offer to review), it does imply that there is a degree of uncertainty in this process. Open reviews may also attract only favourable reviews and Mr Mahawar agrees to this.
  5. It is reassuring to know that such processes have been put in place and they serve to minimize any confusion.
  6. See point 4 above.
  7. I don’t think people want to publish poor articles but they happen to improve on their science in the existing process. If the article was freely delivered to the scientific community, would authors strive to improve? I guess it is like an examination process which makes people work harder. Surely, some people would work hard irrespective of the processes in place. In the process that Mr Mahawar suggests, there is a scope to respond to reviewers and improve but as I have just shown, there is no obligation to respond.  
  8. Certainly, poor articles can be unique if in nothing else then in the lack of science. But they waste the reader’s time particularly prior to reviews. They also waste reviewers’ time in absence of an editorial input.
  9. I have in fact made the point that there is no editorial responsibility in WMC (which is what I meant by abdicating any responsibility). I am suggesting that while cost is important, so is quality and reader confidence.
  10. Mr Mahawar, do you have any evidence that happens regularly? Obviously we both are not at a level-playing field because there are no data in relation to your new concepts while pre-publication peer-review has been in existence for several decades. Surely, the onus is on you to be more specific. When has a top-quality manuscript not being published as a result of author fatigue? It might have happened but you say it happens “far too often”. I find this misleading. Give us the evidence. Let me also elaborate a bit. In relation to radical ideas, there might be a problem with the grant process which is not within the remit of your article. For example, if I need to investigate whether a shorter course of a drug for a given disease is as good as a longer well-established treatment course, there might be an issue in relation to funding (e.g. companies that market the drug may be reluctant to promote a study that might in the end reduce their profits). But if I can get through with the study somehow, I doubt if it would be impossible to publish. I can see a problem with any radical idea in the beginning of the study but not at the end and I certainly would not fear rejection after rejection as you imply although rejection as a result of a journal’s commercial interest can happen. But not rejection after rejection upto the point of author fatigue. Even if editors and reviewers do not publish, they would need to provide some explanation. 
  11. In fact I argue that the scientific community is at a significant disadvantage if critical reviews do not take place in a timely manner and if authors are not obliged to respond in a timely fashion. If scientists do not use the data pending thorough reviews and responses, “time” is out of the equation and one is only left with a manuscript whose fate and impact is uncertain. Mr Mahawar could also have been more polite when he reminds us of his right to use his time as per his needs. I believe that responses must be delivered quickly although revisions to the manuscript might well need to wait until such a time when no further responses are expected (as stated in the author instructions on this website). It is upto Mr Mahawar to honestly ask himself if he was due to respond anyway in the next few hours or days or he would have never done so. My review was posted only because he did not respond. This fact, the author has unfortunately twisted. Mr Mahawar, time is either important or it isn’t.    
  12. I think adopting a radically new therapy is an extreme example but we do make significant changes to our existing policies based on one article depending upon where it is published. In absence of this “evolutionary” signal which is not perfect, we would need strong independent reviews and quick author responses. The readers only look forward to independent, unbiased, critical and timely reviews and timely responses.
  13. I am well aware of these papers and the publishing world has responded to these acts of fraud. For example, processes are now put in place so that a manipulated image might be detected. Nobody has called the current process perfect and my first comment has referred to this fact. 
  14. As I have discussed above, while the manuscript is published instantly on websites such as the WMC, they would still need thorough and quick reviews and responses before uptake. Data suggest that the post-publication process is currently not adequate (Gøtzsche PC, Delamothe T, Godlee F, Lundh A. Adequacy of authors’ replies to criticism raised in electronic letters to the editor: cohort study. BMJ 2010; 341: c3926).
  15. If publication is merely an exercise in communication, I think I am right in using the term “blog” which the author finds dismissive (point 13). It is not just another exercise in communication. It is a part of the scientific process. The scientific communication should aspire to the same standard of science that it wishes to communicate.
  16. This last point has nothing to do with the review process. But Mr Mahawar, you place your own article on your own website as a “featured” article. You may have a commercial interest but you are here to serve the scientific community and not serve your own interest. Your ownership of or your influence on WMC gives you the power but not the right. You only have the right to define the criteria for placing an article as “featured” article. I could not see these criteria on your website.  

Finally, Mr. Mahawar needs to explain why hybrid systems are inadequate.

  • competing interests: Close friend of the author. Previous reviews and comments.
  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science:

    None

  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse
 
post-publication peer review
Posted by Dr. PRATIK PATEL on 10 Feb 2012 09:41:05 AM GMT

The article has shown the  great service by founding WebmedCentral as well as by writing this article. The author has covered many of the loopholes in traditional pre-publication peer review and has shown many of the advantages of post-publication peer review. Post-publication peer review has yet to be recognised in orthodox system of publication. 

  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science: None
  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse
 

I see the following major flaws in Webmedcentral's author-driven, post-publication review model:

1. There is no proper Editor to coordinate the reviews and to evaluate, arbitrate and, where applicable, reconcile and moderate conflicting reviewers' opinions and comments.

2. Reviewers can get biased or influenced by seeing other reviewers' scores and comments before submitting their own scores and reviews.

3. Author-chosen reviewers could be fair and objective, but sometimes, when they know they have been invited by the author (which is the case with Webmedcentral), they could also be unnecessarily lenient and complimentary in their reviews (to appease the author) or excessively harsh and unreasonably tough in their comments and scores (to affirm their detachment and distance themselves from the author, so that they are not accused of being biased).

My two cents...

  • Invited by the author to make a review on this article? :
  • Experience and credentials in the specific area of science: None
  • Publications in the same or a related area of science: No
  • References: None
 
Report abuse