Submited on: 08 Apr 2011 11:22:45 PM GMT
Published on: 09 Apr 2011 06:51:31 AM GMT
 

1 Is the subject of the article within the scope of the subject category? Yes
2 Are the interpretations / conclusions sound and justified by the data? Partly
3 Is this a new and original contribution? Yes
4 Does this paper exemplify an awareness of other research on the topic? Yes
5 Are structure and length satisfactory? Yes
6 Can you suggest brief additions or amendments or an introductory statement that will increase the value of this paper for an international audience? Yes
7 Can you suggest any reductions in the paper, or deletions of parts? Yes
8 Is the quality of the diction satisfactory? Yes
9 Are the illustrations and tables necessary and acceptable? No
10 Are the references adequate and are they all necessary? No
11 Are the keywords and abstract or summary informative? Yes
  • Other Comments:

    Since my last comment on this manuscript, I have analyzed the outcome of peer-review in WebMed Central. Articles posted in WMC in the following categories were selected in the snapshot: General Medicine, General Surgery, Infectious Diseases, Microbiology, Ophthalmology, Pulmonary Medicine, Surgery, and Vascular Surgery. These 8 specialties contributed to more than 50 articles between them. There were 64 reviews (including a few comments) of the 50 articles. Twenty reviews were not considered rigorous because they only tended to applaud the authors without critically reviewing the manuscript. These reviews could not have elicited any revisions or replies from the authors. The remaining 44 made at least some significant comments. Of these, 18 were posted within the last 4 weeks and hence it could be argued that sufficient time has not yet been provided to the authors in order to effectively respond. Thus, 26 reviews were included in the final analysis. No manuscript was revised as a result of the reviews and not one author cared to respond to the 26 comments. Mr Mahawar says that it is upto him to decide how best to use his time. His view is obviously shared by several people.

    The above data drive home the point that from what we have seen so far, authors feel no obligation to respond to the reviewers in this post-publication review system offered by WMC.

     

    Thus the following must be considered:

    1. In the current and more prevalent system of pre-publication review, the authors at least have an option to submit their manuscripts elsewhere if the reviewers decline their work. But in this post-publication review system, the reviewers can do very little. Their comments are simply ignored.
    2. The WMC team claim to have delivered an author-driven system. However, this seems to have transformed into an author-dominated system. What was supposed to change was the mechanism of scientific communication. But in reality, what has changed is ethos.  

    In the light of my above work, I request Mr Mahawar to either provide me with the evidence in a timely manner or humbly make changes to his work. In the interest of brevity, my two main criticisms are:

    1. Mr Mahawar’s contention that there is enough room currently for poorly constructed articles to be published does not tie up with his subsequent comments wherein he hints that radical authors are unable to publish their findings as a result of author-fatigue. I ask once again for evidence. If no evidence is available, he should be balance his views accordingly. Again, this could have occured 50 years back when publications must have been a painful process but there wasn't much scope to publish poor articles either.
    2. Mr Mahawar should reconsider if time is really the issue given his own reluctance to deliver a timely response and given what I have quoted above in relation to the articles on WMC. I find it alarming that manuscripts with no checks and those with checks that are simply ignored could later be quoted as science. Once again, Mr Mahawar would probably say that this already happens but that does not answer my question. That just deflects the issue.  
    On the positive side, 44 out of 64 reviews made significant comments to some extent and this is a good sign for a new idea although they were significant only as much as the quality of articles could allow them to be.
  • Competing interests:
    As previously mentioned
  • Invited by the author to review this article? :
    No
  • Have you previously published on this or a similar topic?:
    No
  • References:
    None
  • Experience and credentials in the specific area of science:

    None

  • How to cite:  Bal A .Outcome of reviews in WMC in light of Mr Mahawar's previous response[Review of the article 'Role of Peer Review in Biomedical Publishing ' by Mahawar K].WebmedCentral 2011;2(6):WMCRW00821
1 2 3 4 5 6 7 8 9
Report abuse
 
The manuscript needs revisions
Posted by Dr. Abhijit Bal on 27 May 2011 09:01:02 AM GMT

1 Is the subject of the article within the scope of the subject category? Yes
2 Are the interpretations / conclusions sound and justified by the data? Partly
3 Is this a new and original contribution? Yes
4 Does this paper exemplify an awareness of other research on the topic? Yes
5 Are structure and length satisfactory? Yes
6 Can you suggest brief additions or amendments or an introductory statement that will increase the value of this paper for an international audience? No
7 Can you suggest any reductions in the paper, or deletions of parts? Yes
8 Is the quality of the diction satisfactory? Yes
9 Are the illustrations and tables necessary and acceptable? No
10 Are the references adequate and are they all necessary? No
11 Are the keywords and abstract or summary informative? Yes
  • Other Comments:

    There are no references, illustrations or tables.
    As the author has not found it necessary to respond to my comments posted several weeks ago, I have now reviewed the manuscript more systematically. I hope that the author would find my input stimulating this time round.

    1. In point no. 1.1, the author states that too many low-quality articles are being published anyway because there are too many journals. Later in point no. 1.4, the author states without citing any evidence that far too many deserving people are not allowed to publish in the existing system. Further, in point no. 1.7 the author states that a proportion of research is lost due to the current process. Certainly, the unpublished and lost articles referred to in points 1.4 and 1.7 are high-quality in the context of the narrative. But if low-quality articles can also be published anyway as stated in 1.1, how does the author justify his comments under points 1.4 and 1.7? Why would high-quality articles not get published?

    2. In point no. 1.2, the author states that in the current system of pre-publication reviews, there is no incentive to engage in any post-publication peer review. If there is such an incentive in post-publication peer review, it seems that the author has not found the incentive strong enough. An analysis of this article's metrics shows that the last review/comment was made on the 16th of April for the manuscript published on the 8th of April. Forty days from the last review/comment, the author has not found it necessary to respond to the criticisms of the readers. I hope that lack of time is the reason for the delay but if the same time was lost in pre-publication review, my feeling is that the author would have concluded that the pre-publication reviews lead to unnecessary delays. One could argue that in this case, misleading statements have been made because of lack of pre-publication reviews. The lack of response also shows how little researchers would care when post-publication review becomes the norm as most people quite rightly are inclined to move vertically rather than horizontally. A pre-publication review process would have forced the author to respond with "blood and sweat" and converted a rhetoric piece into a scientific manuscript. This is a harmless manuscript but if manuscripts with significant impact on healthcare that make misleading arguments remain published on-line either without peer reveiw or if authors of such works do not feel obliged to respond timely, there could be potential harm.

    3. In point no. 1.3, the author states that peer-reviewed articles do not necessarily provide the correct information. This is right but the readers assign a value to the articles based on where they get published (e.g. impact factor). These are of course rough estimates . A general blog such as WMC does not necessarily allow this judgment to be made.

    4. In point 1.12, the author states that peer-review is time-consuming but surely this applies to both, pre and post reviews. Also, if post-publication reviewers can be compensated, why cant pre-publication reviewers be similarly compensated?

    5. Point no. 1.13 deals with the area of mistrust towards researchers. In this context, the author has formed an argument that researchers know more. I want to know if this argument would hold ground in relation to, say, double-blind clinical trials. In double-blind trials, do we start from a position of mistrust towards the investigators or is removing the subjective bias the objective? I believe that the the author has not asked the right question in this context. My above points show that researchers, like the author in this case, although they know more about the topic, like the author does in this case, do get subjectively influenced and do not necessarily ask the right question. They leave significant holes in their manuscripts. "Who knows more?" is not the point. The emphasis is on positionality.

    6. Point no. 5 deals with variations in peer review wherein the author mentions that what is rejected by one is accepted by others. Once again, does that not contradict points 1.4 and 1.7?

    7. Finally, one expects authors to respond in a timely manner and certainly so if articles are highlighted as featured articles (although I could not find what the basis is for including manuscripts in this category).

  • Competing interests:
    The author is a close friend of mine. The topic has been informally discussed. My current review was shown to the author although some changes have been made by me subsequently.
  • Invited by the author to review this article? :
    No
  • Have you previously published on this or a similar topic?:
    No
  • References:
    None
  • Experience and credentials in the specific area of science:

    None

  • How to cite:  Bal A .The manuscript needs revisions[Review of the article 'Role of Peer Review in Biomedical Publishing ' by Mahawar K].WebmedCentral 2011;2(5):WMCRW00781
1 2 3 4 5 6 7 8 9
Report abuse
 
Excellent overview of a significant problem
Posted by Dr. Devon D Brewer on 11 Apr 2011 05:15:33 AM GMT

1 Is the subject of the article within the scope of the subject category? Yes
2 Are the interpretations / conclusions sound and justified by the data? Yes
3 Is this a new and original contribution? Yes
4 Does this paper exemplify an awareness of other research on the topic? Yes
5 Are structure and length satisfactory? Yes
6 Can you suggest brief additions or amendments or an introductory statement that will increase the value of this paper for an international audience? No
7 Can you suggest any reductions in the paper, or deletions of parts? No
8 Is the quality of the diction satisfactory? Yes
9 Are the illustrations and tables necessary and acceptable? Yes
10 Are the references adequate and are they all necessary? Yes
11 Are the keywords and abstract or summary informative? Yes
  • Other Comments:

    The author has done the scientific community a great service by founding WebmedCentral as well as by writing this article. The author touches on many of the pitfalls of traditional pre-publication peer review and highlights many of the advantages of post-publication peer review.

     

    Pre-publication peer review is actually an anti-scientific practice – it impedes the accumulation of scientific knowledge by delaying and frequently censoring what gets recorded in the scientific literature. Pre-publication peer review is often used as a weapon to squelch the work of other researchers who have competing perspectives. The history of science is littered with examples of this tragedy. In fact, this problem may be more common now than before, as the political and commercial consequences of research have never been greater. This is particularly the case when research is funded by governments (also commonly determined in part through peer review before the proposed research has been done) and corporations.

     

    It is important to note that most scientific development and exchange occurs without the constraints of peer review before such development and exchange occur. That is, compared to the pre-publication peer review model, most scientific communication within labs, institutes, departments, other institutions, and informal contexts is characterized by the absence of such prior screening, approval, and censorship. Similarly, seminars, colloquia, and often conferences (especially smaller ones) do not involve pre-exchange peer review. And it was precisely in such settings during which science, as a social process, itself first developed and matured in the 1700s and 1800s. So it is really pre-publication peer review that is the abnormality within the broader historical and sociological frame.

     

    Critics of WebmedCentral specifically and post-publication peer review in general rightly note that some manuscripts published in this system are superficially inferior to most in the traditional, pre-publication peer reviewed literature. That is, some manuscripts published before peer review have not been edited, or edited thoroughly. But the quality of the scientific content in manuscripts published before peer review, in my opinion, seems no different on average from that in the traditional published scientific literature as a whole (including journals of all types and levels of prestige).

     

    The author does not address the fact that most authors are very reluctant to abandon pre-publication peer review for post-publication peer review (although the author has investigated this topic empirically elsewhere – Mahawar et al., 2009, Asian J Surg). The author does mention some of the forces that keep authors invested in the pre-publication review system. It seems that authors' reluctance is perhaps the biggest barrier to the growth of the post-publication review model. So it will be very interesting to see how scientific authors respond to the new publication opportunities in the coming years.

  • Competing interests:
    none
  • Invited by the author to review this article? :
    No
  • Have you previously published on this or a similar topic?:
    No
  • References:
    None
  • Experience and credentials in the specific area of science:

    I have served as an author, peer reviewer (both pre- and post-publication), and editor for over 20 years in the health and social sciences.

  • How to cite:  Brewer D D.Excellent overview of a significant problem[Review of the article 'Role of Peer Review in Biomedical Publishing ' by Mahawar K].WebmedCentral 2011;2(4):WMCRW00657
1 2 3 4 5 6 7 8 9
Report abuse