Skip to content

Instantly share code, notes, and snippets.

@PhilGeek
Created August 7, 2009 19:37

Revisions

  1. PhilGeek revised this gist Aug 7, 2009. 1 changed file with 44 additions and 7 deletions.
    51 changes: 44 additions & 7 deletions Tim Crane on refereeing
    Original file line number Diff line number Diff line change
    @@ -1,13 +1,50 @@
    I've been following some of the internet discussions about people's complaints about journals. The recent thread on Brian Leiter's blog (http://bit.ly/dvZuC) prompted a few thoughts. Tried to post them on the blog but for some reason it was rejected. I didnt wan't to let this bit of structured procrastination (http://bit.ly/LG7Z) go to waste, so ...
    I've been following some of the internet discussions about people's complaints
    about journals. The recent thread on Brian Leiter's blog (http://bit.ly/dvZuC)
    prompted a few thoughts. Tried to post them on the blog but for some reason it
    was rejected. I didnt wan't to let this bit of structured procrastination
    (http://bit.ly/LG7Z) go to waste, so ...

    The Tasang & Frey paper which Chris Bertram links to (http://www.iew.unizh.ch/wp/iewwp280.pdf) is very interesting and shows that whatever problems there are with peer review are not unique to our discipline. The 'accept as is' (i.e. no 'revise and re-submit') proposal in this paper is an interesting line, and like the other proposals mentioned here should probably be tested by some journal or other to see how it turns out.
    The Tasang & Frey paper which Chris Bertram links to
    (http://www.iew.unizh.ch/wp/iewwp280.pdf) is very interesting and shows that
    whatever problems there are with peer review are not unique to our discipline.
    The 'accept as is' (i.e. no 'revise and re-submit') proposal in this paper is
    an interesting line, and like the other proposals mentioned here should
    probably be tested by some journal or other to see how it turns out.

    Peer review is often held up as the gold standard for academic publication, but it's pretty obvious that the whole process has become deeply problematic. One thing which is rarely pointed out (though Tsang & Frey do) is that reviewers are often completely untrained for their task: speaking for myself, I had no idea what counted as a publishable paper when I first started reviewing. New reviewers may have completely unrealistic standards, and they are often inexperienced academics, since the experienced ones are too busy (or let's face it, too world-weary or too grand) to take on this job.
    Peer review is often held up as the gold standard for academic publication,
    but it's pretty obvious that the whole process has become deeply problematic.
    One thing which is rarely pointed out (though Tsang & Frey do) is that
    reviewers are often completely untrained for their task: speaking for myself,
    I had no idea what counted as a publishable paper when I first started
    reviewing. New reviewers may have completely unrealistic standards, and they
    are often inexperienced academics, since the experienced ones are too busy (or
    let's face it, too world-weary or too grand) to take on this job.

    'Training' for reviewers is obviously out of the question, of course. And given the current number of journals and submission habits, journals must continue to rely on experienced reviewers. But I think a step in the right direction would be to put more trust/faith in the editors: the editors need not present themselves as even-handedly taking on board everyone's opinion, as if they were all of equal value. Editors should not feel intimidated by the supposed gold standard of 'peer review' to pass the responsibility for evaluation onto the reviewers. They can dismiss reviewer's comments just as they can dismiss a paper. If you don't like editor X's decisions then there are plenty more journals out there.
    'Training' for reviewers is obviously out of the question, of course. And
    given the current number of journals and submission habits, journals must
    continue to rely on experienced reviewers. But I think a step in the right
    direction would be to put more trust/faith in the editors: the editors need
    not present themselves as even-handedly taking on board everyone's opinion, as
    if they were all of equal value. Editors should not feel intimidated by the
    supposed gold standard of 'peer review' to pass the responsibility for
    evaluation onto the reviewers. They can dismiss reviewer's comments just as
    they can dismiss a paper. If you don't like editor X's decisions then there
    are plenty more journals out there.

    This raises another issue. A number of editors I know have commented that it is difficult to find people to review papers. Surely this has something to do with the increasing number of papers which are submitted, and the still increasing number of journals. I'm sure that the recent decisions to have a moratorium on submissions to various philosophy journals were not taken lightly, but in conditions of considerable pressure.
    This raises another issue. A number of editors I know have commented that it
    is difficult to find people to review papers. Surely this has something to do
    with the increasing number of papers which are submitted, and the still
    increasing number of journals. I'm sure that the recent decisions to have a
    moratorium on submissions to various philosophy journals were not taken
    lightly, but in conditions of considerable pressure.

    Another radical suggestion which philosophy journals might want to consider, in order to cut the number of submissions, is to charge people a fee for submitting a paper. This might discourage people from submitting papers which (even in their heart of hearts they themselves might admit...) are still not quite finished, or not quite ready yet. (Of course, the journal would have to make sure that the money did not go to the rapacious publishers and was spent on making the journal more efficient.)
    Another radical suggestion which philosophy journals might want to consider,
    in order to cut the number of submissions, is to charge people a fee for
    submitting a paper. This might discourage people from submitting papers which
    (even in their heart of hearts they themselves might admit...) are still not
    quite finished, or not quite ready yet. (Of course, the journal would have to
    make sure that the money did not go to the rapacious publishers and was spent
    on making the journal more efficient.)

    I don't imagine this would be very popular with the contributors to the thread on Brian's blog but I thought I would mention it anyway.
    I don't imagine this would be very popular with the contributors to the thread
    on Brian's blog but I thought I would mention it anyway.
  2. PhilGeek created this gist Aug 7, 2009.
    13 changes: 13 additions & 0 deletions Tim Crane on refereeing
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,13 @@
    I've been following some of the internet discussions about people's complaints about journals. The recent thread on Brian Leiter's blog (http://bit.ly/dvZuC) prompted a few thoughts. Tried to post them on the blog but for some reason it was rejected. I didnt wan't to let this bit of structured procrastination (http://bit.ly/LG7Z) go to waste, so ...

    The Tasang & Frey paper which Chris Bertram links to (http://www.iew.unizh.ch/wp/iewwp280.pdf) is very interesting and shows that whatever problems there are with peer review are not unique to our discipline. The 'accept as is' (i.e. no 'revise and re-submit') proposal in this paper is an interesting line, and like the other proposals mentioned here should probably be tested by some journal or other to see how it turns out.

    Peer review is often held up as the gold standard for academic publication, but it's pretty obvious that the whole process has become deeply problematic. One thing which is rarely pointed out (though Tsang & Frey do) is that reviewers are often completely untrained for their task: speaking for myself, I had no idea what counted as a publishable paper when I first started reviewing. New reviewers may have completely unrealistic standards, and they are often inexperienced academics, since the experienced ones are too busy (or let's face it, too world-weary or too grand) to take on this job.

    'Training' for reviewers is obviously out of the question, of course. And given the current number of journals and submission habits, journals must continue to rely on experienced reviewers. But I think a step in the right direction would be to put more trust/faith in the editors: the editors need not present themselves as even-handedly taking on board everyone's opinion, as if they were all of equal value. Editors should not feel intimidated by the supposed gold standard of 'peer review' to pass the responsibility for evaluation onto the reviewers. They can dismiss reviewer's comments just as they can dismiss a paper. If you don't like editor X's decisions then there are plenty more journals out there.

    This raises another issue. A number of editors I know have commented that it is difficult to find people to review papers. Surely this has something to do with the increasing number of papers which are submitted, and the still increasing number of journals. I'm sure that the recent decisions to have a moratorium on submissions to various philosophy journals were not taken lightly, but in conditions of considerable pressure.

    Another radical suggestion which philosophy journals might want to consider, in order to cut the number of submissions, is to charge people a fee for submitting a paper. This might discourage people from submitting papers which (even in their heart of hearts they themselves might admit...) are still not quite finished, or not quite ready yet. (Of course, the journal would have to make sure that the money did not go to the rapacious publishers and was spent on making the journal more efficient.)

    I don't imagine this would be very popular with the contributors to the thread on Brian's blog but I thought I would mention it anyway.