Popular Ludology: Clarifying the Peer Review System

I seem to have not expressed the peer review process we’ll be using well enough as some people seem to not understand. This must be my fault so I’ll clarify with reference to the previous post, and I’ll tighten up the language more later on for reference.

When you submit a paper it will be examined by the editorial staff and those who have previously submitted and had their papers accepted. It will be accepted or rejected on the basis of a simple majority (with 50% being a pass). Either way you should be informed.

This is not part of the review process. You could call this a simple ‘smell test’. Does this paper look/sound/smell like bullshit? Do the person’s credentials check out? The vote should only be necessary if there’s any strong objections to a particular process. This is not really any different to a single paragraph dissertation, written in crayon on toilet paper being rejected. Just more formalised.

If your paper is accepted you will be invited to the Popular Ludology email group. You do not have to accept and participation is not mandatory, but it will allow you to participate further as the journal and effort – hopefully – expand.

This group will form a democratic/meritocratic basis for organisational/journal level change in the future. That is its primary aim, to provide a pool of qualified people to vote on procedure etc.

A month will be given for papers to be accepted.

And please do submit. While the ‘theme’ for issue zero is defining and classifying games, any submissions on any game related topic are welcome. I’ve seen and read some interesting things from less conventional scholars and developers over the last ten months and would love to see some of their work more formally published.

At the end of that period the accepted papers will be collated and published in an ‘alpha draft’.

This is the point at which the actual review process starts.

This alpha draft will be made available publicly to anyone and everyone for open review, criticism and objection.

This is the peer review part.

We’re going with an open peer review for several reasons.

  1. It encourages participation.
  2. It encourages non-academic participation.
  3. It reflects a commitment to openness.
  4. There are many existing criticisms of the blind review process (not least that in the digital age it’s hard to keep).
  5. It’s potentially much, much more rigorous.
  6. It allows the authors to directly participate in the process and with their critics.

You should monitor this feedback and, as you feel may be necessary, make changes, clarifications and extensions to your paper over the following month.

Hopefully writers of papers will examine the feedback that they get and make amendments and improvements accordingly.

The final version of the journal (with any amendments, additions, retractions etc from review) will then be published. Papers may only be forcibly withdrawn against your will if 75% or more of the editorial and previously published authors agree in a vote.

This is where, I think, the confusion arises. This is intended more to be a meta-review process. Examining the criticism and seeing whether it is valid and then acting as a qualified group to remove papers that do not hold up, if the author cannot or will not do so themselves.

Voting procedures will be made public in the journal itself.

It should also be noted that we are aiming for a more rigorous scrutiny than currently exists within organisations like DiGRA, and which appears to have let through many papers and presentations that do not seem to hold up to basic standards. It’s also well established, but denied, that there is a great deal of hostility towards the concept of peer review in some of these groups and that what passes for peer review in the humanities is not at all what most people consider that term or procedure to mean.

I’m sure there’ll be lumps and bumps along the way and that idealism will have to give way to pragmatism at various points, but there’s no harm in aiming high from the start.

A great number of objections received so far seem to be based around the idea that this will be rejected by the existing academic structure or that it needs to be changed to be more in line with existing journals and organisations. Given that a central premise behind the setting up of this journal and in seeing a need for it is that there are severe problems with game studies and game studies groups and structures it would seem to be counter-productive to replicate those same issues for unneeded approval, while trying to fix their problems.

The aim here is not to replicate the efforts of DiGRA etc, but to do something different and useful and while I’m sure material good enough for academia will be produced, their approval is not especially wanted. As a pragmatic and practical resource, developer and designer approval and interest is much, much more important.