研究方法研讨班管理哲学师资培训活动报名

The Developmental Reviewer

Paul S. Hempel
City University of Hong Kong, China
Reviews, and reviewers, seem to be a popular conversation topic among academ-ics. All of us seem to have horror stories to share about ‘idiot reviewers’ who
misunderstood our paper, and we complain about the number of reviews we areasked to write. Journal editors will speak of problems finding conscientious, timely,
and developmental reviewers. While the review process can be painful and timeconsuming, quality reviews are important for Management and Organization Review
(MOR) to succeed in our mission to promote quality research. In order to help you become a better reviewer, I will address three separate issues: why you should
become a reviewer; the responsibilities of a reviewer; and finally some specific guidelines on how to review.
REASONS FOR BEING A REVIEWER
High rejection rates combined with a large number of submissions place great demands on both editors and reviewers (Tsui & Holleneck, 2009). Reviewing
involves a lot of work, but is also an opportunity for early and mid-career scholars to learn and network. Academics on the mainland have told me that they primarily
review papers for editors with whom they already have strong ties, but what gets overlooked is that reviewing develops ties to editors and senior academics. These
ties will not automatically make it easier for you to publish, but it does make it easier to become involved in symposia, seminars, and special issues. Reviewing is
also a wonderful learning opportunity in the craft of research by exposing you to the most recent research. When reviewing you find ‘mistakes’ in articles which you
then realize are also present in your own writing. Finally, it is easier to respond to reviewers when you understand how a reviewer thinks.
bs_bs_banner
Management and Organization Review 10:2, July 2014, 175–181
doi: 10.1111/more.12060
© 2014 The International Association for Chinese Management Research
HOW TO BE A REVIEWER
There are a number of good reasons for becoming a reviewer: professionalism,networking, and learning. However, reviewing remains a challenging task for the
new reviewer. Journals provide reviewer guidelines, and there is also an extensive literature providing key reviewer objectives (e.g., Carpenter, 2009; Colquitt &
Ireland, 2009; Feldman, 2003, 2008; Lepak, 2009; Tsang & Frey, 2007), but new reviewers can be challenged in translating these general guidelines into action, so
I will provide both general guidelines as well as some specific advice on reviewing.General Guidelines for Reviewing a Manuscript General guidelines make clear the key roles and duties of a reviewer. These guidelines apply not only to MOR and the International Association for Chinese Management Research (IACMR) bi-annual conference, but also to reviews for
other journals. In no particular order, you should be mindful of the points sum-marized in Table 1.

The role of the reviewer. Your role is to help the editor decide which articles to publish,and to help authors improve the quality of their paper. Identifying problems is only part of what you should do (Feldman, 2008); you should also help the authors bring forward the strengths present within their paper. This requires both an evaluative and a developmental focus, and a consideration of both strengths and weaknesses within the paper. It is critical that evaluative comments are provided only to the editor, while authors are given developmental comments. When you feel the paper is more suited for a different journal, mention this to the editor rather than in your review. Don’t automatically decline to review papers outside your current research Table 1. Some general guidelines for reviewing Reviewer Role 

• Provide evaluative comments to the editor, developmental comments to the authors

• Identify both strengths and weaknesses
• It is the author’s paper, not yours
Reviewer Voice & Tone
• Be honest, but kind
• Provide actionable advice
• Quantity in a review is not the same as quality
Writing the Review
• Number your comments, and indicate the page/line numbers your comments refer to
• Do some library research
• Clearly indicate whether an issue is major or minor
• Proofread your own review
• For a resubmission, read the other reviews and the editor’s decision
176 Editorial
© 2014 The International Association for Chinese Management Research
topic,asreviewinginotherareasispartofthelearningexperience,particularlyifyou do a bit of additional reading. When a paper is poorly crafted, it is not your respon-sibility to provide research training for authors. In these cases, a good approach wouldbetofocusonafewkeyissues.Finally,rememberthatyourgoalistohelptheauthors put forward their ideas, not write the paper you would have written.Reviewer voice and tone. It is easy to be insensitive in a review, but when a paper has problems it is important to communicate this. Think of the review as a conversa-

tion, and say things you would feel comfortable saying in person. This doesn’t mean you should only give positive comments, but ensure your negative comments are constructive. One way of making your review constructive is to focus on actionable advice, such as suggesting alternative theoretical perspectives or ana-lytical methods rather than merely pointing out errors. Review length is not the same as review quality, and you should focus on fundamental concerns rather than minutiae such as grammatical errors. An overly long review can discourage authors, so it might help to focus on major issues in a first review and then clean up minor issues in a second review.


Writing the review. When you write your review, number your comments, and indicate the page/line numbers you are referring to. In addition, indicate whether you think a concern is major or minor; it is also helpful to indicate whether this can be answered in the response letter or if it belongs in the revised manuscript. These actions help both the editor and the author. For a revised manuscript, read the other reviews and the editor’s decision for a resubmission so that you understand how have authors tried to balance sometimes contradictory reviews, and recognize that while editors might not always agree with you this is neither personal nor taken lightly (Lepak, 2009). Finally, it is helpful to put your review aside for a day or two, and then proofread it one last time before submitting it, as poorly crafted reviews

lack credibility.


Specific Criteria to Use While Reviewing a Manuscript Simply knowing the general responsibilities for a reviewer is not enough to ensure that a quality review follows. A journal like MOR publishes papers covering a broad range of topics, which makes the creation of a detailed listing of criteria impracti-cal. I will instead focus upon a set of questions you could consider while reviewing. The introduction. The first impression created by the introduction can make or break a paper. Each author has a personal style, and each paper has a different story, so while there is no single format for an introduction, there are specific characteristics to look for. First, you will want to see that the research question and objectives are clearly stated, and an interesting paper will begin by identifying a research problem Editorial 177


© 2014 The International Association for Chinese Management Research or question. We commonly speak of ‘research gaps’, but another good mental
framing tool is to think in terms of ‘research needs’. Not all gaps need to be filled, because they might be deemed to be obvious or unimportant. In focusing on
identifying the theoretical or empirical questions that need to be understood, you can help the author develop a stronger case for their contribution. At the end of
the introduction, ask yourself what expectations you have of the paper. When the actual contribution differs from the promised contribution, there is a need to
help the author modify the introduction to make the actual contribution clear. The introduction should conclude by summarizing the unique insights of the authors
rather than with a ‘table of contents’ (i.e., ‘first is the literature review, then . . .’).
The literature review. The literature review provides the motivation for the research project and the theoretical and logical underpinnings of the study. The review must
strike a balance between completeness and focus, and should concentrate on theoretical explanations and processes rather than only definitions and empirical
findings. How recently the literature and source journals being cited were pub-lished is something to consider. Authors drawing upon old research should provide
strong justification for re-examining an old topic, while papers citing sources outside the traditional management mainstream must make the ideas relevant to
management scholars. Rather than being dismissive of citations from lower status journals, consider the merits of the ideas being presented. Your feedback might
also identify relevant literature which the authors have missed.
Hypotheses development. Rigorous theoretical development of hypotheses is the heart of the paper, yet assessing theoretical contribution is both difficult and subjective.
A common fault in hypothesis development is ‘argument by result’, and you will want to nudge the authors towards the ‘how’ and ‘why’ underlying their hypoth-
eses. Hypotheses should not be too obvious, and simple replications rarely make a significant contribution, while the more surprising or counter-intuitive the hypoth-
eses the greater the rigour in theoretical development needed. Reviewers often speak of ‘over-arching theory’, but this shorthand can lead to misleading feedback,
implying a need to make the paper an application of a single parent theory.Instead, it helps to consider whether the paper presents piece-wise theorizing,
explaining individual variable choices and relationships rather than the collective choice of variables and overall structure of the model. Guiding authors towards
holistic or integrative theorizing will make the paper stronger.
Methods and sample. Focusing upon methodological issues is common, particularly with less experienced reviewers, but should rarely be the entire focus of the review.
Using simple methods is not automatically bad, nor does the use of complicated methods immediately make the paper stronger. Instead, it should be a matter of

appropriate methods and samples (Feldman, 2003) which make the results clear 178 Editorial


© 2014 The International Association for Chinese Management Research
and easy to understand, and which don’t introduce the possibility of bias. Methods
and sample must be explained clearly, and when uncommon methods are used the
author should explain the method and the benefits of that method.
Analysis and results. It is necessary to evaluate whether the analysis has been done
correctly. You will want to consider the amount of support found for the hypoth-
eses, but keep in mind that no empirical paper is perfect. It isn’t necessary for
all hypotheses to be supported. Indeed, strong and complete support for a very
complicated or counter-intuitive set of hypotheses merits a higher level of scrutiny.
Requesting further clarification, alternative analyses, or more rigorous theorizing,
are ways in which you might address your concerns here.
Discussion. The discussion section should contain just that, a discussion, not a
restatement of hypotheses and results. The paper’s contribution must be identified,
not as a simple enumerated listing, but in terms of implications for managers
and other researchers. The discussion should have both a past ‘what we have
learned’ orientation, as well as a future ‘what comes next’ orientation, and move
beyond simple calls for generalizability. Obviously, conclusions and implications
must be supported by the findings. The discussion typically includes a brief section
on limitations. However, this should not be an excuse for poor methods, but rather
a discussion of how study design might influence the interpretation of results.
Forminganoverallimpression. Ultimately,youwillneedtomakeareject/revise/accept
decision.Itisimportanttorealizethatthereareno‘perfect’papers,sowhatyouwant
todoishelptheeditorsidentifypaperswhichare‘interesting’.Paperswhichpresent
obvious and predictable findings are generally not very interesting, while unexpec-
ted but believable ideas and findings are interesting. Papers which will influence
futureresearchareinteresting,butitisimportanttorememberthatneitheryounor
the editors really know whether the paper will ultimately prove influential. You will
needtodecidewhetheryoubelievethepapercanbefixed,whethertheauthorshave
theabilitytofixit,andwhethertheresultingpaperwouldbepublishable.Youdon’t
want to give false hope to an author by requesting additional revisions unless you
believe that revisions could lead to a publishable paper.
Unique Characteristics of the MOR Review
Reviewing for MOR is very similar to reviewing for other journals, with only a few
minor points to keep in mind.
The opening paragraph of the review. MOR asks that the first paragraph of your review
focus upon your understanding of the paper’s intent and contribution. If an author
sees that your perceptions do not match their intention, this can indicate that the
introduction will need to be revised in order to clarify their actual intentions.
Editorial 179
© 2014 The International Association for Chinese Management Research
Developmental reviews. In general the reviewer’s role is to develop, not to teach, but
given that part of the mission of the IACMR and MOR is to promote research by
early career scholars, we hope that our reviewers could incorporate some elements
of teaching in their reviewers. Guiding the authors towards answering the ques-
tions listed in Table 2 is one way in which you can help develop such scholars.
Given the importance that the IACMR and MOR give to the developmental
mission, if you have any doubts about a paper, it is probably best to recommend a
revise-and-resubmit, particularly when the paper has an interesting idea which
needs further development.
Language and writing style. While reviewers are certainly free to point out grammati-
cal problems, this should not be the determining factor of the reviewer’s recom-
mendation to the editor unless the quality of the writing is so bad as to limit the
comprehensibility of the manuscript. Focus primarily on the ideas, and secondarily
on the language and grammar, as such matters can be fixed at a later stage.
Table 2. Questions a reviewer should consider while reading a manuscript
The Introduction
• What is the research question/problem/need driving the paper?
• What do the authors promise to deliver in the paper?
• Does the introduction accurately and clearly introduce the paper?
The Literature Review
• Is this a review of ideas and processes, or a review of definitions and findings?
• Is the literature relevant, or is literature reviewed which is not used?
• Is the extant literature accurately related and used?
• Are there related concepts appropriate to discuss?
Development of Hypotheses or Research Questions
• Do hypotheses plausibly follow from the logical argument?
• Are the hypotheses obvious? Original? Counter-intuitive?
• Holistic or piece-wise theorizing? Is the theorizing coherent and logically consistent?
Methods and Sample
• Is the sample appropriate for the hypotheses?
• Are the measures consistent with the theoretical constructs?
• Are the methods appropriate for the data and hypotheses?
• If an unusual method is used, is the method and rationale adequately explained?
Analysis and Results
• Is there enough detail to determine if it was done correctly?
• Do methodological issues lead to potential bias, or simply increase standard errors?
• Do the results look too good? Does it look like the hypotheses were chosen to fit the data?
Discussion
• Is it really a discussion, or just repeat theory and results?
• Are key contributions clearly stated, or do they extend beyond what the results support?
• Is a discussion of limitations just there for appearance sake, or as an excuse for poor research?
Forming an Overall Impression
• Are the ideas presented in the paper interesting or obvious?
• Did the revision address the review? Create new problems?
• Are the problems fixable? If they were fixed, would the paper be publishable?
180 Editorial
© 2014 The International Association for Chinese Management Research
Do papers need to be on China? MOR is the official journal of the IACMR, but a
common misconception is that all papers submitted to MOR must involve China
or Chinese people. MOR is committed to publishing quality research, regardless of
the origin of the authors, data, or ideas. MOR editorial policy is to publish papers
which are relevant to China and other developing economies.
Assessing contribution. MOR is interested in publishing papers which are ground
breaking, and which will have broad influence upon scholars interested in Chinese
or developing economy management. MOR is not interested in publishing research
which makes incremental contributions to existing theory or knowledge. If the
paper presents interesting ideas which are still not fully developed, recommending
a revise-and-resubmit would be appropriate.
CONCLUDING COMMENTS
Being a developmental reviewer can be challenging, because it is usually easier
to say what is wrong with a paper than it is to provide advice on how to improve
the paper. At the same time, reviewing can be a rewarding task. The advice and
guidelines here are really just a starting point, and each reviewer will develop their
own reviewing style, but hopefully this will help you in your own professional
development as a reviewer.
NOTE
I would like to thank Brian Boyd, Kwok Leung, Arie Lewin, Christina Sue-Chan, Anne Tsui, and
Haibin Yang for their comments on early drafts of this paper.
REFERENCES
Carpenter, M. A. 2009. Mentoring colleagues in the craft and spirit of peer review. Academy of Management Review, 34(2): 191–195.
Colquitt, J. A., & Ireland, R. D. 2009. Taking the mystery out of AMJ’s reviewer evaluation form.
Academy of Management Journal, 52(2): 224–228.
Feldman, D. C. 2003. Sense and sensibility: Balancing the interests of authors, reviewers, and editors.
Journal of Management, 29(1): 1–4.
Feldman, D. C. 2008. Building and maintaining a strong editorial board and cadre of ad hoc reviewers. In Y. Baruch, A. M. Konrad, H. Aguinis & W. H. Starbuck (Eds.), Opening the
black box of editorship: 68–74. New York: Palgrave Macmillan.
Lepak, D. 2009. What is good reviewing? Academy of Management Review, 34(3): 375–381.
Tsang, E. W. K., & Frey, B. S. 2007. The as-is journal review process: Let authors own their ideas.
Academy of Management Learning & Education, 6(1): 128–136.
Tsui, A. S., & Holleneck, J. R. 2009. Successful authors and effective reviewers: Balancing supply and
demand in the organizational science. Organizational Research Methods, 12(2): 259–275.