Peer-Review Workshop I

Research Project 1 (RP1) — U7

Andy Weeger

Neu-Ulm University of Applied Sciences

September 3, 2026

Opening Remarks

Give and take

A short lecture on reviewing well, then you review, and are reviewed.

Everything you have built so far, topic, theory type, research question, has been tested only against your own team’s judgement and our feedback; this unit adds the stranger’s reading neither provides by pairing you with a team that does not share your assumptions, working only from what you actually wrote, to check whether your framing holds together.

Agenda

• Peer review rationale 15 min
• Reviewing well 15 min
• Calibration exercise 30 min
• The rubric 5 min
• Roundtable workshop 75 min
• Wrap-up 10 min

Learning Outcomes

By the end of this unit, you will be able to:

• Write a structured, constructive review against an explicit rubric, citing specific claims rather than general impressions.
• Distinguish what a peer review is, feedback aimed at improvement, a check on your reasoning, from what it is not: the final call, or a place for personal point-scoring.
• Respond to a review of your own framing, incorporating each point or explaining, in writing, why you chose not to.

Peer Review Rationale

Basics

Scientific publications are peer-reviewed.

• Before a paper is published, it is reviewed by experts in the field.
• Most often a double-blind process: authors and reviewers do not know each other’s identity.
• The process is managed by an editor, an experienced scholar who selects reviewers and makes the publication decision.

Desk Rejects

Some papers are rejected without even being sent to reviewers. Why?

• Lack of fit: the paper is not in the domain, or not general enough for the outlet. This is not an evaluation of quality.
• Lack of likelihood: the paper does not have a good chance of successfully passing review, and reviewer time is a scarce resource worth protecting.
• Lack of formalities: the paper does not meet submission requirements, or the writing is not yet good enough to review productively.

Outcomes

Submitting work for review leads to one of three outcomes:

Rejection, revision (major or minor), or (conditional) accept.

• Rejection: the outlet will not consider this manuscript, or any revision of it.
• Revise and resubmit: a reworked version may be accepted if resubmitted, typically to the same editor and reviewers. Major revision, extensive changes required, still high risk. Minor revision, comments and suggestions, lower risk.
• Acceptance or conditional acceptance: the work can be published as is, or with minor changes.

Virtually all first-round reviews require significant revision, if they are not rejected outright. Assume the same for your own work: your first draft of any deliverable will not be accepted as is, and extensive revisions are the normal case, not a sign something went wrong (Recker, 2021).

Review Scope

What a review is

• Feedback from your peers, aimed at real improvement.
• The stamp of methodological quality, someone besides you checked the chain of reasoning.

What a review is not

• The final call: reviewers recommend; the editor decides.
• A place for bias, prejudice, or personal point-scoring, though it can feel that way if done badly.

Reviewer Motivation

• Duty: you will receive reviews from others; you owe the same effort back.
• Skill-building: reviewing critically sharpens what you notice in your own framing. Good reviewers write good frames.
• The discipline moves forward: peer review is how a field’s collective standard stays higher than any one person’s.

Reviewing Well

Vague vs. Specific

Vague

“This doesn’t work. Rethink the question.”

Specific

“Your knowledge gap is about measurement inconsistency, but the question asks about causal mechanisms instead. State which one you mean, or explain how the mechanism produces the inconsistency.”

Both comments respond to the same flaw: a research question that does not follow from the knowledge gap the author stated. The vague version names a feeling, doesn’t work, without pointing at what to fix; it forces the author to guess. The specific version names the exact mismatch, quotes the two claims that don’t line up, and gives the author two concrete paths forward. The advice on the next slide operationalizes this difference: aim every comment you write at the specific version’s level of detail.

Process

• Read the framing.
• Read the review rubric (what to look for).
• Read the framing again.
• Note areas for enhancement.
• Wait a few days, if you can.
• Read once more, then write.

Put yourself in the author’s shoes, and offer constructive criticism.

Write reviews you would like to receive: reviews that point toward how to fix what is inadequate, beyond simply flagging it.

Additional Advice

Be polite and conversational, identify strengths first, search for the gem, be concise, be consistent, and be on time.

• Be polite and conversational: write “you,” not “the authors.”
• Search for the gem: find the strongest version of what the team is trying to argue, and help them get there, beyond simply flagging its flaws.
• Be concise: one point per comment, reference specific lines or slides.
• Identify strengths first: open with what genuinely works before turning to concerns.
• Be consistent: your written comments and your overall verdict should tell the same story.
• Be on time: the other team is waiting on you the same way you will wait on them next time.

Calibration Exercise

Before reviewing a teammate’s work, calibrate on someone else’s.

1. Recap the hypothetical AoM micro submission “Responses to Transformational Leadership: Are Some Followers Immune?” and note its strengths and weaknesses yourself.
2. Recap the editors’ sample reviews and compare them with your own assessment.
3. Discuss: where did the sample reviews catch something you missed? Where were they more, or less, generous than you would have been?

20:00

Second Pass

Return to the editors’ sample reviews with the rubric categories in mind: chain integrity, failure type, Gregor type, and lens use.

1. For each comment in the sample reviews, decide which of the four categories it addresses.
2. Flag any comment that fits none of the four. What is it actually checking?
3. Discuss: does the rubric cover everything a good review needs to say, or did the sample surface something worth adding to it?

10:00

The Rubric

Checklist

This checklist replaces a vague “is this good” with four specific, checkable things.

1. Is the chain unbroken? Domain → phenomenon → knowledge gap → research question. Can you follow it without the authors in the room to explain a jump?
2. Which failure type, if any, is present? Monologuing, so-what, solving-the-world, multitude, or one of the five inappropriate-question categories from Good Research Questions.
3. Which Gregor type does the question promise? Analysis, explanation, prediction, explanation and prediction, or design and action, and does the rest of the framing actually aim at that type consistently?
4. Which theoretical lenses are doing real work, and which are decoration? A lens that is named but never used to generate a claim or a boundary condition is not yet earning its place.

This rubric is built directly from the last four units and introduces no new criteria. It operationalizes what you have already learned into something you can apply to someone else’s work in twenty minutes. If a criterion here feels unfamiliar, that is a signal to revisit the corresponding unit before you review, not to skip the criterion.

Roundtable Workshop

Form tables of three teams. Each round has three roles: the Author shares their framing (topic, theory type, draft research question), the Reviewer gives live feedback against the four rubric points, and the Scribe, the third team at the table, writes that feedback down for the Author.

1. Round 1 (20 min): Team A is Author, Team B is Reviewer, Team C is Scribe.
2. Round 2 (20 min): Team B is Author, Team C is Reviewer, Team A is Scribe.
3. Round 3 (20 min): Team C is Author, Team A is Reviewer, Team B is Scribe.

Every team reviews once, is reviewed once, and scribes once, and leaves with written feedback it did not have to write while also listening to it. Keep the Scribe’s notes: you need it for your revision.

60:00

Tables of four: add a fourth round (Team D reviews Team A, Team B scribes), and shorten every round to 15 minutes so the table still finishes in about the same time.  Tables of two: skip the scribe role for that pair; the Author takes their own notes instead.

Suggested shape for each 20-minute round: about 3 minutes for the Author to summarize the framing, 12 minutes for the Reviewer to give feedback against the four rubric points while the Scribe writes, and the remaining 5 minutes for the Author to ask clarifying questions and confirm the Scribe’s notes are accurate before the table rotates.

This replaces a written, solitary review with a live, spoken one. The Reviewer does not have to split attention between talking and writing, and the Author leaves with a legible written record without having had to write it themselves while still processing the feedback out loud.

Read-Aloud Debrief

Each Scribe reads *+one note** from their table aloud. The team that received it says, in one word, whether it is actionable.

20:00

Common Failure Modes

• Halo effect: one strong element, a clean literature summary, a polished slide, makes you read the rest of the framing more favorably than it deserves.
• Confirmation bias: you review to confirm your first impression instead of testing whether the chain still holds up on a second, slower read.
• Anchoring on your own project: you end up flagging differences from what your own team would have done, rather than checking strictly against the rubric.
• Politeness inflation: you soften every point until the written review reads more generous than your private assessment, and the author cannot tell which point actually mattered.

Each failure mode pulls a review away from the rubric and toward the reviewer’s own state of mind. The halo effect and confirmation bias are documented across evaluation settings generally; naming them here is a reminder to check each rubric point independently on a second, slower pass rather than trusting a single overall impression formed on the first read. Anchoring on your own project and politeness inflation are specific to peer-review settings where reviewer and author are close in status and stake: guard against both by writing the review against the rubric, point by point, rather than against a running sense of whether the framing feels fine.

Homework

Revise your Problem Framing Brief in light of the review you received.

For each point raised in your review, either incorporate the change, or write one sentence explaining why you chose not to (to foster your own reflection).

Q&A